Mailfromd Manual (split by chapter):   Section:   Chapter:FastBack: s-expression   Up: Top   FastForward: Copying This Manual   Contents: Table of ContentsIndex: Concept Index

Appendix D Upgrading

The following sections describe procedures for upgrading between the consecutive Mailfromd releases. The absense of a section for a pair of versions x-y numbers means that no specific actions are required for upgrading from x to y.

D.1 Upgrading from 8.5 to 8.6

New configure option --with-dbm allows you to select any DBM flavor supported by GNU mailutils as the default DBM implementation for mailfromd.

D.2 Upgrading from 8.2 to 8.3 (or 8.4)

Versions 8.3 and 8.4 differ only in required minimal version of mailutils (3.3 and 3.4, correspondingly). Apart from that, the following instructions apply to both versions.

In version 8.3 I abandoned the legacy DNS resolver and switched to GNU adns. GNU ands is a standalone resolver library, which provides a number of advanced features. It is included in most distributions. The source code of the recent release is available from http://www.chiark.greenend.org.uk/~ian/adns/adns.tar.gz.

This change brought a number of user-visible changes. In particular, arbitrary limits on the sizes of the RRs are removed. Consequently, the following configurations statements are withdrawn:

runtime.max-dns-reply-a
runtime.max-dns-reply-ptr
runtime.max-dns-reply-mx
max-match-mx
max-callout-mx

Secondly, the new resolver is less tolerant to deviations from the standard. This means that remote DNS misconfigurations that would have slipped unnoticed in previous versions, will be noticed by mailfromd 8.3. For example, a CNAME record pointing to another CNAME is treated as an error.

A new command line option was added to mailfromd and calloutd: --resolv-conf-file. This option instructs the programs to read resolver settings from the supplied file, instead of the default /etc/resolv.conf.

Another user-visible change is in handling of SPF checks. Previously, results if SPF checks were cached in a database. This proved to cause more problems than solutions and was removed in this version. As a result, the following MFL global variables have been withdrawn:

spf_ttl
spf_cached
spf_database
spf_negative_ttl

D.3 Upgrading from 7.0 to 8.0

Version 8.0 is a major rewrite, that introduces a lot of new concepts and features. Nevertheless, it is still able to run the MFL scripts from version 7.0 without modifications.

Note the following important points:

D.4 Upgrading from 6.0 to 7.0

The release 7.0 removes the features which were declared as obsolete in 6.0 and introduces important new features, both syntactical, at the MFL level, and operational.

Unless your filter used any deprecated features, it should work correctly after upgrade to this version. It will, however, issue warning messages regarding the deprecated features (e.g. the use of ‘%’ in front of identifiers, as described below). To fix these, follow the upgrade procedure described in upgrade procedure.

The removed features are:

The MFL syntax has changed: it is no longer necessary to use ‘%’ in front of a variable to get its value. To reference a variable, simply use its name, e.g.:

set x var + z

The old syntax is still supported, so the following statement will also work:

set x %var + %z

It will, however, generate a warning message.

Of course, the use of ‘%’ to reference variables within a string literal remains mandatory.

Another important changes to MFL are user-defined exceptions (see User-defined Exceptions) and the try–catch construct (see try–catch).

Several existing MFL functions have been improved. In particular, it is worth noticing that the open function, when opening a pipe to or from a command, provides a way to control where the command’s standard error would go (see standard error).

The accept function (or action) issues a warning if its use would cancel any modifications to the message applied by, e.g., header_add and similar functions. See Message modification queue, for a detailed discussion of this feature.

The most important change in mailfromd operation is that the version 7.0 is able to run several servers (see conf-server). Dedicated callout servers make it possible to run sender verifications in background, using a set of long timeouts, as prescribed by RFC 2822 (see SMTP Timeouts). This diminishes the number of false positives, allows for coping with servers showing large delays and also reduces the number of callouts performed for such servers.

This release no longer includes the smap utility. It was moved into a self-standing project, which by now provides much more functionality and is way more flexible than this utility was. If you are interested in using smap, visit http://www.gnu.org.ua/software/smap, for a detailed information, including pointers to file downloads.

D.5 Upgrading from 5.x to 6.0

The 6.0 release is aimed to fix several logical inconsistencies that affected the previous versions. The most important one is that until version 5.2, the filter script file contained both the actual filter script, and the run-time configuration for mailfromd (in form of ‘#pragma option’ and ‘#pragma database’ statements). The new version separates run-time configuration from the filter script by introducing a special configuration file mailfromd.conf (see Mailfromd Configuration).

Consequently, the ‘#pragma option’ and ‘#pragma database’ statements become deprecated. Furthermore, the following deprecated pragmas are removed: ‘#pragma option ehlo’, ‘#pragma option mailfrom’. These pragmas became deprecated in version 4.0 (see 31x-400).

The second problem was that the default filter script file had ‘.rc’ suffix, which usually marks a configuration file, not the source. In version 6.0 the script file is renamed to mailfromd.mf. In the absence of this file, the legacy file mailfromd.rc is recognized and parsed. This ensures backward compatibility.

This release also fixes various inconsistencies and dubious features in the MFL language.

The support for unquoted literals is discontinued. This feature was marked as deprecated in version 3.0.

The following features are deprecated: ‘#pragma option’ (pragma-option and ‘#pragma database’ (pragma-database) directives, the legacy style of function declarations (old-style function declarations), calls to functions of one argument without parentheses (operational notation), the ‘#require’ statement (See import, for the new syntax) and implicit concatenation (implicit concatenation). See Deprecated Features, for more information about these.

This release also introduces important new features, which are summarized in the table below:

FeatureReference
ConfigurationSee Mailfromd Configuration.
Module systemSee Modules.
Explicit type castsSee explicit type casts.
Concatenation operatorSee Concatenation.
Scope of visibilitySee scope of visibility.
Precious variablesSee rset.

Mailfromd version ‘6.0’ will work with unchanged scripts from ‘5.x’. When started, it will verbosely warn you about any deprecated constructs that are used in your filter sources and will create a script for upgrading them.

To upgrade your filter scripts, follow the steps below:

  1. Run ‘mailfromd --lint’. You will see a list of warnings similar to this:
    mailfromd: Warning: using legacy script file
     /usr/local/etc/mailfromd.rc
    mailfromd: Warning: rename it to /usr/local/etc/mailfromd.mf
     or use script-file statement in /usr/local/etc/mailfromd.conf
     to disable this warning
    mailfromd: /usr/local/etc/mailfromd.rc:19: warning: this pragma is
     deprecated: use relayed-domain-file configuration statement instead
    mailfromd: /usr/local/etc/mailfromd.rc:23: warning: this pragma is
     deprecated: use io-timeout configuration statement instead
    mailfromd: Info: run script /tmp/mailfromd-newconf.sh
     to fix the above warnings
    … 
    
  2. At the end of the run mailfromd will create a shell script /tmp/mailfromd-newconf.sh for fixing these warnings. Run it:
    $ sh /tmp/mailfromd-newconf.sh
    
  3. When the script finishes, run mailfromd --lint again. If it shows no more deprecation warnings, the conversion went correctly. Now you can remove the upgrade script:
    $ rm /tmp/mailfromd-newconf.sh
    

Notice, that the conversion script attempts to fix only deprecation warnings. It will not try to correct any other type of warnings or errors. For example, you may get warning messages similar to:

mailfromd: /etc/mailfromd.mf:7: warning: including a module file is unreliable and may cause subtle errors
mailfromd: /etc/mailfromd.mf:7: warning: use `require dns' instead

This means that you use ‘#include’ where you should have used ‘require’. You will have to fix such warnings manually, as suggested in the warning message.

If, for some reason, you cannot upgrade your scripts right now, you may suppress deprecation warnings by setting the environment variable MAILFROMD_DEPRECATION to ‘no’ before starting mailfromd. Nonetheless, I recommend to upgrade as soon as possible, because the deprecated features will be removed in version ‘6.1’.

D.6 Upgrading from 5.0 to 5.1

Upgrading from 5.0 to 5.1 does not require any changes in your filter scripts. Notice, however, the following important points:

D.7 Upgrading from 4.4 to 5.0

This version of Mailfromd requires GNU mailutils version 2.0 or later.

Upgrading from version 4.4 to 5.0 requires no additional changes. The major differences between these two versions are summarized below:

  1. Support for ‘MeTA1’.
  2. New ‘Mailutils’ configuration file.
  3. New MFL functions.
    1. Message functions. See Message functions.
    2. Mailbox functions. See Mailbox functions.
    3. Mail body functions. See Mail body functions.
    4. Header modification functions. See Header modification functions.
    5. Envelope modification functions. See Envelope modification functions.
    6. Quarantine functions. See Quarantine functions.
    7. getopt and varptr. See getopt.
    8. Macro access functions. See Macro access.
    9. Character type functions. See Character Type.
    10. New string functions (see String manipulation): verp_extract_user, sa_format_report_header, sa_format_score.
    11. Sequential access to DBM files. See dbm-seq.
  4. Changes in MFL
    1. See variadic functions.
    2. See function alias.
  5. New operation mode: See Run Mode.
  6. Improved stack growth technique.

    The stack can be grown either by fixed size blocks, or exponentially. Upper limit can be specified. See stacksize.

  7. Milter ports can be specified using URL notation.
  8. Removed deprecated features.

    Support for some deprecated features has been withdrawn. These are:

    1. Command line options --ehlo, --postmaster-email, and --mailfrom. These became deprecated in version 4.0. See 31x-400.

D.8 Upgrading from 4.3.x to 4.4

The deprecated --domain command line option has been withdrawn. The short option -D now defines a preprocessor symbol (see Preprocessor Options).

This version correctly handles name clashes between constants and variables, which remained unnoticed in previous releases. See variable--constant shadowing, for a detailed description of it.

To minimize chances of name clashes, all symbolic exception codes has been renamed by prefixing them with the ‘e_’, thus, e.g. divzero became e_divzero, etc. The ioerr exception code is renamed to e_io. See status.mf, for a full list of the new exception codes.

For consistency, the following most often used codes are available without the ‘e_’ prefix: success, not_found, failure, temp_failure. This makes most existing user scripts suitable for use with version 4.4 without any modification. If your script refers to any exception codes other than these four, you can still use it by defining a preprocessor symbol OLD_EXCEPTION_CODES, for example:

$ mailfromd -DOLD_EXCEPTION_CODES

D.9 Upgrading from 4.2 to 4.3.x

Upgrading from 4.2 to 4.3 or 4.3.1 does not require any changes to your configuration and scripts. The only notable change in these versions is the following:

The asynchronous syslog implementation was reported to malfunction on some systems (notably on Solaris), so this release does not enable it by default. The previous meaning of the --enable-syslog-async configuration option is also restored. Use this option in order to enable asynchronous syslog feature. To set default syslog implementation, use DEFAULT_SYSLOG_ASYNC configuration variable (see syslog-async).

The following deprecated features are removed:

  1. #pragma option ehlo statement.

    It became deprecated in version 4.0. See pragma-option-ehlo.

  2. #pragma option mailfrom statement.

    It became deprecated in version 4.0. See pragma-option-ehlo.

  3. The --config-file command line option.

    It became deprecated in version 3.1. See 30x-31x.

  4. Built-in exception codes in catch statements.

    They are deprecated since version 4.0. See 31x-400.

D.10 Upgrading from 4.1 to 4.2

Upgrading to this version does not require any special efforts. You can use your configuration files and filter scripts without any changes. The only difference worth noticing is that starting from this version mailfromd is always compiled with asynchronous syslog implementation. The --enable-syslog-async configuration file option is still available, but its meaning has changed: it sets the default syslog implementation to use (see syslog-async). Thus, it can be used the same way it was in previous versions. You can also select the syslog implementation at run time, see –syslog-async option, for more detailed information.

D.11 Upgrading from 4.0 to 4.1

Upgrading to this version does not require any special efforts. You can use your configuration files and filter scripts without any changes. Notice only the following major differences between 4.1 and 4.0:

D.12 Upgrading from 3.1.x to 4.0

Before building this version, please re-read the chapter See Building, especially the section Using non-blocking syslog.

Starting from the version 4.0, MFL no longer uses the predefined symbolic names for exception codes (previous versions used the ‘&’ prefix to dereference them). Instead, it relies on constants defined in the include file status.mfh (see status.mf).

However, the script files from 3.1 series will still work, but the following warning messages will be displayed:

Warning: obsolete constant form used: &failure
Warning: remove leading '&' and include <status.mfh>

Warning: Using built-in exception codes is deprecated
Warning: Please include <status.mfh>

Another important difference is that pragmatic options ‘ehlo’ and ‘mailfromd’ are now deprecated, as well as their command line equivalents --ehlo and --domain. These options became superfluous after the introduction of mailfrom_address and ehlo_domain built-in variables. For compatibility with the previous versions, they are still supported by mailfromd 4.0, but a warning message is issued if they are used:

warning: `#pragma option ehlo' is deprecated,
 consider using `set ehlo_domain "domain.name"' instead

To update your startup scripts for the new version follow these steps:

  1. Change #pragma option mailfrom value to set mailfrom_address value. Refer to mailfrom_address, for a detailed discussion of this variable.
  2. Change #pragma option ehlo value to set ehlo_domain value. Refer to ehlo_domain, for a detailed discussion of this variable.
  3. Include status.mfh. Add the following line to the top of your startup file:
    #include_once <status.mfh>
    
  4. Remove all instances of ‘&’ in front of the constants. You can use the following sed expression: ‘s/&\([a-z]\)/\1/g’.
  5. If your code uses any of the following functions: hostname, resolve, hasmx or ismx, add the following line to the top of your script:
    #require dns
    

    See Modules, for a detailed description of the module system.

  6. Replace all occurrences of next with pass.
  7. If your code uses function match_cidr, add the following line to the top of your script:
    #require match_cidr
    

    See Modules, for a description of MFL module system.

D.13 Upgrading from 3.0.x to 3.1

  1. The mailfromd binary no longer supports --config-file (-c) option. To use an alternative script file, give it as an argument, i.e. instead of:
    $ mailfromd --config-file file.rc
    

    write:

    $ mailfromd file.rc
    

    For backward compatibility, the old style invocation still works but produces a warning message. However, if mailfromd encounters the -c option it will print a diagnostic message and exit immediately. This is because the semantics of this option will change in the future releases.

  2. If a variable is declared implicitly within a function, it is created as automatic. This differs from the previous versions, where all variables were global. It is a common practice to use global variables to pass additional information between handlers (See HELO Domain, for an example of this approach). If your filter uses it, make sure the variable is declared as global. For example, this code:
    prog helo
    do
      # Save the host name for further use    
      set helohost $s
    done
    

    Figure D.1: Implicit declaration, old style

    has to be rewritten as follows:

    set helohost ""
    
    prog helo
    do
      # Save the host name for further use    
      set helohost $s
    done
    

    Figure D.2: Implicit declaration, new style

  3. Starting from version 3.1 the function dbmap takes an optional third argument indicating whether or not to count the terminating null character in key (see dbmap). If your startup script contained any calls to dbmap, change them as follows:
    in 3.0.xin 3.1
    dbmap(db, key)dbmap(db, key, 1)

D.14 Upgrading from 2.x to 3.0.x

Update your startup scripts and/or crontab entries. The mailfromd binary is now installed in ${prefix}/sbin.

We also encourage you to update the startup script (run cp etc/rc.mailfromd /wherever-your-startup-lives), since the new version contains lots of enhancements.

D.15 Upgrading from 1.x to 2.x

If you are upgrading from version 1.x to 2.0, you will have to do the following:

  1. Edit your script file and enclose the entire code section into:
    prog envfrom
    do
      …
    done
    

    See Handlers, for more information about the prog statement.

  2. If your code contained any rate statements, convert them to function calls (see rate), using the following scheme:
    Old statement: if rate key limit / expr
    New statement: if rate(key, interval("expr")) > limit
    

    For example,

       rate $f 180 / 1 hour 25 minutes
    

    should become

       rate($f, interval("1 hour 25 minutes")) > 180
    
  3. Rebuild your databases using the following command:
    mailfromd --compact --all   
    

    This is necessary since the format of mailfromd databases has changed in version 2.0: the key field now includes the trailing ‘NUL’ character, which is also reflected in its length. This allows for empty (zero-length) keys. See Database Maintenance, for more information about the database compaction.

Mailfromd Manual (split by chapter):   Section:   Chapter:FastBack: Upgrading   Up: Upgrading   FastForward: Copying This Manual   Contents: Table of ContentsIndex: Concept Index