General-Purpose Mail Filter
When using polling functions, it is important to take into account possible delays, which can occur in SMTP transactions. Such delays may be due to low network bandwidth or high load on the remote server. Some sites impose them willingly, as a spam-fighting measure.
Ideally the callout verification should use the timeout values defined in the RFC 2822, but this is impossible in practice, because it would cause a timeout escalation, which consists in propagating delays encountered in a callout SMTP session back to the remote client whose session initiated the callout.
Consider, for example, the following scenario. An MFL script performs a callout on ‘envfrom’ stage. The remote server is overloaded and delays heavily in responding, so that the initial response arrives 3 minutes after establishing the connection, and processing the ‘EHLO’ command takes another 3 minutes. These delays are OK according to the RFC, which imposes a 5 minute limit for each stage, but while waiting for the remote reply our SMTP server remains in the ‘envfrom’ state with the client waiting for a response to its ‘MAIL’ command more than 6 minutes, which is intolerable, because of the same 5 minute limit. Thus, the client will almost certainly break the session.
To avoid this,
mailfromd uses a special instance, called
callout server, which is responsible for running callout
SMTP sessions asynchronously. The usual sender verification
is performed using so-called soft timeout values, which
are set to values short enough to not disturb the incoming session
(e.g. a timeout for ‘HELO’ response is 3 seconds, instead of 5
minutes). If this verification yields a definite answer, that answer
is stored in the cache database and returned to the calling procedure
immediately. If, however, the verification is aborted due to a timeout,
the caller procedure is returned an ‘e_temp_failure’ exception, and
the callout is scheduled for processing by a callout server. This
exception normally causes the milter session to return a temporary
error to the sender, urging it to retry the connection later.
In the meantime, the callout server runs the sender verification again using another set of timeouts, called hard timeouts, which are normally much longer than ‘soft’ ones (they default to the values required by RFC 2822). If it gets a definitive result (e.g. ‘email found’ or ‘email not found’), the server stores it in the cache database. If the callout ends due to a timeout, a ‘not_found’ result is stored in the database.
Some time later, the remote server retries the delivery, and the
mailfromd script is run again. This time, the callout
function will immediately obtain the already cached result from the
database and proceed accordingly. If the callout server has not
finished the request by the time the sender retries the connection,
the latter is again returned a temporary error, and the process
continues until the callout is finished.
Usually, callout server is just another instance of
mailfromd itself, which is started automatically to
perform scheduled SMTP callouts. It is also possible to set up
a separate callout server on another machine. This is discussed in
For a detailed information about callout timeouts and their configuration, see conf-timeout.
For a description of how to configure
mailfromd to use callout
servers, see conf-server.
This document was generated on January 3, 2019 using makeinfo.Verbatim copying and distribution of this entire article is permitted in any medium, provided this notice is preserved.