New User? Register here - Existing Users: Username: Password: [Advanced Login]

 

 

Current Poll

Your preferred Interactive shell?









( 1334 votes ~ 14 comments )

 

Weblog entry #20 for lee

Date header based ACL for Exim
Posted by lee on Wed 30 Aug 2006 at 21:45
Tags: , , ,

I've started to notice more of these sorts of messages in my logcheck mails:

spamd[11313]: Second '70' out of range 0..59 at /usr/share/perl5/Mail/SpamAssassin/Util.pm line 429

I check the mail and, sure enough, the date header specifies a time with 70 seconds. Why should I spend resources running a mail through the full automated content analysis if the spammer can't be bothered even attempting to come up with a plausible date?

I've decided to reject based on malformed Date headers before the mail even gets passed to SpamAssassin. The following can be put in the acl_check_data file before any spam acl. The last check is for "obsolete" RFC822 style dates, which shouldn't be generated by any software written in the last five years - I'm just tagging these for now to assess the impact of blocking them.


deny  condition = ${if !def:h_date: {1}}
      message = Message SHOULD have Date header but does not
      log_message = acl_check_data: No Date header

# note: see RFC2822
# + some folded whitespace must implemented as a single space
# + loose checking on CFWS
# bug: will reject mails sent on a leap second, too bad
# bug: needs to be modified to work after 2099
deny    log_message = acl_check_data: RFC2822 Date invalid "$h_date:"
        message = Date header is invalid
        !condition = ${if match {$h_date:}{\N^\s*((Mon|Tue|Wed|Thu|Fri|Sat|Sun),)?\s*?[0-3]?[0-9] (Jan|Feb|Ma[ry]|Apr|Ju[nl]|Aug|Sep|Oct|Nov|Dec) (19|20)?[0-9]{2} [0-2][0-9](\:[0-5][0-9]){1,2} (?:[+-][01][0-9][0-5][0-9]|UT|[A-Z]{2,3}T|[A-Z])(\s(\(.*\))?)*$\N} {1}}
# note: as above, but requires 4 digit year and 4 digit timezone
warn    log_message = acl_check_data: RFC2822 Date format obsolete "$h_date:"
        message = X-RFC2822-Warning: Date format obsolete
        !condition = ${if match {$h_date:}{\N^\s*((Mon|Tue|Wed|Thu|Fri|Sat|Sun),)?\s*?[0-3]?[0-9] (Jan|Feb|Ma[ry]|Apr|Ju[nl]|Aug|Sep|Oct|Nov|Dec) 20[0-9]{2} [0-2][0-9](\:[0-5][0-9]){1,2} [+-][01][0-9][0-5][0-9](\s(\(.*\))?)*$\N} {1}}

Examples of the sorts of dates being rejected:

Date: Sat, 26 Aug 2006 11:23:55 -0200 EST
Date: , 26 Aug 2006 12:23:00 +0900
Date:  %CURRENT_DATE_TIME
Date: Sat, 26 Aug 2006 12:00:44 0000
Date: Lun, 28 Ago 2006 04:48:16
Date:  Wed, 30 Aug 2006 00:12:24 -05-30
Date: 29 Aug 2006 18:56:70 +0100
Date: Thu, 2 Jan 1997 13:21:00 -0060

 

Comments on this Entry

Posted by Steve (62.30.xx.xx) on Thu 31 Aug 2006 at 15:38
[ Send Message | View Steve's Scratchpad | View Weblogs ]

That is a great idea. I've started using your checks too now.

Steve

[ Parent | Reply to this comment ]

Posted by dkg (166.84.xx.xx) on Sun 3 Sep 2006 at 21:46
[ Send Message | View dkg's Scratchpad | View Weblogs ]
Interesting. i had no idea that the date fields needed to be done in english. For example, the following line looks decent from a non-english perspective:
Date: Lun, 28 Ago 2006 04:48:16
But looking it up in RFC 2822, i see that section 3.3 does actually require an english format of the date.

Actually, reading that RFC more closely could suggest several similar filters.

Be careful also about leap seconds. they're rare, but occasionally 23:59:60 UTC is a valid time. bleh.

[ Parent | Reply to this comment ]

Posted by lee (193.82.xx.xx) on Thu 21 Sep 2006 at 11:32
[ Send Message | View Weblogs ]
Checking through my logs since I implemented this I have dropped several pieces of mail, all from mailing lists.

Two of them appear to have come from mail software on handheld devices, where the hour field is implemented as a single digit, even though it's specified as 2DIGIT in both 822 and 2822. The other has a timezone format not matched by either spec, but no User Agent identification.

Date: Mon, 4 Sep 2006 4:19:00 -0700
X-Mailer: VersaMail(c) 1998-2004 3.1D-1, palmOne, Inc.

Date: Mon, 18 Sep 2006 9:22 -0400
X-Mailer: SnapperMail 1.9.5.01  by Snapperfish, www.snappermail.com

Date: Wed, 13 Sep 2006 20:48:09 WET DST

For the handheld mail, it could be potentially be fixed by running Date: validity checking on the mail submission gateway - and potentially fixing it.

[ Parent | Reply to this comment ]

Posted by rickf (24.24.xx.xx) on Tue 30 Jan 2007 at 19:10
[ Send Message ]
Hi.. Just wondering why the following dates are getting popped out -- in this case they all belong to legit spam, but after reading parts of RFC2822, I just don't see why they're flagged :

Tue, 30 Jan 2007 11:41:59 -0500
Tue, 30 Jan 2007 13:44:28 -0500
Tue, 30 Jan 2007 13:45:03 -0500

Any ideas?

[ Parent | Reply to this comment ]

Posted by rickf (24.24.xx.xx) on Tue 30 Jan 2007 at 19:24
[ Send Message ]
Nevermind.. The exim script is not apparently logging the full date/timestamp that was in the incoming email -- the ones flagged above (found in the rejects file) were like that shown below :

Tue, 30 Jan 2007 13:56:23 -0500 (EST)

I guess it doesn't like the parenthesis around the EST since i didn't see any mention of using (EST) vs EST in the RFC..

[ Parent | Reply to this comment ]

Posted by lee (83.216.xx.xx) on Tue 30 Jan 2007 at 22:54
[ Send Message | View Weblogs ]
That date looks OK to me, I'm not dure why it would get flagged. Anything in parenthesis at the end is fine - in terms of the RFC it's just a comment.

[ Parent | Reply to this comment ]

Posted by rickf (24.24.xx.xx) on Tue 30 Jan 2007 at 23:01
[ Send Message ]
Here's a sample that just came in (and was rejected) -- this was in the exim reject log.. I've so far (to the best of my knowledge) not had any legit emails bounced because of enabling this filter... If you've got any ideas why it's bitching about these, I'd love to hear it..

2007-01-30 14:57:35 1HC1ux-0002Wb-0y H=mail.adrianocele.com [66.97.170.242] F=<holidaydiet@adrianocele.com> rejected after DATA: acl_check_data: RFC2822 Date invalid "Tue, 30 Jan 2007 17:17:49 -0500
Envelope-from: <holidaydiet@adrianocele.com>
Envelope-to: <rickf@ca-flower.com>
P Received: from mail.adrianocele.com ([66.97.170.242])
by srv1.ca-flower.com with esmtp (Exim 4.66)
(envelope-from <holidaydiet@adrianocele.com>)
id 1HC1ux-0002Wb-0y
for rickf@ca-flower.com; Tue, 30 Jan 2007 14:57:35 -0800
P Received: by mail.adrianocele.com (qmail 412 by uid 77) id hnv6ls01g744; Tue, 30 Jan 2007 17:18:00 -0500 (envelope-from <holidaydiet@adrianocele.com>)
Date: Tue, 30 Jan 2007 17:17:49 -0500
F From: "HolidayDiet" <holidaydiet@adrianocele.com>
Subject: Lose Up to 30lbs by Christmas
T To: rickf@ca-flower.com
I Message-ID: <dlxs61waalhfmtjftfdp-g@adrianocele.com>
Date: Tue, 30 Jan 2007 17:17:49 -0500 (EST)
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="=_dLxS61wAAlHfMtJftFDP-g"

[ Parent | Reply to this comment ]

Posted by rickf (24.24.xx.xx) on Tue 30 Jan 2007 at 23:23
[ Send Message ]
One more item of note.. I notice in my regular exim log file lines like the following when it flags one of those "odd" guys :

DATA: acl_check_data: RFC2822 Date invalid "Tue, 30 Jan 2007 18:15:00 -0500

Interestingly enough, there is no end quote mark after the "-0500" which leads me to believe that perhaps someone either put a tab in the date or somesuch tactic which might be causing problems (and might explain why only the spam fails)..

I've had others get kicked out and they look like :

DATA: acl_check_data: RFC2822 Date invalid "Tue, 30 Jan 2007 14:42:23 US/Eastern"

Makes me wonder if either the logging system is doing something odd with the string or perhaps this might be a bug in Exim (with regards to the logging/regex engine) and why it doesn't show the entire string...?

[ Parent | Reply to this comment ]

Posted by lee (83.216.xx.xx) on Tue 30 Jan 2007 at 23:42
[ Send Message | View Weblogs ]
Ah-ha. I see what's happening. The RFC2822 (section 3.6) specifies that there can be a maximum of one "Date:" header. When Exim processes headers that can appear multiple times, for example the Received: headers it concatenates them into a single string.

There are two Date headers here, I think exim is attempting to match
"Tue, 30 Jan 2007 17:17:49 -0500 Tue, 30 Jan 2007 17:17:49 -0500 (EST)".

[ Parent | Reply to this comment ]

Posted by rickf (24.24.xx.xx) on Tue 30 Jan 2007 at 23:47
[ Send Message ]
Cool.. I was wondering about that.. So, I guess their mailer (and this must be common because I've been getting a log of rejects for this same issue today alone) is being naughty (which is fine by me) and putting two dates before connecting to my server.. Very interesting.. Thanks for looking at my wierd logs..

[ Parent | Reply to this comment ]

Posted by rickf (157.127.xx.xx) on Wed 14 Feb 2007 at 20:42
[ Send Message ]
Lee --

If I wanted to modify this macro so that it would bypass (e.g. whitelist) this check for a group of email addresses (or perhaps one or more domains of my choosing), whats the best way to go down that path? I've got one domain that sends out occasional emails that work fine, but emails from their support group and some push-style emails (ones I forced via their website) seem to go out through a different server and fail the date check acl. I'd effectively like to whitelist them (besides telling them that one of their mailservers is spitting out bad dates).. Any suggestions for going down this path would be greatly appreciated!

[ Parent | Reply to this comment ]

Posted by lee (193.82.xx.xx) on Thu 15 Feb 2007 at 10:43
[ Send Message | View Weblogs ]

This is one of those "more than one way of doing it" type questions.

One way might be to set up an ACL that matches the addresses, domains and hosts in question from which you recieve legitimate mail with bad headers, and call it "acl_whitelist_broken_headers" (see acl_whitelist_local_deny in the exim config for an example in setting this up). Then for any ACL checking header syntax, include the line "!acl = acl_whitelist_broken_headers" and it won't apply to any matches.

Exim docs: Access Control Lists

[ Parent | Reply to this comment ]

 

 

Flattr