DynaStop:Dynamic IP Address filtering

DynaStop is a gpl licensed LINUX utility to examine IP4 based addresses for Exim and procmail for the purpose of filtering based upon patternistic rules defined by the administrator.

DynaStop will work with any mail transfer agent/utility (like DSpam) that can run external programs or inline filters (piped I/O) and respond to error codes. I would be most appreciative to anyone doing so and would like to share the medthod of incorporating DynaStop into their mail system. Full Credit and a link will be gladly provided.

This can be a pivotal factor in e-mail filtering and server load management as dynamic IP addresses are typically used for dial-up, dhcp, and DSL accounts. All of which have a designated mail exchange server that all outbound mail flows as defined with many if not most large Internet Service Providers (ISP) such as Road Runner, ATT, Qwest, PacBell, BellSouth, EarthLink, AOL, and many more around the world if their terms of service or acceptable use policy.

With DynaStop, an Exim administrator or procmail administrator/user can filter incoming mail on the basis of whether it is a dynamic IP address, thus reducing system load and resources in processing unwanted mail as all legitimate mail from any given ISP will be sent from a proper mail exchange server.

A discussion form is available at exim-users.org. Many thanks to Kevin Barrett for hosting the forum.

Many thanks to Tom Knaupp of http://tom.knaupp.com/?p=9 for contributions in DynaStop's config files. Tom has provided a list of false positives. His list is in the DynaStop.TomKnaupp.conf file.

Many thanks to Mike Cardwell for developing and testing a macro for Exim that allows it comminucate with the DynaWall server without the use of the DynaStop module and for suggesting the the web update script should be included in the distribution file. The section for the Cardwell Macro is here.

Please visit DynaStop's Freashmeat page, http://freshmeat,net/projects/dynastop, and rate it.

DynaStop requirements:

A full developement system (gcc compiler/libraries)
Shared Memory (kernel level) (Click here for an example of how to activate shared memory)
Semaphores (kernel level)
libpcre (Click here for the libpcre site)

File available for download:

DynaStop vers. 08023.1417 can be downloaded by clicking here for the direct download.
Last updated on Monday, August 18, 2008 at 04:37:29 AM
Tom's Exclusion List can be downloaded by clicking here. Last updated on Monday, May 21, 2007 at 01:37:35 PM.
The BleedingThreats List can be downloaded by clicking here. Last updated on Tuesday, August 19, 2008 at 04:39:40 AM.
The DNS WhiteList can be downloaded by clicking here. Last updated on Monday, August 18, 2008 at 04:37:29 AM.
The CYMRU Bogons List can be downloaded by clicking here. Last updated on Wednesday, May 28, 2008 at 04:37:24 AM.
The Spamhaus Drop List can be downloaded by clicking here. Last updated on Monday, August 18, 2008 at 04:37:29 AM.

How DynaStop was tested and the research involved (The research data is NOT required to run DynaStop):

The research used to develop DynaStop was started in 1999 as part of the BullDog Firewall. BullDog's DNS Database was used to extensively analyze reverse IP address schemes. BullDog is NOT required for DynaStop to work properly and neither is this analysis. Use this only for analytical and research purposes. This process is VERY CPU intensive and not recommended for a gateway server to do (though an internet connection is required for the DNS lookups). The test takes several weeks to complete on a 1.7GHz AMD. You may wish to redirect the output to a file.

Over 373 Million IP addresses were used in testing DynaStop for integrity and stability in identifying dynamic addresses and cross checked false positives. The DNS Database is free for download and based strictly of RFC policies regarding DNS queries. DynaStop can be tested as follows:
A simple script can be used to generate random IP addresses:
#!/bin/bash

i=0

while [ $i -le $1 ] ; do
  let ip1=$RANDOM%256
  let ip2=$RANDOM%256
  let ip3=$RANDOM%256
  let ip4=$RANDOM%256

  echo $ip1.$ip2.$ip3.$ip4
  let i=i+1
done
    
After DynaStop has been installed and configured and the above script has been saved as ipaddr.sh, the following command can be used as a single line command:

./ipaddr.sh 20 | /usr/exim/DynaTest /usr/exim/DynaStop.conf
The columns printed out are as follows (This applies to all DynaTest/DynaCheck output):

Return Code
IP Address
Reverse Domain Name
Wordy result of analysis
Matched components after IP substitution, M:
The orignal configuration line that is to be compared, C:
or a P: and the pattern DynaTest believe will match this IP address
Click here for comparision and verification tables of a given data set.
Of the 373,302,000 IP addresses tested (Aug 1/07) 85% (317,306,720) where evaluated to be dynamic. These numbers do not include IP addresses that do not have a reverse domain name. The estimated 10% ligitimate mail servers that use dynamic IP addresses (based upon user feedback) can easily be excluded leaving 279,976,512 IP addresses (potential spam zombies) that will be blocked/tagged by DynaStop.

Pragmatic Analysis of DynaSyop

The graph on the right is an illustration of this system's spam filtering from December of 2005 to the middle of Febuary of 2006. The timeline prior to the end of January illustrates Exim using only HELO forgery tests (at most 10% of total, averaging around 5% daily). The values on the graph of the daily total percentage of spam identified. Starting in late January (the 27th), DynaStop was brought online. The remainder of January to current shows the impact of Exim with HELO forgery tests combined with DynaStop.

It should be noted that antivirus and other antispam messures are performed external to Exim where resource management has better control. Antispam and antivirus should run in the order of least expensive to most expensive. The expense of any given method is the cost of bandwidth and processing power.

Below is one year of data illustrating this system antispam defenses. It is updated on a daily basis.

Setting up DynaStop is a follows:

Be sure to check the COMPILE script and modify the CPU variable as needed. Below is the Compilation script used to build DynaStop (included in the download):

#!/bin/bash clear # Modify for your CPU using GCC switches export CPU="-march=i686 -mtune=i686" # Required defines export DEF="-D_USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -DDynaWall" export SRC="-D_REENTRANT -D_BSD_SOURCE" # Required Libraries export LIB="-lresolv -lpcre" # Required compilier options export OPT="-w -O3 -funroll-loops -ffast-math -march=i686 -mtune=i686 -fomit-frame-pointer -fno-exceptions" # Get down to business and compile it out. for i in Stop Test ; do gcc -o Dyna$i Dyna$i.c $DEF $CPU $SRC $LIB $OPT strip Dyna$i done ### ### Special defines (Greylisting flavors, case IS important): ### ### Definations: ### RDN Reverse Domain Name ### ### GreyList Activates greylisting functions ### REQUIRED for any greylisting operations ### ### GreyEvery Greylist every NON-DYNAMIC ip address if this flag ### is present. If this flag is NOT present, then ### greylist dynaic IP addresses ### ### AutoGreen Automatically write green listed entries to the ### AutoGreen file. ### ### RDN Only reversible IP addresses are greylisted. ### IP addresses with nor reverse domain name have a ### return code 2 issued. ### Only useful when NOT using GreyEvery. ### ### Uncomment ONE of the following, default Greylisting with AutoGreen ### For NO greylisting, make sure all the SP lines have a # infront of them ### Greylisting only of DYNAMIC IP addresses with a RDN #SP="-DGreyList -DRDN" # Greylisting with AutoGreen saves SP="-DGreyList -DAutoGreen -DRDN" # Greylisting NON dynamic IP address, return codes 1/2 for dynamic IP addresses #SP="-DGreyList -DGreyEvery" # Greylisting NON dynamic IP address, return codes 1/2 for dynamic IP addresses With AutoGreen saves #SP="-DGreyList -DGreyEvery -DAutoGreen" gcc -o DynaWall.greylist DynaWall.shram.c $DEF $CPU $SRC $LIB $OPT $SP strip DynaWall.greylist

After adjusting the CPU parameters to match your hardware and selecting the compile options, simply type:

./COMPILE

If all went well, the prompt will return and no errors will be displayed.

For easy administration, I recommend coping the DynaStop, DynaWall, and DynaTest executables into your Exim base folder (procmail users can change the folder as appropriate, the change does have to be consistent, this includes the DynaStop conf files), like so:

cp DynaWall.shram /usr/exim/DynaWall

cp DynaStop /usr/exim

cp DynaTest /usr/exim

Now on to the DynaStop configuration file. Below is an example of a basic configuration. This is in no means complete or accurate to any system. It is a basic blueprint to tailor to your system's specific needs. An example is included in the download.

### DynaStop configuration file. ### ### Control verbs DynaStop uses are: ### NoFilter - Receipent domain names that do NOT want any filtering ### Exclude - Dynamic patterns that are EXCLUDED at all costs from DynaStop ### ExRange - IP ranges that are EXCLUDED at all costs from DynaStop ### ExCidr - IP CIDRs that are EXCLUDED at all costs from DynaStop ### Pattern - Defines a dynamic pattern to search for ### PatRange - Dynamic IP ranges to search for ### PatCidr - Dynamic IP ranges to search for ### NoMail - A pattern that describes a site you do NOT want to receive mail from ### NoMailRange - IP ranges that are NOT accepted at all costs from DynaStop ### NoMailCidr - IP CIDRs that are NOT accepted at all costs from DynaStop ### Include - Includes a seperate file into DynaStop. Same format as this file and requires full pathname ### ### All patterns ekements must match for the pattern to be matched ### ### Patterns that match dynamic IP addresses ### ### IP Number breakdown: ### ### IP address: 126.12.34.56 ### Reverse IP: softbank126012034056.bbtec.net ### ### $1 - $4 are the decimal IP address parts ### $5 - $8 are the hex IP address parts ### ### $1 = 126 $5 = 7E ### $2 = 12 $6 = 0C ### $3 - 34 $7 = 22 ### $4 = 56 $8 = 38 ### ### See also: ### http://tools.ietf.org/id/draft-msullivan-dnsop-generic-naming-schemes-00.txt ### ### WARNING: This is is based completely on heuristic analysis. ### WARNING: BE SURE YOU EXAMINE IT AND YOUR LOGS VERY FREQUENTLY ### WARNING: TO PREVENT UNWANTED BLOCKING if you use DENY or DROP ### WARNING: in your Exim configuration. USE AT YOUR OWN RISK. ### WARNING: THE CONTENTS IN THIS FILE ARE FOR EXAMPLE ONLY. ### ### WARNING: BE SURE YOU EXCLUDE YOUR OWN LOCAL NETWORKS. ### ### GreyList/Cache timeouts ### ### MaxTTL - Maximum time in HOURS an item remains in the cache MaxTTL 24 ### GreyTTL - Minimium time in MINUTES an IP address has to wait ### to be GreenListed. Ignored if GreyList not enabled GreyTTL 1 ### ### File to write AutoGreen entries. Ignored if AutoGreen ### is not enabled ### AutoGreen /usr/exim/DynaStop.AutoGreen.conf ### ### Expressions must always be first ### Include /usr/exim/DynaStop.Expressions.conf ### ### Load the NoFilter patterns ### Include /usr/exim/DynaStop.NoFilter.conf ### ### Load the Dynamic IP patterns, Excludes, and NoMail sections ### Include /usr/exim/DynaStop.Confirmed.conf Include /usr/exim/DynaStop.Excludes.conf Include /usr/exim/DynaStop.Patterns.conf Include /usr/exim/DynaStop.NoMail.conf ### ### Lists from BleedingThreats.net and webhelper4u.net ### ### I make no warrenties about the stability of these lists. ### USE AT YOUR OWN RISK. Include /usr/exim/DynaStop.BleedingThreats.conf ### ### This is for the AutoGreen feature ### Include /usr/exim/DynaStop.AutoGreen.conf ### ### Tom Knaupp's list of False Positives ### Include /usr/exim/DynaStop.TomKnaupp.conf ### ### DNSWL WhiteList ### Include /usr/exim/DynaStop.WhiteList.conf ### ### Spamhaus DropList ### Include /usr/exim/DynaStop.DropList.conf

This file needs to be saved in the same directory as the DynaStop executable, i.e./usr/exim as DynaStop.conf.

Now we need to start DynaWall, the DynaStop server:

( /usr/exim/DynaWall & )

Be sure to put the above line in your start up scripts before Exim.

Now that DynaStop is configured and ready, we need to configure Exim to use DynaStop. We need to know the return codes from DynaStop for Exim:

0 - Not dynamic/no error
1 - Dynamic address
2 - No reverse address
3 - An address listed as NO mail
4 - Greylisted IP address
5 - Some sort of error, most likely corrupt IP address

Signals that DynaStop (DynaWall Server) respond to:

HUP The HUP signal tells DynaWall to reload the configuration file

USR1 The USR1 signal tells DynaWall to write the statistics and cache files to disk

QUIT The QUIT signal tells DynaWall to shutdown and cleanly leave memory

Now we can begin editing the Exim configuration file.

The following is suitable for testing and diagnostics. ALWAYS test your patterns.

Add ONE of the following to your acl_check_rcpt ACL:

To add a warning header to the message, add the following block:

warn set acl_m7 = ${run{/usr/exim/DynaStop localhost $sender_host_address}}
condition = ${if eq {$runrc}{1}{yes}{no}}
message = X-DynaStop: Dynamic IP address/$sender_host_address

To block a message, add the following block:

deny set acl_m7 = ${run{/usr/exim/DynaStop localhost $sender_host_address}}
condition = ${if eq {$runrc}{1}{yes}{no}}
message = Dynamic IP Address: Please use your provider's designated mail exchange server

To defer a message, add the following block:

defer set acl_m7 = ${run{/usr/exim/DynaStop localhost $sender_host_address}}
condition = ${if eq {$runrc}{1}{yes}{no}}
message = (D1) Please try later...

localhost can be any hostname in your network or on the internet. If you use an external DynaWall server, be sure you check with that system's administrator first. Also, this must be a fully resolvable host name, not an ip address.

Be sure you test your patterns extensively before going to blocking mail. This is a powerful tool and can block all mail inadvertently.

NEVER put this above you abuse and postmaster entries in Exim's configuration.

Restart Exim and you're done. DynaStop is now integrated into Exim and filtering mail according to your settings.

The Cardwell Macro:

At the top of your Exim configuration file, put the following lines:

### The Cardwell Macro
DYNASTOP = ${readsocket{inet:localhost:337}{$sender_host_address\n}{60s}{}{-1}}

Here is an example of the Cardwell Macro in use:
deny condition = ${if eq{DYNASTOP}{1}}
message = Dynamic IP address

For procmail, DSpam, or any other inline filtering method (messsage read from/to stdin/out), the IP address has to be taken from the message. Note that GreyListing at this point serves no value and should NOT be used except at the SMTP level. The command line is slightly different for inline filtering:

DynaStop localhost stdin

The stdin parameter tells DynaStop to get the message from standard input, extract the IP address, and send the message to standard out with the X-DynaStop header set if appropriate and exit with a return code of 0. DynaWall must still be installed as this is the work horse of DynaStop. See above for installing DynaWall.
Procmail recipe for DynaStop:

Add these lines to the begining of your procmailrc (be sure to change paths as appropriate):
:0fw
| /usr/exim/DynaStop 127.0.0.1 stdin

Add these lines to the filtering part of your procmailrc (change Spam as needed):
:0
* ^X-DynaStop:.*D[1-3]
Spam
Procmail will now filter dynamic IP addresses as desired.

All advertisements at the bottom of this page do not reflect the views of the author.
They are only to help offset the costs of maintaining this site.

This page was last updated on Tuesday, 20-May-2008 15:29:27 CDT


	Advertise throughout tanaya.net

HUP	The HUP signal tells DynaWall to reload the configuration file
USR1	The USR1 signal tells DynaWall to write the statistics and cache files to disk
QUIT	The QUIT signal tells DynaWall to shutdown and cleanly leave memory