Filtering traffic based on thousands of IPs efficiently
Posted by uljanow on Wed 4 Jul 2007 at 11:11
Trying to insert 70.000 rules in iptables on a recent machine takes about an hour and going through these rules for each packet is even more of a burden. But iptables can send packets to userspace to be handled there. This article describes how to filter network traffic based on thousands of IPs with a new tool called nfqueue efficiently.
Prerequisites
nfqueue requires a 2.6.14 kernel or later with the option
CONFIG_NETFILTER_XT_TARGET_NFQUEUE enabled (module or build-in). On a standard Debian installation (Etch) the additional packages libnetfilter-queue1 and Install prerequisites Get the Debian nfqueue package and install it IP ranges are specified in p2p, dat, csv text files or in nfq binary format. A p2p format looks like this: A dat file looks like this: (Values less than 127 are dropped.) For available lists take a look at /usr/share/doc/nfqueue/README.lists. Sending packets to userspace is done by using the NFQUEUE target. E.g: From userspace there are basically 3 things one can do with packets. Repeating Packets sends them back to the chain (IN-, OUTPUT or FORWARD)
they came from. Since this could lead to endless loops marking packets
is possible. The other options Accept and Drop are terminating targets. See "man 1 nfqueue" for more details. Get the csv file from webhosting.info Let's assume we want to block the whole US. First we put the ip ranges of
the USA into a nfq binary to make loading faster. The easy way now would be to use the /usr/share/doc/nfqueue/nfqueue.sh script which I will explain later. Updating these values is all that needs to be done: Run: Packets are filtered in the INPUT and OUTPUT chain. For each new connection
(both directions) nfqueue looks if the IP is specified in usa.nfq. If
the IP is found then it gets marked and repeated so that it can be rejected
by iptables. If the IP is not found nfqueue marks the packet to avoid
looping forever and sends it back (repeat again) to be handled by the
rest of the iptables configuration. The script only rejects packets from clients specified in files
and the rest is handled by your iptables configuration. Note that the script rejects packet properly instead of just dropping. There is also an ipset tool from netfiler.org which requires kernel-patching and some scripting to parse the IPs from files and insert them.Installation
aptitude install libnetfilter-queue1 libnfnetlink1
wget http://nfqueue.sf.net/debian/nfqueue_0.11-1_i386.deb
dpkg -i nfqueue_0.11-1_i386.deb
Overview
foo : 127.0.0.1 - 127.0.0.2
127.0.0.1, 127.0.0.2, <0-255>, foo
iptables -I INPUT -p all -j NFQUEUE
Example - Blocking whole Countries
wgethttp://ip-to-country.webhosting.info/downloads/ip-to-country.csv.zip
unzip -c ip-to-country.csv.zip | grep -i usa | \
nfqueue -t repeat -o usa.nfq -
INPUT_FILES=/path/to/usa.nfq
OUTPUT_FILES=/path/to/usa.nfq
nfqueue.sh start
nfqueue.sh stop
nfqueue.sh status
nfqueue.sh Script
What does the script do exactly
Notes
- why do you repeat the USA packets instead of using a DROP target in the nfqueue ? How is this an advantage to just doing it in iptables straight away ?
- What markings are set, and what rules do you set in the iptables config to recognize them ?
[ Parent | Reply to this comment ]
- the reason not to use iptables itself for so many IP ranges is just performance.
Loading a binary with 160K IP ranges takes few seconds with nfqueue. Doing a binary search on merged/unique IP ranges is faster than iptables going through thousands of rules. Since only new packets have to be inspected the penalty of delegating packets to userspace is very low.
- The mark values can be specified through the commandline. The script uses 2 for target matches and 1 for the rest.
The idea is:
If packets (state NEW) are not marked with 1 (! --mark 1) which means that nfqueue hasn't looked at these, they are delegated to nfqueue (-j NFQUEUE). The case where packets match an IP range is described above. In the other case nfqueue marks the packet with 1, so it won't match the rule used to send them to nfqueue (to avoid looping):
iptables -I INPUT 3 -p all -m mark ! --mark 1 -j NFQUEUE
The result is that the packets that should not be rejected are handled by the rest of the iptables rules.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
If N = 70,000 than O(N) (in KS with the module iprange of iptables) is more work than O(Log N) including context swtiches in Userspace.
I haven't any benchmarks because just inserting N rules on a Core2Duo 2G takes an hour and I'm not that patient, except for this one time. :)
But there is an iptables modul named ipset which can deal with lots of IP-addresses, but I haven't tried it out because I don't know how stable these patch-o-magic patches are. Ther are not part of the official kernel yet.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
System: Intel Core 2 Duo T7200 2GHz (which was running with 1GHz all the time),
2GB RAM, 100Mb networkcard
During the tests the complete bandwith of 100Mbit was used (only Upload).
- Using scp to copy one 1500 MByte file to another machine in my LAN with
100Mbit cards without nfqueue
time of scp:
23,42s user 13,75s system 27% cpu 2:17,01 total
- the same copying with nfqueue and 165,000 rules inserted, with the above script which filters in INPUT and OUTPUT chains. Every packet was send to nfqueue, not just the ones with state NEW. Logging of nfqueue was disabled.
time of scp:
42,05s user 21,24s system 45% cpu 2:19,61 total
average cpu usage of nfqueue was:
* thread that was listening on the INPUT chain 7%
* thread that was listening on the OUTPUT chain 11%
The same test with 77,000 rules
time of scp:
42,01s user 21,44s system 45% cpu 2:19,80 total
nfqueue was running with 10% on the OUTPUT thread
and 7% on the other.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
there is also geoip patch in netfilete/iptabes projects.
link
http://www.debian-administration.org/articles/518
Regards to all
[ Parent | Reply to this comment ]
In order to get it working (and not lock a server completely)
you need to change *_MARK variables to some higher number not used by shorewall.
In my case up to 4 were used so I changed to 64 & 65.
[ Parent | Reply to this comment ]
regards
[ Parent | Reply to this comment ]
I guess they are used for traffic priority which I set up through shorewall.
My experience with firewalls is mostly FreeBSD's ipfw.
So I spent all the day learning the iptables internals and figuring out whats in wrong in my simple shorewall setup. I was totally lost in all of this.
Finally I found a script which helped me a lot - iptables-trace.
http://lists.netfilter.org/pipermail/netfilter/2003-March/043088. html
[ Parent | Reply to this comment ]