Filtering traffic based on thousands of IPs efficiently

Posted by uljanow on Wed 4 Jul 2007 at 11:11

Tags: none.

Trying to insert 70.000 rules in iptables on a recent machine takes about an hour and going through these rules for each packet is even more of a burden. But iptables can send packets to userspace to be handled there. This article describes how to filter network traffic based on thousands of IPs with a new tool called nfqueue efficiently.

Prerequisites

nfqueue requires a 2.6.14 kernel or later with the option CONFIG_NETFILTER_XT_TARGET_NFQUEUE enabled (module or build-in). On a standard Debian installation (Etch) the additional packages libnetfilter-queue1 and

Installation

Install prerequisites

aptitude install libnetfilter-queue1 libnfnetlink1

Get the Debian nfqueue package and install it

wget http://nfqueue.sf.net/debian/nfqueue_0.11-1_i386.deb
dpkg -i nfqueue_0.11-1_i386.deb

Overview

IP ranges are specified in p2p, dat, csv text files or in nfq binary format.

A p2p format looks like this:

foo : 127.0.0.1 - 127.0.0.2

A dat file looks like this:

127.0.0.1, 127.0.0.2, <0-255>, foo

(Values less than 127 are dropped.)

For available lists take a look at /usr/share/doc/nfqueue/README.lists.

Sending packets to userspace is done by using the NFQUEUE target. E.g:

iptables -I INPUT -p all -j NFQUEUE

From userspace there are basically 3 things one can do with packets.

  • Accept
  • Drop
  • Repeat

Repeating Packets sends them back to the chain (IN-, OUTPUT or FORWARD) they came from. Since this could lead to endless loops marking packets is possible. The other options Accept and Drop are terminating targets. See "man 1 nfqueue" for more details.

Example - Blocking whole Countries

Get the csv file from webhosting.info

wgethttp://ip-to-country.webhosting.info/downloads/ip-to-country.csv.zip

Let's assume we want to block the whole US. First we put the ip ranges of the USA into a nfq binary to make loading faster.

unzip -c ip-to-country.csv.zip | grep -i usa | \
  nfqueue -t repeat -o usa.nfq -

The easy way now would be to use the /usr/share/doc/nfqueue/nfqueue.sh script which I will explain later. Updating these values is all that needs to be done:

INPUT_FILES=/path/to/usa.nfq
OUTPUT_FILES=/path/to/usa.nfq

Run:

nfqueue.sh start
nfqueue.sh stop
nfqueue.sh status

nfqueue.sh Script

What does the script do exactly

Packets are filtered in the INPUT and OUTPUT chain. For each new connection (both directions) nfqueue looks if the IP is specified in usa.nfq. If the IP is found then it gets marked and repeated so that it can be rejected by iptables. If the IP is not found nfqueue marks the packet to avoid looping forever and sends it back (repeat again) to be handled by the rest of the iptables configuration.

The script only rejects packets from clients specified in files and the rest is handled by your iptables configuration.

Note that the script rejects packet properly instead of just dropping.

Notes

There is also an ipset tool from netfiler.org which requires kernel-patching and some scripting to parse the IPs from files and insert them.

 

 


Posted by vegiVamp (195.177.xx.xx) on Wed 4 Jul 2007 at 14:42
Looks interesting, but I'm not quite clear on how it works...

- why do you repeat the USA packets instead of using a DROP target in the nfqueue ? How is this an advantage to just doing it in iptables straight away ?
- What markings are set, and what rules do you set in the iptables config to recognize them ?

[ Parent | Reply to this comment ]

Posted by uljanow (84.189.xx.xx) on Wed 4 Jul 2007 at 15:46
- using a DROP target is also possible, but rejecting packets has more advantages. Rejecting is done by marking the packet with 2 in nfqueue and using REJECT-rules in iptables with "--mark 2". There is no other way to do this. Marking/repeating packets is also used in ipsec iptables-setups.

- the reason not to use iptables itself for so many IP ranges is just performance.
Loading a binary with 160K IP ranges takes few seconds with nfqueue. Doing a binary search on merged/unique IP ranges is faster than iptables going through thousands of rules. Since only new packets have to be inspected the penalty of delegating packets to userspace is very low.

- The mark values can be specified through the commandline. The script uses 2 for target matches and 1 for the rest.
The idea is:
If packets (state NEW) are not marked with 1 (! --mark 1) which means that nfqueue hasn't looked at these, they are delegated to nfqueue (-j NFQUEUE). The case where packets match an IP range is described above. In the other case nfqueue marks the packet with 1, so it won't match the rule used to send them to nfqueue (to avoid looping):
iptables -I INPUT 3 -p all -m mark ! --mark 1 -j NFQUEUE
The result is that the packets that should not be rejected are handled by the rest of the iptables rules.

[ Parent | Reply to this comment ]

Posted by Anonymous (213.164.xx.xx) on Thu 5 Jul 2007 at 08:09
How is this approach quicker (technically)? How can userspace be quicker?

[ Parent | Reply to this comment ]

Posted by uljanow (84.189.xx.xx) on Thu 5 Jul 2007 at 12:22
Userspace is not quicker, but having different complexities makes a difference.
If N = 70,000 than O(N) (in KS with the module iprange of iptables) is more work than O(Log N) including context swtiches in Userspace.
I haven't any benchmarks because just inserting N rules on a Core2Duo 2G takes an hour and I'm not that patient, except for this one time. :)

But there is an iptables modul named ipset which can deal with lots of IP-addresses, but I haven't tried it out because I don't know how stable these patch-o-magic patches are. Ther are not part of the official kernel yet.

[ Parent | Reply to this comment ]

Posted by Anonymous (216.244.xx.xx) on Fri 6 Jul 2007 at 19:01
does this work in a high traffic firewall (80 Mb in and 60 Mb out) ?

[ Parent | Reply to this comment ]

Posted by uljanow (84.188.xx.xx) on Fri 6 Jul 2007 at 22:09
Some benchmarks...

System: Intel Core 2 Duo T7200 2GHz (which was running with 1GHz all the time),
2GB RAM, 100Mb networkcard

During the tests the complete bandwith of 100Mbit was used (only Upload).

- Using scp to copy one 1500 MByte file to another machine in my LAN with
100Mbit cards without nfqueue

time of scp:
23,42s user 13,75s system 27% cpu 2:17,01 total

- the same copying with nfqueue and 165,000 rules inserted, with the above script which filters in INPUT and OUTPUT chains. Every packet was send to nfqueue, not just the ones with state NEW. Logging of nfqueue was disabled.

time of scp:
42,05s user 21,24s system 45% cpu 2:19,61 total

average cpu usage of nfqueue was:
* thread that was listening on the INPUT chain 7%
* thread that was listening on the OUTPUT chain 11%

The same test with 77,000 rules
time of scp:
42,01s user 21,44s system 45% cpu 2:19,80 total

nfqueue was running with 10% on the OUTPUT thread
and 7% on the other.

[ Parent | Reply to this comment ]

Posted by Anonymous (88.65.xx.xx) on Fri 21 May 2010 at 15:32
Importing 70.000 rules with iptables-restore just takes few milliseconds
Read 5.1. Speed considerations http://www.faqs.org/docs/iptables/saveandrestore.html

[ Parent | Reply to this comment ]

Posted by TRS-80 (203.59.xx.xx) on Fri 13 Jul 2007 at 11:08
There's also IP sets which are an in-kernel iptables target that supports many IPs, TCP/UDP ports and MAC addresses.

[ Parent | Reply to this comment ]

Posted by Anonymous (84.189.xx.xx) on Sat 14 Jul 2007 at 00:55
just for the records, project is renamed to iplist (http://iplist.sf.net)

[ Parent | Reply to this comment ]

Posted by debianex (77.238.xx.xx) on Thu 2 Aug 2007 at 19:48
Hi all,

there is also geoip patch in netfilete/iptabes projects.

link

http://www.debian-administration.org/articles/518


Regards to all

[ Parent | Reply to this comment ]

Posted by mpa (217.172.xx.xx) on Sat 4 Aug 2007 at 22:51
The ipblock script is incompatible with default settings with standard shorewall.
In order to get it working (and not lock a server completely)
you need to change *_MARK variables to some higher number not used by shorewall.
In my case up to 4 were used so I changed to 64 & 65.

[ Parent | Reply to this comment ]

Posted by uljanow (84.189.xx.xx) on Sun 5 Aug 2007 at 14:12
thanks. this will be fixed with iplist-0.14.

regards

[ Parent | Reply to this comment ]

Posted by mpa (217.172.xx.xx) on Sun 5 Aug 2007 at 16:17
In my case there were few --mark rules in the mangle table.
I guess they are used for traffic priority which I set up through shorewall.
My experience with firewalls is mostly FreeBSD's ipfw.
So I spent all the day learning the iptables internals and figuring out whats in wrong in my simple shorewall setup. I was totally lost in all of this.
Finally I found a script which helped me a lot - iptables-trace.
http://lists.netfilter.org/pipermail/netfilter/2003-March/043088. html


[ Parent | Reply to this comment ]

Sign In

Username:

Password:

[Register|Advanced]

 

Flattr

 

Current Poll

What do you use for configuration management?








( 531 votes ~ 7 comments )