Transparent proxies via Squid.
Posted by Steve on Tue 4 Jan 2005 at 16:12
If you've setup a Linux machine as a gateway, standing between you and the internet then you have a lot of options for tweaking it. One of the most common things to setup is transparent proxying via squid.
If you've followed the previous guide on setting up a Linux gateway you will have one machine with two network interfaces, one for internal use (eth0) and one which is publically connected to the internet (eth1).
This gateway machine has a collection of firewall rules which:
- Allow machines "behind" it to make outgoing connections.
- Prevent incoming connections from the internet.
If we wish to setup a transparent proxy server to cache web pages - which would speed up browsing for those machines behind the gateway we need to do two things:
- Install a caching proxy server
- Add some rules to our gateway to seemlessly allow our machines to use it.
The first is simple. As root run:
apt-get install squid
This will install the Squid caching proxy server. This is configured by the file /etc/squid/squid.conf and we will need to make several changes to it.
First of all we need to tell it that we only wish it to listen on the internal interface. Remember that this gateway machine has two networking interfaces, one for the internal LAN and one for the internet.
Pick the one which is internal and add it to the configuration file as follows:
http_port 127.0.0.1:8080 http_port 192.168.1.1:8080
(In my case the internal address is 192.168.1.1, allowing the server to listen on the "loopback" address of 127.0.0.1 is a good idea too, and will be required later).
As well as that we'll need to tell the server what it's hostname is, and which email address is in charge of it for error display, etc:
visible_hostname gateway.my.flat cache_mgr proxy@foo.com
We also need to add the support for the transparent proxying we will be using:
httpd_accel_host virtual httpd_accel_port 80 httpd_accel_with_proxy on httpd_accel_uses_host_header on
The only remaining thing we need to do is to tell Squid which networks are allowed to connect to our proxy server, without this it will refuse all incoming requests.
For a network which is 192.168.1.x internally the following will be fine:
# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS # Example rule allowing access from your local networks. Adapt # to list your (internal) IP networks from where browsing should # be allowed acl our_networks src 192.168.1.0/24 http_access allow our_networks http_access allow localhost # And finally deny all other access to this proxy http_access deny all
If you're using a different network internally then you will need to adjust the addresses appropriately.
That's all the squid setup complete, so now we restart it:
/etc/init.d/squid restart
Now we have a caching proxy server - which you is listening on 192.168.1.1:8080. If you were to enter that into your browser you should see it working - but what we are going to do next is make it transparent.
Nobody behind the gateway should need to do anything, instead it should just magically work (tm ;)
The way we do that is to add a rule to the firewal, which will redirect outgoing requests to the web (port 80) to instead go via the proxy server we've setup on the gateway machine on port 8080.
Add the following towards the end of your firewall rules:
# Transparent proxying iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
This says that anything coming from the internal interface (eth0) which has a destination port of 80 (web) should be redirected to the new squid installation we've made on port 8080.
Reload your firewall and you should have it in place.
To test it you simply need to watch the squid logfile as you browse the web from another machine.
On the gateway machine you can run:
tail -f /var/log/squid/access.log
If you see something like the following when you surf the web on a machine upon your LAN you know it worked:
1104854410.086 159 192.168.1.50 TCP_MISS/302 469 GET http://www.google.com/ - DIRECT/216.239.59.104 text/html 1104854410.217 128 192.168.1.50 TCP_MISS/200 1459 GET http://www.google.co.uk / - DIRECT/216.239.59.99 text/html 1104854410.397 180 192.168.1.50 TCP_MISS/200 9022 GET http://www.google.co.uk /intl/en_uk/images/logo.gif - DIRECT/216.239.59.99 image/gif 1104854415.196 200 192.168.1.50 TCP_MISS/200 1459 GET http://www.google.co.uk / - DIRECT/216.239.59.99 text/html 1104854415.271 74 192.168.1.50 TCP_REFRESH_HIT/304 235 GET http://www.google .co.uk/intl/en_uk/images/logo.gif - DIRECT/216.239.59.99 text/html
[ Send Message | View Serge's Scratchpad | View Weblogs ]
Tweakinq Squid's default listen port (tcp/3128 if I recall well) is not really necessary, as you can as well transparently forward whatever port to it.
An interesting extra might be to
apt-get install dansguardian
Which is a very good filtering engine and plugs to Squid (on tcp/3128) by default, and which listens himself on tcp/8080. Then configure transparent proxy to Dansguardian's port.
--
Serge van Ginderachter
[ Parent | Reply to this comment ]
[ Send Message | View Serge's Scratchpad | View Weblogs ]
--
Serge van Ginderachter
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
I've patched up your comment, and I'll get working on the preview code again - it used to work but I broke it and couldn't understand why.
The ability to edit comments is very definately not going to be allowed, unless it will keep a diff against the previous one - it's too easy to get into arguments where people make each other look trolly by posting inflamatory comments and then editting them later..
Steve
-- Steve.org.uk
[ Parent | Reply to this comment ]
The ability to edit comments is very definately not going to be allowed, unless it will keep a diff against the previous one - it's too easy to get into arguments where people make each other look trolly by posting inflamatory comments and then editting them later..
Some web forums try to deal with this problem by limiting the amount of time during which a user can edit their posts, say 15 minutes. They might then also add the "Last edited on ..." line.
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Steve
-- Steve.org.uk
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
I use a transparent proxy on a gateway machine for 40+ users, and have never seen any problems, but I guess this is just a case of buyer-beware.
Testing is never a bad thing ;)
Steve
-- Steve.org.uk
[ Parent | Reply to this comment ]
Should I be frightened ?
bye.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
do you know of any way of doing this?
[ Parent | Reply to this comment ]
--internet---<eth0---eth1>----internal network
assuming that you have a working transparent proxy according to the tutorial?
to clarify, using the transparent proxy and turning it into a bridge.
firewalling in not neccessary as there is a firewall, dhcp server between internet and eth0
[ Parent | Reply to this comment ]
try this
iptable -A FORWARD -s 192.168.0.0/24 -j ACCEPT
iptable -t nat -A POSTROUTING -s 192.168.0.0/24 -j MASQUERADE
echo 1 > /proc/sys/net/ipv4/ip_forward
or
iptables -t nat -A PREROUTING -s 192.168.0.0/24 -p tcp --dport 80 -j REDIRECT --to-port 3128
echo 1 > /proc/sys/net/ipv4/ip_forward
or
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 3128
echo 1 > /proc/sys/net/ipv4/ip_forward
assuming
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
is your squid.conf file entries)
sunu
[ Parent | Reply to this comment ]
For example, my gateway is 192.168.1.1, and my proxy server is 192.168.1.2, I want to be able to redirect requests on port 80 to 192.168.1.2:8080
[ Parent | Reply to this comment ]
iptables -t nat -A PREROUTING -i eth0 -s ! squid-box -p tcp --dport 80 -j DNAT --to squid-box:3128
iptables -t nat -A POSTROUTING -o eth0 -s local-network -d squid-box -j SNAT --to iptables-box
iptables -A FORWARD -s local-network -d squid-box -i eth0 -o eth0 -p tcp --dport 3128 -j ACCEPT
From http://www.faqs.org/docs/Linux-mini/TransparentProxy.html#s6
Sorted :)
[ Parent | Reply to this comment ]
One such plugin -- and honestly I'm surprised it's gone unmentioned here -- is adzapper, a simple filtering add-on designed to match and block advertisements in all web content that passes through Squid (it apparently works with Apache, too).
Installing it on a sarge machine is as simple as invoking
apt-get install adzapper
and then adding the following line to your /etc/squid/squid.conf :
redirect_program /usr/bin/adzapper.wrapper
Once this is done, simply restart your proxy by issuing a /etc/init.d/squid restart, and voilà! You have a basic ad-blocking setup. Any website you visit (via your squid proxy of course) will now be filtered on the fly so that ad images are replaced with an unvarying placeholder, and ad text is replaced with the words This ad zapped.
Over time this will save you tons of bandwidth, not to mention time spent waiting pages to load.
[ Parent | Reply to this comment ]
you forgot one thing though took me hours
to figure out why it was not working. since i'm a noob
at this.
First, you must create the swap directories. Do this by running Squid with the -z option:
Squid -z
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
That should happen automatically since the startup script /etc/init.d/squid will create them if they are missing ...
[ Parent | Reply to this comment ]
forwarded_for off
if not your private and public IP is visible
--
Best regard
[ Parent | Reply to this comment ]
I've got ipmasq/dnsmasq running for my parents' business. (they're in the boonies w/NO broadband) I could use dial-on-demand, but for their convenience I've built a little perl app that allows them to browse to http://192.168.1.1:8000 and click a button to connect and disconnect. They like that level of control.
For anybody who's curious: The script gets launched from an init script like a daemon, forks like apache and then listens on the configured port for HTTP requests. When somebody clicks the connect or disconnect button, the script just issues an ifup ppp0 or ifdown ppp0 command. It's been working wonderfully for several years. (Though sometimes ifupdown freaks out over dropped connections and has to be reset - that's done through another button)
Anyhow, I'd love to use a transparent proxy to speed up their browsing and things like windows updates, but I've found nothing that's 'seamless' with dial-up.
If anybody reading this can give me some pointers, please reply! Maybe there's something I can do for you as well! :)
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Yes this will work with a dialup link, you'd just need to adjust the firewall rule to use ppp0/whatever instead of eth0.
[ Parent | Reply to this comment ]
I'll have to try it now.
[ Parent | Reply to this comment ]
I have freeradius & mysql & pppoe-server installed
I'd install squid and it's only working when i put the proxy and port in browser
Im trying to make it work in transperant way i'd set the iptables rule and the ip forward too
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080
echo 1 > /proc/sys/net/ipv4/ip_forward
But still it's not working in transperant way.. here is my route -n settings
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
10.67.15.30 0.0.0.0 255.255.255.255 UH 0 0 0 ppp1
10.67.15.31 0.0.0.0 255.255.255.255 UH 0 0 0 ppp2
10.67.15.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 eth1
eth1 is connecting to the router
eth0 is for the network
when i open http://ipaddress:8080/ it works but when i browse it didn't show me the browsing proccess in the access.log
waiting ur response
Dino
[ Parent | Reply to this comment ]
Proxy1: 192.168.100.13:8080 - outside my administrative privileges
Station1: 192.168.1.30 - connected to the Internet via Proxy1
Station2: 192.168.1.31 - not allowed to use Proxy1
I want to configure Station1 to act as a proxy server through which Station2 can get access to the Internet. Something like Proxy1 -> Station1 (intermediate proxy) -> Station2.
Two problems:
1 - Looks like the httpd_accel_* TAGS aren't recognized when squid.conf is parsed:
ltsp-server:~# /etc/init.d/squid restart Restarting Squid HTTP proxy: squid Waiting.................................done. 2006/09/17 05:42:13| parseConfigFile: line 2982 unrecognized: 'httpd_accel_host virtual' 2006/09/17 05:42:13| parseConfigFile: line 2983 unrecognized: 'httpd_accel_port 80' 2006/09/17 05:42:13| parseConfigFile: line 2984 unrecognized: 'httpd_accel_with_proxy on' 2006/09/17 05:42:13| parseConfigFile: line 2985 unrecognized: 'httpd_accel_uses_host_header on' . ltsp-server:~#
2 - Ignoring these errors, squid starts on the server (Station1). The client (Station2) connects to Station1:8080, sends an HTTP request, and the proxy server returns "ERROR 503: Service Unavailable":
ultra1:~# wget http://www.google.com --06:02:09-- http://www.google.com/ => `index.html' Connecting to 192.168.1.30:8080... connected. Proxy request sent, awaiting response... 503 Service Unavailable 06:03:45 ERROR 503: Service Unavailable. ultra1:~#
This is logged on the proxy server machine too:
ltsp-server:~# tail -f /var/log/squid/access.log 1158452477.733 49432 192.168.1.31 TCP_MISS/503 1542 GET http://www.google.com/ - DIRECT/www.google.com text/html 1158452477.733 155009 192.168.1.31 TCP_MISS/503 1542 GET http://www.google.com/ - DIRECT/www.google.com text/html 1158452477.733 34808 192.168.1.31 TCP_MISS/503 1542 GET http://www.google.com/ - DIRECT/www.google.com text/html 1158452707.379 155012 192.168.1.31 TCP_MISS/503 1542 GET http://www.google.com/ - DIRECT/www.google.com text/html ltsp-server:~#
What am I doing wrong? Or is such a setup not possible?
[ Parent | Reply to this comment ]
Basically, the accel directives go away, and http_port becomes
http_port <port> transparent
[ Parent | Reply to this comment ]
I need to add a 'parent' 'cache_peer':
cache_peer 192.168.100.13 parent 8080 0 proxy-only no-query
Additionally, I need to forward all cache requests to my 'parent':
never_direct allow all
Though it's not always necessary, I should also specify which pages never to cache:
acl QUERY urlpath_regex cgi-bin \? asp jsp php cache deny QUERY
Works beautifully for me. Hope this helps.
[ Parent | Reply to this comment ]
I have the same problems with you.
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]
Another great article, that got me up and running in a few minutes. When I googled for those magic words, squid, transparent, proxy, iptables, debian I kinda knew your site would be near the top. Glad I hit this first, it Just Works (tm).
Cheers,
popey
[ Parent | Reply to this comment ]
How do I set the default owner to 'proxy' for the new access.log, or how do I get squid to accept an access.log owned by 'root'?
Regards,
Yousuf
[ Parent | Reply to this comment ]
How do I set the default owner to 'proxy' for the new access.log, or how do I get squid to accept an access.log owned by 'root'?
Regards,
Yousuf
"The Network IS The Computer"
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
Take a look at /etc/logrotate.d/squid - that should be the file that does the rotation.
[ Parent | Reply to this comment ]
The following directives are not available any more in the squid.conf file in the third version of squid
httpd_accel_host virtual
httpd_accel_port 80
httpd_accel_with_proxy on
httpd_accel_uses_host_header on
[ Parent | Reply to this comment ]
New way to enable transparent proxying is to make your http_port line read similar to this:
http_port 192.168.1.1:8080 transparent
Note the word "transparent at the end... Seems this is the only part required. I have mine simply read "http_port 3128 transparent".
[ Parent | Reply to this comment ]
Thanks :)
[ Parent | Reply to this comment ]
QUESTION: I tried this stuffs on my proxy server. But why am i getting the following errors:
| parseConfigFile: line 131 unrecognized: 'iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080' |
| parseConfigFile: line 132 unrecognized: 'iptables -A PREROUTING -t nat -i eth1 -p tcp --dport 80 -j DNAT --to 192.168.2.10:80' |
| parseConfigFile: line 133 unrecognized: 'iptables -A FORWARD -p tcp -m state --state NEW -d 192.168.2.10 --dport 80 -j ACCEPT' |
I hope someone out there could help me. Thanks!
[ Parent | Reply to this comment ]
[ Send Message | View Steve's Scratchpad | View Weblogs ]
These are lines to be added to the end of your firewall - or entered interactively. They should not be added to the squid configuration file.
[ Parent | Reply to this comment ]
# Transparent proxying
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port
8080
Been searching looong for a solution for transparently filtering content on LAN hosts.
THANK YOU VERY MUCH !
[ Parent | Reply to this comment ]
[ Parent | Reply to this comment ]