July 2012: Small update on new versions of squid (squid v 3.2) here
There seems to be a bit of confusion about configuring SQUID to transparently intercept SSL (read: HTTPS) connections. Some sites say it’s plain not possible:
http://www.faqs.org/docs/Linux-mini/TransparentProxy.html#ss2.3
Recent development in SQUID features have made this possible. This article explores how to set this up at a basic level. The SQUID proxy will basically act as a man in the middle. The motivation behind setting this up is to decrypt HTTPS connections to apply content filtering and so on.
There are some concerns that transparently intercepting HTTPS traffic is unethical and can cause legality issues. True, and I agree that monitoring HTTPS connections without properly and explicitly notifying the user is bad but we can use technical means to ensure that the user is properly notified and even gets prompted to accept monitoring or back out. More on this towards the end of the article
So, on to the technical details of setting the proxy up. First, install the dependencies . We will need to compile SQUID from scratch since by default it’s not compiled using the necessary switches. I recommend downloading the latest 3.1 version, especially if you want to notify users about the monitoring. In ubuntu:
apt-get install build-essential libssl-dev
Note : for CentOS users, use openssl-devel rather than libssl-dev
Build-essentials downloads the compilers while libssl downloads SSL libraries that enable SQUID to intercept the encrypted traffic. This package (libssl) is needed during compilation. Without it, when running make you will see the errors similar to the following in the console:
error: ‘SSL’ was not declared in this scope
Download and extract the SQUID source code from their site. Next, configure, compile and install the source code using:
./configure –enable-icap-client –enable-ssl
make
make install
Note the switches I included in the configure command:
* enable-icap-client : we’ll need this to use ICAP to provide a notification page to clients that they are being monitored.
* enable-ssl : this is a prerequisite for SslBump, which squid uses to intercept SSL traffic transparently
Once SQUID has been installed, a very important step is to create the certificate that SQUID will present to the end client. In a test environment, you can easily create a self-signed certificate using OpenSSL by using the following:
openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout http://www.sample.com.pem -out http://www.sample.com.pem
This will of course cause the client browser to display an error:
In an enterprise environment you’ll probably want to generate the certificate using a CA that the clients already trust. For example, you could generate the certificate using microsoft’s CA and use certificate auto-enrolment to push the certificate out to all the clients in your domain.
Onto the actual SQUID configuration. Edit the /etc/squid.conf file to show the following:
always_direct allow all
ssl_bump allow allhttp_port 192.9.200.32:3128 transparent
#the below should be placed on a single line
https_port 192.9.200.32:3129 transparent ssl-bump cert=/etc/squid/ssl_cert/www.sample.com.pem key=/etc/squid/ssl_cert/private/www.sample.com.pem
Note you may need to change the “cert=” and the “key=” to point to the correct file in your environment. Also of course you will need to change the IP address
The first directive (always_direct) is due to SslBump. By default ssl_bump is set to accelerator mode. In debug logs cache.log you’d see “failed to select source for”. In accelerator mode, the proxy does not know which backend server to use to retrieve the file from, so this directive instructs the proxy to ignore the accelerator mode. More details on this here:
http://www.squid-cache.org/Doc/config/always_direct/
The second directive (ssl_bump) instructs the proxy to allow all SSL connections, but this can be modified to restirct access. You can also use the “sslproxy_cert_error” to deny access to sites with invalid certificates. More details on this here:
http://wiki.squid-cache.org/Features/SslBump
Start squid and check for any errors. If no errors are reported, run:
netstat -nap | grep 3129
to make sure the proxy is up and running. Next, configure iptables to perform destination NAT, basically to redirect the traffic to the proxy:
iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j DNAT –to-destination 192.9.200.32:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 443 -j DNAT –to-destination 192.9.200.32:3129
Last thing to be done was to either place the proxy physically in line with the traffic or to redirect the traffic to the proxy using a router. Keep in mind that the proxy will change the source IP address of the requests to it’s own IP. In other words, by default it does not reflect the client IP.
That was it in my case. I did try to implement something similar to the above but using explicit mode. This was my squid.conf file, note only one port is needed for both HTTP and HTTPS since HTTPS is tunneled over HTTP using the CONNECT method:
always_direct allow all
ssl_bump allow all#the below should be placed on a single line
http_port 8080 ssl-bump cert=/etc/squid/ssl_cert/proxy.testdomain.deCert.pem key=/etc/squid/ssl_cert/private/proxy.testdomain.deKey_without_Pp.pem
As regards my previous discussion of notifying users that they are being monitored, consider using greasyspoon:
With this in place, you can instruct greasyspoon to send a notify page to the clients. If they accept this notify page, a cookie (let’s say the cookie is called “NotifySSL”) is set. GreasySpoon can then check for the presence of this cookie in subsequent requests and if present, allow the connection. If the cookie is not present, customers again get the notify page. Due to security considerations, most of the time cookies are only valid for one domain, so you may end up with users having to accept the notify for each different domain they visit. But, you can use greasyspoon in conjunction with a backend MySQL database or similar to record source IP addresses that have been notified and perform IP based notifications. Anything is possible 🙂
David Vassallo's Blog 
Hi, Thanks for the guide it works.
I had to modify: ssl-bump into sslBump in the following
https_port 192.9.200.32:3129 transparent ssl-bump cert=/etc/squid/ssl_cert/www.sample.com.pem key=/etc/squid/ssl_cert/private/www.sample.com.pem
Now, privoxy will not work with following directive enabled:
always_direct allow all
In order to allow squid to parent to another proxy such as privoxy I have to add:
cache_peer localhost parent 8118 0 default no-query no-delay no-digest no-netdb-exchange
never_direct allow all
Is there a way to enable both https and privoxy as “always_direct allow all” kicks privoxy out of play.
Regards
Hi
I would say your best way forward would be to divert only port 80 (HTTP) traffic to privoxy while allowing SSL traffic to go direct (not through privoxy). This is not such a big limiation as it would seem since anyway Privoxy in it’s normal mode does not filter HTTPS. If you look at the provoxy FAQ (http://www.privoxy.org/faq/misc.html),section 4.15, it says:
4.15. How can Privoxy filter Secure (HTTPS) URLs?
Since secure HTTP connections are encrypted SSL sessions between your browser and the secure site, and are meant to be reliably secure, there is little that Privoxy can do but hand the raw gibberish data though from one end to the other unprocessed.
So at least you’ll have SQUID intercepting SSL, and then sending it direct, while HTTP traffic will be passed over to privoxy. To do this, you need to use the following two lines:
acl PrivoxyPorts port 80
always_direct allow !PrivoxyPorts
The first line defines an ACL with destination port 80 for HTTP traffic. The second line will replace the “always_direct allow all” so that only port 80 will be sent to privoxy
Let me know if that works! 🙂
Hi !
Great post man ! Have you developed the script to notify user that they are being monitored?
Not yet 😦 never enough time 😦
A good solution for SSL interception.
But I think clients would receive warnings about SSL certificates, because they are anot the URLs clients requested. Do they?
They would, but that is what the “sslbump” and dynamic ssl generation feature would address. Have a look :
http://wiki.squid-cache.org/Features/DynamicSslCert
Is their a way to have squid acl’s (to filer some traffic) apply to https traffic without caching or decrypting the connection. I’m just trying to block based on URL, but so far, it’s looking like the only way is to that for HTTPS is by blocking the DNS request. I’d just like the same acl’s to apply to HTTPS as they apply to HTTP.
Hi Tim,
There are two possible ways of doing that..
If you setup an explicit proxy, normal http acls will do the trick because a client will first have to request the URL over an unencrypted HTTP CONNECT request. You can see a couple of examples filtering the CONNECT request here:
http://wiki.squid-cache.org/ConfigExamples/Authenticate/Bypass
In transparent mode the only way of filtering without decrypting is putting an ACL on the common name (CN) of an ssl certificate but there are a couple of problems:
* I couldnt find any documentation stating if and how squid supports this (other commercial proxies such as bluecoat do so technically this is definitely possible to implement
* CNs do not always reflect the proper name of the site, a prime example is youtube whose certificate has a google.com CN but then utilises Subject Alternative Name (SAN)
Hope that helps a bit…
Dave
Hi, I just implemented it. You forgot the dynamic certification. Thats what makes Squid this way work as an dynamic trusted man in the middle. Otherwise you’ve to one certificate for each domain. Here’s thewith compile options an the modified configline:
./configure –enable-ssl –enable-ssl-crtd
https_port 127.0.0.1:3129 intercept ssl-bump
generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
options=ALL cert=/etc/apache2/ssl/apache.pem
and dont forget to make the log and cert dirs accessable for squid and certd!
http://wiki.squid-cache.org/Features/DynamicSslCert
Jan F, did you get a working transparent HTTPS proxy?
As far as I can tell from Squid’s wiki, that shouldn’t be possible with SslBump as of yet (http://wiki.squid-cache.org/Features/DynamicSslCert#Limitations)..
Good news is there’s some interesting development going on for this:
http://wiki.squid-cache.org/Features/MimicSslServerCert
http://wiki.squid-cache.org/Features/BumpSslServerFirst
Jan F, did you get a transparent HTTPS proxy working?
As far as I can tell from Squid’s wiki, that shouldn’t be possible with SslBump yet (http://wiki.squid-cache.org/Features/DynamicSslCert#Limitations)..
Good news is there’s some interesting development going on:
http://wiki.squid-cache.org/Features/MimicSslServerCert
http://wiki.squid-cache.org/Features/BumpSslServerFirst
Jan, this blog post was about intercepted HTTPS connections. Support for dynamic certificate generation for intercepted HTTPS connections is expected in Squid v3.3.
Current Squid releases cannot support dynamic certificate generation in interception mode because Squid does not get the origin server domain name at the time it needs to generate the fake certificate for the intercepted client. Your Squid can intercept and bump the SSL connection, but Squid will not generate a matching fake certificate, causing numerous warnings for popular secure sites, even if the user trusts Squid CA. This makes bumping intercepted HTTPS connections with the currently available code inpractical in a general setup.
For more information about the solution to this problem, please see http://wiki.squid-cache.org/Features/BumpSslServerFirst
This is very helpful thanks!!!
for squid 3.1, don’t you mean ‘intercept’ instead of ‘transparent’? also, I get ‘WARNING: transparent proxying not supported’
Yes indeed… I just looked it up and Devs have changed the argument from transparent to intercept – well done in my opinion it better reflects the functionality
Great work thanks!
I have used REDIRECT and that was my mistake.
Now i use DNAT and it works 🙂
Why can’t you use SSLBUMP on squid to handle HTTPS traffic over the internet and deliver content to the client in HTTP. That way you will be able to filter and cache the content. And there are no issues reagarding CLIENT end root certificate
Hi Bilal
Good point, but it wont always work because some of the bigger sites like facebook, google, twitter, etc are starting to implement HSTS (HTTP strict transport security) which forces a browser to use HTTPS rather than HTTP:
https://www.owasp.org/index.php/HTTP_Strict_Transport_Security
Just been testing an sslbump type solution but ran into an HSTS issue for Google: “Your connection is not private” – does HSTS require the MimicSSL feature before it can be filtered?
I met the same issue. I used the peek and slice with squid 3.5.However, for sites without HSTS , it gives a warning that the connection is not private so the user need to manually proceed. For sites with HSTS, the connection is simply refused.
1)Is there any solution to remove that warning?
2)Is there any solution to support sslbump with HSTS?
I have a same problem with HSTS.
Thank you for this article.
Even though it’s a bit old, I want to let you know that the dashes you uses in this line (and possibly others) : “./configure –enable-icap-client –enable-ssl” are not the right one ( – ), which caused me some trouble for almost an hour before I realized it !!
Thanks for pointing that out! 🙂