About these ads

David Vassallo's Blog

If at first you don't succeed; call it version 1.0

SQUID transparent SSL interception


July 2012: Small update on new versions of squid (squid v 3.2) here

There seems to be a bit of confusion about configuring SQUID to transparently intercept SSL (read: HTTPS) connections. Some sites say it’s plain not possible:

http://www.faqs.org/docs/Linux-mini/TransparentProxy.html#ss2.3

Recent development in SQUID features have made this possible. This article explores how to set this up at a basic level. The SQUID proxy will basically act as a man in the middle. The motivation behind setting this up is to decrypt HTTPS connections to apply content filtering and so on.

There are some concerns that transparently intercepting HTTPS traffic is unethical and can cause legality issues. True, and I agree that monitoring HTTPS connections without properly and explicitly notifying the user is bad but we can use technical means to ensure that the user is properly notified and even gets prompted to accept monitoring or back out. More on this towards the end of the article

So, on to the technical details of setting the proxy up. First, install the dependencies . We will need to compile SQUID from scratch since by default it’s not compiled using the necessary switches. I recommend downloading the latest 3.1 version, especially if you want to notify users about the monitoring. In ubuntu:

apt-get install build-essential libssl-dev

Note : for CentOS users, use openssl-devel rather than libssl-dev

Build-essentials downloads the compilers while libssl downloads SSL libraries that enable SQUID to intercept the encrypted traffic. This package (libssl) is needed during compilation. Without it, when running make you will see the errors similar to the following in the console:

error: ‘SSL’ was not declared in this scope

Download and extract the SQUID source code from their site. Next, configure, compile and install the source code using:

./configure –enable-icap-client –enable-ssl
make
make install

Note the switches I included in the configure command:

* enable-icap-client : we’ll need this to use ICAP to provide a notification page to clients that they are being monitored.

* enable-ssl : this is a prerequisite for SslBump, which squid uses to intercept SSL traffic transparently

Once SQUID has been installed, a very important step is to create the certificate that SQUID will present to the end client. In a test environment, you can easily create a self-signed certificate using OpenSSL by using the following:

openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout http://www.sample.com.pem  -out http://www.sample.com.pem

This will of course cause the client browser to display an error:

tranSSL

In an enterprise environment you’ll probably want to generate the certificate using a CA that the clients already trust. For example, you could generate the certificate using microsoft’s CA and use certificate auto-enrolment to push the certificate out to all the clients in your domain.

Onto the actual SQUID configuration. Edit the /etc/squid.conf file to show the following:

always_direct allow all
ssl_bump allow all

http_port 192.9.200.32:3128 transparent

#the below should be placed on a single line
https_port 192.9.200.32:3129 transparent ssl-bump cert=/etc/squid/ssl_cert/www.sample.com.pem key=/etc/squid/ssl_cert/private/www.sample.com.pem

Note you may need to change the “cert=” and the “key=” to point to the correct file in your environment. Also of course you will need to change the IP address

The first directive (always_direct) is due to SslBump. By default ssl_bump is set to accelerator mode. In debug logs cache.log you’d see “failed to select source for”. In accelerator mode, the proxy does not know which backend server to use to retrieve the file from, so this directive instructs the proxy to ignore the accelerator mode. More details on this here:

http://www.squid-cache.org/Doc/config/always_direct/

The second directive (ssl_bump) instructs the proxy to allow all SSL connections, but this can be modified to restirct access. You can also use the “sslproxy_cert_error” to deny access to sites with invalid certificates. More details on this here:

http://wiki.squid-cache.org/Features/SslBump

Start squid and check for any errors. If no errors are reported, run:

netstat -nap | grep 3129

to make sure the proxy is up and running. Next, configure iptables to perform destination NAT, basically to redirect the traffic to the proxy:

iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j DNAT –to-destination 192.9.200.32:3128
iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 443 -j DNAT –to-destination 192.9.200.32:3129

Last thing to be done was to either place the proxy physically in line with the traffic or to redirect the traffic to the proxy using a router. Keep in mind that the proxy will change the source IP address of the requests to it’s own IP. In other words, by default it does not reflect the client IP.

That was it in my case. I did try to implement something similar to the above but using explicit mode. This was my squid.conf file, note only one port is needed for both HTTP and HTTPS since HTTPS is tunneled over HTTP using the CONNECT method:

always_direct allow all
ssl_bump allow all

#the below should be placed on a single line

http_port 8080 ssl-bump cert=/etc/squid/ssl_cert/proxy.testdomain.deCert.pem key=/etc/squid/ssl_cert/private/proxy.testdomain.deKey_without_Pp.pem

As regards my previous discussion of notifying users that they are being monitored, consider using greasyspoon:

http://dvas0004.wordpress.com/2011/02/28/squid-greasyspoon-enhancing-your-proxy-deployment-with-content-adaptation/

With this in place, you can instruct greasyspoon to send a notify page to the clients. If they accept this notify page, a cookie (let’s say the cookie is called “NotifySSL”) is set. GreasySpoon can then check for the presence of this cookie in subsequent requests and if present, allow the connection. If the cookie is not present, customers again get the notify page. Due to security considerations, most of the time cookies are only valid for one domain, so you may end up with users having to accept the notify for each different domain they visit. But, you can use greasyspoon in conjunction with a backend MySQL database or similar to record source IP addresses that have been notified and perform IP based notifications. Anything is possible :)


About these ads

23 responses to “SQUID transparent SSL interception

  1. John April 28, 2011 at 6:44 am

    Hi, Thanks for the guide it works.
    I had to modify: ssl-bump into sslBump in the following
    https_port 192.9.200.32:3129 transparent ssl-bump cert=/etc/squid/ssl_cert/www.sample.com.pem key=/etc/squid/ssl_cert/private/www.sample.com.pem

    Now, privoxy will not work with following directive enabled:
    always_direct allow all

    In order to allow squid to parent to another proxy such as privoxy I have to add:
    cache_peer localhost parent 8118 0 default no-query no-delay no-digest no-netdb-exchange
    never_direct allow all

    Is there a way to enable both https and privoxy as “always_direct allow all” kicks privoxy out of play.

    Regards

    • dvas0004 May 3, 2011 at 10:22 am

      Hi

      I would say your best way forward would be to divert only port 80 (HTTP) traffic to privoxy while allowing SSL traffic to go direct (not through privoxy). This is not such a big limiation as it would seem since anyway Privoxy in it’s normal mode does not filter HTTPS. If you look at the provoxy FAQ (http://www.privoxy.org/faq/misc.html),section 4.15, it says:

      4.15. How can Privoxy filter Secure (HTTPS) URLs?

      Since secure HTTP connections are encrypted SSL sessions between your browser and the secure site, and are meant to be reliably secure, there is little that Privoxy can do but hand the raw gibberish data though from one end to the other unprocessed.

      So at least you’ll have SQUID intercepting SSL, and then sending it direct, while HTTP traffic will be passed over to privoxy. To do this, you need to use the following two lines:

      acl PrivoxyPorts port 80
      always_direct allow !PrivoxyPorts

      The first line defines an ACL with destination port 80 for HTTP traffic. The second line will replace the “always_direct allow all” so that only port 80 will be sent to privoxy

      Let me know if that works! :)

  2. Fabio Schmidt July 27, 2011 at 4:20 am

    Hi !

    Great post man ! Have you developed the script to notify user that they are being monitored?

    • dvas0004 November 19, 2012 at 6:10 am

      Not yet :( never enough time :(

  3. JOHN August 23, 2011 at 6:36 am

    A good solution for SSL interception.
    But I think clients would receive warnings about SSL certificates, because they are anot the URLs clients requested. Do they?

    • dvas0004 November 19, 2012 at 6:09 am

      They would, but that is what the “sslbump” and dynamic ssl generation feature would address. Have a look :

      http://wiki.squid-cache.org/Features/DynamicSslCert

  4. TimK October 20, 2011 at 11:25 pm

    Is their a way to have squid acl’s (to filer some traffic) apply to https traffic without caching or decrypting the connection. I’m just trying to block based on URL, but so far, it’s looking like the only way is to that for HTTPS is by blocking the DNS request. I’d just like the same acl’s to apply to HTTPS as they apply to HTTP.

    • dvas0004 November 19, 2012 at 6:24 am

      Hi Tim,

      There are two possible ways of doing that..

      If you setup an explicit proxy, normal http acls will do the trick because a client will first have to request the URL over an unencrypted HTTP CONNECT request. You can see a couple of examples filtering the CONNECT request here:

      http://wiki.squid-cache.org/ConfigExamples/Authenticate/Bypass

      In transparent mode the only way of filtering without decrypting is putting an ACL on the common name (CN) of an ssl certificate but there are a couple of problems:

      * I couldnt find any documentation stating if and how squid supports this (other commercial proxies such as bluecoat do so technically this is definitely possible to implement

      * CNs do not always reflect the proper name of the site, a prime example is youtube whose certificate has a google.com CN but then utilises Subject Alternative Name (SAN)

      Hope that helps a bit…

      Dave

  5. Jan F. February 26, 2012 at 3:33 pm

    Hi, I just implemented it. You forgot the dynamic certification. Thats what makes Squid this way work as an dynamic trusted man in the middle. Otherwise you’ve to one certificate for each domain. Here’s thewith compile options an the modified configline:

    ./configure –enable-ssl –enable-ssl-crtd

    https_port 127.0.0.1:3129 intercept ssl-bump
    generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
    options=ALL cert=/etc/apache2/ssl/apache.pem

    and dont forget to make the log and cert dirs accessable for squid and certd!

    http://wiki.squid-cache.org/Features/DynamicSslCert

    • ximo March 26, 2012 at 2:47 am

      Jan F, did you get a working transparent HTTPS proxy?

      As far as I can tell from Squid’s wiki, that shouldn’t be possible with SslBump as of yet (http://wiki.squid-cache.org/Features/DynamicSslCert#Limitations)..

      Good news is there’s some interesting development going on for this:

      http://wiki.squid-cache.org/Features/MimicSslServerCert

      http://wiki.squid-cache.org/Features/BumpSslServerFirst

    • ximo March 26, 2012 at 2:50 am

      Jan F, did you get a transparent HTTPS proxy working?

      As far as I can tell from Squid’s wiki, that shouldn’t be possible with SslBump yet (http://wiki.squid-cache.org/Features/DynamicSslCert#Limitations)..

      Good news is there’s some interesting development going on:

      http://wiki.squid-cache.org/Features/MimicSslServerCert

      http://wiki.squid-cache.org/Features/BumpSslServerFirst

    • Alex Rousskov April 12, 2012 at 3:54 pm

      Jan, this blog post was about intercepted HTTPS connections. Support for dynamic certificate generation for intercepted HTTPS connections is expected in Squid v3.3.

      Current Squid releases cannot support dynamic certificate generation in interception mode because Squid does not get the origin server domain name at the time it needs to generate the fake certificate for the intercepted client. Your Squid can intercept and bump the SSL connection, but Squid will not generate a matching fake certificate, causing numerous warnings for popular secure sites, even if the user trusts Squid CA. This makes bumping intercepted HTTPS connections with the currently available code inpractical in a general setup.

      For more information about the solution to this problem, please see http://wiki.squid-cache.org/Features/BumpSslServerFirst

  6. Pingback: wpad/pac or transparent proxy

  7. Pingback: How to block Ultrasurf from squid? - Digit Technology Discussion Forum

  8. WildcArd July 5, 2012 at 5:30 am

    This is very helpful thanks!!!

    for squid 3.1, don’t you mean ‘intercept’ instead of ‘transparent’? also, I get ‘WARNING: transparent proxying not supported’

    • dvas0004 July 5, 2012 at 1:09 pm

      Yes indeed… I just looked it up and Devs have changed the argument from transparent to intercept – well done in my opinion it better reflects the functionality

  9. Pingback: Update: SQUID transparent SSL interception : Squid v3.2 « Dvas0004's Blog

  10. Pingback: Squid + SSL | Sem Memória

  11. xXsamXx1991 December 2, 2012 at 4:56 pm

    Great work thanks!
    I have used REDIRECT and that was my mistake.
    Now i use DNAT and it works :)

  12. Pingback: BYOD Linux Server for Squid, DNS & DHCP

  13. Pingback: SQUID transparent SSL interception | 6PM Solutions

  14. Bilal Malik November 7, 2014 at 8:18 am

    Why can’t you use SSLBUMP on squid to handle HTTPS traffic over the internet and deliver content to the client in HTTP. That way you will be able to filter and cache the content. And there are no issues reagarding CLIENT end root certificate

    • dvas0004 November 7, 2014 at 9:20 am

      Hi Bilal

      Good point, but it wont always work because some of the bigger sites like facebook, google, twitter, etc are starting to implement HSTS (HTTP strict transport security) which forces a browser to use HTTPS rather than HTTP:

      https://www.owasp.org/index.php/HTTP_Strict_Transport_Security

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 170 other followers

%d bloggers like this: