Tagged: ssl Toggle Comment Threads | Keyboard Shortcuts

  • dvas0004 1:21 pm on July 5, 2012 Permalink | Reply
    Tags: , , , squid cache, ssl, ssl server certificates   

    Update: SQUID transparent SSL interception : Squid v3.2 

    In order to keep this blog post a bit more relevant, there have been some improvements since that post was written. Squid v3.2 has been released earlier this year, making ssl interception more seamless and easier. The new features for HTTPS interception can be found while reading through the man page for http_port:

    http://www.squid-cache.org/Versions/v3/3.2/cfgman/http_port.html

    More specifically:

    1. The “transparent” keyword has been changed to “intercept“:

               intercept    Rename of old 'transparent' option to indicate proper functionality.

    INTERCEPT is now better described as:

    intercept	Support for IP-Layer interception of
    			outgoing requests without browser settings.
    			NP: disables authentication and IPv6 on the port.

    2. In order to avoid more certificate errors when intercepting HTTPS sites, squid now can dynamically generate SSL certificates, using generate-host-certificates. This means the CN of the certificate should now match that of the origin server, though the certificate will still be generated using SQUID’s private key:

    SSL Bump Mode Options:
    	    In addition to these options ssl-bump requires TLS/SSL options.
    
    	   generate-host-certificates[=<on|off>]
    			Dynamically create SSL server certificates for the
    			destination hosts of bumped CONNECT requests.When 
    			enabled, the cert and key options are used to sign
    			generated certificates. Otherwise generated
    			certificate will be selfsigned.
    			If there is a CA certificate lifetime of the generated 
    			certificate equals lifetime of the CA certificate. If
    			generated certificate is selfsigned lifetime is three 
    			years.
    			This option is enabled by default when ssl-bump is used.
    			See the ssl-bump option above for more information.

    Looks like the above is an offshoot of the excellent work here: http://wiki.squid-cache.org/Features/DynamicSslCert

    Make sure to use the above two features for smoother HTTPS interception – though remember, always warn users that SSL traffic is being decrypted, privacy is a highly-valued right…

     
  • dvas0004 11:51 am on March 22, 2011 Permalink | Reply
    Tags: , , ssl   

    SQUID transparent SSL interception 

    July 2012: Small update on new versions of squid (squid v 3.2) here

    There seems to be a bit of confusion about configuring SQUID to transparently intercept SSL (read: HTTPS) connections. Some sites say it’s plain not possible:

    http://www.faqs.org/docs/Linux-mini/TransparentProxy.html#ss2.3

    Recent development in SQUID features have made this possible. This article explores how to set this up at a basic level. The SQUID proxy will basically act as a man in the middle. The motivation behind setting this up is to decrypt HTTPS connections to apply content filtering and so on.

    There are some concerns that transparently intercepting HTTPS traffic is unethical and can cause legality issues. True, and I agree that monitoring HTTPS connections without properly and explicitly notifying the user is bad but we can use technical means to ensure that the user is properly notified and even gets prompted to accept monitoring or back out. More on this towards the end of the article

    So, on to the technical details of setting the proxy up. First, install the dependencies . We will need to compile SQUID from scratch since by default it’s not compiled using the necessary switches. I recommend downloading the latest 3.1 version, especially if you want to notify users about the monitoring. In ubuntu:

    apt-get install build-essential libssl-dev

    Note : for CentOS users, use openssl-devel rather than libssl-dev

    Build-essentials downloads the compilers while libssl downloads SSL libraries that enable SQUID to intercept the encrypted traffic. This package (libssl) is needed during compilation. Without it, when running make you will see the errors similar to the following in the console:

    error: ‘SSL’ was not declared in this scope

    Download and extract the SQUID source code from their site. Next, configure, compile and install the source code using:

    ./configure –enable-icap-client –enable-ssl
    make
    make install

    Note the switches I included in the configure command:

    * enable-icap-client : we’ll need this to use ICAP to provide a notification page to clients that they are being monitored.

    * enable-ssl : this is a prerequisite for SslBump, which squid uses to intercept SSL traffic transparently

    Once SQUID has been installed, a very important step is to create the certificate that SQUID will present to the end client. In a test environment, you can easily create a self-signed certificate using OpenSSL by using the following:

    openssl req -new -newkey rsa:1024 -days 365 -nodes -x509 -keyout http://www.sample.com.pem  -out http://www.sample.com.pem

    This will of course cause the client browser to display an error:

    tranSSL

    In an enterprise environment you’ll probably want to generate the certificate using a CA that the clients already trust. For example, you could generate the certificate using microsoft’s CA and use certificate auto-enrolment to push the certificate out to all the clients in your domain.

    Onto the actual SQUID configuration. Edit the /etc/squid.conf file to show the following:

    always_direct allow all
    ssl_bump allow all

    http_port 192.9.200.32:3128 transparent

    #the below should be placed on a single line
    https_port 192.9.200.32:3129 transparent ssl-bump cert=/etc/squid/ssl_cert/www.sample.com.pem key=/etc/squid/ssl_cert/private/www.sample.com.pem

    Note you may need to change the “cert=” and the “key=” to point to the correct file in your environment. Also of course you will need to change the IP address

    The first directive (always_direct) is due to SslBump. By default ssl_bump is set to accelerator mode. In debug logs cache.log you’d see “failed to select source for”. In accelerator mode, the proxy does not know which backend server to use to retrieve the file from, so this directive instructs the proxy to ignore the accelerator mode. More details on this here:

    http://www.squid-cache.org/Doc/config/always_direct/

    The second directive (ssl_bump) instructs the proxy to allow all SSL connections, but this can be modified to restirct access. You can also use the “sslproxy_cert_error” to deny access to sites with invalid certificates. More details on this here:

    http://wiki.squid-cache.org/Features/SslBump

    Start squid and check for any errors. If no errors are reported, run:

    netstat -nap | grep 3129

    to make sure the proxy is up and running. Next, configure iptables to perform destination NAT, basically to redirect the traffic to the proxy:

    iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 80 -j DNAT –to-destination 192.9.200.32:3128
    iptables -t nat -A PREROUTING -i eth0 -p tcp –dport 443 -j DNAT –to-destination 192.9.200.32:3129

    Last thing to be done was to either place the proxy physically in line with the traffic or to redirect the traffic to the proxy using a router. Keep in mind that the proxy will change the source IP address of the requests to it’s own IP. In other words, by default it does not reflect the client IP.

    That was it in my case. I did try to implement something similar to the above but using explicit mode. This was my squid.conf file, note only one port is needed for both HTTP and HTTPS since HTTPS is tunneled over HTTP using the CONNECT method:

    always_direct allow all
    ssl_bump allow all

    #the below should be placed on a single line

    http_port 8080 ssl-bump cert=/etc/squid/ssl_cert/proxy.testdomain.deCert.pem key=/etc/squid/ssl_cert/private/proxy.testdomain.deKey_without_Pp.pem

    As regards my previous discussion of notifying users that they are being monitored, consider using greasyspoon:

    http://dvas0004.wordpress.com/2011/02/28/squid-greasyspoon-enhancing-your-proxy-deployment-with-content-adaptation/

    With this in place, you can instruct greasyspoon to send a notify page to the clients. If they accept this notify page, a cookie (let’s say the cookie is called “NotifySSL”) is set. GreasySpoon can then check for the presence of this cookie in subsequent requests and if present, allow the connection. If the cookie is not present, customers again get the notify page. Due to security considerations, most of the time cookies are only valid for one domain, so you may end up with users having to accept the notify for each different domain they visit. But, you can use greasyspoon in conjunction with a backend MySQL database or similar to record source IP addresses that have been notified and perform IP based notifications. Anything is possible🙂


     
    • John 6:44 am on April 28, 2011 Permalink | Reply

      Hi, Thanks for the guide it works.
      I had to modify: ssl-bump into sslBump in the following
      https_port 192.9.200.32:3129 transparent ssl-bump cert=/etc/squid/ssl_cert/www.sample.com.pem key=/etc/squid/ssl_cert/private/www.sample.com.pem

      Now, privoxy will not work with following directive enabled:
      always_direct allow all

      In order to allow squid to parent to another proxy such as privoxy I have to add:
      cache_peer localhost parent 8118 0 default no-query no-delay no-digest no-netdb-exchange
      never_direct allow all

      Is there a way to enable both https and privoxy as “always_direct allow all” kicks privoxy out of play.

      Regards

      • dvas0004 10:22 am on May 3, 2011 Permalink | Reply

        Hi

        I would say your best way forward would be to divert only port 80 (HTTP) traffic to privoxy while allowing SSL traffic to go direct (not through privoxy). This is not such a big limiation as it would seem since anyway Privoxy in it’s normal mode does not filter HTTPS. If you look at the provoxy FAQ (http://www.privoxy.org/faq/misc.html),section 4.15, it says:

        4.15. How can Privoxy filter Secure (HTTPS) URLs?

        Since secure HTTP connections are encrypted SSL sessions between your browser and the secure site, and are meant to be reliably secure, there is little that Privoxy can do but hand the raw gibberish data though from one end to the other unprocessed.

        So at least you’ll have SQUID intercepting SSL, and then sending it direct, while HTTP traffic will be passed over to privoxy. To do this, you need to use the following two lines:

        acl PrivoxyPorts port 80
        always_direct allow !PrivoxyPorts

        The first line defines an ACL with destination port 80 for HTTP traffic. The second line will replace the “always_direct allow all” so that only port 80 will be sent to privoxy

        Let me know if that works!🙂

    • Fabio Schmidt 4:20 am on July 27, 2011 Permalink | Reply

      Hi !

      Great post man ! Have you developed the script to notify user that they are being monitored?

    • JOHN 6:36 am on August 23, 2011 Permalink | Reply

      A good solution for SSL interception.
      But I think clients would receive warnings about SSL certificates, because they are anot the URLs clients requested. Do they?

    • TimK 11:25 pm on October 20, 2011 Permalink | Reply

      Is their a way to have squid acl’s (to filer some traffic) apply to https traffic without caching or decrypting the connection. I’m just trying to block based on URL, but so far, it’s looking like the only way is to that for HTTPS is by blocking the DNS request. I’d just like the same acl’s to apply to HTTPS as they apply to HTTP.

      • dvas0004 6:24 am on November 19, 2012 Permalink | Reply

        Hi Tim,

        There are two possible ways of doing that..

        If you setup an explicit proxy, normal http acls will do the trick because a client will first have to request the URL over an unencrypted HTTP CONNECT request. You can see a couple of examples filtering the CONNECT request here:

        http://wiki.squid-cache.org/ConfigExamples/Authenticate/Bypass

        In transparent mode the only way of filtering without decrypting is putting an ACL on the common name (CN) of an ssl certificate but there are a couple of problems:

        • I couldnt find any documentation stating if and how squid supports this (other commercial proxies such as bluecoat do so technically this is definitely possible to implement
        • CNs do not always reflect the proper name of the site, a prime example is youtube whose certificate has a google.com CN but then utilises Subject Alternative Name (SAN)

        Hope that helps a bit…

        Dave

    • Jan F. 3:33 pm on February 26, 2012 Permalink | Reply

      Hi, I just implemented it. You forgot the dynamic certification. Thats what makes Squid this way work as an dynamic trusted man in the middle. Otherwise you’ve to one certificate for each domain. Here’s thewith compile options an the modified configline:

      ./configure –enable-ssl –enable-ssl-crtd

      https_port 127.0.0.1:3129 intercept ssl-bump
      generate-host-certificates=on dynamic_cert_mem_cache_size=4MB
      options=ALL cert=/etc/apache2/ssl/apache.pem

      and dont forget to make the log and cert dirs accessable for squid and certd!
      http://wiki.squid-cache.org/Features/DynamicSslCert

    • WildcArd 5:30 am on July 5, 2012 Permalink | Reply

      This is very helpful thanks!!!

      for squid 3.1, don’t you mean ‘intercept’ instead of ‘transparent’? also, I get ‘WARNING: transparent proxying not supported’

      • dvas0004 1:09 pm on July 5, 2012 Permalink | Reply

        Yes indeed… I just looked it up and Devs have changed the argument from transparent to intercept – well done in my opinion it better reflects the functionality

    • xXsamXx1991 4:56 pm on December 2, 2012 Permalink | Reply

      Great work thanks!
      I have used REDIRECT and that was my mistake.
      Now i use DNAT and it works🙂

    • Bilal Malik 8:18 am on November 7, 2014 Permalink | Reply

      Why can’t you use SSLBUMP on squid to handle HTTPS traffic over the internet and deliver content to the client in HTTP. That way you will be able to filter and cache the content. And there are no issues reagarding CLIENT end root certificate

    • putt1ck 6:10 pm on March 26, 2015 Permalink | Reply

      Just been testing an sslbump type solution but ran into an HSTS issue for Google: “Your connection is not private” – does HSTS require the MimicSSL feature before it can be filtered?

      • feiyultd 5:33 am on June 10, 2015 Permalink | Reply

        I met the same issue. I used the peek and slice with squid 3.5.However, for sites without HSTS , it gives a warning that the connection is not private so the user need to manually proceed. For sites with HSTS, the connection is simply refused.

        1)Is there any solution to remove that warning?
        2)Is there any solution to support sslbump with HSTS?

        • kiribool 9:58 am on June 15, 2015 Permalink

          I have a same problem with HSTS.

    • Clément Fiere 12:25 pm on March 27, 2016 Permalink | Reply

      Thank you for this article.

      Even though it’s a bit old, I want to let you know that the dashes you uses in this line (and possibly others) : “./configure –enable-icap-client –enable-ssl” are not the right one ( – ), which caused me some trouble for almost an hour before I realized it !!

c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: