David Vassallo's Blog

If at first you don't succeed; call it version 1.0

Category Archives: Security

Using Let’s Encrypt manual mode

Let’s Encrypt is a service sponsored by web giants such as Facebook, Google Chrome, and Cisco (https://letsencrypt.org/). Let’s Encrypt has recently gone into public beta and is extremely easy to use. It has some modules already built in to integrate directly with popular webservers like Apache. In my case, I had two use cases where I needed to use manual mode – Installing the certificate on a Citrix Netscaler and on an NGINX reverse proxy. This necessitates using let’s encrypt in manual, standalone mode, which is still very easy to use. First, make sure you have the following requirements installed (I was using a linux machine):

  • Git client
  • Python 2.7

The process from here is very simple:

  1. Install letsencrypt-auto (the command line client for let’s encrypt) by using the git clone command as specified here: https://letsencrypt.readthedocs.org/en/latest/intro.html#about-the-let-s-encrypt-client
  2. Cd into the “letsencrypt” folder and run the following command:
    ./letsencrypt-auto certonly --manual

This will set off a text based wizard which will guide you through the process. It will first ask for the domain you wish to encrypt:

dvas0004@dvas0004-HP-ProBook-450-G2: ~-letsencrypt_349

Next, it will give instructions on how to write a simple script that will run a simple python-based webserver. The purpose of this is so that the let’s encrypt system can verify that you do really own the servers that respond to the domain just entered. As you can see from the code below, this simply creates a randomly generated filename under the url “http://your-domain.com/.well-known/acme-challenge”

dvas0004@dvas0004-HP-ProBook-450-G2: ~-letsencrypt_350

Pretty simple and elegant. You run the above command on your webserver – so obviously make sure your DNS hosts and NAT forwarding is setup correctly. You do not necessarily have to run the python webserver, if another webserver like IIS is already setup, you can manually copy/paste the contents into the file in the .well-known/acme-challenge folder.

PS, another tip that will avoid you thinking you went crazy, when importing the certificates into Netscaler, it wont recognize the private key without some OpenSSL magic. First off, to install the certificate, I used: Traffic Management > SSL > Certificates

Selection_352

And click the “install” button. From here you can upload the “fullcertchain.pem” and “privkey.pem” files, however, first please transform the privkey.pem file using the following:

All that said, it took 2 minutes to generate a free, 90 day SSL certificate… much easier, cheaper, and trustworthy than the normal SSL providers in my opinion… Kudos to Let’s Encrypt!!

ELK : exporting to CSV

Note: the following requires the “jq” json parser, available from: http://stedolan.github.io/jq/

1. Run the desired query through the Kibana WebUI

2. Expand the additional options pane by clicking on the arrow underneath the graph as indicated in the below screenshot:

blog_csv_export

3. Select “Request” and copy the request displayed:

blog_csv_export_2

4. Open a linux terminal and use the following command, pasting the above where indicated, and changing the template name as appropriate:

curl -XGET ‘http://192.168.12.68:9200/template_name-*/_search?pretty’ -d ‘—–PASTE HERE———‘ | jq -r ‘.hits.hits[]._source | del(.tags) | [to_entries[]] | map(.value) | @csv’ > /tmp/output.csv

Note: “size” in the copy/pasted text needs to be modified according to how many records are to be exported.

The csv output will be stored in the /tmp/output.csv file, which can be downloaded via SFTP and manipulated as necessary

Note: Since ELK deals with unstructured data, it may be the case that the csv file does not have the same number of columns for each entry, especially if different types of records are queried. This is to be expected and is by design. In order to get a list of column names for each row, the following command can be run:

curl -XGET ‘http://192.168.12.68:9200/template_name-*/_search?pretty’ -d ‘—–PASTE HERE———‘ | jq -r ‘.hits.hits[]._source | del(.tags) | [to_entries[]] | map(.key) | @csv’ > /tmp/output.csv

The above command will generate the /tmp/output.csv file but will contain column headings for each row rather than actual data.

Follow

Get every new post delivered to your Inbox.

Join 261 other followers