Elasticsearch & Java: Tips for faster re-indexing

Notes based on some feedback: Elasticsearch seem to be pushing the REST client rather than using the native Java client... to future proof your code you may be better off going down this route. Why not just use the Re-Index API? Although it's still considered experimental, this may be a good option if you dont have … Continue reading Elasticsearch & Java: Tips for faster re-indexing

Advertisements

Elasticsearch & Python: Tips for faster re-indexing

Some valuable lessons learned while going through an elasticsearch re-indexing exercise. (For the uninitiated, re-indexing data basically means getting large volumes of documents from elasticsearch, enriching or changing the data within each document, and then sending these back). The below assume python and the use of the python elasticsearch client (https://elasticsearch-py.readthedocs.io/en/master/). 1. When retrieving data, use the … Continue reading Elasticsearch & Python: Tips for faster re-indexing

Signing GMail Messages with the Estonian eID PKI Card (Part 2)

In a previous blog post we explored how to sign messages using the Estonian eID card. In this video, we demonstrate how a receiver who got a signed email message would be able to verify that the email really did come from the advertised sender. https://youtu.be/Nx-jRPIAyEM I have uploaded the revised code to github, please … Continue reading Signing GMail Messages with the Estonian eID PKI Card (Part 2)

Nugget Post: Pouchdb-find regex example

Here's an example of using the excellent pouchdb-find plugin's (https://github.com/nolanlawson/pouchdb-find) regex features. I couldn't really find a good example in the documentation, but looking at the source code we come up with something like this: https://gist.github.com/dvas0004/a67f1093bc323d7556ed Note that the $regex command cannot be run on an indexed field, so this will be a slow query … Continue reading Nugget Post: Pouchdb-find regex example

Lessons learned: Gephi not starting up

Scenario: When using Ubuntu Trusty (14.04.3LTS) Gephi 0.82 initializes but gets stuck on "Loading Cached Objects" Solution: The default Gephi startup bash script does not correctly initialize the JAVA path (it uses $jdkhome) The solution is simply to add the following to the beginning of the startup script (~/gephi/bin/gephi) export JAVA_HOME=/usr/lib/jvm/default-java jdkhome=$JAVA_HOME The full script is … Continue reading Lessons learned: Gephi not starting up

AlienVault ELK Integration

In the last couple of blog posts[1][2] we've been exploring how to use the ELK stack as a forensic logging platform. We also had a couple of posts on deploying some AlienVault features [3][4]. In this post we explore a quick and easy way to integrate between the two systems. Apart from the flexible querying … Continue reading AlienVault ELK Integration

Building a Logging Forensics Platform using ELK (Elasticsearch, Logstash, Kibana)

During a recent project we were required to build a "Logging Forensics Platform", which is in essence a logging platform that can consume data from a variety of sources such as windows event logs, syslog, flat files and databases. The platform would then be used for queries during forensic investigations and to help follow up … Continue reading Building a Logging Forensics Platform using ELK (Elasticsearch, Logstash, Kibana)