Elasticsearch REST API: JEST upsert

I've already written about tips and tricks when using the Elasticsearch Java API. The Elasticsearch REST API has been going from strength to strength, and it seems that going forward the Elasticsearch team will focus more on the REST API than the native JAVA client. At the time of writing however, the official java REST library … Continue reading Elasticsearch REST API: JEST upsert

Advertisements

Nugget Post: Insert an alert / rule name in emails sent from ElastAlert

ElastAlert is a fantastic python module that makes it easy to setup near - realtime alerts and responses to entries of interest. A quick piece of information  which I didnt find clearly documented was how to insert an alert name into a custom subject line sending email alerts.  A typical custom email alert rule file would include … Continue reading Nugget Post: Insert an alert / rule name in emails sent from ElastAlert

Lessons learned: Gephi not starting up

Scenario: When using Ubuntu Trusty (14.04.3LTS) Gephi 0.82 initializes but gets stuck on "Loading Cached Objects" Solution: The default Gephi startup bash script does not correctly initialize the JAVA path (it uses $jdkhome) The solution is simply to add the following to the beginning of the startup script (~/gephi/bin/gephi) export JAVA_HOME=/usr/lib/jvm/default-java jdkhome=$JAVA_HOME The full script is … Continue reading Lessons learned: Gephi not starting up

ELK : exporting to CSV

Note: the following requires the "jq" json parser, available from: http://stedolan.github.io/jq/ 1. Run the desired query through the Kibana WebUI 2. Expand the additional options pane by clicking on the arrow underneath the graph as indicated in the below screenshot: 3. Select “Request” and copy the request displayed: 4. Open a linux terminal and use the … Continue reading ELK : exporting to CSV

Building a Logging Forensics Platform using ELK (Elasticsearch, Logstash, Kibana)

During a recent project we were required to build a "Logging Forensics Platform", which is in essence a logging platform that can consume data from a variety of sources such as windows event logs, syslog, flat files and databases. The platform would then be used for queries during forensic investigations and to help follow up … Continue reading Building a Logging Forensics Platform using ELK (Elasticsearch, Logstash, Kibana)

AlienVault: Monitoring individual sensor Events Per Second [EPS]

In a distributed AlienVault environment, it is important to be able to monitor individual sensor's output. In our case, the requirements was to: Monitor each sensor's generated events over a configurable interval If the number of generated events of a sensor goes below a configured threshold, then notify the user via email There are several … Continue reading AlienVault: Monitoring individual sensor Events Per Second [EPS]

OSSEC event loss troubleshooting

There is a general consensus that OSSEC will lose events in the event that the main OSSEC server goes offline for whatever reason ( [1] , [2] ) - be it the service is stopped, a network disconnection, or anything in between. However, there doesn't seem to be much information on when exactly even loss can occur, for … Continue reading OSSEC event loss troubleshooting