About these ads

David Vassallo's Blog

If at first you don't succeed; call it version 1.0

Category Archives: Open Source

Getting started with Neo4J and security data analysis

During a recent study module for a MSc I am undertaking we discussed the importance of continuous monitoring of data sources as part of a sound security defensive strategy. This lead me down a very interesting path, eventually culminating in my discovery of an entire subset in security discipline many refer to as “Security visualization”. There are plenty of examples on the internet as regards this discipline, one of the best links I found is “secviz”, located here:

http://secviz.org/

My focus during this was actually trying to use graph databases to visualize security data, which in it’s raw form is simply CSV logs from our corporate firewall. I chose to explore graph databases via Neo4J, an extremely popular, open source, graph database. This article is intended as a quick reference or high level overview to any who’d like to get started in this field, and also as a reminder to my future self as to what I did so I can pick it up again sometime soon…

Getting Neo4J installed on ubuntu was child’s play. Copy/pasting the debian install script provided from the Neo4J site was all it took. A quick overview of cypher (Neo4J’s declarative language) is also easy for those coming from a SQL background. What did require thorough reading was the procedure on how to import CSVs into Neo4J, found here:

http://docs.neo4j.org/chunked/stable/cypherdoc-importing-csv-files-with-cypher.html

the firewall logs I used in this study came from a Palo Alto firewall, specifically the threat logs. These logs give the source (attacker) and destination (target) IP address of an attack, as well as the name of the attack. So first step is to open the raw CSV file, and whittle it down to the important columns, ending up with the following (ip addresses changed to protect the innocent):

 

Source Destination Threat
192.168.7.6 192.168.1.2 Microsoft RPC Endpoint Mapper(30845)
192.168.9.5 192.168.9.4 SSH2 Login Attempt(31914)
95.169.160.24 1.2.3.4 DNS ANY Request(34842)
202.43.161.114 1.2.3.5 Win32.Conficker.C p2p(12544)
195.190.146.40 5.4.7.8 Suspicious or malformed HTTP Referer field(35554)
192.68.7.106 192.168.21.2 Microsoft Windows SMB Negotiate Request(35364)
192.16.2.200 92.168.123.3 Microsoft RPC Endpoint Mapper(30845)

– Warning –

The logs can be quite voluminous, and I didn’t pay much attention to the various optimizations that can and should be done (like indexing), so some queries can take a significant amount of time to execute.

First step is to import that csv file into Neo4J. The cypher code I used was:
CREATE INDEX ON :IP(address)
USING PERIODIC COMMIT
LOAD CSV WITH HEADERS FROM "file:///home/davidvassallo/Downloads/log.csv" AS csvLine
MERGE (source:IP { address: csvLine.Source })
MERGE (destination:IP { address: csvLine.Destination })
MERGE (attack:Threat { attack: csvLine.Threat })
CREATE (source)-[:USED]->(attack)-[:AGAINST]->(destination)

In the first part we load the appropriate CSV file (note the triple slash in the beginning for Linux systems), and loop through each line of the csv file, storing it in the variable csvLine. Breaking down a MERGE command, we have:

blog_neo4j_1

The first “MERGE” line basically instruct Neo4J to create nodes of type IP, with an attribute/property called address from the “Source” column of the CSV file. If the node already exists, then don’t create a new one (hence the use of MERGE instead of CREATE). We do the same for the Destination column in the next line, and finally we create nodes of type “Threat” out of the threat column in the CSV line. Last but not least, the CREATE lines creates the edges (or relationships) between the source, destination, and attack. There are multiple ways to do this, but in my head it came down to:

“A source used an attack againstdestination

One notices that the cypher instruction very closely resembles the english sentence I wrote. In the cypher instruction, nodes are enclosed in parenthesis, while relationships are enclosed in square brackets. With that out of the way, we can get to visualizing our data. After pointing our browser to http://localhost:7474, we load up all nodes of type “threat”:

blog_neo4j_2

Click to enlarge

Note how the generating cypher command is also give to the user (number 2 above). Even at this stage, without running any queries, we can get some really useful information, by double clicking on the threat nodes. To give a concrete example, it’s worth while analyzing the following picture:

click to enlarge

Click to enlarge

You see two attackers (on the right hand side) which launched a variety of attacks designed to try to break into RDP. I checked the firewall and it really does have RDP open for some reason (a definite plus to using security visualization… auditing your own traffic). You see that one attacker (80.82.65.60) is probably a poor schmuck who got infected by a worm. Morto is a worm that propagates using RDP. The other attacker on the top of the diagram is a more organized attacker…. First he did reconnaissance and found an open RDP port (that’s the “Ncrack RDP” scan) , then tested if the scan was actually correct by trying to use the actual RDP client (that’s the microsoft rdp initial attempt), then, finding out he couldnt guess the password, he launched a full blown brute force attempt against the server.

I haven’t even begun to scratch at the surface of Neo4J…. there’s plenty more to discover, but I’ll leave you with some very useful queries I constructed while on the learning journey:

--- Find sources and targets of SSH2 login attempts:
MATCH (a)-[r]-(b) WHERE a.attack =~ "SSH2.*" OR b.attack =~ "SSH2.*" RETURN a,b LIMIT 25
--- Node Traversal (who attacked this node id)
--- the node ID can be determined by clicking once on the node
--- it will appear in parenthesis in the black box that appears on the right
START n=node(233)
MATCH (x)-[r]->(n)
RETURN n,x
-- who used attack id 78, return only distinct attackers
START n=node(78)
MATCH (a)-[r]->(n)
RETURN DISTINCT(a),n
LIMIT 500

References:

About these ads

Android Linux / Raspberry Pi Bluetooth communication

This post explores my attempt to get bidirectional bluetooth communication going between an android device and a linux computer (in this case a Raspberry Pi running Raspbian). This post is part of the documentation of knowledge acquired while building an “AquaPi” – a raspberry pi that will act as a control center for an aquarium. Here is a video showing a working example:


https://www.youtube.com/watch?v=hvykOO73LbM


Hardware:

Software:

  • On the raspberry pi: setting up and using pybluez

Note: these instructions have also been confirmed to work on an ubuntu workstation with in-built bluetooth support.

First, plugin the bluetooth dongle and fire up the raspberry pi. Next, install the pre-requisites to get bluetooth support on the raspberry pi by running:

sudo apt-get install bluez python-bluez

There are plenty of guides in the internet on how to get bluetooth working, but the only method that worked consistently for me is the following:

1. Disable bluetooth pnat support as there seems to be a bug which stops proper operation with pnat enabled. Full details can be found here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=690749

A workaround is to add the following to /etc/bluetooth/main.conf:
   DisablePlugins = pnat

Not following the above will give errors similar to:

 (111, 'Connection refused')

Incidentally, the “Steps to Reproduce” section of the bug link are a very good way of testing your bluetooth stack and making sure you have the proper groundwork in place

2. Make note of the following two commands which will be useful when pairing your raspberry pi to the android phone:

sudo hciconfig hci0 piscan [make your device discover-able]
sudo hciconfig hci0 name 'Device Name' [change your device name to something else you fancy]

3. Download the following python script someplace on your raspberry pi. The script is a modified version of the “simple rfcomm server” script found under /usr/share/doc/python-bluez/examples/simple/rfcomm-server.py

Note that as previously stated, this script stems from a build to control an aquarium. So you may need to modify the if/elif/else statements beginning at line 60 to do something more meaningful in your particular case. Simply run the server via:

sudo python /path/to/script.py
  • On android: testing bluetooth and writing an app

Before proceeding, you must make sure to pair your android device to your raspberry pi. To do this, ensure that your raspberry pi is discover able (see step 2 above), and search for the device in the bluetooth settings in your android. After successful pairing, you should be able to continue.

1. Install “BlueTerm” from the playstore (https://play.google.com/store/apps/details?id=es.pymasde.blueterm). This will allow you to test your bluetooth setup up this stage. Your raspberry pi should accept connections and even respond to typed commands

2. Write an android app to establish a bluetooth rfcomm channel and send commands to the raspberry pi as well as read information. Following is a sample main activity from the mobile app written for the video demo:

Notes on the android script:

  • Line 33: note the UUID. This must match the UUID being advertised by the rfcomm server – which is the python script (line 46)
  • Lines 31-51: this is a function which will send information to the raspberry pi (hence the android here is acting as a “client”) . If not already connected, the function creates an RFCOMM socket, gets a writeable stream, and pushes the message through the socket. The message to be sent is passed as an argument to the function
  • Lines 67-141: this is the runnable (thread) which is used to send and receive information over the bluetooth RFCOMM channel. We need to run this in a seperate thread so as to avoid ANR (application not responding) notifications, and hence leaving the GUI responsive. The thread first sends a message using the sendBtMsg function (line 77), and then opens an input stream and waits for response data. The app waits for response data by creating a buffer (of 1024 bytes in this case – line 95). When data is received, the app checks if the delimiter character is present (in this case the delimiter character is the exclamation mark [!] – ascii character 33 – line 27). We implement this pattern so as to be sure that we received the entire intended data stream before displaying the information to the end user. Obviously, you may want to change my choice of using ! as a delimiter to something more suited to your needs.
  • Line 113: here we set a text view to the received data from the raspberry Pi
  • Line 129: if the data is received and displayed to the user, close the bluetooth connection and break out of the listener loop to finish the thread
  • Lines 144-179: here we are simply attaching onClickListeners to buttons on the GUI to send commands to the raspberry pi. These differ only in the string which is sent over the bluetooth RFCOMM connection to the raspberry pi
  • Lines 184-185: here we check if bluetooth is enabled. If not, we fire off an intent to ask the user to allow bluetooth use
  • Lines 188-199: go through the list of paired devices, and find the raspberry pi. Note, you may need to change the name in the code if you didnt use the standard (line 193). Once the raspberry pi is found, assign it as the device to be used (line 196)
Follow

Get every new post delivered to your Inbox.

Join 155 other followers