Analyzing credit card transactions using machine learning techniques – 3

Introduction In a previous article, we explored how PCA can be used to plot credit card transactions into a 2D space, and we proceeded to visually analyse the results. In this article, we take this process one step further and use hierarchical clustering to automate parts of our analysis, making it even easier for our … Continue reading Analyzing credit card transactions using machine learning techniques – 3

Analyzing credit card transactions using machine learning techniques – 2

Principal Component Analysis - Introduction and Data Preperation Principal Component Analysis [PCA] is an unsupervised algorithm which reduces dimensionality and is widely used. A good visual explanation can be found here: http://setosa.io/ev/principal-component-analysis/ As mentioned in our previous article, Correspondence Analysis  works exclusively on categorical data. In contrast, PCA accepts only numerical data. This means our data … Continue reading Analyzing credit card transactions using machine learning techniques – 2

Analyzing credit card transactions using machine learning techniques

Introduction In this 3-part series we'll explore how three machine learning algorithms can help a hypothetical financial analyst explore a real data set of credit card transactions to quickly and easily infer relationships, anomalies and extract useful data. Data Set The data set we'll use in this hypothetical scenario is a real data set released … Continue reading Analyzing credit card transactions using machine learning techniques

Nugget Post: Reactive Functions to parse nested objects

Note this article assumes familiarity with the Observer Pattern / Reactive Programming as described here: http://reactivex.io/ Some APIs return complex nested JSON objects. For example, take this cleaned up sample response from ElasticSearch (which incidentally is used to build the "Data Table" visualization): https://gist.github.com/dvas0004/8f3427955a5bb21213c864d30094d072 Note the structure of the object. Within the top level "aggregations" object … Continue reading Nugget Post: Reactive Functions to parse nested objects

Lessons Learned: Winlogbeat & Forwarded Events – no event description

Scenario: Shipping Azure Cloud Logs to an Elasticsearch Cluster The Azure Log Service [AZLog ] audits events across your Azure Cloud infrastructure, and sends these to a central log collector. It leverage the Windows Event Forwarding subsystem to do this, meaning that the collector server will be able to view the AZLog alerts via the … Continue reading Lessons Learned: Winlogbeat & Forwarded Events – no event description

How to create a “heatmap” graph network visualization

What we're after @CyberSiftIO we've been going through an exercise of adding "confidence levels" to our visualizations. In other words, how confident is the CyberSift engine that an alert really is an anomaly/outlier? The above screenshot shows one of the ways we visualize the output from this exercise. Each blue node is an internal PC/Server, while … Continue reading How to create a “heatmap” graph network visualization

How we built the CyberSift Attack Map

Recently we launched a small site called the "CyberSift Attack Map" hosted at http://attack-map.cybersift.io. Any one involved in the InfoSec industry will be instantly familiar with the site:   It's basically a map of attacks which either trip some rule in a signature based IPS such as SNORT, or land in a honeypot. In this article we'll list … Continue reading How we built the CyberSift Attack Map