FloCon 2018

Graph analysis can capture relationships between IPs and can be used to identify and rank anomalous IPs from NetFlow data. If NetFlow data is collected at the edge of the network, as often is the case, internal and external roles of IPs and relationships between them are either unknown or incomplete. Inferred relationships between the external IPs can add context that can provide insights of this coordination between the nodes.

This paper will focus on scalable and flexible techniques for applying graph analytics on various types of logs that have bipartite structure, as well as methodologies to further narrow returned results to anomalous/outlier cases that may be indicative of a cyber security event. Examples of this type of data include internal/external IP addresses, client-server data, and/or user-service data. Operational use-cases that leverage these techniques with bro logs conn view, SMTP view, RDP view, and Kerberos view will be presented. A specific use case with internal/external IP flow data is the ability to identify IPs and infer their roles that are involved in Distributed Denial of Service (DDOS) attack where a large number of nodes are synchronized to collectively send small packets to a target service. These nodes often send small enough packets to make it past firewall and intrusion detection system barriers to disrupt a service that is provided by the network. Project Chanology conducted by Anonymous [1], Project Rivolta conducted by Mafiaboy [2], and the attack on the website of the Georgian President by Russia in 2009 [3] are examples of famous DDOS attacks on enterprise, high-profile networks.

The specific algorithms presented that infer relationships and highlight anomalous IPs or users, henceforth referred to as nodes, include unipartite graph projections, community detection, page rank, and other first order graph features. The nearest neighbor algorithm is used to identify the most anomalous nodes in a particular community or across an entire network. A novel framework for building directional graphs from unipartite graph projections first infers the relationships between the nodes. Community detection is then used to identify groups of nodes that are more similar to each other than the rest of the network. Finally, first order graph features such as page rank, projected degree, and community size are fed to the nearest neighbor algorithm to identify anomalous nodes across the network. Post-processing methods on the set of anomalous nodes discovered in this manner to develop explanations of the anomalies will also be presented.

Attendees Will Learn: Automated methods to identify anomalies in cyber networks with data collected at the edge of a network (or other bipartite network)