Network traffic analysis for IR: Threat intelligence collection and analysis

Introduction

While any security professional can call themselves an analyst, the full scale and scope of the cyber-threat intelligence analyst is often underestimated. Especially when paired with other cyber-incident response and detection tools and programs, the practice of cyber-threat intelligence can help organizations to track, identify and deal with increasingly sophisticated threats against their business. While malware is the tool, the real threat is a human one and a cyber-threat analyst can use network traffic analysis and other data sources to establish methods to counter human threats and bolster their organization’s defenses.

On a day-to-day basis, cyber-threat intelligence can help network defenders to understand failed and successful methods employed by adversaries, supply that and other information to security operations professionals, and provide actionable data to incident response teams to assist in scoping, mapping and responding to intrusions. In other words, cyber-threat intelligence based on both network traffic and other data sources can help organizations to prioritize their risk and inform all aspects of their security programs.

Learn Network Traffic Analysis for Incident Response

Get hands-on experience with nine courses covering how to collect, identify, extract and analyze network traffic.

Start Learning

So just what does cyber-threat intelligence entail and what is its role in incident response? This article will explore these topics while also laying a foundation for the practice itself.

Cyber-threat intelligence overview

Cyber-threat intelligence can be both a complex and a simple concept. At its core, cyber-threat intelligence means the analysis of the information relating to cyberthreats, both real and potential. This information can include the source of the threat, the type of technical threat, its delivery method, the damage it can or has caused, and many other pieces. 

Because it is impossible for any one organization to understand the full scale of modern cyberactors, cyber-threat intelligence analysts combine their own organizational data — ranging from network traffic to risk and vulnerability reports — with data shared by governments, private businesses and data clearinghouses to further supplement their analysis and security programs.

As a result, the intelligence created can be used by an organization for tactical defense, to identify technical signatures or indicators of specific kinds of malware, operational data that may lead to preventing or assessing a threat or making strategic security changes in order to make comprehensive changes in an organization’s security posture. Understandably, cyber-threat intelligence and analysis is then a cyclical process, which has been presented in a threat intelligence life cycle

Cyber-threat intelligence life cycle

One of the best ways to understand cyber-threat intelligence is by thinking about it as a cycle comprising analytic techniques that take into account the work of decades of experience from government, private sector and military organizations. The individual phases of the life cycle are direction, collection, processing, analysis, dissemination and feedback. 

Each of these parts bring together a variety of skill sets, roles, responsibilities and even separate underlying methodologies (e.g., risk analysis or vulnerability management), but this article will highlight the role of network traffic collection and analysis.

Direction

A logical place to enter the cyber-threat intelligence life cycle is the direction phase, where goals for the threat intelligence program are set and requirements developed. Specifically, organizations should be identifying and prioritizing the assets and processes to be protected, the potential impacts to their operations if they were interrupted or exploited and the types of information that need to be collected in order to monitor security and evaluate performance. This is also the time to identify the roles that need to be established to make the program work and the tools and resources that need to be in place.

Once the overall vision for your organization’s threat intelligence collection is set, requirements can be set and metrics for collection can be identified so the security tools in place can begin to monitor the assets that are identified. With these and the priorities established, analysts can focus their research and analysis on potential vulnerabilities and vectors adversaries may exploit to attack their organization.

Collection

Once the direction of the threat intelligence program is established, organizations can then begin the process of collecting the information and data aligned to the intelligence requirements and metrics. The information can be collected through a variety of methods, including: 

• External data sources, such as data clearinghouses, reports or industry groups
• Data feeds from cybersecurity vendors
• Data from in place security tools, network devices and system logs
• Analysis from news websites and forums

Having the requirements in mind will help to refine this otherwise overwhelming amount of data and it can include both raw data from system metadata through to finished reports and surveys from experts. 

Processing

This phase about the cyber-threat intelligence collection life cycle is about consolidating, simplifying and formatting the data generated and collected by the organization to make it consumable and actionable. Because network data can often include terabytes of network logs from various network devices, feedback from security tools (e.g., alerts, blocked intrusion attempts) and performance data (network traffic flows over time, IP addresses) as well as information from internal and external professionals (help desk reports, industry group findings, regulatory alerts), all of the information needs to be consolidated and vetted for reliability. 

Analysis

The analysis phase turns all of the collected data into intelligence that can be used to make decisions. These can include tactical decisions such as beginning an investigation into anomalies or potential threats or steps needed to immediately block an on-going attack. The analysis phase can also be used to help with more strategic decisions like how to improve security controls, additional investments that need to be made in security tools,  

Specific to network traffic analysis, analysts can begin with either a bottom-up or a top-down approach. For the former, analysts begin their research by starting with an event of interest — an alert from a firewall or a piece of data from network flow — and then follow it as the traffic moves to other network devices. For example, an analyst can follow a website outage back to IDS alerts or server logs to find possible indications of malicious activity.

Alternatively, an analyst can conduct their work from the top down, which entails tracking broader trends in network traffic that fall out of the normal ranges. For example, if network traffic suddenly deviates from a usual 9:00 a.m. to 5:00 p.m. trends, analysts could be alerted to anomalies that drive them to check network logs, firewalls and IDS systems for further evidence of suspicious activity.

Dissemination

This phase focuses on communicating the cyber-threat intelligence output to the required internal and external stakeholders. This can include management teams, fellow internal security professionals, external cybersecurity vendors and groups, government regulators, employees and even customers. For each of these groups, organizations need to understand the best methods to share the data, the format, any context that is needed to be presented with the data and the next steps and impacts that may be felt from the intelligence. 

Feedback

The final phase of the cyber-threat intelligence collection life cycle is used to update and refine future collection requirements and metrics. This can impact the types of data collected, the tools and roles in place to collect and analyze that data, where that data is collected from and where that data is shared. This phase is ultimately about making regular adjustments to an organization’s intelligence collection approach and tools. 

Understanding and responding to a cyberthreat

Another way to understand the evolution of a cyber-attack is through the cyber-kill chain, developed by Lockheed Martin. This approach or methodology to understanding how an adversary identifies and harms an organization has for years helped information security professionals establish strong access controls and countermeasures. The cyber-kill chain comprises seven stages or phases:

1. Reconnaissance
2. Weaponization
3. Delivery
4. Exploitation
5. Installation
6. Command & control (C2)
7. Actions on objectives

Although it was originally developed following a military model to identify, prepare, engage and destroy a target, the cyber-kill chain has since been adopted by security professionals and threat intelligence analysts to better anticipate and recognize threats. Specifically, threat intelligence analysts can use the structure of the cyber-kill chain to gather actionable threat intelligence to either improve organizational security or respond in the wake of an attack. Outputs from this effort can include lists of potential threat actors, identification of malware or malicious IP addresses, indicators of compromise (IOCs), and tactics and techniques used by potential adversaries. 

Let’s explore how threat intelligence can be useful at each of the seven stages of the cyber-kill chain.

1. Reconnaissance

Before any attack can begin, a hacker must first select their potential victim that aligns to their motivations and resources. Whatever their reasons, threat intelligence already plays a role in this first stage, using external resources to attempt to identify potential attacks and close up vulnerabilities taken advantage of in other attack scenarios. 

Furthermore, threat intelligence can be used to predict which kinds of attackers — ranging from disgruntled employees to competitors and hacktivists — are likely to target an organization. Combined, this can help an organization to prioritize their resources and align their security devices to meet their potential threats.

2. Weaponization

Once an attacker identifies their victim, research about their network and security practices can begin. When doing this work, attackers can use passive network reconnaissance tools and techniques, elementary probes of network defenses, and even attempts to brute-force passwords or exploitation of known vulnerabilities to test for security controls. It is during this phase that the exploit and the vulnerability is selected by the attacker.

Attackers can also use the dark web and forums to learn more about a target’s behavior, structure or network to determine the best time to strike. In both of these cases, threat intelligence can be used to investigate abnormal behavior in their network as well as read what hacker communities are saying about the equipment they have in place so they can be proactively warned to a more robust attack.

3. Delivery

After the target is locked in, the exploit chosen and the vector research completed, the target organization and their threat intelligence analysts have little they can do other than trust in their security tools and procedures. This is why analysts focus so much on how they can help their organizations be as prepared as possible based on current attacker tactics, techniques and procedures (TTPs). With this knowledge, analysts can help to close down known vulnerabilities, prepare their staff for the social engineering attacks and monitor for expected attacks.

The installation phase, where the malware is introduced to the target network, is also a key place for analysts to reengineer an attack. At this phase, intelligence analysts can understand the conditions that were in place during the initial exploitation, the type of malware used, and impact that tightening or changing certain security protocols would have had on the attack.

4 & 5. Exploitation and installation

This part of the cyber-kill chain sees the attacker gain a foothold in a target network, a key milestone for the hacker as well as your network defense and intelligence program. This is because an attacker is exposing their IP addresses, tools, domains, hashes and behavior, all associated with the initial malicious activity and hopefully recorded by the target’s IDS and other network devices. With this information, cyberintelligence analysts can either be alerted to abnormal activity and then work to contain and end the attack before it spreads or, at worst, rewind the attack back to this point so technical controls can be tightened to prevent a future attack. 

6. Command-and-control (C2)

As malware enters your network, it comes down to its command-and-control functionality to give hackers the ability to move from just having a foothold to escalating their potential for damage. 

These communications between the implant and the attacker must utilize the victim’s network devices, meaning IP addresses, ports and other data can continue to be collected and analyzed for future defensive purposes. Even better, if network analysts were able to identify information about the attacker’s C2 servers and any co-opted devices, this malicious traffic can be watched, contained and eliminated. 

7. Actions on objectives

Once a hacker has reached this stage, it is going to be very difficult to prevent damage from occurring. Depending on the motivation of the actor, this could be anything from stealing customer data to defacing a website or taking money or financial information. But from a threat intelligence perspective, not all is lost.

A postmortem intelligence analysis can be used to help identify future IOCs and behaviors that refine security procedures, contain persistent threats and allow organizations to attempt to proactively alert stakeholders and customers about potential impacts. Analysts can also share signatures, network traffic logs and other data with external organizations so they can be better prepared for an attack against their assets. 

Conclusion: Bringing it all together

While having comprehensive, around-the-clock situational awareness of your network has been made dramatically easier with the help of monitoring devices and tools, it takes threat intelligence analysis to know how to make sense of all the data being generated. 

With the right approach, analysts can use network traffic and devices to help detect malicious activity and attempt to catch cyberattacks as early as the reconnaissance phase or, at worst, perform the needed forensics to reverse-engineer an attack to contain the damage and share it with peers. Ultimately, the threat intelligence can continuously provide value to any organization at any stage of the cyber-kill chain, regardless of the maturity of their security program.

Learn Network Traffic Analysis for Incident Response

Get hands-on experience with nine courses covering how to collect, identify, extract and analyze network traffic.

Start Learning

Sources

1. The National Cyber Security Centre, ncsc.gov.uk
2. The Cyber Kill Chain, Lockheed Martin
3. Traffic Analysis for Network Security: Two Approaches for Going Beyond Network Flow Data, Software Engineering Institute