What is CIDR? Chasing the detection long tail

Cyber attacks are constantly evolving and can be divided into three stages: unknown, emerging, and known/variants. In the unknown stage, new attacks are launched and there is limited information available about them. These attacks are often undetected and come to light only after the perpetrators have achieved their objectives and the impacted organization commissions an incident response firm to analyze the attack.

In the emerging stage, information about the attack trickles in as researchers start to analyze and publish information about the attack. This information, though helpful, is usually incomplete and may not be sufficient to detect all future occurrences of the attack.

Once enough information is available, automated detection tools, such as EDR, NDR, and nextgen AV, incorporate it into their systems. However, attackers are constantly changing their methods to evade detection, leading to a game of cat and mouse between the attackers and defenders. Defenders must be constantly updating their detection methods to stay ahead of the attackers, making the distribution of threats a challenge.

The long tail of cyber attacks refers to the vast array of skills and capabilities required to match the adversary and detect all possible threats across the three stages. It is a complex and ever-evolving landscape that requires continuous learning and adaptation to stay ahead of cyber threats.

The problem of detection

The problem of detecting cyber threats can be broken down into three main areas: known threats and their variants, emerging threats, and unknown attacks that include custom targeted attacks.

Detecting known threats and their variants

Known threats are those that have been previously identified and documented. These include viruses, malware, and other types of attacks that have already been discovered and analyzed. However, these known threats can evolve and change over time, creating new variants that can evade detection. This is why it’s important to have a system in place that can detect known threats and their variants in real time.

Detecting emerging threats

Emerging threats are those that are just coming to light, and security tools may not have the ability to detect them yet. These threats are often new and unknown, making them harder to detect. As more organizations and individuals get affected, threat researchers can analyze and start producing intelligence about them that is helpful in detecting these threats. This is why it’s important to have a system in place that can detect emerging threats and provide early warning of potential attacks.

Detecting custom, targeted attacks

Custom, targeted attacks are those that are designed to target specific individuals or organizations. These attacks are often highly sophisticated and can evade detection for a long time. They may never come to the notice of the research community and can attack completely clandestinely. This is why it’s important to have a system in place that can detect custom, targeted attacks and provide early warning of potential attacks.

By understanding these three areas of detection, organizations can develop a comprehensive security strategy that includes a combination of automated detection, intelligence-based detection, and expert-led detection to protect against known, emerging, and custom targeted threats.

Automated detection

Detecting known threats and their variants is an important aspect of cybersecurity. These threats can be detected using automated and autonomous detection mechanisms such as antivirus (AV) and endpoint detection and response (EDR) tools. These tools use available signatures and observe file behaviors through a rules engine or machine learning engine to provide coverage for a wide variety of these threats. However, limited time and resource availability at the time of detection can impair the ability to detect even modestly evasive attacks.

To address this limitation, a backup system that can analyze files using multiple signals, including static analysis, behavior analysis, and anomaly analysis, is required. This system must be able to supply all the necessary computing and time resources to perform deep analysis and the analysis must be done continuously as new information pertaining to detecting threats becomes available. This approach allows for more comprehensive detection of known threats and their variants. The system must continuously analyze existing files as new intelligence becomes available in order to ensure that the system can detect new variants of known threats as they arise.

Stairwell Mal-eval is a feature of the Stairwell platform that allows customers to quickly identify malicious files in their environment and take corrective action. Mal-eval is a verdict that Stairwell assigns to each file it ingests. Backed by multiple AV engines, Stairwell curated threat feeds, static and dynamic signals, and our proprietary machine learning models, mal-index provides a reliable way to identify malware that may lie hidden on your devices.

Stairwell Mal-eval is designed to do a deep and continuous analysis of executable content. An out-of-band detection mechanism, Mal-eval has availability of time and resources to do a complete analysis of files before reaching a verdict. This is different from a typical endpoint-based malware detection system that has to work within the constraints of time (milliseconds) and user experience expectations (cannot use more than a certain percentage of the CPU time).

Intelligence-led detection

Intelligence-led detection is a method of detecting threats as new threat intelligence becomes available. This information can come from various sources such as research blogs, threat providers, or internally generated by a team. The biggest challenge with this approach is that while there are a lot of sources for threat intelligence, it can be difficult to effectively use this information for detection.

For example, if you receive a YARA rule or a domain name as an indicator of compromise from threat intelligence, it can be difficult to identify if you have matching content in your environment. Running a search across all your files looking for the contents of the YARA rule or the domain name can be prohibitively expensive, and running any type of search thousands of times a day is not feasible. Additionally, without a system for sifting through all matches, it can be difficult to identify truly malicious files from false positive matches.

To effectively operationalize threat intelligence, a system should meet the following criteria:

• It should be able to automatically match incoming threat intelligence across all the files from an organization at scale and out of band, potentially in a cloud environment to ensure there is no impact on individual devices.
• It should provide full context around all the matching files and tools to quickly narrow down the list of files to truly malicious files.
• It should not require highly skilled threat intelligence analysts to perform these tasks.

By meeting these criteria, organizations can effectively use threat intelligence to detect emerging threats and stay ahead of potential attacks.

Stairwell provides broad capabilities for using threat intelligence for detection and response activities. Customers can bring their own IoCs or work with the various threat reports provided by Stairwell. Stairwell can look for these IoC hits across all of the executable content in a customer’s environment and find any matching files.

Moreover, threat reports typically have a limited set of IoCs that researchers have been able to get their hands on. It is likely that more IoCs exist out there that may get discovered later. Stairwell uses technology to extend the list of IoCs by finding variants (like) IoCs that might be interesting within the context of the threat report. This superset of IoCs can then be used by Stairwell to identify any attacks more comprehensively across the organization.

Expert-led detection

Expert-led detection is a method of detecting truly novel or targeted attacks that are difficult to detect using known threat intelligence and tactics, techniques, and procedures (TTPs). Organizations responsible for national security or securing multinational financial institutions often hire large teams of threat researchers to hunt for attacks at scale. These researchers create hypotheses on how an attacker could potentially try to attack their organization, then gather data to analyze if the attackers are in fact, using something similar to attack them.

However, this approach has several challenges. The costs associated with these efforts can be high and may not justify the outcomes. Additionally, it can be prohibitively costly for a typical enterprise to replicate this approach. Based on today’s available technologies and scarce expertise, a significant improvement in tools, automation, and data infrastructure is required to make expert-led detection viable for organizations. Nonetheless, the need for this type of detection remains as it is essential to detect truly novel or targeted attacks.

Stairwell customers can massively scale their threat-hunting efforts using the data set and platform capabilities. Stairwell provides threat analysts the ability to express their hypotheses using a combination of YARA rules, IoCs, and object metadata, find all the matches in the wild and in their own environment, and go through our threat-hunting workflow to identify real attacks. Identification of these attacks helps create persistent detections to detect any future occurrence of these attacks.

CIDR platforms: Solving for a complex task

Detecting cyber threats is a complex task that requires a comprehensive security strategy. Adversaries are becoming increasingly sophisticated, and the tools and techniques used by attackers are advancing as well. Automated threat detection using commonly available tools is no longer enough to protect organizations from cyber attacks. A new thinking is required where detection can be performed across time windows, away from the adversary’s reach, and not constrained by the time and resources available for the point-in-time detection provided by the existing tooling.

A Continuous Intelligence, Detection, and Response (CIDR) platform can help organizations detect cyber threats and keep their organization secure by combining the best of automated detection, intelligence-based detection, and expert-led detection to detect across the long tail of cyber attacks. With a CIDR platform, organizations can have around-the-clock analysis of actual files stored indefinitely as evidence, providing the security team with the capabilities and visibility across all time horizons they need to confidently protect their organization from the risks of advanced attacks.