This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more anonymous collaborator.
In this blog (#4 in the series), we will start to talk about the elephant in the room: how intel becomes detections (and, no, it is not trivial)
Detection Engineering is Painful — and It Shouldn’t Be (Part 1)
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more…medium.comDetection Engineering and SOC Scalability Challenges (Part 2)
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more…medium.comBuild for Detection Engineering, and Alerting Will Improve (Part 3)
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more…medium.comDetection Engineers are often picky with the intelligence they receive … and for good reasons. Incomplete, too high-level or overly specific data leads to long analysis time, bias and ultimately inconsistent detection quality and detection coverage gaps. On the other hand, the intel team may process so much intelligence data that it can be a real challenge to understand what the detection team is even expecting. So, in many cases there is an “intelligence — detection gap”, of sorts. This is our topic in this part of the blog series.
To start, in the Detection Engineering Maturity Matrix by Kyle Bailey, intel is referenced on the upper maturity level of the “Threat Operations” Category — and historically this has held true. In some cases, Detection Engineering (DE) teams do not get valuable threat intel at all, and have to do their own threat research as a result (if this is not like that in your organization, congratulations!). But only working in tandem allows us to reach best-in-class results for detection!
Today, let’s explore what’s wrong (for detection, specifically!) with much intelligence data provided, what’s good intel (for DE purposes), and how we can work better together (but do look at the previous posts Part 1, Part 2, Part 3 even if this one is most interesting for you…)
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more anonymous collaborator.
In this blog (#4 in the series), we will start to talk about the elephant in the room: how intel becomes detections (and, no, it is not trivial)
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more…medium.com
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more…medium.com
This blog series was written jointly with Amine Besson, Principal Cyber Engineer, Behemoth CyberDefence and one more…medium.com
Detection Engineers are often picky with the intelligence they receive … and for good reasons. Incomplete, too high-level or overly specific data leads to long analysis time, bias and ultimately inconsistent detection quality and detection coverage gaps. On the other hand, the intel team may process so much intelligence data that it can be a real challenge to understand what the detection team is even expecting. So, in many cases there is an “intelligence — detection gap”, of sorts. This is our topic in this part of the blog series.
To start, in the Detection Engineering Maturity Matrix by Kyle Bailey, intel is referenced on the upper maturity level of the “Threat Operations” Category — and historically this has held true. In some cases, Detection Engineering (DE) teams do not get valuable threat intel at all, and have to do their own threat research as a result (if this is not like that in your organization, congratulations!). But only working in tandem allows us to reach best-in-class results for detection!
Today, let’s explore what’s wrong (for detection, specifically!) with much intelligence data provided, what’s good intel (for DE purposes), and how we can work better together (but do look at the previous posts Part 1, Part 2, Part 3 even if this one is most interesting for you…)
Intelligence is Often Lacking in TI
A frequent observation in CTI and DE team relationships is that DE is asking CTI what they should focus on, what threat they should address first, and CTI cannot deliver precise answers beyond a bundle of ATT&CK techniques (which are largely the same for the large majority of threat actors and organizations). As we’ve seen in our previous blog post, ATT&CK is not sufficient for more technical DE needs and for detailed planning of the priority detection engineering activities.
In some lucky cases, useful intel is often forwarded directly without processing. Similarly, if the IR team is effective and mature, they will send useful data back to CTI and DE teams. In this case, naturally, it’ll always be after the fact (i.e. after an incident), where the DE target is to develop ahead of an incident.
These observations beg the question — what is good intel for detection engineering purposes? For the most part, we can derive some common bad practices:
- 💅 Just ATT&CK techniques: just giving a set of technique ID to a DE is not valuable intel (“Attackers used T1548“ — “Great! What do we do with it!”) : it may take weeks in best cases to properly develop detections for the scope of a single technique on one platform, and technique-level detections are very hard to reach without resorting to “false positive”-prone approaches. Realistically, most of the time, only well-known procedures will be covered.
- 🧐 Generalist “intel”: everyone knows brute force is a bad thing — but what did the actor do exactly that we could detect? DE teams need technical threat descriptions. There is a large set of ways to brute force something and an ever larger set of ways to detect the resulting activities.
- 💔 Hard to parse with detection in mind: Nicely documented CTI reports have a bad tendency of containing the entire attack path and a lot of other exciting yet irrelevant for detection details. For DE teams, that now means breaking down the report into a dozen singular threat elements that each needs to be evaluated and processed into detection.
- 🍂 Lack of variety: Yes, Email and Windows are exploited left and right. But we also have Cloud, SaaS, Remote Workers and other unique threat vectors to look into. If your company uses multiple platforms, but your intel team feeds your DE team with Windows stuff only, attackers will get in and you won’t know it.
- ⚖️ No value add, just copy from the feed: CTI ultimate purpose in an organization is not to compile external content (some of questionable provenance), but to analyze threats through the prism of your business operations and IT systems in place. “Detect threats that matter” is easy to say, but really hard to do!
- 🫠 Doesn’t understand the threat: An intel analyst should deeply understand the threat they communicate to DE to accelerate further processing, and ensure enough information is delivered. DE is a process, where any threat can be processed into detection, but only as long as we understand the issue at hand. Ideally, bi-directional CTI-DE comms are established…
- 🐢 Sloooow: DE aims at moving at the speed of intelligence, thus it needs intel which is closely following the latest developments. Intel done for research can be slow, but intel done for detection cannot be.
A frequent observation in CTI and DE team relationships is that DE is asking CTI what they should focus on, what threat they should address first, and CTI cannot deliver precise answers beyond a bundle of ATT&CK techniques (which are largely the same for the large majority of threat actors and organizations). As we’ve seen in our previous blog post, ATT&CK is not sufficient for more technical DE needs and for detailed planning of the priority detection engineering activities.
In some lucky cases, useful intel is often forwarded directly without processing. Similarly, if the IR team is effective and mature, they will send useful data back to CTI and DE teams. In this case, naturally, it’ll always be after the fact (i.e. after an incident), where the DE target is to develop ahead of an incident.
These observations beg the question — what is good intel for detection engineering purposes? For the most part, we can derive some common bad practices:
- 💅 Just ATT&CK techniques: just giving a set of technique ID to a DE is not valuable intel (“Attackers used T1548“ — “Great! What do we do with it!”) : it may take weeks in best cases to properly develop detections for the scope of a single technique on one platform, and technique-level detections are very hard to reach without resorting to “false positive”-prone approaches. Realistically, most of the time, only well-known procedures will be covered.
- 🧐 Generalist “intel”: everyone knows brute force is a bad thing — but what did the actor do exactly that we could detect? DE teams need technical threat descriptions. There is a large set of ways to brute force something and an ever larger set of ways to detect the resulting activities.
- 💔 Hard to parse with detection in mind: Nicely documented CTI reports have a bad tendency of containing the entire attack path and a lot of other exciting yet irrelevant for detection details. For DE teams, that now means breaking down the report into a dozen singular threat elements that each needs to be evaluated and processed into detection.
- 🍂 Lack of variety: Yes, Email and Windows are exploited left and right. But we also have Cloud, SaaS, Remote Workers and other unique threat vectors to look into. If your company uses multiple platforms, but your intel team feeds your DE team with Windows stuff only, attackers will get in and you won’t know it.
- ⚖️ No value add, just copy from the feed: CTI ultimate purpose in an organization is not to compile external content (some of questionable provenance), but to analyze threats through the prism of your business operations and IT systems in place. “Detect threats that matter” is easy to say, but really hard to do!
- 🫠 Doesn’t understand the threat: An intel analyst should deeply understand the threat they communicate to DE to accelerate further processing, and ensure enough information is delivered. DE is a process, where any threat can be processed into detection, but only as long as we understand the issue at hand. Ideally, bi-directional CTI-DE comms are established…
- 🐢 Sloooow: DE aims at moving at the speed of intelligence, thus it needs intel which is closely following the latest developments. Intel done for research can be slow, but intel done for detection cannot be.
🧬IOCs only get you so far
CTI teams spend a considerable amount of money building an infrastructure that captures, stores, and distributes IOCs to various intel consumers (including whatever detection platforms you use, SIEM, EDR, etc). For detection engineers, handling those IOC to scan incoming logs, search historical ones or enrich existing alerts is a very efficient catch-all. However, most threats nowadays are highly effective at avoiding simple markers and naive rules.
Modern malware, domain generation algorithms and other approaches introduce fuzziness in the behavior monitoring systems will see, and require different approaches to be effectively detectable. Starting from studying the exact procedures seen in the wild, we can model the common behaviors into low to high level detection objectives that remain highly targeted (and thus without resorting to trying to detect an entire ATT&CK technique, which may be costly and lengthy compared to the detailed intel received).
For example, a registry key used for persistence may change continuously and not be a great way to detect, but the presence of certain patterns in the registry value (certain non-latin characters, mentions or prefixes), very indicative of a certain actor may be great detection objective to successfully and quickly detect an entire campaign. But coming up with this takes skill, time and good intel!
For CTI to deliver a higher value-add to creating new detections, it must have a stronger focus into compiling intelligence frequently, and delivering the correct level of technical knowledge to detection engineers. Level of abstractions matters more (“they brute forced” is too much abstraction, but “they tried this password 37 times” is definitely too little)
CTI teams spend a considerable amount of money building an infrastructure that captures, stores, and distributes IOCs to various intel consumers (including whatever detection platforms you use, SIEM, EDR, etc). For detection engineers, handling those IOC to scan incoming logs, search historical ones or enrich existing alerts is a very efficient catch-all. However, most threats nowadays are highly effective at avoiding simple markers and naive rules.
Modern malware, domain generation algorithms and other approaches introduce fuzziness in the behavior monitoring systems will see, and require different approaches to be effectively detectable. Starting from studying the exact procedures seen in the wild, we can model the common behaviors into low to high level detection objectives that remain highly targeted (and thus without resorting to trying to detect an entire ATT&CK technique, which may be costly and lengthy compared to the detailed intel received).
For example, a registry key used for persistence may change continuously and not be a great way to detect, but the presence of certain patterns in the registry value (certain non-latin characters, mentions or prefixes), very indicative of a certain actor may be great detection objective to successfully and quickly detect an entire campaign. But coming up with this takes skill, time and good intel!
For CTI to deliver a higher value-add to creating new detections, it must have a stronger focus into compiling intelligence frequently, and delivering the correct level of technical knowledge to detection engineers. Level of abstractions matters more (“they brute forced” is too much abstraction, but “they tried this password 37 times” is definitely too little)
😤 Massive PDFs Ain’t Intelligent!
CTI teams are encouraged by their TIP (threat intelligence platform, which sources and catalogs third-party threat reports) to transform multiple PDF into a single one, presumably so that it would be easier for humans to read. This is perhaps a vestige or non-cyber intelligence, which was often compiled in long paper reports.
Such aggregated PDFs are, frankly, an annoyance for most detection use cases! In fact, for good DE teams, this is a superb method to make them go mad, as they parse the report in weeks (yes, weeks) into individual objects, then research each one of them to identify detection opportunities. It discourages prioritizing particular aspects of the report, since the entire analysis blocks a DE, which then needs to present and act upon a large body of work, instead of neatly packaged work items. Consider that intel is only “intelligent” if it is processed in a way that doesn’t require further manual work and more analysis to understand.
CTI teams are encouraged by their TIP (threat intelligence platform, which sources and catalogs third-party threat reports) to transform multiple PDF into a single one, presumably so that it would be easier for humans to read. This is perhaps a vestige or non-cyber intelligence, which was often compiled in long paper reports.
Such aggregated PDFs are, frankly, an annoyance for most detection use cases! In fact, for good DE teams, this is a superb method to make them go mad, as they parse the report in weeks (yes, weeks) into individual objects, then research each one of them to identify detection opportunities. It discourages prioritizing particular aspects of the report, since the entire analysis blocks a DE, which then needs to present and act upon a large body of work, instead of neatly packaged work items. Consider that intel is only “intelligent” if it is processed in a way that doesn’t require further manual work and more analysis to understand.
Structuring expert teams
Detection Engineering is such a novel function that the reality is simply that many CTI teams are unprepared for the additional work needed to support such a function that intends to build on intelligence, rather than simply read it.
As a result, many DE teams today actually work very independently, combining threat intelligence and threat modeling into their processes, where in a target operating model, both of these inputs should be expected from threat intelligence analysts. In many cases, when a CTI team exists, they can’t scale to the DE challenge (and, yes, there are beautiful examples of effective CTI-DE collaboration at scale).
Creating new services in the CTI functions thus requires thinking about the organization structure, and how to make teams integrate. Next in the series? An operations model! Very fun!
Related blog posts:
Detection Engineering is such a novel function that the reality is simply that many CTI teams are unprepared for the additional work needed to support such a function that intends to build on intelligence, rather than simply read it.
As a result, many DE teams today actually work very independently, combining threat intelligence and threat modeling into their processes, where in a target operating model, both of these inputs should be expected from threat intelligence analysts. In many cases, when a CTI team exists, they can’t scale to the DE challenge (and, yes, there are beautiful examples of effective CTI-DE collaboration at scale).
Creating new services in the CTI functions thus requires thinking about the organization structure, and how to make teams integrate. Next in the series? An operations model! Very fun!
Related blog posts:
