Friday, September 25, 2020

Posts From Beyond The Grave: How To Impress / Annoy An Analyst During A Briefing

My old $employer blog has vanished and a lot of content of value to the community went down with it. Naturally, I do not own the IP and I…

My old $employer blog has vanished and a lot of content of value to the community went down with it. Naturally, I do not own the IP and I cannot go to archive.org and bring it back to life.

However, I will make an exception for this post. Because it (and this is my ego talking, natch) exudes pure awesomeness!

— — — — — — — — — start repost — — — — — — — — -

About a year ago, I crowdsourced a collection of best/worst tips for Vendor Briefings (a one hour presentations from a technology vendor to a Gartner analyst) from other analysts, and now I finally found time to blog it, thanks for some motivation from the Twitterverse. The list is organized as DOs and DON’Ts for the vendors, listed in semi-logical order.

DO NOT:

  • Do not present a disorganized slide deck, full of typos and outdated information [extra: do NOT misspell “HIPAA”]
  • Do not (NOT!) go to a VB without any materials such as slides; this negatively affects the retention of the shared information by the analyst
  • But also, do not bring a 110 slide deck and then fly through it, see the above point re: analyst retention of the material
  • Do not spend time talking about your team — in fact, don’t tell me anything I can read on your ”about us” page [we are analysts, NOT investors, a brief “who we are” slide is OK]
  • Do not go so fast that an analyst cannot ask questions, but also do not stop every 3 minutes to ask whether an analyst have any questions
  • Do not be vague about what your product/service actually does in real life [this, as you can easily guess, is a biggie!]
  • Do not focus excessively on marketing and “the story you tell” at a cost of product functionality, specific problems you solve and precise reasons why your approach is superior to alternatives
  • Do not spend too much time [or: any time] on industry statistics about how bad the threat is. We see this every time. Do focus on explaining the specific problem your product/service tackles
  • Generally, do not ask for feedback on Vendor Briefings, we are not allowed to provide it — please schedule a client call if you are client for that
  • Do not allude to having many customers and then say “but they are so secretive we cannot share anything about them”
  • Do not assign somebody who is unqualified, disorganized and arrogant to the task of doing a Vendor Briefing
  • Similarly, do not bring “a CTO” who cannot answer technical questions; because if your CTO is fake, perhaps your entire product is?
  • Do not name drop other analysts’ names and then imply that they really liked your product
  • Do not lie to an analyst such as by saying that you are fascinated by their brilliant research while it becomes clear you actually never read it
  • Do not ask for a briefing under NDA, unless you have some really, really, really good reasons for it [A.C. — this is very rare!]
  • Do not focus on bashing your competition [such as with rumors and innuendo], but do provide a crisp, fact-based comparison to the competition you fight in the field.

DO:

  • Do read the official Vendor Briefing instructions first!
  • Do bring a well-organized slide deck that you actually plan to follow (please no “let’s skip to slide 48”). Some analysts recommend 10–20 slides, I think a bit more is acceptable, as long as they tell a coherent story.
  • Provide a copy of your presentation in advance [we all prefer it, but some of us feel much strongly about this one!]
  • When you provided slides in advance, actually follow the slides provided during the briefing [this sounds silly, but yes, we have seen vendors not do it…]
  • Do use a good quality phone line, and seek to minimize airport, dog, kid, pet hyena noises [A.C. — I do tell people “you sound like VOIP” sometimes, but I would never complain about the dog barking…]
  • You should make sure you tell the problem you solve and what makes you different in the first 10 minutes of the briefing. Be sure to describe this as a unique value for end users, not a unique marketing story.
  • Do ask an analyst what he wants to hear the most, but don’t be shocked if he says “I just want to know what you do and how” [this is true if this a new product space for them]
  • Do express your views on the market history, evolution, trends and your role in all this — but do not give us the market basics.
  • Ideally, share why we need yet another vendor, in this market, to do this [if you are a player in a very competitive market]. A free tip: if you cannot explain this to an analyst, you definitely will not explain it to a prospect….
  • If you are going to do a demo, please make sure it works. Include a realistic demo, i.e. with data, a scenario that represents how your customers use your product,
  • For a product new to the analyst, show how the product addresses specific problems and customer use cases.
  • If you are going to focus on your product strategy, please understand what the word means: a real strategy defines the actions you are taking to grow, help clients and beat the competition.
  • Do provide a fact-based slide comparing your approach to alternatives, direct competitors or even the old way of solving the same problem [all told, is your tool really better than doing it manually]
  • Do remember what you presented to this analyst in the past [we don’t like an introductory briefing 2 months after your introductory briefing…]
  • If an analyst asks a question, please answer concisely, and take no more than 30–60 seconds doing so.
  • Do be clear about what your product strategy is and try to actually articulate this in a Briefing
  • Do follow-up if you promised to send the materials later, share whitepapers, customer references, etc
  • Do cover the exact problems you solve, broad product architecture, top competitors you see (and, please don’t say THERE ARE NONE!), customer use cases, your largest deployments, roadmap [that you plan to actually follow!], etc.
  • Some of us care about your pricing and your pricing strategy, but some [typically us GTP analysts] don’t. Ask an analyst whether they want to know your pricing mechanics.
  • Most of us care at least somewhat about your number of production customer and/or revenue. Please share at least something on this even if you are very secretive
  • If you want to really impress, perhaps even shed some light on what lessons you learned selling, deploying and helping customer operate your product.

Thanks to those Gartner analysts who contributed [sorry, the list of names would be too long to list here, but extra thanks go to Lawrence Pingree for his help organizing the list]

An obligatory cryptic reference: my ROTC instructor once told me that in PSYOPS, we do not lie when we talk about easily verifiable things… perhaps vendors need to take notice of this tip as well…

Other useful materials on briefing analysts can be found at the links below [note that not all of the below agree on all the details, since analysts have different focus areas] — please add your favorite resources on this to comments and I will update the post:

Originally published at Medium.

Posted by at
Labels:

Wednesday, September 23, 2020

Agreed, this one has more gaps and spaces to research/figure out.

Agreed, this one has more gaps and spaces to research/figure out.

Originally published at Medium.

Posted by at
Labels:

Agreed, this one has more gaps and spaces to research/figure out.

Agreed, this one has more gaps and spaces to research/figure out.

Originally published at Medium.

Posted by at
Labels:

Chronicle Detect is Here

A lot of people ask me how Chronicle is doing inside Google Cloud (TLDR: doing well), and I wanted to share some good news. Also, I wanted…

A lot of people ask me how Chronicle is doing inside Google Cloud (TLDR: doing well), and I wanted to share some good news. I also wanted to reveal some of our lessons building our threat detection capabilities (that we just released).

If you recall, we announced our YARA-L detection language at RSA 2020. Naturally, many people loved it, and our capabilities have grown since then. Here is what we learned and then built as a result:

  • Our initial YARA-L implementation laid the foundation for Detect, with the data foundation (where we stitch events together into a stateful timelines) and of course the scale for historical and real-time detections. Now we have multi-event operations; we added sequence awareness; aggregation and windowing that work well on petabytes of data we have.
  • We did start with ATT&CK mappings for our detection content, but now our rules have additional magic as well: we can refer to low prevalence artifacts straight from a rule without any additional work by the client; this works really well for some tricky detections.
  • Another common question was about existing rule creation approaches like Sigma — and now we have a way to convert Sigma rules into YARA-L. In fact, this will allow us to use public Sigma code and get some of their content into Chronicle (following this vision).
  • Obviously, some people asked to see how we use our unique threat intelligence for detections. This is now accomplished by a detection feed built by our threat research team, Uppercase.
  • Finally, our approach conceptually follows the idea I covered here as “detection as code.” Specifically, it is much easier to version our YARA-L detection content , map it to frameworks, create and reuse modules, and convert to/from Sigma (for cross-vendor/cross-tool usage).

Here is one detection example (as a narrative and not as raw YARA-L):

Give me all the documents opened through outlook.exe that were followed by a child process that made a network connection to a low prevalence domain and then creating and launching a process with a low prevalence hash.

Note that it makes references to “low prevalence domain” and “low prevalence hash”; these are just magically created by the system and don’t require any user action. This allows us to run powerful rules that mix “known bad” detection and anomaly detection gracefully.

Together this makes Chronicle Detect work well for the kinds of advanced, complex, and subtle threats our customers face today.

So, we now have a decent argument that our detection engine and rule approach are better and help clients implement a modern “detection as code” approach if they desire. However, we also still have an unbeatable argument that our scale/performance are the best.

Calls to action:

Originally published at Medium.

Posted by at
Labels:

Chronicle Detect is Here

A lot of people ask me how Chronicle is doing inside Google Cloud (TLDR: doing well), and I wanted to share some good news. Also, I wanted…

A lot of people ask me how Chronicle is doing inside Google Cloud (TLDR: doing well), and I wanted to share some good news. I also wanted to reveal some of our lessons building our threat detection capabilities (that we just released).

If you recall, we announced our YARA-L detection language at RSA 2020. Naturally, many people loved it, and our capabilities have grown since then. Here is what we learned and then built as a result:

  • Our initial YARA-L implementation laid the foundation for Detect, with the data foundation (where we stitch events together into a stateful timelines) and of course the scale for historical and real-time detections. Now we have multi-event operations; we added sequence awareness; aggregation and windowing that work well on petabytes of data we have.
  • We did start with ATT&CK mappings for our detection content, but now our rules have additional magic as well: we can refer to low prevalence artifacts straight from a rule without any additional work by the client; this works really well for some tricky detections.
  • Another common question was about existing rule creation approaches like Sigma — and now we have a way to convert Sigma rules into YARA-L. In fact, this will allow us to use public Sigma code and get some of their content into Chronicle (following this vision).
  • Obviously, some people asked to see how we use our unique threat intelligence for detections. This is now accomplished by a detection feed built by our threat research team, Uppercase.
  • Finally, our approach conceptually follows the idea I covered here as “detection as code.” Specifically, it is much easier to version our YARA-L detection content , map it to frameworks, create and reuse modules, and convert to/from Sigma (for cross-vendor/cross-tool usage).

Here is one detection example (as a narrative and not as raw YARA-L):

Give me all the documents opened through outlook.exe that were followed by a child process that made a network connection to a low prevalence domain and then creating and launching a process with a low prevalence hash.

Note that it makes references to “low prevalence domain” and “low prevalence hash”; these are just magically created by the system and don’t require any user action. This allows us to run powerful rules that mix “known bad” detection and anomaly detection gracefully.

Together this makes Chronicle Detect work well for the kinds of advanced, complex, and subtle threats our customers face today.

So, we now have a decent argument that our detection engine and rule approach are better and help clients implement a modern “detection as code” approach if they desire. However, we also still have an unbeatable argument that our scale/performance are the best.

Calls to action:

Originally published at Medium.

Posted by at
Labels:

Monday, September 21, 2020

Can We Have “Detection as Code”?

One more idea that has been bugging me for years is an idea of “detection as code.” Why is it bugging me and why should anybody else care?

One more idea that has been bugging me for years is an idea of “detection as code.” Why is it bugging me and why should anybody else care?

First, is “detection as code” just a glamorous term for what you did when you loaded your Snort rules in cvs in, say, 1999? Well, not exactly.

What I mean by “detection as code” is a more systematic, flexible and comprehensive approach to threat detection that is somewhat inspired by software development (hence the “as code” tag). Just as infrastructure as code (IaC) is not merely about treating your little shell scripts as real software, but about machine-readable definition files and descriptive models for infrastructure.

Why do we need this concept? This is a good question! Historically, from the days of first IDS (1987) to the sad days of “IDS is dead” (2003) and then to today, detection got a bit of a bad reputation. We can debate this, to be sure, but most would probably agree that threat detection never “grew up” to be a systematic discipline, with productive automation and predictable (and predictably good!) results. In fact, some would say that “Your detections aren’t working.” And this is after ~35 years of trying …

Detection engineering is a set of practices and systems to deliver modern and effective threat detection. Done right, it can change security operations just as DevOps changed the stolid world of “IT management.” You basically want to devops (yes, I made it a word) your detection engineering. I think “detection as code” is a cool name for this shift!

As you see, this is not so much about treating detections as code, but about growing detection engineering to be a “real” practice, built on modern principles used elsewhere in IT (agile this, or DevOps whatever).

Now, to hunt for the true top-tier APTs, you probably need to be an artist, not merely a great security engineer (IMHO, best threat hunting is both art and science, and frankly more art than science….). But even here, to enable “artistic” creativity in solving threat detection problems we need to make sure those solutions function on a predictable layer. Moreover, for many other detection pursuits, such as detecting ransomware early, we mostly need automated, systematic, repeatable, predictable and shareable approaches.

OK, how do we do “detection as code”? How would I describe the characteristics of this approach?

  • Detection content versioning so that you can truly understand what specific rule or model triggered an alert — even if this alert was last July. This is even more important if you use a mix of real-time and historical detections.
  • Proper “QA” for detection content that covers both testing for broken alerts (such as those that never fire or those that won’t fire when the intended threat materializes, and of course those that fire where there is no threat) and testing for gaps in detection overall. “False positives” handling, naturally, get thrown into this chute as well.
  • Content (code) reuse and modularity of detection content, as well as community sharing of content, just as it happens for real programming languages (I suspect this is what my esteemed colleague describes here). As a reminder, detection content does not equal rules; but covers rules, signatures, analytics, algorithms, etc.
  • Cross-vendor content would be nice, after all we don’t really program in “vendor X python” or “big company C” (even though we used to), we just write in C or Python. In the detection realm, we have Sigma and YARA (and YARA-L too). We have ATT&CK too, but this is more about organizing content, not cross-vendor writing of the content today.
  • I also think that getting to cross-tool detection content would be great, wherever possible. For example, you can look for a hash in EDR data and also in NDR; and in logs as well. SIEM alone won’t do.
  • Metrics and improvement are also key; the above items will give you plenty of metrics (from coverage to failure rates), but it is up to you to structure this process so that you get better.
  • While you may not be looking at building a full CI/CD pipeline for detections to continuously build, refine, deploy and run detection logic in whatever product(s), I’ve met people who did just that. To me, these people really practice detection as code.
  • Finally, I don’t really think this means that your detections need to be expressed in a programming language (like Python here and here or Jupyter notebooks). What matters to me is the approach and thinking, not actual code (but we can have this debate later, if somebody insists)

Anything else I missed?

For our recent SANS paper / webcast, that mentioned this topic, we crafted this example visual:

Source: recent SANS paper.

Finally, let’s cattle-prod the elephant in the room: what about the crowd that just does not want anything “as code”? They also don’t like to create their own detections at all. In fact, they like their detections as easy as pushing an ON button or downloading a detection pack from a vendor? This is fine.

Personally, I’ve met enough security people who run away screaming from any technology that is “too flexible”, “very configurable” and even “programmable” (or: “… as code”) because their past experience indicates that this just means failure (at their organization). However, to detect, you need both a tool and content. Hence, both will have to come from somewhere: you can build, buy, rent, but you must pick.

Now, upon reading this, some of you may say “duh … what is not painfully obvious about it?” but I can assure you most people in the security industry do NOT think like that. In fact, such thinking is alien to most, in my experience. Maybe they think detection is a product feature. Or perhaps they think that detection is some magical “threat” content that comes from “the cloud.”

Hence, “detection as code” is not really an approach change for them, but a more philosophical upheaval. Still, I foresee that threat detection will always be a healthy mix of both an engineering and a creative pursuit….

Thoughts?

P.S. Thanks to Brandon Levene for hugely useful contributions to this thinking!

Originally published at Medium.

Posted by at
Labels:

Friday, September 11, 2020

Sort of, yes.

Sort of, yes. SIEM/UEBA (and log analysis) still remains the most broad and flexible of the pillars, but not without its challenges, to be sure

Originally published at Medium.

Posted by at
Labels:

Sort of, yes.

Sort of, yes. SIEM/UEBA (and log analysis) still remains the most broad and flexible of the pillars, but not without its challenges, to be sure

Originally published at Medium.

Posted by at
Labels:
Subscribe to: Posts (Atom)

Dr Anton Chuvakin

Dr Anton Chuvakin