Wednesday, March 26, 2014

How to Use Threat Intelligence with Your SIEM? [BACKUP FROM DEAD GARTNER BLOG]

 NOTICE: after Gartner killed ALL blogs in late 2023, I am trying to salvage (via some of the most critical blogs I've written while working there, and repost them with backdates here, for posterity. This one is about SIEM and TI.

SIEM and Threat Intelligence (TI) feeds are a marriage made in heaven! Indeed, every SIEM user should send technical TI feeds into their SIEM tool. We touched on that subject several times, but in this post will look at in in depth. Well, in as much depth as possible to still make my future paper on the topic a useful read :–)

First, why are we doing this:

  • Faster detection – alerting on TI matches (IPs, URLs, domains, hashes, etc) is easier than writing good correlation rules
  • Better context – alert triage and incident investigation becomes easier and information is available faster
  • Threat tracking and awareness – combining local monitoring observations, external TI and [for those who are ready!] internal TI in one place.

What log data do we need?

  • Most common log data to match to TI feeds: firewalls logs (outbound connection records … my fave logs nowadays!), web proxy logs
  • Also used: netflow, router logs or anything else that shows connectivity
  • NIDS/NIPS (and NBA, if you are into that sort of thing) data (TI matching here helps triage, not detection)
  • ETDR tools can usually match to TI data without using a SIEM, but local endpoint execution data collected in one place marries well to TI feeds.

Where would TI data comes from (also look for other TI sources):

  • SIEM vendor: some of the SIEM vendors are dedicating significant resources to the production of their own threat intelligence and/or TI feed aggregation, enrichment and cleaning
  • Community, free TI feeds: CIF format comes really handy here, but CSV can be imported just as well (some lists and information on how to compare them)
  • Commercial packaged feeds from the TI aggregator (it may even have pre-formatted rules ready for your SIEM!)
  • Commercial TI providers of original threat intelligence.

Obviously, using your SIEM vendor TI feeds is the easiest (and may in fact be as easy as clicking one button to turn it on!), but even other sources are not that hard to integrate with most decent SIEM tools.

Now, let’s review all the usage of TI data inside a SIEM:

  • Detect owned boxes, bots, etc that call home when on your network (including boxes pre-owned when not on your network) and, in general, detect malware that talks back to its mothership
  • Validate correlation rules and improve baselining alerts by upping the priority of rules that also point at TI-reported “bad” sources
  • Qualify entities related to an incident based on collected TI data (what’s the history of this IP?)
  • Historical matching of past, historical log data to current TI data (key cool thing to do! resource intensive!)
  • Review past TI history as key context for reviewed events, alerts, incidents, etc (have we seen anything related to this IP in the past TI feeds?)
  • Review threat histories and TI data in one place; make use of SIEM reports and trending to analyze the repository of historical TI data (create poor man’s TI management platform)
  • Enable [if you feel adventurous] automatic action due to better context available from high-quality TI feeds
  • Run TI effectiveness reports in a SIEM (how much TI leads to useful alerts and incidents?)
  • Validate web server logs source IP to profile visitors and reduce service to those appearing on bad lists (uncommon)
  • Other use of TI feeds in alerts, reports and searches and as context for other monitoring tasks

So, if you are deploying a SIEM, make sure that you start using threat intelligence in the early phases of your project!

Posts related to this research project:

Wednesday, March 19, 2014

Our Team Is Hiring Again: Join Gartner GTP Now!

It is with great pleasure that I am announcing that our team is HIRING AGAIN!

Join Security and Risk Management Strategies (SRMS) team at Gartner for Technical Professionals (GTP)!

Excerpts from the job description:

    • Create and maintain high quality, accurate, and in depth documents or architecture positions in information security, application security, infrastructure security, and/or related coverage areas;
    • Prepare for and respond to customer questions (inquiries/dialogues) during scheduled one hour sessions with accurate information and actionable advice, subject to capacity and demand;
    • Prepare and deliver analysis in the form of presentation(s) delivered at one or more of the company’s Catalyst conferences, Summit, Symposium, webinars, or other industry speaking events;
    • Participate in industry conferences and vendor briefings, as required to gather research and maintain a high level of knowledge and expertise;
    • Perform limited analyst consulting subject to availability and management approval;
    • Support business development for GTP by participating in sales support calls/visits subject to availability and management approval;
    • Contribute to research planning and development by participating in planning meetings, contributing to peer reviews, and research community meetings

In essence, your job would be to research, write, guide clients (via phone inquiries/dialogs) and speak at events. Also, we do list a lot of qualifications in the job req, but you can look at my informal take on them in this post.


P.S. If the link above fails, go to and search for “IRC26388

P.P.S. If you have questions, feel free to email me – I cannot promise a prompt response, but I sure can promise a response.

P.P.P.S This is cross-posted from my Gartner blog.

Related posts:

Monday, March 03, 2014

Monthly Blog Round-Up – February 2014

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:
  1. Simple Log Review Checklist Released!” is often at the top of this list – the checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version)
  2. Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? Well, you be the judge.
  3. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports (the paper link is now restored!) – also see this SIEM use case in depth.
  4. My classic PCI DSS Log Review series is popular as well. The series of 18 posts cover a comprehensive log review approach, useful for building log review processes and procedures, whether regulatory or not. It is also described in more detail in our Log Management book.
  5. “SANS Top 6 Log Reports Reborn!” is a new post that announces that many people’ work on best log reports has finally been published as “The 6 Categories of Critical Log Information” (with a subtitle of “Top 6 SANS Essential Categories of Log Reports 2013”)

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog:

Current research on threat intelligence:

Past research on using big data approaches for security:

(see my published Gartner research here)

Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013.

Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

Dr Anton Chuvakin