Why Your Security Data Lake Project Will FAIL! [BACKUP FROM DEAD GARTNER BLOG]

NOTICE: after Gartner killed ALL blogs in late 2023, I am trying to salvage (via archive.org) some of the most critical blogs I've written while working there, and repost them with backdates here, for posterity. This one reminds the world that I was a huge skeptic of security data lakes.

 Beats me, but for some reason organizations think that they can build A SECURITY DATA LAKE and/or their own CUSTOM BIG DATA SECURITY ANALYTICS tools. Let me tell you what will happen – it will FAIL.

Cue the data swamp jokes. Mention data pond scum. Discuss pissing in the data pool. The result is the same – it likely won’t work.

OK, let me tone this down a bit – it will be successful (however this is defined) for 0.1% of those who try [the percentages are approximate and are meant to increase the dramatic impact of this post, not to share data]

Why am I so adamant about it? During our UEBA research we encountered several organizations that are migrating from DIY/custom security analytics to COTS (typically to UEBA as it has matured). What truly shocked us was that some organizations reported that they did have a custom security analytics project running for a few years – but it is now being shut down due to “huge effort – low value” combination. What was even more shocking, some of the organizations were essentially in a “Fortune 50” class, presumably those global technology elites. It didn’t even work for [some of] them…. The QotD [modified to remove any possible relation to the client] was “we wish we’d never discovered Hadoop – we wasted years of trying to make a security analytics capability out of it.”

Motivated by the use of cheap hardware, reduced data redundancy (store one copy – so wow!) and promise of advanced analytics they went for it … and mostly FAILED.

Some of the reasons for failure or relative lack of success included:

1. Dirty data – you throw stuff in and then cannot use it; a #1 “fail-cause” (great story about it)
2. Trouble with collecting data – SIEM vendors spent 10+ years debugging their collectors for a reason…
3. Trouble with accessing data – data went in – plonk!- and now nobody knows how to get it out to do analysis (great story here)
4. No value beyond collection – the data lake was created and filled with data, so it is there just in case, but any subsequent project phases stumbled
5. No value beyond keyword search – data lake was created to enable advanced analytics, but ultimately delivered only basic keyword search of logs
6. No threat detection value – this happened when somebody hired a big data company to build a security data lake; they build all the plumbing and said “ah, security use cases? you do it!” and left
7. Failure to conceptualize and define the security analytics use cases – OK, we will now detect threats… OK, how? Well, nobody knows and no time to experiment. And see #1 – dirty data
8. Security analytics use case design much harder then expected
9. Much higher bar for analytics and big data expertise talent requirements and failure to acquire said talent.

(note that some are overlapping and/or related)

As we say here, “Given the simplicity of the technical characteristics of a data lake, it shouldn’t come as a surprise that getting value out of this concept is entirely dependent on the availability of advanced programming and analytics skills.” For security, you also need to add threat analysis skills to the mix.

In essence, the only successful project type (and this is not really security analytics, not by a long shot) was “install ELK, throw logs in, search for keywords.” This works well, but this is NOT what they aspired for – not even close. Not even in the same realm.

To conclude, successful custom big data security analytics efforts remain rare outliers, like a flying car. My 2012 post was full of hope – and sadly it didn’t work out. At this point, it is very clear to me that DIY or open source is NOT the way to go for security analytics. Sure, we will continue watching both Spot and Metron, but frankly at this point I am a skeptic.

So, short summary: open source – based log aggregation – sure, custom security analytics – only worked well for a very select few. If you still want to try, feel free to review this for some ideas (if you read it, provide feedback here!). It seems like this document will NOT be updated anytime soon…

Related blog posts on security analytics:

• Our Security Analytics and UEBA Papers Published
• Ok, So Who Really MUST Get a UEBA?
• On UEBA / UBA Use Cases
• UEBA Clearly Defined, Again?
• What Should Your UEBA Show: Indications or Conclusions?
• UEBA Shines Where SIEM Whines?
• The Coming UBA / UEBA – SIEM War!
• Next Research: Back to Security Analytics and UBA/UEBA
• Sad Hilarity of Predictive Analytics in Security?
• On Unknown Operational Effectiveness of Security Analytics Tooling
• Now That We Have All That Data What Do We Do, Revisited
• Killed by AI Much? A Rise of Non-deterministic Security!
• Those Pesky Users: How To Catch Bad Usage of Good Accounts
• Security Analytics Lessons Learned — and Ignored!
• Security Analytics: Projects vs Boxes (Build vs Buy)?
• Do You Want “Security Analytics” Or Do You Just Hate Your SIEM?
• Security Analytics – Finally Emerging For Real?
• Why No Security Analytics Market? <- important read for VCs and investors! Works in 2017 too, mostly.
• More On Big Data Security Analytics Readiness
• 9 Reasons Why Building A Big Data Security Analytics Tool Is Like Building a Flying Car
• Big Analytics” for Security: A Harbinger or An Outlier?

Monthly Blog Round-Up – February 2017

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:

1. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010, so I have no idea why it tops the charts now!) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on developing security monitoring use cases here!
2. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” … 
3. “Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” (also aged a bit by now) is a companion to the checklist (updated version)
4. This month, my classic PCI DSS Log Review series is extra popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ even though it predates it), useful for building log review processes and procedures, whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (now in its 4th edition!) – note that this series is mentioned in some PCI Council materials.
5. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the SIEM project (well, a program, really) costs at an organization (a lot more details on this here in this paper).

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog [which, BTW, now has about 5X of the traffic of this blog]: 

Recent research on security analytics and UBA / UEBA:

• Ok, So Who Really MUST Get a UEBA?
• On UEBA / UBA Use Cases
• UEBA Clearly Defined, Again?
• What Should Your UEBA Show: Indications or Conclusions?
• UEBA Shines Where SIEM Whines?
• The Coming UBA / UEBA – SIEM War!
• Next Research: Back to Security Analytics and UBA/UEBA
• Sad Hilarity of Predictive Analytics in Security?

Miscellaneous fun posts:

• Security in 2025 – Extrapolate or Bust?
• All My Research Published in 2016
• Threats Inside vs Insider Threat
• Can I Detect Advanced Threats With Just Flows/IPFIX?
• No, Virginia, It Does NOT Mean That! (detection vs prevention)
• Security: Automate And/Or Die?
• Defeat The Casual Attacker First!!
• On “Defender’s Advantage”

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016.
Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on August 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.
Previous post in this endless series:

• Monthly Blog Round-Up – January 2017
• All posts tagged monthly

Monthly Blog Round-Up – January 2017

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:

1. This month, my classic PCI DSS Log Review series is extra popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ even though it predates it), useful for building log review processes and procedures, whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (now in its 4th edition!) – note that this series is mentioned in some PCI Council materials.
2. “Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” (also aged a bit by now) is a companion to the checklist (updated version)
3. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on developing security monitoring use cases here!
4. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” … 
5. “An Open Letter to Android or “Android, You Are Shit!”” is an epic rant about my six year long (so far) relationship with Android mobile devices (no spoilers here – go and read it).

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog [which, BTW, now has about 5X of the traffic of this blog]: 

 

Current research on security analytics and UBA / UEBA:

• Ok, So Who Really MUST Get a UEBA?
• On UEBA / UBA Use Cases
• UEBA Clearly Defined, Again?
• What Should Your UEBA Show: Indications or Conclusions?
• UEBA Shines Where SIEM Whines?
• The Coming UBA / UEBA – SIEM War!
• Next Research: Back to Security Analytics and UBA/UEBA
• Sad Hilarity of Predictive Analytics in Security?

Recent research on deception:

• Our “Applying Deception Technologies and Techniques to Improve Threat Detection and Response” Paper is Published
• APT-Ready? Better Threat Detection vs Detecting “Better” Threats?
• Better Data or Better Algorithms?
• Tricky: Building a Business Case for A Deception Tool?
• It Is Happening: We Are Starting Our Deception Research!
• “Deception as Detection” or Give Deception a Chance?

Miscellaneous fun posts:

• Security in 2025 – Extrapolate or Bust?
• All My Research Published in 2016
• Threats Inside vs Insider Threat
• Can I Detect Advanced Threats With Just Flows/IPFIX?
• No, Virginia, It Does NOT Mean That! (detection vs prevention)
• Security: Automate And/Or Die?
• Defeat The Casual Attacker First!!
• On “Defender’s Advantage”

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016.

Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on August 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

• Monthly Blog Round-Up – December 2016
• All posts tagged monthly

Annual Blog Round-Up – 2016

Here is my annual "Security Warrior" blog round-up of top 10 popular posts/topics in 2016. Note that my current Gartner blog is where you go for my recent blogging, all of the content below predates 2011.

1. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not.
2. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on developing security monitoring use cases here!
3. “Simple Log Review Checklist Released!” is often at the top of this list – the checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version)
4. My classic PCI DSS Log Review series is always hot! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ in 2017 as well), useful for building log review processes and procedures , whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (out in its 4th edition!) 
5. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the SIEM project (well, a program, really) costs at an organization (a lot more details on this here in this paper).
6. “Top 10 Criteria for a SIEM?” came from one of my last projects I did when running my SIEM consulting firm in 2009-2011 (for my recent work on evaluating SIEM tools, see this document)
7. “How to Write an OK SIEM RFP?” (from 2010) contains Anton’s least hated SIEM RFP writing tips (I don’t have any favorite tips since I hate the RFP process)
8. “An Open Letter to Android or “Android, You Are Shit!”” is an epic rant about my six year long (so far) relationship with Android mobile devices (no spoilers here – go and read it).
9. “A Myth of An Expert Generalist” is a fun rant on what I think it means to be “a security expert” today; it argues that you must specialize within security to really be called an expert.
10. Another old checklist, “Log Management Tool Selection Checklist Out!”  holds a top spot  – it can be used to compare log management tools during the tool selection process or even formal RFP process. But let me warn you – this is from 2010.

Disclaimer: all this content was written before I joined Gartner on August 1, 2011 and is solely my personal view at the time of writing.  For my current security blogging, go here.

Also see my past monthly and annual “Top Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015.

Monthly Blog Round-Up – December 2016

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:

1. “An Open Letter to Android or “Android, You Are Shit!”” is an epic rant about my six year long (so far) relationship with Android mobile devices (no spoilers here – go and read it).
2. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on developing security monitoring use cases here!
3. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” … 
4. “Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” (also aged a bit by now) is a companion to the checklist (updated version)
5. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ as well), useful for building log review processes and procedures, whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (now in its 4th edition!)

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog [which, BTW, now has about 5X of the traffic of this blog]: 

 

Current research on security analytics and UBA / UEBA:

• UEBA Clearly Defined, Again?
• What Should Your UEBA Show: Indications or Conclusions?
• UEBA Shines Where SIEM Whines?
• The Coming UBA / UEBA – SIEM War!
• Next Research: Back to Security Analytics and UBA/UEBA
• Sad Hilarity of Predictive Analytics in Security?

Recent research on deception:

• Our “Applying Deception Technologies and Techniques to Improve Threat Detection and Response” Paper is Published
• APT-Ready? Better Threat Detection vs Detecting “Better” Threats?
• Better Data or Better Algorithms?
• Tricky: Building a Business Case for A Deception Tool?
• It Is Happening: We Are Starting Our Deception Research!
• “Deception as Detection” or Give Deception a Chance?

Miscellaneous fun posts:

• All My Research Published in 2016
• Threats Inside vs Insider Threat
• Can I Detect Advanced Threats With Just Flows/IPFIX?
• No, Virginia, It Does NOT Mean That! (detection vs prevention)
• Security: Automate And/Or Die?
• Defeat The Casual Attacker First!!
• On “Defender’s Advantage”

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015.

Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on August 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

• Monthly Blog Round-Up – November 2016
• All posts tagged monthly

An Open Letter to Android or “Android, You Are Shit!”

Dear Android:

I know you are an operating system and probably cannot (yet?) read on your own. However, recent events compelled me to write this letter to you; an idea for it literally came to me in a dream.

You see, I have carried an Android phone in my pocket since 2010, for almost six years. First Sony Experia X10 (eventually running a venerable Android 2.3.7), then another phone and then finally a Google Nexus 4 and now Google Nexus 5X (sporting Android 7.1.1). At some point, I traded an iPad for a Google Nexus 9. A [sort of] Android Amazon Fire is my living room Android. I have convinced my wife to start using Android as well and she became a fan too. This represents a multi-year love affair with you, dear Android.

In fact, dear Android, I often had to defend you from packs from rabid Apple fanboys, generally with good results - I either won or we had a draw. Over the years, I had to defend my mobile technology choices from many people: “No, it is NOT an iPhone, it is a Nexus”, “Yes, I chose Android because I like it more than iPhone, not because it is cheaper”, “Yes, I think Google Now is way more useful than Siri”, etc, etc. I’ve counter-attacked with arguments about “closed Apple ecosystem”, “one stupid button” and “overpriced devices.”   As a person who follows information technology, I am aware of Android many strengths such as better background processes and multi-tasking, security improvements, flexible user interface, Google Now integration, etc.

However, as I am writing this, my beloved Nexus 5X is no longer with me. In fact, recent events have triggered some soul-searching and ultimately this letter. While doing my soul-searching, I realized that my love affair with you, Android, has some strong dysfunctional notes. You see, I think I always suspected that you are shit.

Over the years, I’ve been using my Android devices carefully and thoughtfully – I never rooted them, never sideloaded apps [well, not to my main personal phone], and I even tried to minimize my use of non-Google applications, etc.  However, as I recall my experiences with Android over the last six years, I am saddened to report that you, Android, never really worked quite right.

In fact, I distilled my reasons to calling you “shit” to one key point: I have never really trusted you, because you have never worked reliably enough to earn such trust.

Indeed, my Sony phone will sometimes crash and reboot, or freeze (“battery out” was the only cure). I of course explained it by “growing pains of Android, the new mobile OS”…after all you were just in v.2., practically a baby. My Nexus 4 used to crash and shut down as well; apps will often drain the battery to zero without any warning.  Furthermore, even nowadays, my Google Nexus 9 tablet (running Android 7.1.1) will occasionally just shut down out of the blue – I just had to restart it earlier today.  A few days before my Nexus 5X untimely death - just 1 year and 9 days after purchase, the phone rebooted when I launched a Camera app. Such random reboots and crashes were not common with my Nexus 5X, but they did happen periodically.  And then finally, my Nexus 5X entered an endless reboot loop a few days after the 7.1.1 OTA update and now has to be replaced. No troubleshooting steps helped.

OK, Google, you want to blame the hardware, perhaps? My experiences over the last 6 years sap the energy from this argument. I used the hardware from 3 different makers, all running Android, all having stability problems.

You see, Android, I don’t care about improved malware protection, faster UI and about the fact that you are “really Linux.” I don’t care about your growing market share.  An OS that cannot stay up is shit OS. And, you, my dear OS friend, is shit.

In fact, as my employer gave me an iPhone (first 4S and now 5S), a peculiar pattern of behavior developed in my life: if I absolutely, positively had to call an Uber on a dark and stormy night, I will stash my work iPhone in my back pocket, just in case. If I have to show a boarding pass to a permanently angry TSA agent, I will print it or use an iPhone. In fact, I was not even aware of this “if it has to happen – use iPhone” pattern until my wife asked me about why I was printing another boarding pass and I said “Ok, I guess I can use an iPhone for that” – and so I realized that I just won’t trust my Android device with this.

Dear Android, you may be a full-featured OS now, but you are just not mission critical. In fact, you are the opposite of that – you are iffy. And the only reason for why a version SEVEN (not version TWO with growing pains, mind you) will not achieve this reliability is obvious to me – you are shit.

Android, I’ve never really trusted you and I don’t trust you now. I’ve lived with you since your version 2.1 to a current 7.1.1. The only way you can still have "growing pains" after so many years is that you are a shit OS.

Despite all that, dear Android, I will take one more chance with you. When my Google Nexus 5X is repaired and then hopefully continues working for a while, I will stick to using you. But, sorry, no promises beyond that point!

Respectfully ... but distrustfully,

Dr. Anton Chuvakin

(as a consumer, NOT as a technology analyst!)

Monthly Blog Round-Up – November 2016

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:

1. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on developing security monitoring use cases here!
2. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” … 
3. “Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” (also aged a bit by now) is a companion to the checklist (updated version)
4. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ as well), useful for building log review processes and procedures, whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (now in its 4th edition!)
5. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the costs of a SIEM project (well, a program, really) at an organization (much more details on this here in this paper).

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog [which, BTW, now has about 5X of the traffic of this blog]: 

 

Current research on security analytics and UBA / UEBA:

• UEBA Shines Where SIEM Whines?
• The Coming UBA / UEBA – SIEM War!
• Next Research: Back to Security Analytics and UBA/UEBA
• Sad Hilarity of Predictive Analytics in Security?

Recent research on deception:

• Our “Applying Deception Technologies and Techniques to Improve Threat Detection and Response” Paper is Published
• APT-Ready? Better Threat Detection vs Detecting “Better” Threats?
• Better Data or Better Algorithms?
• Tricky: Building a Business Case for A Deception Tool?
• It Is Happening: We Are Starting Our Deception Research!
• “Deception as Detection” or Give Deception a Chance?

Past research on SOC:

• SOC Webinar Questions Answered
• Our “How to Plan, Design, Operate and Evolve a SOC” Paper Is Published
• About The Tri-Team Model of SOC, CIRT, “Threat Something”
• New Research Starting Soon: Threat Intel, SOC, etc
• Your SOC Nuclear Triad

 

Miscellaneous fun posts:

• Threats Inside vs Insider Threat
• Can I Detect Advanced Threats With Just Flows/IPFIX?
• No, Virginia, It Does NOT Mean That! (detection vs prevention)
• Security: Automate And/Or Die?
• Defeat The Casual Attacker First!!
• On “Defender’s Advantage”

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015.

Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

• Monthly Blog Round-Up – October 2016
• All posts tagged monthly

Monthly Blog Round-Up – October 2016

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:

1. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on security monitoring use cases here!
2. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” … 
3. “Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version)
4. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the costs of a SIEM project (well, a program, really) at an organization (much more details on this here in this paper).
5. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ as well), useful for building log review processes and procedures, whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (now in its 4th edition!)

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog [which, BTW, now has about 5X of the traffic of this blog]: 

 

Upcoming research on security analytics:

• Next Research: Back to Security Analytics and UBA/UEBA
• Sad Hilarity of Predictive Analytics in Security?

Currect research on deception:

• APT-Ready? Better Threat Detection vs Detecting “Better” Threats?
• Better Data or Better Algorithms?
• Tricky: Building a Business Case for A Deception Tool?
• It Is Happening: We Are Starting Our Deception Research!
• “Deception as Detection” or Give Deception a Chance?

Recent research on SOC:

• Our “How to Plan, Design, Operate and Evolve a SOC” Paper Is Published
• About The Tri-Team Model of SOC, CIRT, “Threat Something”
• New Research Starting Soon: Threat Intel, SOC, etc
• Your SOC Nuclear Triad

 

Miscellaneous fun posts:

• Threats Inside vs Insider Threat
• Can I Detect Advanced Threats With Just Flows/IPFIX?
• No, Virginia, It Does NOT Mean That! (detection vs prevention)
• Jumping Security Maturity FAIL!
• Security: Automate And/Or Die?
• Defeat The Casual Attacker First!!
• On “Defender’s Advantage”

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015.

Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

• Monthly Blog Round-Up – September 2016
• All posts tagged monthly

Monthly Blog Round-Up – September 2016

Here is my next monthly "Security Warrior" blog round-up of top 5 popular posts/topics this month:

1. “Why No Open Source SIEM, EVER?” contains some of my SIEM thinking from 2009. Is it relevant now? You be the judge.  Succeeding with SIEM requires a lot of work, whether you paid for the software, or not. BTW, this post has an amazing “staying power” that is hard to explain – I suspect it has to do with people wanting “free stuff” and googling for “open source SIEM” … 
2. “New SIEM Whitepaper on Use Cases In-Depth OUT!” (dated 2010) presents a whitepaper on select SIEM use cases described in depth with rules and reports [using now-defunct SIEM product]; also see this SIEM use case in depth and this for a more current list of popular SIEM use cases. Finally, see our 2016 research on security monitoring use cases here!
3. “SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?” is a quick framework for assessing the costs of a SIEM project (well, a program, really) at an organization (much more details on this here in this paper).
4. My classic PCI DSS Log Review series is always popular! The series of 18 posts cover a comprehensive log review approach (OK for PCI DSS 3+ as well), useful for building log review processes and procedures , whether regulatory or not. It is also described in more detail in our Log Management book and mentioned in our PCI book (now in its 4th edition!)
5. “Simple Log Review Checklist Released!” is often at the top of this list – this aging checklist is still a very useful tool for many people. “On Free Log Management Tools” is a companion to the checklist (updated version)

In addition, I’d like to draw your attention to a few recent posts from my Gartner blog [which, BTW, now has about 5X of the traffic of this blog]: 

 

Currect research on deception:

• Tricky: Building a Business Case for A Deception Tool?
• It Is Happening: We Are Starting Our Deception Research!
• “Deception as Detection” or Give Deception a Chance?

Recent research on SOC:

• About The Tri-Team Model of SOC, CIRT, “Threat Something”
• New Research Starting Soon: Threat Intel, SOC, etc
• Your SOC Nuclear Triad

Recent research on threat intelligence:

• How to Grow to Strategic Threat Intel Consumption?
• Baby’s First Threat Intel Usage Questions
• How a Lower Maturity Security Organization Can Use Threat Intel? 

Miscellaneous fun posts:

• Threats Inside vs Insider Threat
• Can I Detect Advanced Threats With Just Flows/IPFIX?
• Sad Hilarity of Predictive Analytics in Security?
• Anton’s Favorite Threat Hunting Links
• No, Virginia, It Does NOT Mean That! (detection and prevention)
• Jumping Security Maturity FAIL!
• Security: Automate And/Or Die?
• Defeat The Casual Attacker First!!
• On “Defender’s Advantage”

(see all my published Gartner research here)
Also see my past monthly and annual “Top Popular Blog Posts” – 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015.

Disclaimer: most content at SecurityWarrior blog was written before I joined Gartner on Aug 1, 2011 and is solely my personal view at the time of writing. For my current security blogging, go here.

Previous post in this endless series:

• Monthly Blog Round-Up – August 2016
• All posts tagged monthly