How to Write an OK SIEM RFP?

Ok, some people think consultants are supposed to make money off helping enterprises write RFPs, but I am busy enough and so it goes. This is what happens if Anton is stuck in a metal tube for 5 hours in seat 1A

Question: How do I go about writing a SIEM or log management RFP?

Quick answer: don’t. This “purchase method” is probably equally hated by vendors and end-users. As somebody who was “volunteered” to help sales folks with 1600 page (yes, really!) and smaller RFPs more than once during my “vendor years,” I can tell you that – with a tongue firmly in cheek:

a) if you ask a vague question in your RFP, you will get either a “Yes” or a nice blurb taken from a random location in vendor datasheet

b) if you ask a question starting with “How…?”, you will get a nice blurb taken from a vendor datasheet

c) if you ask a silly question (“do you have an Albanian language interface?”), you will get either a “Yes” or a nice blurb taken from a random location in vendor datasheet

d) if you ask a question that is impossible to answer (“Can your product handle the high load?”), you will likely get a “Yes” – surprise!

e) if you ask an honest question that might cast a product in a negative light (“will you every lose log data?”), what do you think you will get….?

See a theme emerge here?  Note that I am not trying to imply that any particular vendor would lie in their RFP responses – the term here is “defensible creative exaggeration.” BTW, what do you think happens when a standard enterprises RFP template collides with a standard vendor RFP “boiler plate” response? Boom! The explosion of high-grade concentrated idiocy…  And if you think that I am a bit cynical about this whole thing, than maybe you are correct… making sausage for a long time does distort one’s personality a wee bit

Despite the above, there are two exceptions to this rule of not doing RFPs:

1. You are obligated to do a RFP (government, etc)
2. You’d like to use your RFP as a chance to distill and focus your SIEM/LM requirements.

Let’s address them both at the same time. If you are case #1 above, you should really turn it into case #2.  As you recall (if not, review these posts here), one of the most important things an organization should do before buying a SIEM is to set its own goals, requirements, use cases, etc. BTW, this recent SIEM presentation stresses the same point – esp. see slide 16 and around.  This older presentation has some things to avoid at the product selection stage – see “worst practices” 1-4.

So, based on my experience on both sides of the RFP “interface”, here are some of my SIEM RFP tips:

• Keep it short! If you cannot express what you need in under 10 pages, go back and rethink it. “Every time an organization releases a 500+ page RFP, God kills an intern.” Yes, that very intern who is tasked with responding to that monster, of course.
• Start from your REAL main reason for getting a SIEM, your problem statement – monitor PCI DSS CDE, perform IDS/IPS alert analysis, monitor servers for suspicious logins, protect web applications via log correlation, etc
• Include your use cases – which simply means to describe how you plan to use the system and what you expect the system to do for you. Some examples are shown here  (more high level) and here (more detailed inside the whitepaper).
• Based on your goals and use cases (and that is important!), describe SIEM product functionality that is essential for your mission: agentless collection, bandwidth throttling, rule-based correlation, visual dashboards, trend reports, whatever…
• Include log sources / devices that you absolutely need supported and what you mean by “supported” (e.g. parsed, normalized, categorized, suitable for correlation, covered by default correlation rules, updated promptly when log source changes, etc). This area is notorious for extra-high volume of “creative exaggeration” (“of course we support VidgetMaster 7.2! – via our generic LogMahgic 1.0 (TM) collector … which  dumps log files right into storage without analysis … and then rotates them to oblivion within 7 days”)
• Avoid or reduce the usage of vague terms: ”scalable”, “high”, “flexible”, “effective”, “advanced”, “automatic”, “proper”, etc. Why tempt the other side unnecessarily?
• Clarify most other terms, even those that look clear to you: “correlation”, “reporting”, “keyword search”, “trend”, “responsive”, etc
• Size the environment before writing an RFP, as we discuss in LogChat #2. Baseline your log sources for 2-4 weeks to get your average EPS rate then include both the volume of data and number of log sources that you absolutely need supported. Also, specify response time for reports and searches while you are at it.
• Make phases of your SIEM project clear up front – don’t say “400,000 devices and 4,000,000 EPS enterprise-wide.” I got news for you – you probably will never get there… Be very clear about your Phase 1-2 and simply keep later phases in mind for the coming years.
• Try hard to avoid idiotic statements (sorry!): “Vendor MUST specify their efforts and processes to guarantee that products and services provided will completely satisfy us or exceed our expectations” (quote from a real RFP)

And – hold on to your pants – despite the above effort you should be prepared to take the responses with a HUGE grain of salt. One of my contacts on the enterprise side put it simply: “of course we ignore all the specifics in RFP responses.” With this approach to RFP writing, you WILL still benefit even if you don’t read the responses…

Finally, a more useful question than “how do I write a SIEM RFP?” is “how do I buy the right SIEM for my organization?” Keep this in mind while tuning your RFP. Or just retain me to help – a $20k consulting project is known to sometimes save an organization from  a $500k SIEM failure….

Possibly related posts:

• Log Management Tool Selection Checklist Out!
• "So, What Should I Want?" or How NOT to Pick a SIEM-III? 
• On Choosing SIEM
• I Want to Buy Correlation” or How NOT to Pick a SIEM?
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• Logging, Log Management and Log Review Maturity
• Log Management + SIEM = ?
• On SIEM Complexity
• SIEM Bloggables: SIEM Use Cases
• Whitepaper with detailed SIEM use cases (using a particular SIEM as an example)
• Log Management / SIEM Users: "Minimalist" vs "Analyst"
• All posts labeled SIEM

Log Management / SIEM Users: “Minimalist” vs “Analyst”

Just a random piece of some research project I did at some random point :-) In discussions at RSA 2010 conference, somebody mentioned that SIEM, log management and other monitoring/detection security product users are split into two major categories: one actually uses the product while the other “buys it for compliance” and then eventually uses … as a doorstop, for example.

And I actually had  an old presentation about this that was offered as strategic guidance to my consulting client (a vendor).

Here is that picture and text: two types of SIEM/log managements users that your solution has to make happy:

“Minimalist” SIEM/LM User

•Still evolves from “logs are dirt” to raw collection of log data

•Pure compliance focus – “deliver me from evil… eh… auditors” (or assessors, in case of PCI DSS)

•Collecting logs is the primary “activity”; not even thinking about log review yet

•Checkbox mentality is rampant among that type of user (sometimes, “correlation” is one of the checkboxes, sadly)

•Less mature; needs more hand-holding when deploying the product (might not want any help though…)

“Analyst” SIEM/LM User

•Evolved to “so we have them collected – now what?”; stuck now and not sure how to use “all that data”

•“Compliance+” or even pure security/operational focus; for example, SOC operation

•Using logs – review, analysis, at the very least investigations

•Explore and use logs mentality, focuses on getting the value of the data and solving problems

•More mature; needs more “cool tools”

So, before you plan/design/build your solution, think what is the primary user type… but keep in kind that to be truly successful you might need to entice both.

Enjoy!

Possibly related posts:

• SIEM Bloggables
• The Myth of SIEM as "An Analyst-in-the-box" or How NOT to Pick a SIEM-II?
• Log Management + SIEM = ?

 

Live Test of FUD Value: Pro/Con?

Can you achieve “security goodness” (and common good) by FUD and [possibly] stretching the truth? Let’s debate this one!
The story started here with this letter from an unknown-but-now-infamous PoS (=point of sale aka cash register – for the non-PCI crowd) vendor about using Windows 2000 after Microsoft EOL’s it and no more security updates come. The letter makes an argument that any OS no longer supported by the vendor will be automatically out of compliance. StorefrontBacktalk, that covers retail tech (and payment security), has a good story on this here. They say:

“For your overflowing folder marked “Ludicrous PCI Scare Tactics That Too Many People Believe” comes a renewed effort from some security vendors to say that out-of-date operating systems this year will cause instant PCI non-compliance. ”

So, the statement about “no security patches –> no PCI compliance” clearly does not hold water. It is what is known as “a lie.” Compensating controls can definitely be used in this case and PCI Council even has a FAQ entry about this very subject (quote: “Systems that use operating systems that are no longer supported with new security patches by the vendor, OEM, or developer are not necessarily out of compliance.”)
However!
While embedded and highly “cut down” Windows 2000 can be “made secure” (with whatever definition: secure enough to run while directly connected to the Internet) even in the absence of patches (especially if some whitelisting software is deployed), I personally will trust neither a typical merchant nor a typical PoS vendor to actually do it. If I were a QSA in this case, I’d accept heavy OS changes plus no user access plus host firewalling plus application whitelisting as adequate compensating controls. However, I doubt that this is the case for most of those “W2K holdouts.”  So, IMHO, that outdated stuff “must die” since it puts everyone at risk (think: botnets). If their W2K install dies together with the merchant – then so be it.
Overall, many security folks treat merchants resisting PCI DSS as either stupid or malicious and irresponsible (or both). The merchants, on the other hand, are simply trying to survive and run their businesses. However, at what cost to society? Every one of those W2K boxes CAN BE (and, in many cases, probably IS) used to attack other sites (think: SCADA) and spread malware. Still, is lying the right tactic to get them to upgrade?
For me, this is a hard call to make.
What do you think? “Go FUD!” or “Truth and W2K Rulez!”?
Possibly related posts:

• “Top PCI DSS Security Marketing Annoyances”
• “More On Security Vendors and PCI DSS”
• “A Treatise on FUD”

On SIEM Complexity

I love Laura Ries (@lauraries). Not in that way, but I think she is the source of non-trivial marketing awesomeness (despite her iPhone fiasco).

In any case, here are three pictures from her recent presentation:

Note that on the 3rd picture she uses the line that I’ve heard many times, but never fully accepted: “Changing the reality doesn’t change the perception.”  This is pretty darn profound – and darn hard to accept for folks of the scientific or engineering persuasion.

What is has to do with Security Information and Event Management (SIEM)? You know, “SIEM is very complex.” Everybody “knows” it.

At a conference in Scotland last week, I was leading a roundtable on SIEM  and I started the discussion with this provocative question: “Is SIEM ‘a MUST-have’ or ‘a nice-to-have’?” No offense to my friends at SIEM vendors (you know I love you too :-)), but 100.00% of those who responded chose “nice-to-have” and, respectively,  0.00% picked “must-have.” One person added that it would indeed be “nice”, but only if it will solve problems that his organization is having and at a reasonable cost. And another person stated that “SIEM is very complex” (with the implicit assumption that anything that complex cannot really be mandatory). And yet another got upset that his auditor requested that he “needs to get correlation” without explaining what it was…

BTW, yet another person brought tears to my eyes by saying that “on the other hand, log management IS a MUST-have” for incident response, accountability and other uses, but this is a different story altogether.

So, “simple to use SIEM” is “a luxury Hyundai” or (new meme alert!) “an anti-Unicorn” – you might find it, smell it, touch it, BUT still refuse to believe in it. That, my friends, is why (deep insight alert!) enterprise SIEM vendors don’t have much success with their SIEM appliance offerings (note that I am talking about their SIEM appliances and not their log management appliances; those are doing fine). Remember that “old school” SIEM vendors all started with ambitions of being an “HP OpenView of security” (EPIC FAIL alert!) which exudes pure complexity…

Personally, I’ve seen some decent attempts to make appliance SIEM easier, but my suspicion is that today the theme of “SIEM is complex” is exceedingly powerful and mere reality will not overcome it.

What can we do about it?

First, if we are to believe esteemed Ms Ries, fighting it with facts will not work. Perception will beat reality and you into bloody pulp. So, “but, dude, our SIEM really IS SIEMmple” won’t fly.

Second, we can focus on how amazingly NICE it is, without being a MUST-have. Stop obsessing about your SIEM not being a MUST-have like, say, iPhone or, say, Twitter :-) In many cases other than SOC building, SIEM’s purchase justification is fuzzy at best, despite more than 10 years of concerted vendor efforts with “ROI”, “TCO”, etc. Or such justification is based on a compliance shortcut which then backfires. In fact, “SIEM is not for everyone” might not be a bad slogan to use… or maybe “grow up to SIEM!” BTW,  I’ve heard of cases where SIEM was deployed even before NIPS/NIDS (or at the same time), and this shows that some organizations place fairly high priority on it.

Third, we can we sidestep the whole “must vs nice” debate and focus on specific problems that SIEM solves. You know, well-tuned correlation engine really can tell you about “bad shit” happening on your network! And it can simplify your daily workflow. And enrich logs with a lot of useful context information. And help with incident response (well, log management is better in that case).  If SIEM focuses on solving particular problems and solving them well, then the customer will have to decide whether solving that problem is a must for them or would just be nice. And the whole debate will change in a useful direction!

Fourth, you can focus on log management. Easy, huh? :-) And then decide which of your customers are ready for SIEM and who think of it as “sufficiently nice”  to deploy – then you can have them “grow up to SIEM.” Log management is – or, at least, at some point will be – for everybody who has logs and that is pretty much everybody…

 

Finally, I’d like to invoke a curse of unspeakable evil: if you sell an appliance SIEM that has a  license-based “hard throttle” which causes you to silently (!) drop incoming logs when 10% (!) of the EPS rate [that your customer paid for!] is exceeded and you are reading this post, please die a painful and embarrassing death. You are an offense to common sense! Also: dear appliance SIEM buyers – please ask your vendor what happens if the EPS rate that you paid for is exceeded…

Possibly related posts:

• SIEM Bloggables
• All SIEM posts

Top PCI DSS Security Marketing Annoyances

I figured I’d offer some free consulting (via this blog post :-)) to all those folks that try to market some security thingy using PCI DSS.

As it happens, I just read one too many PCI-focused whitepapers and my annoyance at some vendor’s ignorance boiled over the top. This post is the result:

1. Don’t misspell PCI DSS. It is not “PCI DDS”, and even not “PCIDSS.” BTW, if you want to impress PCI literati, make sure that “PCI DSS” has a space, while “PA-DSS” has a dash.
2. Most definitely, do not pretend that you address ALL PCI DSS requirements for the only reason of wanting to look good.
3. You cannot “automate PCI.” Don’t market it and don’t sell it … or people will call you on it. Admittedly, you can automate a lot of it, but not all (think “policy and process”).
4. Please don’t say “PCI compliancy!” This is just another synonym of “I am a buffoon.” BTW, if you offer “free PCI compliancy”, then you are both a buffoon and an idiot :-)
5. Don’t call QSA (Qualified Security Assessor) “an auditor.” That “A” does NOT stand for “auditor” and PCI on-site assessment is not the same as, say, SOX audit.
6. Further, if you want to market to QSAs or ASVs, spent a few minutes learning what they actually do, which is which, etc. Helpful hint: QSA is not the same as a pentester :-( As per Requirement 11.3, QSA must ”obtain and examine the results from the most recent penetration test to verify that penetration testing is
   performed,” not go and ”just do it.”
7. “Ongoing compliance” theme is awesome. Sadly, a majority of your customers don’t do it like this (to their own loss – this why it is sad). They still have assessment-time rush, pleasing the assessor approach and checklist-oh-we-are-DONE! mentality. If you want to sell continuous compliance, you need to educate them first!
8. Don’t pretend that “PCI is about data encryption.” It is not! If you have to have some simple one-liner, use “PCI is about not having card data sitting around” instead.
9. Please don’t write whitepapers that are structured like this: “section 1: this is PCI”, “section 2: this is our shit”, “section 3": our shit is great” (and, no, it has only very, very tenuous relation to PCI DSS…). Specifically, don’t say “these are PCI-compliant features of our security product.”
10. If you mention cloud computing in your PCI marketing materials, think – very hard! - whether the rest of the content has ANY relationship whatsoever to it…
11. Finally, if you are building the dreaded matrix of how your product magically makes everything PCI compliant, try differentiating between features that directly satisfy requirements vs those that enable somebody to eventually reach compliance vs those that simplify compliance validation. Your users and their QSAs will thank you for it!
    
12. UPDATED (thanks Walt!) There is no such thing as "PCI certified" either. PCI validated is what you are likely trying to say. TO add to this, there are no "PCI validated" products, only companies or organizations.
13. UPDATED (thanks Walt!) Please also forget about "selling into PCI DSS market." This is simply your insanity talking :-) PCI might be a driver, might be some other motivation for buying stuff, might be the regulation du jour, etc. But it .. is ... not ... the ... market.

Overall, unless your goal is humorous relief of people working on PCI projects, please pay attention…

Enjoy!

Possibly related posts:

• All PCI DSS coverage.

Obligatory “added everywhere” posts :-)

• I am not at Qualys anymore and looking for the next big security idea to work on! I might be available for consulting projects…

When “Solutions Before Problems” Approach is OK?

So, they say that dumb overeager salespeople push “what they have” no matter “what the customer needs” – and, more often  than not, end up with BOTH an annoyed customer and some damage to their employer’s brand (yes, it might be all about his/her personal sleaziness, but it DOES damage the employer’s brand!) On the other hand, it is said that a smart salesperson will always inquire about “what problem does the customer have?” and then position/describe his wares accordingly, IF they are indeed a fit for his needs.

I happen to agree with this and think that problems should be visible before solutions are unpacked. Other people mention it as well (recent example from Andy’s blog and its continuation, and then here and again here; read it – its fun!)

However,  what happens when a customer insists: “tell me what ya have!”  There are, curiously, many versions of that, when a customer confronts you with something like this:

• “You guys are experts; tell me what I need to be doing ‘to be OK’”
• “Please tell me which options I should enable”
• “Just give me a document explaining how I can “be secure” using your product”
• “You tell me which one is the best!”

(all above examples are fictitious, but “inspired by true stories”)

I can fight it (and I did fight it on a few occasions in the past, actually, insisting on problem description), but it creates a bizarre paradox:

“Customer is always right” + “problems before solutions” + “customer wants to hear about solutions first” = ?

Just sharing an observation… 

On Idiots and Logs

How on Earth can someone even utter the phrases "scalable log management" and "Microsoft Access for data storage" in one sentence? OMG, OMG, OMG...

MS Access, for God's sake! I wonder if they tried storing logs in Excel spreadsheets?

Yeeeeesh.

PCI Is "Made Easy" -> Hilarity Ensues

Note the "humor" tag on this post!

Read in the recommended order:

1. Mike R lashes out at security marketing message "PCI Made Easy"
   
2. "The victim" of said lashing makes good fun of Mike R (especially read the comments, where an idea to rename the webcast into "PCI Compliance Made Easy, But Not Too Easy As To Avoid Becoming Linkbait For Mike Rothman" is floated...)
3. Mike R responds by professing his love of "baloney" (and salami) and then pans with "No vendor can [make it easy] because security is neither easy, simple or affordable."

All in good fun :-)

But What Does It ACTUALLY DO?

A great follow-up to my post On Security Marketing: Marcus Ranum rants on what stateful firewalls "actually DO." He says:

"One of the fun questions I used to ask my firewalls tutorial
attendees (back in the day) is: What is a stateful inspection firewall? I.e.: what does it DO?

The answers are usually illuminating. Nobody seems to actually know." (more here)

I think if you are buying a security product, you should always know WHAT IT ACTUALLY DOES!

And if you hear, "Oh, it does, you know, 'risk management'!" - you know what to do (hint: it includes a rotten egg, throwing and running away - in whatever order you prefer ...) :-)

UPDATE (12/22/2007): this is NOT about stateful inspection, this is about a) bad marketing and b) opaqueness of some security vendors about what they do. Come on!

Possibly related posts:

• On Security Marketing

On Security Marketing

Answer one simple [but slightly philosophical :-)] question for me, please: if you look at a security company website and, upon reading it, you have no idea whatsoever what they ACTUALLY DO, does it lower your opinion of their technologies?

Or, in other words, are you able to notice good technology through the sleaze-veil of bad marketing?

Here are some fun examples from recent history which kinda explain what I am talking about:

• 'we just added a GRC feature'
• 'we moved into the risk management business'
• 'we simplify threat monitoring'
  

UPDATE: this is in part inspired by this MJR post to fw-wiz.
UPDATE2: another very fun example today from Mike: '"unrivaled SaaS technology that represents a tipping point in enterprise security." Who writes this stuff, and how can I get some of what they are smoking?'

On Marketing Desperation

OMG, ROFL! :-) - Ahhh. Marketing Desperation. | securosis.com: "You can always smell desperation. It has a certain… quality that gently waifs into the nasal cavity, tickling those very nerves that are too oft neglected in our sanitary society. You know, the same ones that pick up the odor of sewer crap. What’s odd is that this smell is extruded not only by the truly desperate, but by those whose self esteem is so battered that they crave every bit of validation they can beg off the nearest passerby.

Today, thanks to Alan Shimel, we see probably the most amusing act of desperation I’ve ever witnessed. One of his competitors bought the Google AdWords for Alan’s name."

UPDATE: now, I am offended here! Why aren't people doing it on my name? :-)

Dedicated to iPhone Users Everywhere ...

A humorous quote from Laura Ries, related to iPhone price drop: "If the iPhone were truly living up to anything near its hype, then Jobs is the dumbest person alive. Lesson number one in business school: You don’t drop the price on winners."

Is it time to say that she was right and everybody else wrong? Or not yet?

Simply Cannot Resist...

Eddie is my friend and all, but I simply cannot resist, but ROFL at that: "... says Eddie Schwartz, CSO and vice president of marketing for NetWitness."

It is official folks! Security = marketing :-)

On Awards

Think this (and more here) doesn't happen in enterprise security land? He-he, think again ...

UPDATE: which security publication is known for such reviews?

On Crossing the Chasm

Fun post from Andy on "crossing the chasm." Be careful, if your success is only due to early technology adopters, you can run out of them :-) And then, if you cannot sell to "normal people," you are f*cked ...

Why Some Vendor Webcasts Suck?

So, I was attending this vendor's webcast related to log management the other day and it sucked ... pretty bad.

OMG, why oh why some vendors think that their customers and prospects are stupid!? How many times a "C{T|M}O hybrid" can utter "changing threat landscape" and "you now have a legal obligation to protect information" (or even such gems of deep thinking as "success requires commitment of resources AND effort" :-) Gee, monkey, I really thought I need only resources, but not effort!) and still retain any semblance of credibility?

WTF? If his mouth is "presenting solutions" to the audience's problems, but his brain keeps thinking "buy our crap!!!," the webcast will likely result in audience being both bored and aggravated ...

I usually take offense when security pros in the trenches call vendor people "vendor scum" (or even here), but after this webcast I think I why now. Mike, please create your "Selling to Pragmatic CISO" ASAP and then jam it up the /.../of these folks!

On CooL

Do I have a secret marketing gene? Maybe that bastard causes me to really like this presentation on "Unlocking Cool." Needless to say, I am also a major fan of "Pattern Recognition."

Marketing Folks Discover Mr FUD :-)

This blog post from a marketing blog has some interesting thoughts on marketing using fear, using an example of North Korea. Fun quote:

'What is North Korea’s Core Purpose: To keep the current regime in power.

How might its Marketing Department make this happen?

Let's start with a unique value proposition (UVP). What differentiates North Korea from its competitors (Iran, the Taliban, Hezbollah, etc.), each vying for its spot in the world marketplace and doing everything possible to maintain power?

Without products, services, money to invest or a charming personality, if I were offering advice I would recommend fear. North Korea is a scary place and not much else. So, our UVP for North Korea is:

"We guarantee that our promise to change the world is unlike any other. Buy now or pay us later."'

But do read on.

On Competitive Differentiation

This paper has some enlightening info on innovation and competitive differentiation. Specifically, it covers three ways one can achieve competitive differentiation

"1. Operational Excellence aka Cost Leadership
Provide middle-of-the-market products at the best price and the least hassle.

2. Product Leadership
Provide the best product, period. Continue to innovate year after year.

3. Customer Intimacy
Provide unique solutions to customers by virtue of intimate knowledge of their needs. "

So, which one is your company doing? :-)