Monday, March 07, 2011

SIEM Resourcing or How Much the Friggin’ Thing Would REALLY Cost Me?

One of the ugliest, painfulest, saddest issues with SIEM is resourcing. Yes, that SIEM appliance might set us back $75,000 in hard earned security budget dollars, but how much more will we have to spend in the next 3 years deploying, integrating, using, tuning, cursing, expanding the thing? How much manpower will the new operational procedures (example) cost us? And if we actually build a SOC or “a virtual SOC”, how much will we have to spend on an ongoing basis to get the value and benefits? In fact, how much will the coffee cost if we have to work 20 hours in a row recovering that crashed SIEM database partition?

These and other questions are super-important for every SIEM and log management project. And the time has come for me to reveal my secret knowledge of SIEM resourcing. OK, that’s a joke – it is not a secret, just a bunch of things that are often unpleasant for many SIEM buyers, users and sellers to hear.

So:

NEWSFLASH! SIEM costs money. “Free” SIEM (example) costs money too, BTW. Let’s try to delve into what those costs are. I will be not-quite-scientific in regards to real “hard money” costs (e.g. software license purchasing) and “soft costs” costs (e.g. staff time costs), but I will try to clearly mark each kind of SIEM cost below.

First, assumptions and limitations:

  • This is NOT “SOC staffing” , but simply “running a SIEM” staffing. SOC implies more processes and more tasks and a broader mission.
  • Assumes in-sourced, traditional “buy and run” SIEM; outsourced, co-managed/co-sourced cost model would look different.

Here is the rough cost model for some of the most common SIEM cost categories:

  1. Initial “hard” costs 
    1. SIEM license costs: base price +per user, per node, per EPS, per CPU (and per CPU core), etc costs – however your favorite vendor charges
    2. For software SIEM, also hardware, OS, database costs for as many servers as you need
    3. If any, mandatory 3rd party software license costs (occasionally, agents, reporting tools, etc)
    4. If chosen, vendor or consultant deployment services costs. If not chosen, staff time for deployment will pop out in soft costs below!
    5. Vendor training or 3rd party training on logs, log management,  SIEM, etc
    6. Additional external storage (in most cases)
  2. SIEM ongoing, operating “hard” costs
    1. Various SIEM vendor services: support (typically mandatory), ongoing professional services costs
    2. Personnel to operate a SIEM: from part of FTE (very small scale, few use cases for a SIEM) to 1 FTE (small appliance deployments) to many FTEs of various roles (+much more for SOC staffing if live monitoring is implemented). 0 FTE for SIEM = SIEM project FAIL with 100.00% probability.
  3. Periodic or occasional “hard” costs
    1. Various SIEM vendor services: professional services, custom development work for device integration (some of these may go into soft costs if done internally – for advanced organization or those experienced with SIEM already)
    2. Periodic staff training on SIEM operation and tuning
    3. Specialty staffing: DBA, sysadmins, node sysadmins, in-house developers for custom connectors, Crystal Reports administrator (yuck!), etc – some of these might go into “soft” costs if “poaching” existing personnel time
    4. Deployment expansion costs: same as initial costs, but for additional systems, software, hardware, etc as you grow; these sneak up really fast if SIEM is licensed using many dimensions such as user+CPU+node+server+something else.
    5. External storage expansion costs – yes, you will buy more storage if your volume grows, and log retention time stays the same (e.g. 1 year for PCI DSS)

On the other hand, here are some of the “soft” costs, such as time expenditures by existing resources:

  1. Initial “soft” costs 
    1. Deployment time for the SIEM project – allocate more time if deploying purely using internal personnel, not vendor or consultant
    2. Log source configuration and integration – this will likely take way more than simply installing the tool. This is what makes SIEM deployment projects go for months in complex, distributed organizations with many silos.
    3. Initial tuning, content creation  and adapting the tool to your environment  (however lightweight it may be)
    4. Training and other staff time costs to jumpstart the operation (Congratulations! You bought ta SIEM. Now you need to operate it…)
  2. SIEM ongoing, operating “soft” costs
    1. Report review and other ongoing monitoring tasks – from 24/7 to daily to weekly
    2. Alert response and escalation; SIEM implies correlation and automated alerting
    3. Other “using SIEM” tasks such as reviewing the dashboards
    4. Uptime maintenance tasks i.e. caring for your SIEM as well as storage – backups, updates, minor troubleshooting, etc
  3. Periodic or occasional “soft” costs
    1. SIEM rule tuning, reports creation, dashboard customization, new log source integration, other ongoing SIEM tasks
    2. Periodic training and related staff time costs
    3. Expansion: same as initial soft costs

As was suggested by a discussion on SIEMusers.org (shhh…the site is not ready for launch yet), it is useful to separate soft costs  that are mandatory FOR SIEM operation from those which commonly arise FROM SIEM operation. The most obvious example is incident response due to increased awareness of network and system activities, delivered by your SIEM.

”Soft” costs that commonly result from SIEM:
  1. Added cost of incident response: more incidents are likely to be detected
  2. Resulting incident remediation costs and even cost of new technologies deployed for preventing the discovered issues
  3. Other department personnel time for dealing with issues revealed by SIEM – the soft costs do leak out of the security department to IT and even beyond (legal, HR, etc).

Anything big I missed that you experienced? BTW, in my experience, I have seen the total cost of a SIEM project (hard + soft) range from 10% of SIEM license costs (for shelfware SIEM “deployments”) to a mind-boggling 20x of license cost.

P.S. Finally, if you want to really annoy Anton, mention “SIEM ROI.” If you do that, I will send you to Gal Shpantser for a mandatory “why he avoids SIEM!” class Smile

Possibly related posts:

Dr Anton Chuvakin