Dr Anton Chuvakin Blog (Original)

Security writing since the mid-2000s. New posts are mirrored from Anton on Security on Medium.

On SIEM Tool and Operation Metrics

By Anton Chuvakin
Historical Archive Note: This post was originally published on the Gartner Blog Network on June 17, 2014 by Anton Chuvakin.
Original URL: http://blogs.gartner.com/anton-chuvakin/2014/06/17/on-siem-tool-and-operation-metrics/ | Archive.org Snapshot

While some people whine that “their SIEM deployment has failed”, how the hell do they know? I’ve met some folks who spent 8 digits (that’s EIGHT digits!) on SIEM and they are as happy as pigs in clover. They think that SIEM is the best security investment they’ve ever made, for realz.

Measuring SIEM health and operations is still an emerging art, and there is no set of accepted SIEM metrics. The core SIEM team has to define success criteria at the planning stage and periodically check for progress in regard to these criteria (source). However, collecting logged event numbers, correlated event numbers, the related rules enabled, the number of incidents handled and even the number of changes implemented as a result of SIEM monitoring has proven useful for many organizations. These do allow for basic measuring of SIEM tool and program performance. For example, if the volume of collected and correlated logs have decreased dramatically, maybe the tool usage is waning.

Measuring SIEM impact on incident recovery time (similar to the operational mean time to repair [MTTR] metric) and on incident severity also present great evidence of more strategic SIEM success. Even better, a reduced incident discovery window, if observed, can provide a great boost to an SIEM program. The number of cases open for investigation as a result of SIEM and the potential incidents resolved at early stages are useful metrics as well. The number of alerts handled for each analyst allows the organization to track team performance and not just tool performance.

Select SIEM tool metrics:

  1. Event collection rate, EPS (average, maximum – per log source, per type, etc)
  2. Event processing/analysis rate, EPS (average, maximum)
  3. Total log storage, GB (in SIEM, log management)
  4. Log source count (by type, region, log volume, etc)
  5. Alerts triggered count (per time unit, by target, by type, by rule, etc)
  6. SIEM resource usage (CPU, RAM, disk)

Select SIEM operation metrics:

  1. Alerts handled (per analyst, per rule, per target, etc)
  2. Alert response timing [such as time from triggering to review, then to first action, then to closure or escalation (by alert type, by target, by analyst, etc)] <- some call this metric “the only one that matters”
  3. Incidents opened based on SIEM alerts (by time unit, by analyst, by target, etc)

[note the word usage above, these are “select”, not “top” metrics – I feel that I don’t know enough at this stage to proclaim knowledge of the best or top metrics!]

Care to suggest more? Which ones you find the most useful?

Select recent blog posts related to SIEM: