The Mystery of SOC Metrics: What Are We Actually Measuring?

Historical Archive Note: This post was originally published on the Gartner Blog Network on April 16, 2015 by Anton Chuvakin.
Original URL: http://blogs.gartner.com/anton-chuvakin/2015/04/16/the-mystery-of-soc-metrics-what-are-we-actually-measuring/ | Archive.org Snapshot

Every CISO wants a dashboard with SOC key performance indicators (KPIs). But almost every metric currently used in SOC reporting incentivizes the exact opposite of good security.

The Perverse Incentive of Ticket Speed:

If you measure your Tier 1 analysts on "Alerts closed per hour" or "Average time to close ticket," what do you get? You get analysts who rapidly close complex, ambiguous alerts as "False Positive" so they hit their hourly quota, while spending 45 minutes on trivial, obvious port scans.

Rethinking SOC Performance:

• MTTD & MTTR Limitations: Mean Time to Detect only measures what you actually caught. It tells you nothing about the attacker who dwelt on your domain controller for 9 months undetected.
• Measuring Detection Coverage: Are you expanding visibility into lateral movement, credential access, and living-off-the-land techniques?
• The Tuning Feedback Loop: How many alert types are reviewed and suppressed each week to reduce cognitive fatigue?

A mature SOC is not an IT helpdesk. It is an engineering and investigative lab.