Tuesday, September 18, 2007

On Using SIEMs with Log Management

A fun piece which kinda highlights why some people deploy both SIEM (for correlation) and log management (for 100% log collection and analysis).

Interesting quotes: "Can I, algorithmically ahead of time, guarantee that the system will “think” about every event I want it to? With almost every single correlation methodology Ive seen - especially including [SIEM's] default methodology - the answer is a resounding “NO”." and also "This methodology failure means that you cannot go back and do formal analysis on an incident that has passed through [SIEM] without the original raw events and significant manual labor except by sheer luck"

Thus, you have a SIEM for selective real-time security analytics via correlation and risk scoring AND a log management to have a full archive of all logs for incident response and in-depth analysis.

Jack posted a detailed clarification here (and in comments below); still, I'd say that if we live in the world of SELECT statements and limit ourselves to database / structured data analysis, we are by definition missing things that are:
  • filtered away on the agent/collector/connector
  • not parsed into the database by vendor's choice
  • not parsed by mistake / agent bug
  • not retailed for long enough on the database ...

Dr Anton Chuvakin