Dr Anton Chuvakin Blog (Original)

Security writing since the mid-2000s. New posts are mirrored from Anton on Security on Medium.

On Large-scale SIEM Architecture

By Anton Chuvakin
Historical Archive Note: This post was originally published on the Gartner Blog Network on July 25, 2012 by Anton Chuvakin.
Original URL: http://blogs.gartner.com/anton-chuvakin/2012/07/25/on-large-scale-siem-architecture/ | Archive.org Snapshot

How would YOU architect a SIEM deployment for this FICTITIOUS (but real-world-inspired …) large corporate environment:

  • About 30,000 events/second ongoing rate (this is NOT a peak rate, but a rate measured and then averaged over the course of 24 hours)
  • 15 separate sites, most in US but some in Europe and Asia; a few datacenters and a few regional offices (large and small)
  • Log source mix is a diverse blend of firewalls, network devices, NIPS, Windows servers, Unix/Linux servers, web proxies, and also web servers and select database servers
  • Retention policy is 30 days for log data used for operational security analysis and 1 year for searchable full archives
  • The use case is a combination of near-real-time monitoring (via correlation rules and whatever other analytic features the SIEM has) AND incident investigations (via searches, reports and whatever store data analytics the SIEM has) + maybe some compliance reports to spice it up Smile The monitoring efforts will focus on both outside attackers, malware as well as possible insider abuse.
  • A few analysts will be using the tool simultaneously most of the time.

So, here is a mental exercise for you:

  • How would you architect it?
  • Where would you place the collectors? How many?
  • How would you plan storage – single or distributed?
  • Where the main correlation system (or systems) will be deployed?
  • How will the system deal with outages in collection and maybe even storage?
  • How will its performance be tracked over time?
  • How will it scale with increasing volumes (logs tend to grow)?
  • What OTHER information will you need to architect it?

By the way, if you tell me that one appliance will handle the entire environment and no other software/hardware will be needed, a filter will be implemented to send further communication to /dev/null Smile

Related posts: