The famous SANS Log Management Survey 2010 is out; grab the document here [PDF], some highlights follow below: 
- One of the highlights this year is “there is room for improvement in the ability of log management systems to deliver value from logs being collected, specifically in the areas of searching (where 36% of respondents reported problems), and analysis (where 34% had problems)”
- It seems like collection beast has been beaten back by most organizations ”in 2005, respondents reported their biggest problem was simply collecting log data, in
2010, collection was cited by only 10% as the biggest problem.” Looks like we continue to nicely evolve along the log maturity curve. - ”The top reason for collecting logs was to “Detect/prevent unauthorized access and insider abuse,” with 63% of respondents rating this as most critical.“ Compliance, BTW, got the 2nd place with 41%. BTW, I think this is a fluke since the survey unexplicably had two of the identical options “meet regulatory requirements” and “ensure regulatory compliance." These two together beat the current response leader!
- As expected, “organizations aren’t installing log management systems for their troubleshooting value, yet an increasing
number are finding troubleshooting assistance to be a useful added bonus.” The magic of “Compliance+” model where you buy from compliance (say PCI DSS) and then use for security, troubleshooting, etc is alive and well! - Another long expected finding is that “we’re also seeing a large increase in the amount of log data being collected from a variety of other types of sources.” (there is a curve for that too). Log management is not only about firewalls, routers, switches, IDS/IPS, servers anymore (collection rates for these run in 90-95% range)
- And here is an OMFG: “Last year [2009], 41% of respondents collected log data from homegrown applications. This year, the definition was expanded to include both homegrown and commercial applications, and 73% of
respondents say they’re collecting logs from these sources.” Mind-blowing indeed! I’d never guess that 73% of all organization collect application logs today. I am sure folks at Brannan St are preparing for an IPO, given this news. - Similarly shocking is a finding that ”Logs from desktops are also being collected—49% of respondents report collecting data from desktops.” And so is “48% of respondents indicated they collect log data from “Physical devices (badge access systems, plant control systems).” This is great! This totally makes me think we live in the 21st century :-)
- Even though “The amount of log data being collected is growing at the rate of 15-20% per year,” log retention is up, as expected, “This year, the largest group retained logs for one to two years.” However, survey says that even some PCI DSS impacted organizations store logs for less than 90 days. Sorry, guys, you are just not compliant :-) And don’t blame the council!
- At the end, the survey teams gives some useful tips such as “Know Your Data” and “Know Your Logs,” and, specifically, “Get to learn what normal is for your organization”
So, enjoy the survey! BTW, the webcast (part 2) focused on the use of the data – which is more fun (here is a link to part 1 webcast which focused on collection). The rumor is, however, that the recording for part 2 might not be available… :-)
