More Log Management Questions - Answered!

I did this VERY fun webcast with WhiteHatWorld this week and a lot of good questions about log management came up. I am answering them here for my readers. BTW, LogLogic product-specific questions can be found on LogLogic website; I am not answering them here.
 
Q1: Is a preferred log management program to consolidate the log data and then allow us to review them?
A1: The answer is "Yes!" for a vast majority of use cases consolidating logs work better than the silo'ed approach. Also, this will be answered in  longer dedicated post within a few days (link TBA).
 
Q2: Is it feasible to use a log management tool to try to determine whether application events / failures are being caused by infrastructure issues?
A2:Wow, fantastic! The answer  to this is "Yes, if you have the right logs collected." In most cases,  to get to the bottom of such issues requires having BOTH application (e.g. PeopleSoft or Oracle) and infrastructure logs (e.g. Windows or Solaris).
 
Q3: What the typical retention schedule for logs which might be required logs for compliance issues?
A3: I wish I can give a simple answer for this, but there is none. Well, PCI DSS makes it simple: 1 year for logs from in-scope systems. Other regulations are not as clear and the numbers, or - more often! - guesses at such number range from 90 days to 7 years and more.  90 days to 1 year is a common retention policy for security (on the longer side of this range) and operationally (on the shorted side of this time range) useful logs. Check this out for a few ideas for long long you might need the logs.
 
Q4: Once you have logged the events, what do you do with them?
A4: Well, I was about to laugh it off since it truly opens up a Universe of questions, issues, challenges, etc. But here is my attempt at a short answer (like, less than a book :-)): a) you collect the logs and now you can search thru them in case you need to b) you summarize them and notice the trends - overall know what is going in your environment c) you analyze them in real time to trigger alerts on "critical" log messages - failures, attacks, etc.  See this slide deck for some useful pointers.
 
Q5: Why do I create a log policy? 
A5: Log policy is a clear and simple document that show what you log on each system (and why): it helps you to configure logging across all the systems as well as helps to know what information you have in your environment (should an auditor ask, for example). A log policy also defines log retention, log review practices, etc. NIST 800-92 Guide to Security Log Management  [PDF] is a good source of info on this subject.
Enjoy!

Technorati tags: log management, logging