Monday, April 02, 2007

Top 11 Reasons to Collect and Preserve Computer Logs

I've been wanting to create those for a loooooong time and finally - here they are (you can guess I've been on a long flight :-)). Some are admittedly tongue-in-cheek, but useful nonetheless. So, enjoy Anton's "Top 11 Reasons to Collect and Preserve Computer Logs", presented in no particular order:

  1. Before anything else, do you deal with credit cards? Patient info? Are you a government org under FISMA? A financial org? You have to keep'em - stop reading further.
  2. What if there is a law or a regulation that requires you to retain logs - and you don't know about it yet? Does the world "compliance" ring a bell?
  3. An auditor comes and asks for logs. Do you want to respond "Eh, what do you mean?"?
  4. A system starts crashing and keeps doing so. Where is the answer? Oops, it was in the logs - you just didn't retain them ...
  5. Somebody posts a piece of your future quarterly report online. Did John Smith did it? How? If not him, who did? Let's see who touched this document, got logs?
  6. A malware is rampant on your network. Where it came from? Who spreads it? Just check the logs - but only if you have them saved.
  7. Your boss comes and says 'I emailed you this and you ignored it!!' - 'No, you didn't!!!' Who is right? Only email logs can tell!
  8. Network is slow; somebody is hogging the bandwidth. Let's catch the bastard! Is your firewall logging? Keep the info at least until you can investigate.
  9. Somebody added a table to your database. Maybe he did something else too - no change control forms were filed. Got database log management? How else would you know?
  10. Disk space is cheap; tape is cheaper still. Save a log! Got SAN or NAS? Save a few of them!
  11. If you plan to throw away a log record, think - are you 100% sure you won't need it, ever? Exactly! :-) Keep it.

Have more? Feel free to suggest your own reasons below!

Coming soon: "Top 11 Reasons to Look at Your Logs"

Technorati tags: , , , ,

4 comments:

yoshi said...

I have issues with 1, 2, and 11. These rules suggest that the organization doesn't have any sort of documented log retention policy or suggests that you don't know what rules govern your organization. I find this troubling. Not knowing what rules govern your organization is worse than not logging. Keeping around logs because you don't know why is not an answer. Any mature IT organization would already have this defined. Talk to the policy/compliance group or your lawyers. They should already have an opinion on how long logs should be kept.

Most regulations state that you should have documentation that states how long logs should be kept and that documentation is what the auditors will audit you too. There are exceptions to this of course. My last organization had an entire process around document retention. Our logs (fw, ids, network, etc) were included and any changes to logging retention had to be approved by a board. In addition - logs must be purged when you say they will be purged. This was for legal and compliance reasons.

I personally have not seen any reason for keeping around logs longer than 6 months or a year unless there was a documented rule that stated I need to do so for a given system (and there has been). So - know the rules that govern your organization and get an outside opinion.

Anton Chuvakin said...

"These rules suggest that the organization doesn't have any sort of documented log retention policy or suggests that you don't know what rules govern your organization. I find this troubling."

No kidding; these are indeed much more important than just logging. However, I still see a lot of orgs who have neither a well-documented policy about logging nor the logging itself. The "Top 11 Reasons..." are meant to prod this into action!

Indeed, a site-specific documented logging policy driven by well-understood and applicable regulations trumps any list you can see online, including this one :-)

Anton Chuvakin said...

"I personally have not seen any reason for keeping around logs longer than 6 months or a year unless there was a documented rule that stated I need to do so for a given system (and there has been)."

Indeed, most operationally-driven log storage almost never exceeds 6 mos and 1-3 months is even more common...

Anonymous said...

true. 6 months is a long time for a log. But in the same vein as "keep them because you don't know that you won't need them" - why not keep forever(ish)?

If 6 months of logs fit on a DVD - write them to it. File it. Maybe I'm in a small business but the idea that 6 months of logs would fill a CD boggles me, surely 4G is enough to permanently archive 6 months of ascii. Doing so is trivial in terms of time and cost.

I suspect if you've got more than a DVD's worth of logs then you are in a business that won't notice the cost of a permanently archived tape every 6 months.

Dr Anton Chuvakin