Friday, August 25, 2006

On Common Event Format (CEF)

So, Common Event Format info is out. Do I like it? Do I think it sucks? Does it matter? Here is what I think.

There was a lot of talk about such standards. On our corporate blog we said that "Mary-Ann Davidson, CISO of Oracle, has been promoting an audit log standard for years. Others include a spring initiative by NIST to launch Common Logging Interchange Format. SANS deserves credit for picking up the ball where NIST left off. They brought together a wide range of users and "loggies" to debate standards at the recent log management summit [where I presented as well]. And, Amrit Williams from Gartner also published on the topic - such as his May 2006 Gartner publication #G00139205 on log output standards." And there are always IDMEF (largely RIP), SDEE (dormant? dead?) and WELF (not too relevant and not going anywhere fast) ...

Obviously, I'd also prefer something more vendor neutral, but, what is more important I think we should first go for a content, not format standard like CEF. I usually like to structure it like this - one can standardize logs in:

  1. Format
  2. Transport
  3. Content

IDMEF? CSV? CEF? WELF? You are talking formats here; slashes, dashes, pipes, spaces, tabs and name=value pairs.

Syslog UDP 514? SQLNet? HTTPS? This is about transport, how log bits travel from there to here and back :-)

How about content? There is really no credible log content standard that defines what is in the logs (and what it means) and not how they are formatted, apart from a long dead and forgotten CIEL. And that is where the greatest need is (and has been for a while)!

I can spew XML and CSV just as well as the other guy, but this is not the problem that people our there are facing. Our log parsing capabilities allow us to chew through any of the current log formats, admittedly, with varying degrees of difficulty. However, understanding logs, whether parsed or free-form English is still a challenge for security and operations folks. If some industry group can standardize the log content, the world would be a much better place. And SANS did pick up this challenge so there is hope that this will finally move forward.

So, as a result, everyone would be able to understand, just by looking at each log record, whether it relates to access failures, system changes, attacks, service restarts, system rebooting, account creation, etc via a clear and open log taxonomy, and to do all that without diving into thousand-page documents.

Can it happen? Yes, if we make it happen!

tags: , , ,

Dr Anton Chuvakin