NIST EMAP Workshop–Aug 2011

A lot of good work on logging standards as well as standards for the “surrounding areas” (correlation rules, parsing rules, etc) will happen at this first-ever NIST workshop on EMAP.

Please mark your calendars to save the date for an EMAP Developer Workshop to be held August 29-30, 2011 at the NIST Campus in Gaithersburg, Maryland.  We are still formalizing the agenda, but topics to be covered will include:

· Discussion of target use cases and requirements as identified by EMAP working group.

· CEE Overview and in-depth discussion of current issues.

· Discussion of EMAP component specifications and issues/questions for the community.

· Discussion of EMAP roadmap and connections with other efforts within security automation.

We are in the process of standing up a registration page and creating the agenda.  A teleconference line will be provided for those who cannot attend in person.  More details to come in the near future, we hope to see you there.

If you are dealing with logs and SIEM (such as building, or even using the tools) and care about standards, please consider attending – but only if you will contribute!

Possibly related posts:

• NIST EMAP is Out
• CEE posts

NIST EMAP Out

As those in the know already know, NIST has officially released some EMAP materials the other day (see scap.nist.gov/emap/). EMAP stands for “Event Management Automation Protocol” and has the goal of “standardizing the communication of digital event data.” You can think of it as future “SCAP for logs/events” (the SCAP itself is for configurations and vulnerabilities). Obviously, both twin standards will be “Siamese twins” and will have multiple connection points (such as through CVE, CPE and others).

In reality, SCAP and EMAP are more like “standard umbrellas” and cover multiple constituent security data standards – such as CPE, CVE, CVSS, XCCDF, etc for SCAP and CEE for EMAP. As the new EMAP site states:

The Event Management Automation Protocol (EMAP) is a suite of interoperable specifications designed to standardize the communication of event management data. EMAP is an emerging protocol within the NIST Security Automation Program, and is a peer to similar automation protocols such as the Security Content Automation Protocol (SCAP). Where SCAP standardizes the data models of configuration and vulnerability management domains, EMAP will focus on standardizing the data models relating to event and audit management. At a high-level, the goal of EMAP is to enable standardized content, representation, exchange, correlation, searching, storing, prioritization, and auditing of event records within an organizational IT environment.

[emphasis by me]

While CEE team is continuing its work on the log formats, taxonomy, profiles and other fun details of logging events, the broader EMAP effort creates a framework around it as well as proposes a set of additional standards related to correlation, parsing rules, event log filtering, event log storage, etc. The released deck [PDF] has these details as well as some use cases for EMAP such as Audit Management, Regulatory Compliance, Incident Handling, Filtered Event Record Sharing, Forensics, etc.

In the future, I expect EMAP to include event log signing, maybe its own event transport (run under CEE component standard) as well as a bunch of standardized representation for correlation (via CERE component standard) and parsing rules (via OEEL) to simplify SIEM interoperability as well as migration.

Everything public to read on EMAP is linked here (2009), here (2010), here, etc [links are PDFs], if you are into that sort of reading. SIEM/log management vendors, please pay attention - some of you have already started implementation of this stuff….

CEE Log Standard for Dummies!

We wrote a very clear and concise note about Common Event Expression (CEE) approach to log standards. Even marketing people can read it

Quoted from here:

“I'd like to make you aware that the CEE editorial board has published a short overview white paper describing the overall CEE effort including the problems and approaches that CEE is taking.

If you want a quick summary of what CEE is and how the different parts of the effort work, we'd encourage you to take a look at this paper.
The document is available for download in PDF form on the CEE web site: http://cee.mitre.org/documents.html.

And as always, we'd encourage your feedback- please feel free to post any comments to the CEE discussion mailing list at cee-discussion-list@lists.mitre.org. “

P.S. Posted by a scheduler. I am away from the computers – responses to comments will be delayed.

Log Standards and Future Trends

As some of you know, I’ve done this BrightTalk Log Management web conference the other week. My presentation was about “Log Standards and Future Trends.” Here is an embed of my presentation with voice.  If you just want this slides, go check the Slideshare version.

A BrightTALK Channel

Enjoy!
Possibly related posts:

• Log Awesomeness - On August 19!
• CEE Architecture Overview FINALLY Out!
• CEE Update - Aug 2010

CEE Architecture Overview FINALLY Out!

The future of logging is finally here! Common Event Expression (CEE) team releases CEE Architecture Overview [PDF] for public comments. HUGE thanks to MITRE side of team for finally clearing all the hurdles and releasing “our baby.”

The Common Event Expression (CEE™) Architecture Overview document defines the structure and components that comprise the CEE event log standard. This architecture was developed by MITRE, in collaboration with industry and government, and builds upon the Common Event Expression Whitepaper. This document defines the CEE Architecture for an open, practical, and industry-accepted event log standard. It provides a high-level overview of CEE along with details on the overall architecture and introduces each of the CEE components including the data dictionary, event taxonomies, syntax encodings, and profiles. The CEE Architecture is the first in a collection of documents and specifications, whose combination provides the necessary pieces to create the complete CEE event log standard. 

We encourage community members to offer feedback on this document on the CEE
Email Discussion list. You may also contact us directly at cee@mitre.org.

Again, the document is at: http://cee.mitre.org/docs/CEE_Architecture_Overview-v0.5.pdf

The day we were working towards for nearly five years (!) has finally come and more of CEE is revealed to the world! Of course, detailed specifications are still in development and we will release them when they are ready for public review.

Possibly related posts:

• All posts tagged CEE
• My first post on CEE in April 2007

CEE Update – Aug 2010

Reposted (from here) for those who don’t monitor public Common Event Expression (CEE) log standard effort.

“First of all, let me answer the question that is in all of your heads: No, CEE is not dead... After a slow start, the work pace has picked up over the past couple of months. I apologize for the extremely long delay in posting any details regarding the state of CEE.

The CEE Board [that’s us – A.C. :-)] has been working on bringing the CEE Dictionary, Taxonomy, and Syntax requirements together. We have drafted specification documents detailing the CEE Architecture as well as the initial Dictionary and Taxonomy specifications. These documents are the first of the CEE v0.5 (previous versions were internal) document series and have already been shared with some of you for your feedback. We are waiting for the final authorization before they are posted to the CEE website for your review.

This authorization is expected to be granted next week. A follow-up e-mail will be sent to this discussion list as well as the CEE Announce list notifying you that the documents were posted along with the URL where said documents may be obtained. It is important to note that these are only the first iterations of many. We need your help in improving CEE.
 

In addition to helping critique and improve the CEE documents, we are also requesting your help in building and improving the CEE Use Cases. Within the next week or two, you will also receive the beginnings of a list of CEE Use Cases for review.”

Sign up for the public CEE list to see more. Meanwhile, the effort for log standardization makes another step!

The Right Place for Information on Common Event Format

… is this place: http://www.arcsight.com/solutions/solutions-cef/ This is where you can request it via email.

The reason for this post? A lot of Google search traffic for “common event format” lands here on my blog (see picture) and the link above is the correct place to go to.

Now, if those are generic searches looking for some kind of log or event format, then you want CEE (when we finish it, actually)

Possibly related posts:

• “On Common Event Format (CEF)” (summary: CEF serves its purpose well, but CEE is needed to solve broader log problems)
• “Finally, Common Event Expression (CEE) is Out!!!” (OMG, I just realized that our work first became public in 2007…)

Presentation from NIST SCAP

As mentioned before, here is my presentation from 5th Annual IT Security Automation Conference in Baltimore, MD on Oct 26-29, 2009.

LogChaos: Challenges and Opportunities of Security Log Standardization

View more presentations from Anton Chuvakin.

Onto CSI2009 tomorrow; my PCI presentation there is going to be totally awesome :-) It will feature … The Devil!!!

Also, events notes from the NIST event will follow later.

See you there!

Speaking at NIST Security Automation Conference

Just a quick FYI – I will be speaking about log standards at 5th Annual IT Security Automation Conference by NIST. 

The info about the conference:

”Conference Date:

October 27th and 28th, 2009

Location:

Baltimore Convention Center, Baltimore, MD

Registration:

Conference Registration Form

Agenda:

DRAFT Conference Agenda”

I guess if I end up being employed by that time, I will make this my first speaking op at a new employer :-)

Finally, if you are in Baltimore, let’s meet up.

Possibly related posts:

• Log Standards Make A HUGE Step
• RSA 2009 Impressions, Part IV or The Rest of RSA 2009
• CEE White Paper Out (Finally!!!!!!!!!!)
• My older log standards slide deck.
• All posts on CEE log standard.

Obligatory “added everywhere” posts :-)

• I am not at Qualys anymore and looking for the next big security idea to work on! I might be available for consulting projects.

Log Standards Make A HUGE Step

Quietly, sometimes with tiny steps and sometimes with long delays for deep meditation :-), the case for log standardization moves forward. Recently, at RSA2009 and BlackHat2009 meetings, the CEE team has managed to achieve a breakthrough and actually resolve many of the highly debated issues of taxonomy, definitions, formats and even of the future CEE compliance program. Of course, not all of the trouble spots are resolved – the log standardization remains as devilishly hard as it always has been!

For example, here is how far the Common Event Expression has progressed:

“Status
------
MITRE is continuing work on the Common Event Expression (CEE) standard
in conjunction with the Editorial Board and various organizations.
The past months have been spent on the drafting and validation of a
proposal for the initial CEE Specification.
This specification was submitted to the Editorial Board last month.
MITRE is currently working at rolling in the comments received from
the Board, and expect to have a new draft for their review in the next
couple of weeks.
Once the Board has approved the specification, the specification will
be posted to the CEE Community for feedback. We expect this to occur
within the next month. Our goal is to have final proposal that the
community can agree to by the end of 2009. “

and

“Proposed Specification
----------------------
MITRE in collaboration with industry and government offer the Common
Event Expression (CEET) Architectural Proposal for the Core
Components as the basis to standardize event logs from electronic
systems. This paper builds on the CEE proposal summarized in the
Common Event Expression Whitepaper by defining the core components'
architecture needed to enable collaborative efforts in the creation
of an open, practical, and industry-accepted event interoperability
standard for electronic systems.
This specification summarizes CEE and provides details on the
architecture of the core components including the data dictionary,
syntax specifications, and event taxonomies. This proposal is the
first in a collection of documents and specifications. The
combination of the documents and specifications provides the
necessary pieces to create a complete event log standard, which can
be mapped against the four components of CEE: Transport, Syntax,
Taxonomy, and Log Recommendations. “

Finally, CEE and the - previously thought to be lost - cause for log standardization now have a secret weapon: the EMAP, SCAP’s evil twin.

You thought government will mandate which health insurance you’d have? Ha-ha-ha, how about what logs you’d have? ;-) [but unlike health insurance, that would be a good thing!![

Possibly related posts:

• TBA
• All posts about CEE

RSA 2009 Panel on Log Standards

Yes, the words “log” and “standard” in one sentence; alongside each other, in fact :-)

 

Session Code: HOST-304

Session Title: Common Event and Log Standards: Leveling IT's Tower of Babel

Scheduled Date/Time:  Thursday, April 23 02:10 PM @ Purple 304

Abstract:  The IT industry suffers from a lack of standards for event, log, and audit information. Regulatory requirements to retain, protect, and destroy log data continue to increase. Organizations also need better situation awareness and cost control across complex IT security event horizons. The good news is that standards efforts are underway, which this session will detail.

Moderator:

• Daniel Blum, Senior VP, Principal Analyst Burton Group

Panelist:

• Anton Chuvakin, Director of PCI Compliance Solutions, Qualys
• David Corlette, GRC Solution Architect
• Eric Fitzgerald, Senior Program Manager, Microsoft
• Mary Ann Davidson, Chief Security Officer, Oracle

Attendance is mandatory :-)

CAG Out!

OMFG... is this the most ambitious project in security (eh... maybe not :-)) or what?

"Consortium of US Federal Cybersecurity Experts Establishes Baseline Standard of Due Care for Cybersecurity – The Top Twenty Most Critical Controls" (brief)

Here is the first thing I thought about it:

• 2000 - SANS Top 20 vulnerabilities comes out (in its early form)
• 2009 - CAG "Top Twenty Most Critical Controls" comes out.

Now, think:

• Does it mean we are moving towards "control-based" security?
• Does it automatically mean we are moving away from "risk-based" security?
• How many times the term "risk management" is mentioned in a full CAG doc?
  

Finally, some misc highlights:

On vulnerabilities: "Verify that vulnerability testing of networks, systems, and applications are run no less than weekly. Where feasable, vulnerability testing should occur on a daily basis."

On logs: "Validate audit log settings for each hardware device and the software installed on it, ensuring that logs include dates, timestamps, source addresses, destination addresses, and various other useful elements of each packet and/or transaction." (CEE gets mentioned here too)

On web apps: "Test [production - A.C. ] web applications for common security weaknesses using web application scanners prior to deployment and then no less often than weekly as well as whenever updates are made to the application."

On integrity checking: "In particular, most endpoint security solutions can look at the name, file system location, and/or MD5 [yes, MD5, really!] hash of a given executable to determine whether the application should be allowed to run on the protected machine."

In any case, go read the CAG.

UPDATE: LOLCATS on CAG.

It Is With Great Trepidation...

... that I announce that one of the true adepts of the ancient art of log analysis has - FINALLY!!! - joined the blogging world.

Sanford's first post "Why standards?" starts thus: "I’ve often wondered about the viability of broad vendor adoption of a log standard" (read more)

If some of you - you know who you are! - think that my blog is sometimes shallow in its coverage, that it doesn't have enough regexes and SQL commands, you HAVE TO go and subscribe to Sanford's; this will be deeeeep, since Sanford probably forgot more about logs than most of us would ever know (BTW, I am serious). Also BTW, Sanford is a Log Data Architect at LogLogic.

Lately, there have been much more people blogging about logs (I will post some new resources in a bit), but this is truly an event of the century...

CEE White Paper Out (Finally!!!!!!!!!!)

Don't you dare make fun of my "Finally!!!!!!!!!!" in the title. We've been waiting for the release to happen for a "few" months already.

In any case, Common Event Expression (CEE) standard takes a major step forward: our whitepaper is finally public (page, PDF)

"Provides a detailed introduction to the Common Event Expression (CEE) initiative to create an open community-developed event interoperability standard for electronic systems. The paper describes the scope of the problem; explains how CEE’s Common Log Transport (CLT), Common Log Syntax (CLS), Common Event Expression Taxonomy (CEET), and Common Event Log Recommendations (CELR) will provide the framework for a community consensus in log transportation, log syntax, event representation, and event logging recommendations for various log sources and scenarios; examines the benefits and illustrates them in two use cases; reviews CEE in comparison to past efforts; and offers a roadmap to creating the CEE Language Specifications."

We have been working on this baby for a long time, but it was "in approval" for loooonger....

Why Is ISO2700x Hot in UK, but Not in US?

First, something hilarious: I was teaching this brief course on logs overseas and touched upon  a  subject of ISO17799. So, having recently read how many companies in the US were ISO17799 certified, I asked my audience whether they could guess what the number was. One guy volunteered an answer, after some hesitation: "Less then 50%?"

That's "percent", folks :-)

I said to him: "You are right!" and laughed - "It is indeed less then 50!" 50 as in "count" (I read somewhere at the time that 49 companies were certified US-wide)

So, ISO17799 is hot in some countries: UK, Japan, Russia (where it is a basis for a set national standards), many others. But not in the US.

I have long been puzzled about this. What's the story?

The most likely explanation is that every security manager worth his salt read ISO17799 documents and then used the ideas and material in his own policies, procedures, etc. On the other hand, he sees no motivation whatsoever to invest in certification - since nobody is making him do it (no equivalent of a PCI auditor is standing nearby with a big axe...)

Another explanation that due to longer history of security management in the US (compared to other countries), home-grown approaches took root and no external standard will dislodge them?

Yet another hypothesis goes like this: in the US, it is more important to do a good job [managing security] than to be "standards-compliant." Is the opposite true in Europe and Asia? I dunno...

Or maybe ISO stuff is seen as "that Euro thing?" Exotic like a Hungarian chick, but just as relevant :-)

Any ideas? UK scene, any ideas? Do you care for ISO17799 at all? As a useful document to read or a something to be certified in?

View on CEE from Burton

Burton's Dan Blum has posted an interesting article "Prospects Brightening for a Common Event Standard," which covers CEE log standard progress as well as connections between CEE and OpenXDAS effort (which aims to standardize OS audit events - one of the most chaotic domains of the Log Realm :-))

CEE Log Standard Update

OMG, I forgot to blog it! :-) Here is my "CEE Logging Standard: Today and Tomorrow" presentation given at Security Forum & Identity Management Forum meeting at 17th Enterprise Architecture Practitioners Conference by OpenGroup in San Francisco (Feb 2008)

In this presentation I explained the CEE logging standard to the OpenGroup folks and also shared where the standard effort currently is.

Another CEE standard update: public mailing list archives are public and online (thanks, MITRE folks!) - now you can watch all the pillow fights we have on the list :-)

CEE Finally Official

After all the discussions, the official website is up: cee.mitre.org.

Who Benefits from Log Standards? Part II - Application Developers

As I promised, I will post another blurb on log standards following the first: Who Benefits from Log Standards? Part I - Log Management Vendors

Just as the previous one, this comes from the still-upcoming CEE whitepaper (yes, official website is still upcoming as well). Here is the quote that covers the benefits of log standards (in this case, CEE):

"Event Producers (vendors & products)  [A.C. - i.e. platform and application software vendors as well as network gear developers whose products generate logs] will be able to decrease cost associated with logging and reuse log libraries. Vendors could move away from encouraging developers from picking log messages on a closest-fit basis from a limited, product-specific message index. Furthermore, the generation of these log messages could be bases on a single API call. Also product interoperability will increase with the others who speak with the same event expressions, resulting in satisfied customers. "

So, in other words, it is not only the  log management people who will benefit: software vendors will have an easier life with logging; this applies even more to smaller vendor and even in-house IT teams who often (always?) struggle with how to do logging right in their applications ...

Technorati tags: logging standards, log management

Fun Logging Event: InfoSec World 2008 Log Management Summit

I will be speaking at this fun event called "Log Management Summit" by MISTI at InfoSec World 2008. Come over!

Main Conference InfoSec World 2008:

• Wednesday, March 12, 2008 9:45 AM - 11:15 AM

Session I8: The Five Mistakes of Security and Compliance Log Analysis

The Log Management Summit:

• Thursday, March 13, 2008 10:00 AM - 11:00 AM

Session LM3: Logs for Incident Response

• Thursday, March 13, 2008 3:15 PM - 4:15 PM

Session LM7: Emerging Log Standards

• Thursday, March 13, 2008 4:15 PM - 5:00 PM

Session LM8: Panel Discussion: Avoiding Log Overkill: How Much is Enough?

Full info here in the PDF brochure - http://www.misti.com/PDF/174/21737/OS08_bro.pdf