Monthly Blog Round-Up - November 2007

I saw this idea of a monthly blog round-up and I liked it. In general, blogs are a bit "stateless" and a lot of good content gets lost since many people, sadly, only pay attention to what they see today.

So, here is my next monthly "Security Warrior" blog round-up, top posts and comments by topic.

1. Oh, wow! I am proud to announce that one of my favorite pieces is indeed the most popular this month: "Protecting Logs from Admins: A Lost Battle?"
2. My Top11 logging lists take the next spot:  Top 11 Reasons to Collect and Preserve Computer Logs and  Top 11 Reasons to Look at Your Logs (the third list, Top 11 Reasons to Secure and Protect Your Logs, was not quite that popular ...) - and, yes, ONE more list is coming: "Top 11 Reasons to Analyze Your Logs!"
3. Same as during the last few months, the "fallout" from being featured on a high-profile programming site continues to drive loads of traffic.  The topic that got such a huge boost was anti-virus efficiency. Thus, these posts with same theme of anti-virus efficiency were the most popular: Answer to My Antivirus Mystery Question and a "Fun" Story, More on Anti-virus and Anti-malware, Let's Play a Fun Game Here ... A Scary Game, The Original Anti-Virus Test Paper is Here!, Protected but Owned: My Little Investigation  as well as a final entry about my own switch away from AV: A Bit More on AV  and Closure (Kind of) to the Anti-Virus Efficiency/Effectiveness Saga
4. Next on the top list is my other most favorite piece of writing: Ideal Log Management Tool?
5. And, finally, my logging polls! Yes, they are popular too. In fact,  Poll Results: Which Logs Do You Collect? takes the final, 5th spot in this Top 5! :-)

See you in December :-)  

Possibly related posts / past monthly popular blog round-ups:

• Monthly Blog Round-Up - October 2007  
• Monthly Blog Round-Up - September 2007
• Monthly Blog Round-Up - August 2007

 

Technorati tags: monthly

"High-level" vs Fluffy vs Dumb vs Irrelevant?

Very fun emerging blog discussion on executive data presentation:

First: "What the client is telling me is that their execu-types can’t handle 5-6 word sentences and I have to be more concise and drop it down to 1-3 words per bullet. [...] I find it alarming at many levels that the executives running fortune 100 or 500 companies can only comprehend at a 1st grade level."

Second: "While I agree with Jim and share his pain (I have given a few exec-level presentations in my time), I also think there is another underlying cause for this: basically, people do not want their execs to know what is going on."

Third: " ... executives at big companies can't comprehend at a 1st grade level. That's a load of crap. Fortune class executives understand exactly what the issues are. The sad truth is that relative to security, for the most part, they just don't care. So we don't need to dumb down our presentations, WE NEED TO MAKE THEM RELEVANT."

Anton Security Tip of the Day #13: Into the Darkness ... or The Ominous World of Unix Binary Audit Logs

Following the new "tradition" of posting a security tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along and join the initiative. One of the bloggers called it "pay it forward" to the community.

So, Anton Security Tip of the Day #13: Into the Darkness ... or The Ominous World of Unix Binary Audit Logs

In this tip, we will take a peek at one of the most esoteric areas of logging: Unix binary audit logs. Solaris BSM and Trusted Solaris auditing is the least unknown :-) example of it, even though other Unix vendors have similar auditing capabilities - see this for HP-UX Audit and this for IBM AIX audit. Linux kernel audit is also pretty much the same thing. If you look for information on 'Solaris BSM audit logs' , you'd find plenty of tips on how to enable such logging, a little on how to manage/rotate the log files, a bit on how to survive the resulting data deluge and ALMOST NOTHING on what to do with the log data, which is kinda sad :-) After looking at BSM logs for a while, I developed an opinion that nobody has ever looked at them on a regular basis :-)

So, let's assume you enabled Solaris BSM kernel audit for user "root" and few other "interesting" users (there is no per-object logging in Solaris; other Unix'es do have it) via the following commonly recommended per-user configuration in /etc/security/audit_user:

root:lo,ad,fw:no

anton:all,-all:no

jsmith:all,-all:no

This config pretty much records all the actions by the users listed. Now, you have audit files growing like shrooms in you /var/audit. What good does it give us?  First, we need to convert the binary audit files into text - something along the lines of

# auditreduce -A  /var/audit/20071127193515.not_terminated.SunUltra10 | praudit -l > /tmp/sol_box_11272007

will do. Now what? In this tip we will learn how use the audit logs to see who is trying to copy sensitive files off the system.

First, who is connecting out - lets's search the logs for 'connect' calls (if you are using LogLogic for it, use Index Search for this task; if not, grep will have to do, but be prepared to wait). A few recommended searches:

• "connect AND 172.16.10.*"   or "connect AND NOT 172.16.10.*"   (to look for connection to specific IPs or to the outside networks) or simply 'connect AND username'

Here is an example found (with connect, IP and user in bold):

header,103,2,connect(2),,Tue Nov 27 11:36:46 PST 2007, + 193 msec,argument,1,0x4,so,socket,0x0002,0x0002,0x80d6,SunUltra10,0x0016,10.1.1.41,subject,root,anton,other,anton,other,29902,29720,0 1611 172.16.0.173,return,success,0

At this point we already know the user name of the user who run that connecting process since it will be in the results (you can also the user to search as I showed above).

Next, what are those connections - let's try to uncover which programs actually connected (BSM logs don't make that easy). Let's search for process starts in the same time frame:

• "execve AND NOT ls AND NOT <whatever other commands you don't care to see>" will give you a list of started programs.

Example:

header,124,2,execve(2),,Tue Nov 27 11:36:46 PST 2007, + 115 msec,path,/usr/bin/scp,attribute,100555,root,bin,136,1573,0,subject,root,anton,other,anton,other,29901,29720,0 1611 172.16.0.173,return,success,0

Notice that both records have the same timestamps. Sadly, time and parent process ID ( which is in our case 29720) is all that correlates them together.

Finally, what file was affected (i.e. copied off the system via scp in this case) - more digging is in order; we again use the process ID and time. The easiest is to search for a file name or browse all records around the same time frame (might be A LOT!):

• "*secret.zip* AND anton" will work; we can add the above process ID and look for "anton AND 29720" (but expect a lot of data since this is a shell process ID)

For example:

header,135,2,open(2) - read,,Tue Nov 27 11:36:47 PST 2007, + 900 msec,path,/tmp/not-so-secret.zip.gz,attribute,100600,anton,other,0,32743959,18446744073709551615,subject,root,anton,other,anton,other,29901,29720,0 1611 172.16.0.173,return,success,4

What do we know now? This user connected to this system and MAYBE copied this file via, MAYBE via scp. How cool is that? (A: not cool at all, since we are not sure!)

To conclude, if you can avoid dealing with Solaris BSM logs, please do so :-) On a more serious note, you now know why these logs were called "the ugliest logs ever."

Even more seriously (but still pretty humorously), these logs are a classic example of trees that make every effort to obscure the forest, because they record syscalls and not processes or user actions (and connect, execve and read are all logged separately). There are also many, many more idiosyncrasies (and, in fact, idiocies) where these come from :-)

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Security Tips of the Day.

Technorati tags: log management, logging, logs, tips

 

On Security Marketing

Answer one simple [but slightly philosophical :-)] question for me, please: if you look at a security company website and, upon reading it, you have no idea whatsoever what they ACTUALLY DO, does it lower your opinion of their technologies?

Or, in other words, are you able to notice good technology through the sleaze-veil of bad marketing?

Here are some fun examples from recent history which kinda explain what I am talking about:

• 'we just added a GRC feature'
• 'we moved into the risk management business'
• 'we simplify threat monitoring'
  

UPDATE: this is in part inspired by this MJR post to fw-wiz.
UPDATE2: another very fun example today from Mike: '"unrivaled SaaS technology that represents a tipping point in enterprise security." Who writes this stuff, and how can I get some of what they are smoking?'

Fun Logging Event: InfoSec World 2008 Log Management Summit

I will be speaking at this fun event called "Log Management Summit" by MISTI at InfoSec World 2008. Come over!

Main Conference InfoSec World 2008:

• Wednesday, March 12, 2008 9:45 AM - 11:15 AM

Session I8: The Five Mistakes of Security and Compliance Log Analysis

The Log Management Summit:

• Thursday, March 13, 2008 10:00 AM - 11:00 AM

Session LM3: Logs for Incident Response

• Thursday, March 13, 2008 3:15 PM - 4:15 PM

Session LM7: Emerging Log Standards

• Thursday, March 13, 2008 4:15 PM - 5:00 PM

Session LM8: Panel Discussion: Avoiding Log Overkill: How Much is Enough?

Full info here in the PDF brochure - http://www.misti.com/PDF/174/21737/OS08_bro.pdf

Who Benefits from Log Standards? Part I - Log Management Vendors

I received a fun comment to my old post on CEF (here): "But your problem is that if all vendors agree on the same [log] format (and content), your company (LogLogic) won't earn as much money."

This comment represents a common misconception that I've heard a few times before. In reality, nothing is further from the truth - and our efforts to lead the establishment of log standards prove that as well!

I wanted to formulate my response to this, but then I remember that I already had to do it once, for the still-upcoming CEE whitepaper (yes, official website is still upcoming as well). Here is the quote that covers the benefits of log standards (in this case, CEE):

"Benefits for Event Consumers (vendors & products) [A.C. - what we mean here by 'event consumers' are log management and SIEM vendors who "eat up" the logs] will not have to worry about handling a different event syntax and description for each new version of each product, since these discrepancies should be non-existent in products supporting this standard. There would be no longer a need to employ an event mapping team to manually interpret and handle the different events produced by different devices. Additionally, the consumers can produce better, more accurate analysis because of the availability of detailed, meaningful information.  "

So, in other words, log management people won't have to spent so much time and effort fighting artificial challenges imposed by diverse logging formats, fuzzy log contents and illogical log transport options and can find the real, more interesting peaks to climb: automate making meaningful conclusion from log data,  predicting future faults and issues and overall enabling the log data to be used for a wide variety of goals in security, system and network operations and - yes!- compliance as well.

 

Technorati tags: log management, logging standards, logs

A Bit of Ego-Googling

Google for "most popular security blogs"

Risk vs Risk

I was talking to somebody and this following paradox occurred to me:

1. Obviously, all organizations have [some] risk of being successfully attacked by "malicious hackers" (and other threat actors) and thus suffering [some] financial loss (this just reminded of it too)
2. However, few know what is the chance and amount of said loss, so they ignore it (absorb it) and do nothing
3. As a result, all sorts of bad things happen (insert TJX, VA, etc) so governments and other bodies create requirements: the "C"-word is born (compliance)
4. What results is a new threat actor for the organization: an auditor or regulator. Fortunately, this one does have a specific, quantifiable loss amount (e.g. PCI DSS fine) and a more measurable chance of "a successful hit" on the organization
5. Thus, people pay more attention to this new threat factor since they can grasp what the loss might/will be: they choose to act on the imposed requirements
6. What results is that their security is improved by a still unknown amount and some money is wasted as well
7. "Are there yet?" At this point, go back to item #1 and run through this again! Again! Again!

Sounds stupid? This seems to be the world we live in ...

Comment away!

UPDATE: looks like Richard has been thinking about something similar here where he talks that sometimes "achieving compliance may cost more than potential damage" [from an attack].

UPDATE2: Symantec risk report [PDF] mentioned here uses the same approach as Pete says: "To suggest that compliance has its own risk is to acknowledge auditors/regulators as a threat. [...] I am pretty sure they are serious but I suspect auditors and regulators don't see it that way."

UPDATE3: Auditor is a risk? You bet! - says Guerilla CISO here. He then asks a really, really good questions with a really, really bad answer :-) : "Do you think people care about compliance, or do they care more about not being caught out of compliance?"

Technorati tags: security, risk, musings

Poll: What Do You Do With Collected Logs?

Time for another fun logging poll: What Do You Do With Collected Logs?

Vote here!

This is my Logging Poll #3, links to past polls:

• Poll#2 "Why Collect Logs?" (vote here, results so far, my analysis)
• Poll #1 "Which Logs Do You Collect?" (vote here, results so far here, analysis here)

How Do You Predict The Future?!

And you thought all this prediction business is just random opinion vomit ... nooooo. This is science, folks! :-) Read this amazing post (no, really!) from Richard Stiennon - a quote: "There are three types of trend analysis that I have used in the past to predict the future: geographic, temporal, and intuitive."

I admit to being guilty of overusing the #3: "Intuitive trend analysis relies most heavily on the ability of the futurist to absorb all there is to know about a topic and then predict what the logical outcome will be."

Read more here.

Interesting Post: Tracking Install/Uninstall Thru Logs

A quote: "In these days of malware, spyware, and compliance regulations, a lot of admins are looking to track the installation of unauthorized programs, and/or the removal of required programs from client desktops. There are actually several events you can look for in both the Application Event Log and Security Event Log that will help you do this." The Windows events involved are

Read on.

New Paper: "Computer forensics in the age of compliance"

Another one of my "... in the age of compliance" papers published here. A quote: "In previous articles, I've discussed log management and incident response in the age of compliance. It's time to cover a separate topic that has connections to both log analysis and incident management, but is different enough to justify its own article: digital forensics."

Others are linked below:

• Intrusion detection in the age of compliance
• Log management in the age of compliance
• Incident management in the age of compliance

Nobody Is That Dumb ... Oh, Wait X

A perfect, perfect, perfect candidate for my centennial :-) (i.e. number X ...) "Nobody Is That Dumb ... Oh, Wait" piece. Who is the assclown this time? It is "ISC squares" :-) who sue Google AND Yahoo! because of "hosted" content.

A quote: "The most interesting aspect of this case is that ISC2 also sued Google and Yahoo for trademark infringement for hosting content that contained Degraphenreed's impermissible CISSP usage. Specifically, the complaint alleges that Google hosted six blogs that contained the CISSP mark (at least 2 of which contained the term in the blog title), and that Google refused to take down these blogs after the plaintiff's notice."

OMG! Yes, we all saw CISSPs who only know computers in the form of AOL email and who can't spell "I-D-S", but this is in a class of its own. Stupidity definitely reaches a new low with this piece of news ...

Hey, CISSPs, are you embarrassed of what your organization is doing? Or maybe this is taught as "IP protection" in those bootcamps :-)

On "The 7 Reasons why Businesses are Insecure!"

Yes, yes, I know - blogging frenzy and stuff. But I am done ... well, just one little piece which is pretty much required reading: "The 7 Reasons why Businesses are Insecure!"

It starts from a painfully obvious, but elusive: "#1 Management and Governance - If the CEO and Senior Officers of the business do not ultimately own the responsibility and accountability for the security of the business, then it just does not get the appropriate attention."

Even though it covers the very basics, I enjoyed reading it. And so should you :-)

Logs -> Metrics

A brief pointer to Gunnar Peterson blog: note how many fun metrics you can produce from logs (application logs, in this case).

E.g.
"Metric: Anomalous behavior
Source: application logs
Expression: known good vs anomalies"

Many others here.

You'll be hearing more about application log analysis for SOA in the not-so-near future :-)

TJX "Compliant" Until Dec 2008

This needs to be blogged, at least for posterity :-): "The latest is that Visa knew that TJX had “extensive security problems,” but chose to let them off the hook to become PCI compliant until 2009" and "Ironically — while hackers were happily stealing a lot of PEOPLE’s personal and financial information — Visa wrote TJX telling them they would be holding off from fining them as long as they were diligent in fixing the problem."

More details here and here. Irony (and stupidity) in action, PCI inaction :-)

On Satellite Warfare

Wow, this is pretty amazing: "... This month's Discover looks at "eight ways to kill a satellite" -- from space nukes to hack attacks -- and ranks "the threat level of each." The magazine declares "an interceptor launched from Earth, guided to its target by a ground-based laser or radar system" to be 'the gravest threat.'"

.. which is what most people would think. However, the journalist writing the blurb disagrees: "Most of the smart folks I've talked to about this think satellites' biggest vulnerability comes from the orbiters' "need for constant housekeeping from the ground," as MSNBC puts it. Those orders from back here on Earth are eminently spoofable."

So, is it the 21st century or what? :-) Hacking things is now easier than blowing them up...?

Some People Ask Me ...

... why do I hang up when recruiters call me and suggest I take a CSO position with whatever company. Stories like that, make it pretty obvious. Thanks, but no thanks - I enjoy my vendor-side challenges and I prefer to keep it that way...

A quote from the story: " Jim Elste, former security manager at Nevada’s Department of Information Technology, found out in June that there was no system to track the CDs and that the data was unencrypted. [...] Elste says that his attempts to alert the state about the problem led to his being fired. He is appealing his termination, basing the appeal on whistle-blower statutes."

Another Old Presentation: NIST 800-92 Log Management Guide in the real world

I am releasing another one of my old presentations that I don't plan to reuse: "NIST 800-92 Log Management Guide in the Real World." It was presented as SANS @ Night earlier this year.

Enjoy!

Anybody Cares to Share a Copy of This?

Hey, anybody cares to loan me a copy of this doc: "ANSI X9.95:2005Trusted Time Stamp Management and Security"?

I will owe you one "beer equivalent" :-)

On "Passing PCI Subversively"

Very interesting post on "Passing PCI Subversively"

The main idea is: the crappier your scanner vendor ('ASV') is, the more compliant you are :-) And you can help them even more :-) by blocking some of the scans...

Hmmmm....

Stuff like this rarely hits the mainstream media. WTF?!

New Honeynet Paper Out: "Know Your Enemy: Behind the Scenes of Malicious Web Servers"

Another fun piece from the Honeynet Project: "Know Your Enemy: Behind the Scenes of Malicious Web Servers" KYE paper.

Prediction Season is Upon Us! Arghhh!

Remember December? :-) Yes, prediction season is almost upon us (hoooooray!!!). So, I see some early activity on this front: Nick is Reviewing Last Year’s Predictions while some folks scream about Looming Online Security Threats in 2008.

I am starting my prediction recorder now: http://del.icio.us/anton18/security+predictions+2008

See how we all did last year: http://del.icio.us/anton18/security+predictions+2007

Predict on!

Logging Poll #2 Analysis

Oh, shucks! My latest logging poll "Why Collect Logs?" (vote here, results so far) is hugely unpopular :-), unlike the previous one (vote here, live results here, analysis here) - it received only 1/10th of the results!

Why? I am guessing:

• entries too long
• too many entries
• real people hate 'why' questions :-)
  

In other words, I dunno 'why' you hated it :-) Here is the analysis so far, based on just 41 votes (unlike 350 in the previous one)

First, people collect logs mostly for operational reasons - the winner is "We need logs to troubleshoot system/network failures, errors and other availability issues." Combined with #2 reason, that was also operational, security is overall #2 reason for log collection while regulatory compliance is #3.

Second, PCI DSS is the only compliance reason that motivated my readers to collect logs (SOX was a remote second; others - non-existent) - no surprises: PCI DSS directly mandates log management in its Requirement 1o.

Third, as far as security use of logs is concerned, investigations of attacks beats detection of attacks (12:7)

Next poll up soon!

Full Paper on Database Log Management Posted

Finally, my database log management paper (first mentioned here) is released. Head to InfosecWriters and enjoy it! Direct link to PDF here.

Abstract: "Database security have been capturing more and more attention in recent years, even though most of the security issues surrounding databases existed since the first day commercial database systems were introduced in the market in the 1980s.

Nowadays, database security is often seen as containing the following principal components: access control to database software; Structures and data; database configuration hardening; database data encryption; database vulnerability scanning.

It is interesting to see that logging and auditing underline all of the above domains of database security. Indeed, the only way to verify what access control decisions are being made and who views what data from the RDBMS is to look at the authentication logs. Database configuration hardening includes enabling and increasing the auditing levels. Similarly, data encryption might be verified by log and configuration review. And, vulnerability exploitation usually leaves traces in logs despite what some say (the challenge is more often with understanding what the log said and not with having the logs) In recent years, insider attacks gathered more attention than periodic outbreaks of malware; and database logging happens to be in the forefront of this fight against insider attacks. Database systems are usually deployed deep inside the company network and thus insiders are usually has the easiest opportunity to attack and compromise them, and then steal (or “extrude” as some would say) the data."

Read on here [PDF]!

My Next SANS Lunch and Learn:

As I promised here, I am retiring my previous award-winning :-) presentation "Choosing Your Log Management Approach" and replacing it with a new one. Watch out world, here it comes! :-)

"LogLogic Lunch and Learn Presentation - 'Worst Practices' of Log Management
Wednesday, December 12th, 2007 at SANS CDI 2007 in Washington, DC
12:30pm - 1:15pm

Want to learn all the terrible mistakes and pitfalls that await you on the path to log management nirvana? Attend "'Worst Practices' of Log Management" presentation by LogLogic that covers all the things that can go wrong while planning, evaluating, deploying and running a log management solution. Insufficient planning, unrealistic expectations, choosing tools on price alone, lack of logging configuration guidance are among such "worst practices." Each common "worst practice" will be accompanied by suggestions to avoid the errors and do things correctly!"

BTW, SANS also warns: "Please arrive early to guarantee a seat, the lunch session is limited to the first 100 attendees." This is for real, folks! Many of my lunch and learns at SANS were crowded so do sign up!

And, see you in DC!

A Desperate Plea ...

If you are somehow involved with selecting log management tools, running them, building them or doing whatever with the, you MUST run, not walk and attend this SANS webcast called "WhatWorks in Event and Log Management: Simplifying Global Log Management at Rockwell Automation."

You'd think that I am promoting LogLogic here, but that is not the point: the point is that this webcast illuminates the process of choosing a tool [their pre-LogLogic tool!] without sufficient thinking and then regretting it, and then suffering immensely while trying (and failing) to make to work, somehow.

I was almost tearful :-) when I was reading the webcast transcript: these fine folks were pretty much fucked by their previous logging vendor, who promised and never delivered and basically led these folks toward failure through daily frustration and pain ... They are polite in the webcast, but HUGE pent-up frustration just blows through ('It [log management tool] shouldn’t overwhelm you from an administration standpoint' and 'them not being mature enough was a major issue' and 'is it was an incredibly complex application', etc).

This and other experience will also be featured in my new, upcoming SANS Lunch and Learn presentation called "'Worst Practices' of Log Management" (first show at SANS CDI 2007 in DC in December)

And, finally, I know who their previous vendor was, but I ain't talking publicly ... If you do buy from them, I am sorry too say, you ARE an idiot!

What Would be "DRM Good News"?

Fun bit from TechDirt: "Still More Bad DRM News"

Some people never learn - and it is a good thing: a quote: "Blu-Ray discs employ two different DRM technologies. One, called AACS, was cracked back in January. The other, called BD+, was supposed to provide an added layer of "security" and differentiate the format from HD-DVD. Now, to no one's surprise, a company called SlySoft has announced a BD+ crack."

The conclusion is deeply, painfully, truly obvious: "DRM is bad for everyone: technology companies, copyright holders, and their customers."

More On Armed Robots

Kewl quote from this: " We now know that there are robotic cars smart enough to drive themselves around a city. The next step: give those vehicles automatic weapons, of course. "

Of course! :-)

Nobody Is That Dumb ... Oh, Wait IX

Wow, a perfect example for my "Nobody Is That Dumb ... Oh, Wait" series: ultimate, deep, abysmal degree of confusion about "threats", "risks", "attack targets", "exposure" and "vulnerabilities."

A perfect cause to scream "Richard, get them!!!" :-)

So, the "threats" are: AJAX, Google Apps, Social networks, virtualization, virtual words, etc. Holy cow, I am deeply "threatened" by Google Apps - they eat babies and stuff :-)

On a more serious note, these are maybe "emerging attack targets" or maybe "risky emerging technologies," but come on!!! threats?! Gimme a break! How dumb is that!

Protecting Logs from Admins: A Lost Battle?

One of the truly horrible, horrible, horrible :-) challenges of log management is obtaining trusted logs (see my log trust pyramid) of administrator activities (sometimes broadened to cover so-called "privileged users"). Many consider it to be the "lost battle" of logging. However, logging administrator access and actions is more important than ever today (and it is one of the few workable way to deal with insider attacks)

So, let's have a little table where I try to summarize how to protect the C-I-A of various logs from administrators and privileged users (some successful and some very hard to pull off). Note that in case of databases and application, we need to protect the log from the DBA and application admins and not from the underlying server platform admins.

UPDATE: table below might look weird; see the full table here.

	"C" - prevent admins from reading logs	"I" - prevent admins from changing logs	"A" - prevent admin from disabling logging
Standard Unix	Forget it! Maybe stealth logging (sebek)	Remote logging via syslog to another server, append-only log files (via RBAC)	Forget it! But this is logged and thus can be detected (also: stealth logging)
Windows server	Forget it! Maybe stealth logging (sebek for Windows)	Pull the logs ASAP to a central server	Forget it! But this is logged and can be detected (also stealth logging)
Databases	DBA activity log stored outside the database (append-only access)	DBA activity log stored outside the database (append-only access)	DBA activity log stored outside the database
Firewalls and network gear	Remote logging via syslog to another server - no local logging	Remote logging via syslog to another server	Forget it! But this is logged and can be detected
IDS/IPS boxes	Remote logging to another server - no local logging	Remote logging to another server, inaccessible to admin	Forget it! But this is logged and can be detected
Misc enterprise applications	App admin log outside the app (not readable to application user)	App admin log outside the app (only appendable by the application user)	Forget it! But this is logged and can be detected

(just in case, full table is here)

Comments?

Technorati tags: log management, logging, logs, security

Insider Mom Musings

This has some interesting musings on insider attacks and risks, for example: "... my mom is really not capable of launching an internet based attack against a F500 enterprise. However, when she was an office manager, I am reasonably sure she had the ability to do lots of bad things."

Things of that sort help keep the insider angle at the center of attention. It leads me to a scary thought sequence:

a. Is the percentage of unethical people at a major F500 company any different than the entire world? Probably not - or not by much...

b. Thus, for every million of script kiddiez, you have, say, 10 "bad apples" at your org

c. Also, your firewalls/IDS/IPS/NBA/whatever will stop 99%-100% of the above millions

d. Your defenses will probably stop NONE of the 10 "insiders" (and they know where everything is...)

So, who is the bigger risk? I bet on the "evil mom."

Log Management "Strategy:" Built ->Suffer->Suffer->Suffer

Now, I often pondered (here and here, for example) why people chose to build something to do log management and then realize that vendors are in business of doing it for a good reason :-) Namely, that their stuff works perfectly ... until it doesn't. And then mayhem ensues!

Our brilliant field engineer, Dimitri McKay, has to face such situation pretty often and he is frustrated as well: "So tell me why day after day I see administrators and engineers hard nosed about building their own solution? Why do I have to show them day after day that there’s a much more efficient way of managing those logs which will free up resources for a zillion other things."

I would guess that for every million of dollars paid to log management vendors, there are 10 (ten!) millions flushed down the drain by building home-grown logging tools that then get abandoned and decay there on the side of the road ... sad indeed.

More Insider Threat Fun :-)

Interesting post from Richard AND very interesting comments related to insider threat (and also on why so few care about 0nwned boxes on their network - my favorite subject of fascination ...)

"The bottom line is that aside from separation of duties, enforcement of LUA, and monitoring there is not much you can do for an insider threat. Its an almost wholly reactive situation. Of course, vendors don't see it that way :-)"

On "Log Analysis vs. Insider Attacks"

My recent paper "Log Analysis vs. Insider Attacks" is published by the "ISSA Journal" (November 2007 issue). PDF is here (ISSA membership required).

As I mentioned many times, logs are incredibly useful in case of an insider attack investigation.

On "Log Analysis Across System Boundaries for Security, Compliance and Operations"

My recent paper called "Log Analysis Across System Boundaries for Security, Compliance and Operations" is published. It is more on the "basics" side, but useful reading nonetheless. Read on!

Interesting Log Survey Report

Interesting data from here: "More than 40% of large organizations collect at least 1TB (terabyte) of log data on a monthly basis while 11% collect more than 10TB of data each month. Additionally, nearly one-fourth of large organizations collect data from 1,000 or more sources (i.e. security, networking, and IT devices and applications)."

Also: "More than one-fourth of large organizations expect their log file data capacity to "increase substantially" over the next 12 months."

And even: "... log data analysis is used beyond security threat management alone. Business managers, IT operations, compliance administrators, and "C-level" executives are increasingly using log data analysis to monitor numerous business and IT metrics."

And finally: "We are witnessing a paradigm shift where log file collection and processing becomes a discrete service-based architecture and acts as the foundation of a new IT-based data warehousing/business intelligence capability."

Wow, I couldn't have written it better....

Is your log management vendor ready (we are)? Or maybe you are one of those poor misguided souls who are still hoping that you SIEM vendor will solve your log management challenges? :-)

Anomaly Detection Failures?

A fun read from Bruce Schneier: "We've opened up a new front on the war on terror. It's an attack on the unique, the unorthodox, the unexpected; it's a war on different. If you act different, you might find yourself investigated, questioned, and even arrested -- even if you did nothing wrong, and had no intention of doing anything wrong."

Now, narrow it down from society to your network? Do you "fight the different" on your network? Is it working well? Just a thought ...

But Will YOU Detect It?

Thought of the moment, inspired by this here ("80 GBytes of stored data improperly retained by TJX were transferred in this manner. TJX did not detect this transfer."): are you sure that you, my dear reader, would have detected it on your network? Do you log all such activities and send the data to a log management tool?

Ideal Log Management Tool?

The idea came from Jeremiah Grossman (here) when he described "The Best Web Application Vulnerability Scanner in the World" thus: "Within a few moments of pressing the scan button it’ll find every vulnerability, with zero false positives, generate a pretty looking report, and voila you’re compliant with GLBA, HIPAA, and PCI-DSS. Of course, we all know such a web application scanner is simply not possible to create for a variety of reasons."

So, let's imagine the idea log management application.

1. Logging configuration: the ideal log app will go and find all possible log sources (systems, devices, applications, etc) and then enable the right kind of logging on them according to a high level policy given to it (required: God-like powers)
2. Log collection: it will collect all the above logs securely (and without using any risky super-user access ) and with little to no impact to networks and systems (required: God-like powers)
3. Log storage: it can security store the above logs in the original format for as long as needed and in a manner allowing quick access to them - in both raw and summarized/enriched form (required: plenty of hardware)
4. Log analysis: this ideal application will be able to look at all kinds of logs, known to it and previously unseen, from standard and custom log sources, and tell the user what they need to know about their environment and based on their needs: what is broken? what is hacked? where? what is in violation of regulations/policies? what will break soon? who is doing this stuff? The analysis will power all of the following: automated actions, real-time notifications, long-term historical analysis as well as compliance relevance analysis (required: AI)
5. Information presentation: this tool will distill the above data, information and conclusions generated by the analytic components and present then in a manner consistent with the user's role: from operator to analyst to engineer to executive. Interactive visual and drillable text-based data presentation across all log sources. The users can also customize the data presentation based on their wishes and job needs, as well as information perception styles (required: nothing more than a bunch of daring UI designers)
6. Automation: the ideal log management tool will be able to take limited automated actions to resolve discovered and confirmed issues as well as generate guidance to users so that they know what actions to take, when full-auto mode is not appropriate. The responses will range from full-auto actions to assisted actions ('click here to fix it') to issuing detailed remediation guidance. The output will include a TODO-list of discovered items complete with actions suggested, ordered by priority (required: AI + some luck + some user stupidity :-))
7. Compliance: this tool can also be used directly by auditors to validate or prove compliance with relevant regulations by using regulation-specific content and all the collected data. The tool will also point at gaps in data collection as it applies to specific regulations that the user is interested in complying (required: God-like powers)

In other words, this magic black box will have crap shoveled from one side and will have answers to questions about the meaning of Life :-) coming out the other side...

What? :-) Am I nuts? Well, can I dream for a second? :-)

Technorati tags: logs, log management, logging, future, vision

Monthly Blog Round-Up - October 2007

I saw this idea of a monthly blog round-up and I liked it. In general, blogs are a bit "stateless" and a lot of good content gets lost since many people, sadly, only pay attention to what they see today.

So, my 3rd monthly "Security Warrior" blog round-up, top posts and comments by topic.

1. Same as last month AND the month before, the "fallout" from being featured on a high-profile programming site continues to drive loads of traffic.  The topic that got such a huge boost was anti-virus efficiency. Thus, these posts with same theme of anti-virus efficiency were the most popular: Answer to My Antivirus Mystery Question and a "Fun" Story, More on Anti-virus and Anti-malware, Let's Play a Fun Game Here ... A Scary Game, The Original Anti-Virus Test Paper is Here!, Protected but Owned: My Little Investigation  as well as a final entry about my own switch away from AV: A Bit More on AV  and Closure (Kind of) to the Anti-Virus Efficiency/Effectiveness Saga
2. Same as last month, my post introducing Another Presentation: FINAL Full Log Mining Slides made it into Top5. It is indeed a very fun presentation, which summarizes a few years of my logging research. I released it since I got a little bored with researching structured data analysis; a lot of logs we look at today are anything but :-) One of our more enlightened competitors called logs "semi-structured" data.
3. My link to A Sensible Piece on Logging in PCI DSS made it to the top as well. Also check the "PCI  Compliance" book Logging chapter.
4. As I suspected, my little blurb with user profiling ideas called More on 'root' FTP or ""0wned? Again?"" (and Simple User Profiling) is popular. But wait! - there will be more :-) I am writing a little tool that uses LogLogic API to actually do it!
5. Again, in a bizarre twist of fate, the new latest blurb in my humorous saga Nobody Is That Dumb ... Oh, Wait VII made the top list. It proves the point: laughing at others' stupidity is still popular :-) If you doubt it, I have some bridge... ehh... anti-spyware company stock to sell you :-)
6. My third Top11 list Top 11 Reasons to Secure and Protect Your Logs is important to highlight here (it is officially # 6 out of the Top5 for October :-))

See you in November :-)

Technorati tags: round-up, monthly

And Whatever ...

Mike R hits whatever :-) in the bullseye: "...and whatever GRC means. Yes, I know what the acronym stands for, but I don't know what it means."

Richard On "Proving" Security

Richard Bejtlich + long insightful post = a must read.

It takes you on a fun journey from "Are you secure? - Yes (silence)" all the way to "Are you secure? - Yes, we do not have any indications that our systems are acting outside their expected usage patterns, and we thoroughly collect, analyze, and escalate a variety of network-, host-, and memory-based evidence for signs of violations. We regularly test our detection and response people, processes, and tools against external adversary simulations that match or exceed the capabilities and intentions of the parties attacking our enterprise."

BTW, also check out a slightly bizarre discussion of SEM (SIM, SIEM) in the comments.