This is my PERSONAL blog, as as of August 1, 2011, it focuses on personal matters and various things I find to be fun.
Monday, June 30, 2008
Evil BETAs Attack!
Yes, it is available today (as beta maybe - but then again "all software is beta").
Yes, it is free.
Yes, it works ... well, when it does.
Yes, you can trust, say, your email to it (who cares when it is made public, really! :-))
And then the same programmer mindset trickles up to the software that controls your aircraft engine.
That WAS you.
The more I think about it, the more I like the idea of software manufacturers' liability (succinctly described in "Geekonomics"); I suspect that everything bad that might come with it will probably still be better than what we have now (or will have soon...)
Fun Reading on Logs and Log Management
I am amazed (no, AMAZED!) about how many people now write about logs; it is definitely not "the original logging evangelist" anymore :-) Here is a quick sample, useful for those struggling with logs (aka "everybody" :-))
- A very fun read from Patrick Mueller (ex-Neohapsis now turned lawyer): "Facing The Monster: The Labors Of Log Management." I am happy that log management has been finally granted a monster status :-)
- I am happy to see that one of the "five questions to ask before sending your data in the cloud" is "Will I have access to logging and auditing data?" This is indeed a big deal (well, it will be soon) and you will be hearing more about this. I call this "a case of log ransom," since you might need to pay the ransom to see what is "yours" - the logs
- Again on leaving [some] logs behind. Remember, the point is not that "collecting all" is a good idea, it is that figuring what to pick is IMPOSSIBLE, while "collecting all" is simply very hard :-)
- This is hot stuff: "Ten reasons you will be unhappy with your SIM solution" (no, I didn't write it :-), but this is mine)
- Why HA for log management from our star engineer. Those thinking about the reliability of their logging systems should read it.
- Fun info on web server log analysis for different purposes.
- "Why Logs and Logging Matters - Part 1" and "Why Logs Matter - Part 2, A Letter" present really good intro logging for compliance and other purposes (even specifically saying "what you do with the logs that matters.")
- "Smart Business Leaders Support Effective Log Management Practices and Necessary Resources" from Rebecca Herold is a nice basic piece, especially for those outside the circle of logging literati.
- More from Sanford on logging standards: "Drawing Lines", an awesome post indeed.
- A MUST read on SIEM and log management from Greg Shipley (I promise this is a coincidence! :-)) In this piece, Mr Neohapsis drop kicks more than a few "latest generation" SIEM tools. Guess which product review mentions "pain" 3 times on one page :-)
- Finally, this is also worth a read: "Ode to Log Management" where Mr Baum laments logs being pigeonholed in to "another IT management tool" silo despite their broad relevance. He is right - but focusing on one use case after another works...
Thursday, June 26, 2008
You Are "A Security Idiot" If ...
- Misspell both HIPAA and SOX (how the f does one misspell SOX?)
- Confuse "risks" and "threats"
- Think that "Trojan is a vulnerability" AND "DoS is a vulnerability"
- Quote "Insiders are 80%" without thinking for one darn second
- Think that a loss of "$20 million is catastrophic to any company"
- Talk about "NIST compliance"
- Consider IDS a network security control
- Shout that "perimeter is dead"
To be explained later :-)
Wednesday, June 25, 2008
11 Signs That Your SIEM Is A Dog or "Raffy, You Killed SIM!"
Prerequisite: read this (thanks Raffy). Stop reading right before you reach the last line though :-) Then maybe read this too (thanks anonymous).
Next, insert appropriate morbid jokes <here> for "IDS is dead", "NAC is dead", "GRC is dead", everybody is dead... WTF? Are we at the cemetery or what? Is "dead" dead? Yeah, but it came back as a zombie :-) So, "dead" is a "living dead" "dead" now. Ha*3.
Finally, think! Why were you thinking of buying a SIEM? 'Cause the big "G" in the sky said so? And while you are thinking, check these fun points out:
- Does your SIEM require 17 beefy servers to operate? How many gallons of foreign oil have to go up in smoke to power that mammoth up? And you know what happened to mammoths, don't you?
- If your "high-performance" SIEM appliance can only run 5 correlation rules at the same time, what "high" do they mean, really? Hold this thought....
- Is five field engineers, two developers and CTO enough to install it? Who else needs to help? Ah, sorry, I missed the DBA :-)
- Do you know when "If CustomVariable17 = Value5" condition matches? Will you still remember it in a year?
- Can you tell "taxonomy" from "ontology"? You can now? Good for you. Are you more secure now? More efficient? Compliant?
- How many shifts of security analysts do you have watching the shiny consoles 24/7? If zero, then why - oh - why those consoles are running in the first place? "If a tree falls..." - you know how this one ends. Correct! You get hit by the bough.
- When was the last time you built a custom agent for parsing and normalizing, say, SAP logs? Did it work? What did you do after it didn't? Cried? And did it help? Then a burly vendor SE showed up, charged you $37,600 and left? Happy now?
- Do you automatically correlate IDS/IPS alerts with vulnerability data ... for client-side attacks? Really? :-)
- There are dozens of firewall, IDS/IPS, router, etc brands, each with its own log type. This is actually simple! But there are thousands upon thousands of applications in use today. Some have logs. All are different. Care to build rules for that? Now you finally know why SIEM vendors don't parse their own Java logs (no shit!)
- Do you know what "threat x vulnerability x random()" equals to? Yup, it still equals random(). Automated prioritization, you say?
- Do you know why some SIEM vendors are migrating to IT GRC now? So they can go and die there ... quietly.
All in all, I have to agree with Raffy to a large extent! The world has evolved - and SIEM has not. It might not be dead (as old attacks and defenses never really die and large organization still build and man massive SOCs where SIEM is "a must"), but in this age of web application hacking, CSRF and XSS, phishing, PCI DSS, massive bot armies, client-side 0-days, stealth malware, etc, paying $x,000,000 for a pile of ugly Java code is insane ... As a result, SIEM has greatly diminished in importance and has become just one small thing you might do with logs and some other data. What made it so? Mostly implementation complexity - but a slew of other factors mentioned above as well.
So, consider this instead:
- Compliance? "Sorry, buddy, you need this for compliance, not that. "
- Want to simplify your incident response? Get log management and fly through all your logs, not crawl through some of them.
- Have a very real need to dig into your logs for troubleshooting or tracking that pesky user? Log management works.
Now, what if you have a latent and vague desire to "correlate something" and a million nice greenbacks to flush down the drain? OK, go get your SIEM toy for $780,000 + 20% maintenance/year ... a true bargain (price valid today only).
Finally, I would like to end this on an optimistic note. Do we need more intelligence to analyze the log data we have collected? Of course! Do we have a widest set of log use cases from today's security to tomorrow's regulations? You bet. And, for you Raffy, I'd add "... we also have other data to analyze together with logs." So, can we "reinvent SIEM?" Yes, I think so! It just hasn't been done yet ... For now, just use log management.
Will Idiocy Ever End?
I lamented and ranted and rambled about it (here, here, here), but I am still shocked. I come from academic background myself and it is unthinkable to me that a research physicist today will write a thesis on 2nd Law of Newton or will set to prove that objects tend to fall down while dropped. Or that they, in fact, "fall up."
However, that is the type of stuff I see in academic security papers that I occasionally get to review. Based on our FIRST conversation, other people who happen to retain ties to academia are reporting the same: research work that confuses "phishing" with "fast flux networks" (thanks Jose), inventing a new intrusion detection "paradigm, " and all sorts of other bizarre crap continues to be cooked and submitted to publications.
When will this end? Why can't you people tackle REAL problems? Or at least useful and hard classic problems? Or, at the very least, learn WTF is going on the real world of operational security before you do ANYTHING? The maybe you stop saying things like "in general, IDS is considered to be a security tool" as if it was some kind of Zen wisdom (a quote from a pathetic excuse for a paper that I reviewed recently...)
Sunday, June 22, 2008
Will You Buy It Yourself?
Is somebody insane? :-) What has the world come to?
Friday, June 20, 2008
Instant Classic: A First Coffee Machine Hacked Remotely
Hack my coffee -> security taken seriously; screw ROI.
So, CAN We Have DLP?
Can we have DLP - data leak prevention?
Well, can we have IDS? How about IPS? Can we really "prevent intrusions?" Can we really "control access to our networks?"
The answer to "can we have DLP?" is actually pretty simple: if you think "DLP = box that prevents all data leaks" (and you also think that deploying IPS will "prevent intrusions"), then we can't. Forget it.
But blame the idiots who called it "leak prevention" - if you think that "DLP will prevent all leaks" - sorry, but you are one of them! :-) If you treat "L" not as "leak" but as "loss" and hope that "DLP will prevent all data loss, whether intentional or not," you are an even BIGGER one.
So rambling about "Can DLP Really Stop All Leaks" is pretty silly. No, it can't. Pondering "Is DLP Possible" is just as silly. No, complete prevention of all leaks is impossible, with OR without DLP technology. Go read Mike R instead :-)
Why seemingly smart people behave in such childish manner? I dunno. Scratch all that. Instead ask:
Is today's cutting-edge DLP technologies USEFUL?
And the answer is "Hell yeah!"
If you see how much "fun" sensitive content goes over email (corp and personal web-based), gets uploaded to forums, channeled over IM file transfers, FTP'ed somewhere, you'd scream for one of these boxes. Accidental leaks, email address typos, non-malicious leaks, blatant disregard of security policy for the sake of "productivity", even phishing, "wholesale data theft" and amateur "employee hackers" probably account for 10x (100x?) more damage (in direct losses, brand damage, embarrassment and - yes! - non-compliance fines AND loss frequency) than "uber-hackers" (who might indeed go thru your DLP box like hot knife thru butter.) And if an advanced DLP box does one day stop some determined insider theft, that's just icing on the cake.
That is why smart people don't call it "DLP" - they call it "content monitoring and filtering." This sounds much less sexy, but much more useful. The boxes that will show up on your doorstep will still have "DLP" labels, but what they will do for you is really content monitoring and filtering. And even though it will not stop all data theft, DLP box will likely prove useful more than once...
Finally, all rants about any preventative AND monitoring technologies should really end the same: go refresh your incident response plans.
Possibly related posts:
It Is With Great Trepidation...
Sanford's first post "Why standards?" starts thus: "I’ve often wondered about the viability of broad vendor adoption of a log standard" (read more)
If some of you - you know who you are! - think that my blog is sometimes shallow in its coverage, that it doesn't have enough regexes and SQL commands, you HAVE TO go and subscribe to Sanford's; this will be deeeeep, since Sanford probably forgot more about logs than most of us would ever know (BTW, I am serious). Also BTW, Sanford is a Log Data Architect at LogLogic.
Lately, there have been much more people blogging about logs (I will post some new resources in a bit), but this is truly an event of the century...
See You in Vancouver at FIRST 2008
It is a great pity that I won't be able to spend more time at the conference as I have another one on Tuesday :-( - a "can't miss" kind since it is related to CEE.
Also, Honeynet members in attendance are planning a meet-up. Come find us there Monday night...
CEE White Paper Out (Finally!!!!!!!!!!)
In any case, Common Event Expression (CEE) standard takes a major step forward: our whitepaper is finally public (page, PDF)
"Provides a detailed introduction to the Common Event Expression (CEE) initiative to create an open community-developed event interoperability standard for electronic systems. The paper describes the scope of the problem; explains how CEE’s Common Log Transport (CLT), Common Log Syntax (CLS), Common Event Expression Taxonomy (CEET), and Common Event Log Recommendations (CELR) will provide the framework for a community consensus in log transportation, log syntax, event representation, and event logging recommendations for various log sources and scenarios; examines the benefits and illustrates them in two use cases; reviews CEE in comparison to past efforts; and offers a roadmap to creating the CEE Language Specifications."
We have been working on this baby for a long time, but it was "in approval" for loooonger....
Wednesday, June 18, 2008
Network + Security = ?
"most people believe that networks are becoming harder to secure every day"
Am I missing smth?
Tuesday, June 17, 2008
Fun Reading on Security - 4
Instead of my usual "blogging frenzy" machine gun blast of short posts, I will just combine them into my new blog series "Fun Reading on Security." Here is an issue #4, dated June 17, 2008.
So my next iteration of fun reading on security, logging and other topics.
- "Security-as-control" vs "security-as-assurance" - a very useful idea (more here), which is often confused with bad results (e.g. "secure" software = has password authentication OR has has no overflow bugs)
- Rich Mogul grabs GRC by the balls and kicks it, hard, again. A Burton Group guy comes and helps him by doing a nice roundhouse kick in its butt. Still, it doesn't die, as more people kick it ... Maybe 'cause Andy "loves or hates it?"
- Good advice from Andy IT Guy: "We need to step back from time to time and evaluate what we are doing to determine if it still makes sense." (more)
- BBC on cloud security, actually interesting. More on the same subject, albeit with a dumb name
- Breach disclosure laws and security study by CMU, that SANS called idiotic ("What a silly study. It measures the wrong outcome. What matters about data breach notification is what it does to the quality of defenses.") AND "badly flawed" as well. More fun comments on it are here. More discussion of this complicated subject. Rick kicks it too here.
- Along the same line, "Data breaches at retailers are the top cause of credit and debit card theft, accounting for about 20% of all incidents." Wow!
- "The biggest issue in both Audit and IT is a lack of strategic thought." (maybe) When I read it, it reminded me of the old wisdom from Ms Trunk: "if you think you are a 'strategist' - check maybe you think that 'cause your execution sux"
- A very fun read: "Facing The Monster: The Labors Of Log Management." I am happy that log management has been granted a monster status :-)
- Role of compliance for SCADA security puzzles me: think about it - you need a law to make people protect systems that control utilities EVEN THOUGH you already demonstrated (kind of) that hackers can explode generators remotely. So, people fear fines from regulators more than exploded power generators? Yep.
- Is it time to regulate the security of cloud computing?
- "How to Sell Security" by Bruce Schneier - a MUST read. BTW, FUD is NOT dead, and won't be dead. Ever!
- OMG, this is huge and will grow: PCI Compliance and Virtualization (think "only one primary function per server" mandated in PCI). Same source on costs of PCI (also fun!) - still, IMHO, PCI is cheaper than properly securing your environment ... And while we are on the subject of PCI, check out Rich's "The Good (Yes, Good) And Bad Of PCI" and the discussion that followed.
- New wave of compliance is incoooooooooooooming. Take cover!!!
- Please shut up about ALL security being rolled into the network. Hoff says it best here. If you want to join this bandwagon, say "all NETWORK security will be in the network." (you'd probably still be wrong, but less embarassed :-))
- Finally, some "Unintentional hilarity" from David: this is sooooo the world we live in :-)
Thursday, June 12, 2008
Ideal Tool to Solve Real Problems ... of the Near Future?
Remember my write-up about an ideal log management tool?
Somebody asked me: "That's great that you have such a clear vision of a future log management technology - but tell me first what future business problems will such 'ideal tool of the future' solve?"
First, I laughed and said: "Dude, look around, will ya? :-) There are plenty of log-related problems today which we are not even close to solving. We need to solve the problems of today first, before we can get to solving the future problems..."
So, what I consider to be the biggest log-related problems of today?
- Not knowing what to log - whether for compliance, tracking attackers or troubleshooting system problems. Remember all the comedy about "Tell me EXACTLY what to log for PCI?" If not, reread it!
- Log volume - there is too darn many log messages (seriously, 100,000 each second is a lot of log - but there is more at large companies!), and, which is worse, a lot of them are of unknown value to the users (might be useful, might not - but you never know in advance); thus, log clutter networks, systems and brains of security/system analysts.
- Log diversity - logs all look different (at least while standards are being developed) and no single person have the skill set to understand more than a few types. PIX admin groking SAP logs? No way!
- In light of the above, just pure bad logs are also a major challenge - logs that miss a key piece of info (like the infamous "login failed" without the username...) or are useless in some other way are sadly common.
- How about getting the logs from all the nooks and crannies where they are stuck (think application logs here) - it is a problem if you want to achieve (expand, rather) your operational awareness of applications.
- Finally (not really, the list can go on and on), making sense of logs in an automated fashion is still a #1 challenge (IMHO) - we are getting better creating tools for humans to go thru logs (via reports and search), but log->conclusion process still requires a human, and a darn smart one.
Now, when you read the above think "end user", not "log management vendor" challenges (I plan to post about these later). My idea of an ideal tool will seek to solve these and others.
Along the same line, this picture from 4th SANS Log Management Survey shows how people perceive the logging challenges:
as well as my logging challenges poll (analysis here):
Now, let's think of logging problems of the near future, say in 2 years.
But you'd have to wait for the next post for this :-)
Tuesday, June 10, 2008
Thanks For A Wonderful T-shirt!
I would like to thank the splunk team for sending me the wonderful t-shirt (picture of its front side is on the right)
Thursday, June 05, 2008
Curious Factoid on Hated Creations
There is a lot of fun and useful material in both, but here is a little painful factoid: 73% of people who created their own home-grown log management tools hate them :-)
Possibly related posts:
It Changed My Life: My Review of "Geekonomics"
As I am sitting here - yes, you guessed right! - on a plane, I cannot stop thinking about the book "Geekonomics"(book site) which I just finished reading (earlier impressions here and here). The way it ends, BTW, just kicks you in the balls, hard (look up what Mr Petrov did on Sept 26, 1983 and why, if you are already curious)!
Call me easily impressible, call me naive, darn, call me "out of touch with current security issues," but this book struck a major, major chord with me. It really did.
Now, I have experienced as much poor quality and insecure software as the next guy. I am never ever surprised about some feature in MS Office (or other application, really) just flat out not working or not working as expected or not working every time.
I suspect that, by now, every human on Earth who ever laid their hands on a computer knows:
software = might NOT work.
Now, we expect roads, bridges, toasters, chainsaws, bicycles, cars (until they put software in them...) to work and work they do. And if they don't - the company who manufactures them usually makes them work for us fast - or goes away, cut down by the "benevolent" axe of capitalism. Now, software is totally different (my thinking about this one).
And everybody knows it. But nobody was brave enough to take a hard look at this and analyze how that simple fact affected, affects and will affect our society. And, for my extra-paranoid readers: "... and how it might end that very society."
This book might not reveal any secrets about how software works to an IT professional (it will reveal how law works though!), but it will explain why bad software is everywhere, why we are stuck with it, why it will not improve by itself and - sorry for a hysterical note here! - how we might all fucking die because of it. It then unemotionally predicts why more people will certainly die because of bad software. It studies the complicated dynamics of today's software market such as who is more at fault for bad software - buyers who agree to buy or vendors who make it (or both). It also suggests that many of today's regulations and compliance "thingies" are a little misguided (e.g. in a battle a PCI DSS-compliant enterprise and a 0-day-wielding hacker, any sane person will bet on an 0-day). It is also very well-written; it won't bore an experienced IT or security pro and it will not overwhelm a mere IT user.
First, it explains why the software is the "foundation of our civilization" today, and how it will be more so in the future. Next, it casts a look at "innovation" and ponders how innovation-driven software development relates to the fact that users don't touch 90% of features of a typical software. In the third chapter is presents the view of the "0wned world" where "only the stupid [cybercriminals] get caught." Next chapters looks at how government oversight works in other areas (e.g. FDA), how it might work - and how it might fail (and did fail in the past). While doing it, the book dispels the "government will just make it worse" myth (basically, because some things are really bad and quickly streaming towards worse already). The amazing chapter 5 gives the clearest explanation of litigation (torts, etc) that I have ever seen (the book is worth reading just for chapter 5 alone!). Chapter 6 takes a super-pessimistic look at open-source software (no comment - just read it). Finally, several possible future - "the way forward" - is discussed.
Another thing I would like to mention about this book is that a reader should keep in mind that it is not about "insecure" software: it is about bad quality, unsafe software in general and less about "hackable" software. The author chose to not make this distinction very clear, perhaps on purpose.
So, everybody in software business, security business - in fact, just everybody who uses a computer - MUST READ THIS BOOK! Seriously, understanding the point made there might be a matter of life or death for some (all?) of us.
As a conclusion, if you want the visual image of the future to end my review, here it is: it is not "Terminator" future (where machines kill people out of evil) that we must fear and work to prevent, but "Robocop" future (where they do due to software bugs).
Go read the darn book! And support liability for software manufactures. Also, in a few days, check this out (not yet but hover over the link to get a preview...)
New Thing? Phishing for Airline Miles?
Something new every day? Or phishers are getting desperate and will now take miles, not only cash? :-)
Tuesday, June 03, 2008
Logging Poll #8 Analysis: Needed Log Context
In my poll #8, I asked a question: what information is most important when analyzing a particular log record. Live results are here and final count is also below:
What can we conclude?
First, good documentation never hurts :-) - indeed, the most popular information to look for when facing a new log record is documentation on what it means. While some software vendors are great in this regard, many other don't bother documenting their logs or document them only when customers complain.
Second, I was not sure that the second popular choice would be "Other logs from about the same time (this and other systems)." This strongly points at huge value of cross-device log analysis (see this recent log entry on that), where all the logs are consolidated and analyzed together (it goes without saying that time is synchronized OR at least corrected across those logs). Indeed, if you are confused about a log and documentation is not available, reviewing "what else was/is going on?" is smart. Trusting log time stamps across many systems is also key for that.
Third, having IP addresses in logs is great, but human-readable names are better: IPs in logs needs to be mapped to DNS or Netbios names. Indeed, given that often such names reveal where the system is, who might own it, what its function is, etc this information is not just a mapping, but true log information enrichment.
Fourth, so, what's next? The above 3 top responses are indeed universally useful, but the next choice digs deeper: flows, packets, connections and other network information does complement logs and is often studied in combination with logs (e.g. see a strange log entry then go see who connected to the system at that time or where the system itself connected to).
Fifth, next comes a group of pretty much everything else: other logs from the same system, logs about the same system as well as loosely defined 'similar' log entries. These come handy, but are not top choices. In fact, from this I conclude that a lot of additional context information is needed to make sense of a confusing log entry.
Sixth, what was surprising? I thought that identity lookups (e.g. IP to real name or other user identity information) would score higher. I also suspect that people were confused by "logs ABOUT the same systems" (what I meant is, for example, use firewall logs that mention the system which log we are now analyzing) and this should score higher.
Seventh, anything fun in the "Other" category? Yes, there were a few insightful ones: first, results of a Google search (supposedly for the info from the log entry in question)! Very true indeed. Also named were logs from the same daemon/program (how can I miss it?), logs from previous incidents and information on the logging system owner. All very useful indeed. Thanks for good ideas!
Finally, a brief message to people that work for a certain log-related vendor of ill repute who keep polluting my polls: if I catch you, I will kick you in the butt :-) Or, better, I will hammer you with a big and heavy log (you know, the wooden kind) over your miniscule heads ...
Past logging polls and their analysis:
Monday, June 02, 2008
Monthly Blog Round-Up - May 2008
I saw this idea of a monthly blog round-up and I liked it. In general, blogs are a bit "stateless" and a lot of good content gets lost since many people, sadly, only pay attention to what they see today. This is what is driving an idiotic campaign of such "news" as "hackers increase hacking", "compliance is hard" or "awareness of virtualization grows."
So, here is my next monthly "Security Warrior" blog round-up of top 5 popular posts and topics.
- First time this month, my logging polls took #1 spot! Specifically, a controversial Windows Log Collection Poll (which is a poll #7) sits highest among the Top5 posts (closely behind are poll #6 about logs that people actually look at as well as poll #5 about logging challenges). Poll #8 analysis is coming up tomorrow, BTW...
- As expected, the post called "Reverse Compliance or "Logs as Proof of Incompetence?"" tops the charts as well. It is about, "reverse compliance", which is a motivation to purposefully avoid technologies that have a chance of telling you that you are NOT in compliance.
- My quick post on data leak 'prevention' ("In Passing on DLP") is popular as well. Indeed, DLP is a very interesting segment of security market and there is plenty of innovation happening there.
- ISO17799/27002 might not be hot in the US, but discussing why it is not IS indeed hot. WTH? Well, "Why Is ISO2700x Hot in UK, but Not in US?" is in Top5.
- Again, people googling for "open source SIEM" have pushed this post (this tiny blurb) to top5. This ancient post from 2 years ago (!) years ago explains why an open source SIEM will NOT emerge soon, if ever.
See you in June!
Possibly related posts / past monthly popular blog round-ups:
- Monthly Blog Round-Up - April 2008
- Monthly Blog Round-Up - March 2008
- Monthly Blog Round-Up - February 2008
- Monthly Blog Round-Up - January 2008
- Monthly Blog Round-Up - December 2007
- Monthly Blog Round-Up - November 2007
- Monthly Blog Round-Up - October 2007
- Monthly Blog Round-Up - September 2007
- Monthly Blog Round-Up - August 2007
Cross-Device-Type Log Management vs Device-Specific Log Management
Now, I have to first admit that, in general, dealing with logs on a device-specific basis is a cruel joke. What I mean here is when you gather Windows logs in one place, Linux logs in another place, database logs in yet another place; all in different formats, all in different systems not connected to each others, all managed by different people who don't talk to each other (and sometimes hate each other). Yuck! Basically, this situation is "logs at their worst": all different, all disjointed and, as a result, all next to useless.
However, there are rare situations where you can choose device-specific log management approach (and still not look like a money- and time-wasting and idiot :-)). For example, you might be motivated by the fact that tools that can handle one specific type of log data (e.g. Windows-only, web server-only or Cisco PIX-only) are usually many times less expensive than cross-device log management tools. The table below clarifies it:
|Use Case vs Approach||No log consolidation - logs remain on systems they are produced||Device-specific log consolidation and analysis||Cross-device log consolidation and analysis from all log sources|
|Alerting based on log strings (keywords) that indicate security or operational problems||Impossible or tremendously hard to manage across many systems||Acceptable - alerts on each log type are handled by different teams||Superior - all logs are available for analysis when the alert is triggered|
|Reviewing logs for troubleshooting server problems||Acceptable - server logs are sufficient for||Better - one can also look at logs from other similar servers||Better - but same as device-specific log analysis since only one type of logs needs to be reviewed|
|Log review for compliance with PCI DSS||Not acceptable - log management is mandated by Req 10||Impossible or very inefficient - as many types of log data needs to be collected and reviewed||Optimal - all PCI relevant logs can be collected and reviewed in one system|
|Looking for records of a specific user activity||Impossible or tremendously hard since hundreds of systems might need to be searched||Inefficient - several different systems needs to be accessed to review all records of user's activities (and then data needs to be manually correlated)||Optimal - one query gives all traces of the user activity|
|Log review for incident response or forensics investigation||Impossible or tremendously hard since hundreds of systems might need to be searched for evidence||Inefficient - several different systems needs to be searches for evidence and then data manually correlated||Optimal - all log evidence can be found, reviewed and analyzed on one system, neither hundreds, nor several|
Also, while looking at logging tools, one needs to make a distinction between tools that can collect all sorts of logs but only allow you to analyze one log type at a time (e.g. sawmill) vs tools that can collect all sorts of logs AND allow you to analyze all of them together (e.g. LogLogic). The former tools still fall under "device-specific log management" despite their ability to gather hundreds of different log types.
The bottom line: in most cases, cross-device, uniform log management provides huge value, especially if your motivation for log management is compliance or incident response.