Thursday, March 29, 2007

Do You Enjoy Searching?

Now, do you enjoy typing stuff into Google and seeing results come up? I kinda doubt it - you probably need to "find something" rather than feel an unstoppable urge to "search for something." At this point some might say - ah, this is just semantics. I assure you - it is not.

Let's look at something like using Watson or Inxight vs just using Google. In the first case, you are given a spade (Google prompt) and are told to dig until you either a) unearth what you need or b) get tired and leave or c) find something else and get distracted (sounds sad, but ADD epidemic is rampant indeed). In the second case, a smart application (such as Watson) tell you "hey, look at this - this is relevant to what you are doing [and it is!]; check it out." You are given that something you'd dig for, not a digging tool. And note that it is given to you without any digging, which is cool.

So, why are we discussing it here? This discussion has a direct link to computer log analysis and log management. Lately, I've met a few deluded folks who equate "log analysis" with "log searching." At first glance, it sounds truly ridiculous: why would someone enjoy searching logs? :-) OK, a need to find something specific in logs does arise (e.g. to answer targeted question such as "what did this user do?", "what specific systems were attacked?", etc), but I still see log searching as a somewhat secondary and somewhat painful route to answers of your logging questions. And, to top it off, you need to analyze the results for relevance. Yes, Virginia, it means more work... for you!

What is the other way? For example, in one presentation on text mining ("Introduction to Text and Web Mining" by Je Wei Liang) we find this curious picture:

It applies to logs perfectly! I'd rather have questions such as "what do I need to know" (and more specifically, "what's wrong?", "what requires attention?", "what's not normal?", etc) discovered, answered and presented by an automated system, than go for a digging session... A nice dashboard with coffee in the morning containing the answers to the above questions for my environment beats any "log search" over the head with the big stick :-)

Technorati tags: , ,

Dr Anton Chuvakin