Anton Security Tip of the Day #10: Email Tracking Through Logs

Following the new tradition of posting a security tip of the week (mentioned here, here ; SANS jumped in as well), I decided to follow along. One of the bloggers called it "pay it forward" to the community.

So, Anton Security Tip of the Day #10: Email Tracking Through Logs

Email tracking - oh, need I say more? :-) A nightmare for privacy fans - an "evil" weapon of lawyers and HR. Email tracking raises concerns that vary from a severe inability to do it all the way to having too much ability to do it. In this tip, we will focus on the following scenario: your boss says she just sent you an email; you never received it. What's the story?

First we need to search our Sendmail server logs (this is that case where searching logs is appropriate) for the email address of the sender (unfortunately, we rarely can search the logs of the remote mail server which sent - or didn't send! - the message in question). One thing to note before we start searching is to limit the time frame to the one close to the reported problem (let's say to the same day).

So, we search for messages containing from=boss@examples.com (since we want the sender address) at January 11 (if you just search for boss@examples.com, the volume of unrelated results will be much higher) and find something like this:

Jan 11 11:45:49 ns1 sendmail[28022]: j0BGjkc28022: from=boss@example.com, size=376597, class=0, nrcpts=1, msgid=Pine.LNX.4.61.0501111145450.17081@boss.example.com, proto=SMTP, daemon=MTA, relay=esafe1.example.com[10.211.15.69] (may be forged)

Is this our message (we are not quite sure since we don't see a recipient)? Why isn't it here?! What do we do next? Well, this is where things get a little ugly (for those unfamiliar with Sendmail logs).

Next, we need to search the logs for this "mysterious" string "j0BGjkc28022" from the previous log record and find this:

Jan 11 11:46:00 ns1 sendmail[28025]: j0BGjkc28022: to=employee@example.com, delay=00:00:14, xdelay=00:00:11, mailer=local, pri=405901, dsn=2.0.0, stat=Sent

So, it looks like our mailserver have indeed sent the message, but what happened to it afterwards? Where did it go next? Given that "employee" has a mail account on the example.com system, most likely the message went into his mail spool (something like /var/spool/mail folder)

Next? If the employee uses local shell access to a mail server and Pine or other similar mail client to read mail (really unlikely, but possible - mostly at University systems), we need to see whether it exists in user's local mail folders, such his /home/username/Mail folder or something similar.

But what if the employee uses POP3 or IMAP to read mail? In this case, we have more logs to go through ... However, most POP3 server logs will not help us determine the fate of a specific message, but just to confirm that the user checked and/or downloaded mail. We will leave POP3 log fun for the next tip...

So, in today's basic tip we covered one simple scenario of email tracking through logs.

Also, I am tagging all the tips on my del.icio.us feed. Here is the link: All Security Tips of the Day.

Technorati tags: security tips, logs, logging, log management, chuvakin

On Mobile Malware II

My previous post on mobile malware seems to have struck a cord. This post is updated with all the comments that I received (some contradicting each other!), organized in the same manner.

Why mobile malware will be a scourge of the future?

• there are many more cell phones than PCs - great opportunity for global infections and overall wireless mayhem (and robbery?)
• many cell phones/PDAs nowadays are always connected to a TCP/IP based network (more will be in the future)
• there are other fun avenues of possible spread, including Bluetooth, IR, MMS, mobile-to-PC, memory card, etc
• new methods of commercializing mobile malware are being invented (e.g. via paid SMS and MMS)
• cell phones are tied to automatic billing (!)
• many mobile devices contain data which is perfectly usable for social engineering and other attacks (address books, messages, personal notes, etc)
• much faster evolution of malware - its creators learned the lessons from the PC world
• PCs will become more secure and more security tools will be deployed there, so the attackers will shift focus
• very bad "OS" security on mobile platforms ("Win95-grade")

Why mobile malware will be nothing much?

• there is no standard cell phone platform (like Windows in the PC world), even though there are some contenders (Symbian, Windows Mobile)
• sorry, but malware is commercial now and there is not much directly monetizable data to steal from a typical mobile phone
• similarly, mobile platforms are limited in both user and system functionality
• moreover, people PAY using their PCs (thus, phishing, pharming, etc) and mobile devices are not widely used for that (yet?)
• data is still secondary to voice on most modern mobile platforms (exception: Blackberry); thus, if your attack affects data only, the phone is still pretty usable

Other interesting thoughts (see raw comments) explain why it is not a scourge now (e.g. not too many phones are capable of even running malware).

Overall, after seeing the comments, I started to lean more heavily towards "run for your life" scenario - but it will hinge on more payments being done with/via cellphones directly. So, maybe those mobile anti-malware startups are worth something :-)

More thoughts? Anybody from an AV vendor, trying to sell mobile anti-virus tools now?

UPDATE: Kevin Mandia says: "Mandiant: First I would ask the nontechnical question, “Where is the money going?” Because where the money goes, the attacks follow. I would imagine if people start using mobile devices for online banking and credit transactions, then
mobile devices may be the next target. " Interesting!

UPDATE: a dirty lie? "Spanish police have arrested a 28-year-old man on suspicion of creating and spreading a virus that affected more than 115,000 top-of-the-range mobile phones."

Technorati tags: security, malware

Data-at-Rest, Data-in-Motion AND Data-in-Use?

Very good idea: adding data-in-use to data-in-motion and data-at-rest.

"What about Data In Use? Can your users print, copy, move, and otherwise twiddle the data they have access to?"

And, data-centered security is something many talk about and few do (for now) - this is a great ongoing theme and a reliable trend which will continue for a while.

Networks of Tomorrow

Very fun, worth spending time on

1. Phase I - phone system
2. Phase II - packet switching Internet
3. Phase III - view the video.

What comes next? Conversation vs dissemination? Why phase III is not about caching content? What congests the current Internet? These and other fun questions in the video.

This phase III is coming fast and it has some fun implications for security, of course.

UPDATE: here is the conclusion side

Puzzle vs Mystery: Great Reading to Think About

Puzzle vs Mystery: two types of problems we face out there in the world and why it is counterproductive to approach one as the other ... It has this fun quote, also: " Solving puzzles is useful for detection. But framing mysteries is necessary for prevention."

Somebody Asked Me ...

Somebody asked me once: why smart folks like Marcus Ranum make jokes about IETF ("IETF's terrible fruitless attempts" :-)) This is one shining example why: Internet draft for "geographic location in syslog."

On CooL

Do I have a secret marketing gene? Maybe that bastard causes me to really like this presentation on "Unlocking Cool." Needless to say, I am also a major fan of "Pattern Recognition."

Risk: Love It or Hate It

We (many of us...) in the infosec profession have a deeply disfunctional relationship with risk. Here is one of the shining examples why (quoting from this dailydave post, accenting is mine)

'One of the panelists began talking about defining "Acceptable Risk Levels" within organizations. (These were CIO's, CTO's, CSO's etc for multi billion/million dollar companies.) When I heard these people speaking I realized that they never got into anything specific.

Instead it was as if they were just talking about ideas that they briefly read about in magazines or online articles. So I decided to ask them something specific.

My first question to them was "In order to properly understand your acceptable risk level you must first understand the threats faced by your business, correct?" They all nodded in agreement. My second question to them was "Where do you get your threat intelligence?"'

As reported, it all was downhill from there ... :-) Read on

Apps as Employees

"If your app was an employee, what kind of employee would it be?" An ass-kisser? A clueless guy/GUI :-)? Brilliant but temperamental? An undecider?

On Security Architecture

Just linking for posterity - great info on security architecture in support of business goals.

"The purpose of the security architecture blueprint is to bring focus to the key areas of concern for the enterprise, highlighting decision criteria and context for each domain."

On Mobile Malware: Yay or Nay!?

When I see stuff like this, I think "mobile malware" will be huge. But then I think rationally for a bit, and it seems like it won't :-) So, let me think about it a bit more since many folks are planning to launch companies to fight the coming onslaught of the mobile malware ... 

Why mobile malware will be a scourge of the future?

• there are many more cell phones than PCs - great opportunity for global infections and overall wireless mayhem (and robbery?)
• many cell phones/PDAs nowadays are always connected to a TCP/IP based network (more will be in the future)
• there are other fun avenues of possible spread, including Bluetooth, MMS, etc
• new methods of commercializing mobile malware will be invented (ah, make that "are being invented NOW")

Why mobile malware will be nothing much?

• sorry, but malware is commercial now and there is not much to steal from a typical mobile phone (Tiny porn? Address book? Eh?)
• similarly, mobile platforms are limited in both user and system functionality
• data is still secondary to voice on most modern mobile platforms (exception: Blackberry); thus, if your attack affects data only, the phone is still pretty usable
• there is no standard cell phone platform (like Windows in the PC world)

BTW, a few related, some mildly hilarious, links:

• 2006: Year of the mobile malware
• Mobile Malware: The Brief History
• Mobile Malware Evolution
• This is Not the Year of Mobile Malware

The last link also explains: "A great deal of changes need to take place before any year can claim that title: convergence of platforms, greater degree of integration, other targets become less attractive because they become harder to hack, an increase in data and transaction values of mobile targets; the emergence of a premier vulnerability on mobile platforms."

So, what do you think?

Technorati tags: security, malware

More On Trusting the Software

So, just as I suspected, this post "Are You Mad?!" did generate some controversy. Let me add some liquid hydrogen to the fire and then splash some liquid oxygen: the unscrupulous programmer mentioned in the previous post does not have to code any sophisticated vulnerabilities in or bring any funky backdoor covert channel encrypted stego shells. In fact, it is better if he doesn't.

All he needs to do, really, is to enable a way to change the data.

Yes, just having a way to replace an obscure string X75JKHJ56 with another obscure string X75JKHJ67  in some obscure  database is rumored (don't you love those! every security conference worth its salt has plenty of rumors exchanged!) to be sufficient to make sure that the wrong part will be installed by a repair service during the ongoing aircraft maintenance! Do you believe this? Maybe I do and maybe I don't. I will certainly not stop flying just because some [very smart] application pentester shared this story with me. But the point is made: if you trust software (and software developers), your ass will be 0wned before you know it.

So, what is the answer? Yes, you guessed it right, open source is one possible answer. What are others? I dunno - future will tell.

Technorati tags: security, software, future

Log Management Tops the Charts ...

Infosecurity Magazine has some fun bits on log managementrecently. They said: "First-generation security information management systems rarely lived up to the expectations raised by their proponents. [...] The investment in using these early tools was justified only in rare situations (!) where extraordinary technical professionals amplified their value. During 2006, however, increasing regulatory demands caused buying interest in log management to skyrocket. "

More Terms from Logging Glossary Published

As I mentioned here, I started publishing the LogLogic Logging Glossary. Here are the terms and definitions published so far:

1. Alert
2. Audit Logging
3. Context Information

More are sure to come, just watch our blog.

My Presentation from CONFidence 2007 /Teaser/ Posted

Here is a teaser version of my CONFidence 2007 presentation "Log Forensics" Enjoy! It did get some rave reviews (in Polish :-)) I will certainly be speaking more about log forensics as it seems to be of great interest to many people and there is some confusion about what it is.

BaySec in TODAY!

Reminder: BaySec, an informal meet-up of information security professionals in San Francisco and Bay Area happens today, May 16th, 7PM @ Zeitgeist Bar in San Francisco, CA

UPDATE: I missed it due to a cold, but I was told it was fun!

My Presentation from CEIC2007 /Teaser/ Posted

Here is a teaser version of my CEIC 2007 presentation "Integrating Log Analysis into Your Incident Response Practice." Enjoy!

On "Incident management in the age of compliance"

I wrote this fun paper "Incident Management in the Age of Compliance" that covers "the basics of incident response and related them to three major regulations that directly affect the specifics of setting up incident response capabilities."

Here is also a bit of musing inspired by the above: I've been doing a lot more of a strategic, management-level writing, possibly at the expense of bits-bytes-hexes writing. Should I go back and finish that outbound firewall log analysis paper...? :-)

CrapPoint

So, sometimes I highlight something that I found insightful or fun. But in other cases I like to point something stupid. For example, this IDS-related article deserves a prestigious spot in the later category. E.g.

* "that many IDS systems are already becoming less reliant on signatures, and using rule-based engines instead"
* "The product [IDS] isn't really changing"
* "he expects IDS/IPS products to be packaged with honeynet technology as well"
etc.

The paper also has a pretty "shiny" collection of frustratingly obvious.

* "This generation of IDSs is getting better." - as opposed to getting worse? :-)
* "IDS data is being used as part of intelligence-collection for forensics" - wow, this is deep insight from an expert right there ! :-)

So, was Mr Schultz misquoted or did he lose touch from being in academia for too long? Judging by his book ("Intrusion Detection and Prevention") is has to be the latter ...

Competent vs Likable

Penelope Trunk say in her interview:

"Question: Is it more important to be competent or likable?

Answer: People would actually rather work with someone who is incompetent and likeable than competent and unlikable. Most people nod in agreement when they read this. It’s the unlikable people who form arguments in their head.

But there’s more. At work, if you are unlikable, people start thinking you are less competent. So stop thinking you can skate by on your genius IQ because you can’t."

Is this the end of the world as we know it? Or not? Can you guess what I've been reading lately? (Answer: Ayn Rand)

Are You Mad? Are We All?

Imagine you bought something.

• You rely on it with your business, with your very livelihood. Sometimes even with your life.
• There is no warranty whatsoever on what you bought.
• And you don't know what's inside the box.
• Also, you cannot look inside the box, in fact, it is illegal.
• You might not have have heard about the seller before and you have no particular reason to trust him.

Are you totally and irreversibly mad? How can you do it?! If you are not mad, aren't you criminally negligent? Or just very, very, very stupid.

However, we all are. We all bought software at least once in our lives ...

This blurb is inspired by some discussions I had at CONFidence 2007 Conference (where I presented on "Log Forensics" in front of about 180 people). Another related fun thought I picked there is that the most scary cyber-criminal of the future is not a spammer, a scammer, a phisher or a pharmer, and not even a good ole "cracker" - it is an unethical software engineer, who changes the code slightly to introduce a weakness (or a full-blown backdoor or a logic bomb) and later uses or sells this knowledge. In light of the above characteristics of software purchases, think billions stolen in one shot, think ruined companies, think stock market manipulation, think direct physical damage (and, yes, real cyberterror), etc. We do live in interesting times ...

Technorati tags: security, philosophy, chuvakin, software

On “Is your PC virus-free? Get it infected here!”

“Is your PC virus-free? Get it infected here!”

UPDATE: Yo, security awareness fans :-), do you think you can handle this one?

Have Phun Phishing ...

Very fun (ehmm, "phun" :-)) interview with a phisher.

If you are looking some extra income ("3k to 4k a day"), phishing works! :-)

It ends with this: "Lazy web developers are the reason I’m still around pishing." No kidding!

UPDATE: do read the comments!

Logging Glossary: All the Terms and Then Some

Yeah, I know a lot of folks frown when people blog about the fact that they will blog about something :-) In fact, I often think that it totally sux. Why am I doing this then with this post? Good question...maybe it is just my mood of the moment.

So, I hereby pre-announce that I will start blogging pieces of LogLogic Logging Glossary: a definitive list of all terms logging with detailed explanations.

Are you dying to know when? Well, soon :-)

Technorati tags: log management, logging

On Visitor "Logs"?

But think about it - are your computer security logs just as open as those? Ouch, scary! Time to get some log management :-) Oops, did I just use FUD to promote log management? You bet! :-)

Log Management Leads to System Manageability

Check out this fun piece of research from - wait for it! - an Open Source Lab of Microsoft that, among other things, touches why log management is an essential piece of better system manageability.

Visualization Fans, Read This

A sample quote: "Chart-based encryption -- data goes in, no information comes out" :-)

A sample picture: I LOVE this picture from the site. I was thinking that very thing for a looooooooong time :-)

Read on!

Closure (Kind of) to the Anti-Virus Efficiency/Effectiveness Saga

So, this anti-virus efficiency/effectiveness saga is kinda closed with this post AND the discussion that followed.

I suspect that the end is peaceful indeed:
= you likely still need AV,
= AV can be bypassed by attackers (and could have been bypassed in the past),
= the chance of such sad event is probably (but not scientifically) higher now than in the past, given proliferation of many types of "commercial" malware.

Did I summarize it right? :-)

And, finally, Kurt has a point here: malware and anti-malware really isn't my top specialty :-(

Presenting at CEIC 2007

Just a quick note: I am speaking at CEIC 2007 tomorrow. If you are in Vegas, come see me, BUT - oh, horror! - my presentation is at 7:30AM (times like this must be outlawed... :-))

BaySec!!!

An informal meet-up of information security professionals in San Francisco and Bay Area: May 16th @ Zeitgeist in San Francisco, CA: mark the date now.

Unlike other meet-ups, you will not be expected to pay dues, "join up", or stomach another vendor spiel to attend [well, maybe later ... if you really want to... and if you feed said vendor a pint or two :-)].

In other words, if you work in security and live in Bay Area (or the City itself), you MUST be there on May 16th :-)

And, of course, you want to know who will be there: the "In crowd" or the local "Dorks Anonymous"? Well, both :-) Here are some of the people who will be attending: here, here, and here.

UPDATE: somebody asked me "should I bring my girlfriend?" And I said: "can she write exploits?" You guess the rest ...

Few Bits on Log Management Trends

Some time before the recent SANS Log Management Summit, somebody asked me: What are the top three trends in the log analysis industry?

I figured why not also post my answer for all to see. So, here they are (slightly edited for clarity):

• Rapid increase in the breadth of log sources that people care for (and thus collect data from): it used to be just firewall and IDS logs, then servers, and now it is expanding to all sorts of log sources - databases, applications, etc (see more information on this here)
• This might sound boring, but it is still a major trend: more regulations, governance frameworks and standards will cover logs and logging. Just look at recent PCI, NIST 800-92 and a few others (including my very favorite - CEE  where work is just starting up
• There is also a trend towards auditing more access and more activity through logs; for example, few of the file server, storage or database vendors cared much about logging, but now they do (well, some do and some start to :-)). What used to be just about "access to information" is now evolving into "auditable access info." More discussion of this is here.

Comments? Additions? Criticism? Silence? :-)

Technorati tags: log management, logging, trends, chuvakin

From the "Nobody is That Dumb .... Oh, Wait" Department

A quote from this will suffice: "It has long been assumed that corporate computers are relatively free of bots, pieces of malicious code hidden on a computer without the owner's knowledge to perform spamming or other undesirable activities."

More on Web Vulns

I did wonder about it, and now it seems R.F.P. has too. A quote from his interview:

"ap: What policy to apply in the case of public site vulnerability research? Should the researcher avoid it completely, apply the rfpolicy or the full-disclosure way is viable too?

rfp: Funny, because I was just mulling this over recently. It’s one thing to have a security problem in something you control, such as a device or a piece of software installed locally. There’s the potential for you to enact a workaround or introduce another mitigating control.

Public websites are another matter. The only one who can fix the problem is typically the web site. There’s no mitigating strategy users can usually do other than forego use of the site. You think everyone is going to cease to use MySpace because they have an XSS hole? No way.

So thinking that it’s better to tell the world about a security problem in a public site than to tell the site owners is being part of the problem, and not the solution. Again, full disclosure is a tool, and is a worst-case/last-ditch scenario after all else fails."

So, WHAT is the solution then? Does ANYBODY know?

It Was All Him, That Bad Boy 10.11.2.3

As security people we are used to answering questions such as "Who attacked that system?" with a curt "Oh, it was 10.13.13.13." But is the IP address really a who? No, really, is it? I seriously doubt that an auditor, a judge or a lawyer will agree that "an IP address is a who."

Where am I going with this? I think the time when we start making broader use of identity traceback to link the faceless, inhuman :-) IP addresses to a nice (or nasty, as the case may be :-)) warm-blooded humans, who actually press the buttons and write programs.

In fact, PCI tells that that we already MUST. Requirement 10.1 says that the organization must "establish a process for linking all access to system components (especially access done with
administrative privileges such as root) to each individual user." At the same time, one of the DoJ papers on "Computer Records and the Federal Rules of Evidence" mentions this as well in the section on challenges to computer evidence integrity: "Although handwritten records may be penned in a distinctive handwriting style, computer-stored records consist of a long string of zeros and ones that do not necessarily identify their author."

So, making that IP to person connection is important, so how do we do that? Well, let's see who knows who you are:

• a DHCP server does, somewhat: it can link a dynamic IP to a static Windows name (still not a person name, but for workstations it might sometimes be related)
• an Active Directory server does, a bit more
• a NIS or an LDAP server kinda does as well
• Other sources of such info can be used as well (more on this in the future)

Thus, the identity traceback challenge is not really in the lack of info, but in coming up with ways to link the pieces together in an automated and reliable way.

As I mentioned in my RSA 2007 impressions, "identity" is a hot buzzword now; I expect people to start making more use of such identity info for identity traceback...

UPDATE: of course, I saw this (do you think I am asleep or what? :-))

UPDATE - 7/25/2007: here is another fun blurb on this very subject - "automatically associating learned user accounts with IP addresses. " It does it in a really cool way: "It accepts normalized logs from several dozen different authentication log sources, extracts the user name and originating IP address and then creates a log if the user identity is new, or if an existing "user to IP address" relationship has changed."

Technorati tags: log management, logging, logs, identity, chuvakin

My Impressions of SANS Log Management Summit 2007

So, as I mentioned before (here and here), I attended a recent SANS Log Management Summit 2007. Here are my notes and impressions.

First, I really enjoyed the event, both from speaking and listening point of view. I did two panel presentations (one on log management implementation issues and one "vendor shootout") as well as my "award-winning" :-) Lunch and Learn presentation on choosing a log management approach: buy, build or outsource (or combine!). Vendor shootout panel also was pretty exciting (unlike one last year where one of our competitors sent a sales weenie masquerading as a "CTO" who spouted drivel for, like, 15 minutes :-)); we got some fun questions (like "name one feature that you do WORST!?") as well as fun answers. While cynics might grumble that vendors only go to such events for competitive information, the panel was genuinely interesting and thought-provoking.

Among other interesting observations, I noticed that logging for operational uses was much better represented and more frequently mentioned compared to last year; logging for security and compliance were certainly there in full force, but logging for operational uses, which is the oldest, classic use for logs, seems to be making a comeback and people really buy (or build - and then suffer :-)) log management tools to deal with the challenge.

On the market side, Summit pretty much proved that there is a log management market now with its own players, requirements, use cases, etc. At the same time, buyers are much more aware of what they are actually signing up for when they call a log management vendor. Still, I saw a share of people who made really bizarre decisions in regards to their logging...

Also, I was excited that Stephen Northcutt mentioned the MITRE CEE logging standard project, that I've been involved in (since before its name got changed from a more exciting one - guess it! :-) - to this bland "CEE"...) Log standards are definitely way overdue. Even a simple "what should be logged" recommendation (part of CEE, of course) will come incredibly handy.

Finally, somebody asked me how did the summit compare to last years? I liked this one a bit more; content was a bit more useful (even though, obviously, not new to me) and there was much less confusion about what log management actually is (e.g. much less SIEM dirt was thrown around :-)).

Other people's summit notes are here (I am sure more will be posted soon and I will update this)

Technorati tags: security, conference, chuvakin, logs, log management

Agreed - Totally Hate It! :-)

Yes, hate it indeed.

On "Top 10 Internet Crimes of 2006"

Just reporting for posterity :-) Also, a must for those fond of statistics.

"The Internet Crime Complaint Center filed its annual report last month, but didn't get the attention it deserved. A look inside offers some revealing statistics on the darker side of the Web.This is the sixth annual report by the U.S.-based center, which is run by the FBI and the National White Collar Crime Center."

More Vendor on Vendor Action

Want more vendor on vendor action? This one is kinda fun: one unnamed security vendor "hacking" (well, sort of) another unnamed security vendor to harass their customers.

The reason? As dumb as a hidden form value and HEX "encryption". You think nobody is that dumb [in security business in 2007]? Think again...

People vs Technology vs People

Here is an interesting flip on the usual "people vs technology." Joanna points that recently way too much focus was on the people side of the security equation and many started to think that "the only problem in the security field that mankind is facing today is… that we’re too stupid and we do not know how to use the technology properly."

Guess what? If you have a perfect security awareness program and all your IT users are ex-NSA "security conditioned" personnel, you can still get owned thru a 0day. Yes, even though it will sound painfully obvious to many, "even if we were perfectly trained to use the technology and understood it very well, we would still be defenseless in many areas."

Also, lately I've been reading a bit on risk management and thus seeing the words "risk assessment pseudo-science" probably made her post even more appealing to me :-)