We still fail at log analysis

Recently I've been working a couple of data analysis projects, and writing  
some software in support of that. Much of it has direct application to 
automated log analysis, alerting, and decision-support. While I am still tweaking, 
I have been pleased with those results.

Which is a Good Thing, because we need to be lot better at it than the data 
suggests we currently are. Good data are scarce, but the Verizon Data Breach 
Reports do provide some. Exactly what is reported each year, and the format 
in which it is reported changes each year. To some extent it has to; the 
landscape changes rapidly.

Back in 2010

• 86% of victims had evidence of the breach in their log files
• 3% of breaches were discovered by log analysis or review
• 4% were detected by the combination of event monitoring and log analysis (This is a drop from the 6% of 2009)
• 30% were in compliance with PCI Requirement 10: "Track and monitor all access to network resources and cardholder data." A better number than the abysmal 5% in 2009

Fast forward to the report for 2012 (published in 2013), where the data 
are again presented in a slightly different way. Overall, detection via logs was 
1%, broken into undefined Small (nothing reported), Large (4%), and Overall (1%).

There was no figure for how many victims had evidence of the breach in their logs, 
but there is no reason to believe it is substantially different than the 86% 
reported in 2010. So it would appear that there is significant room for 
improvement in log analysis.

I think we can all agree that the worst-case scenario is to not only suffer a 
breach, but to have it discovered by an external party. Anyone doing incident 
response is (or should be) aware that the clock is ticking. If it's public, 
there could be a lot of people watching it tick.

Perhaps it's time to look at your log analysis systems again, including a check 
to ensure that the system is inclusive enough. It's common for organizations to 
not even know where all the logs are. The problems can be as varied as that 
they're being written by unfamiliar or misconfigured software, or systems 
being installed incorrectly or surreptitiously. 

If any of that is found, the problems are obviously more extensive than just logs.

First Actionionable Item From Snowden NSA Leaks

Here it is, the Fourth of July, and I am putting up a post. That is dedication!

Actually, the barbeque is going, and the spud salad and the rest of it is done. Even though the recent Oregon heat wave has broken, I don't want to be sitting next to that barbeque, since I cleverly placed it in the hottest, most uncomfortable place available. I really need to take care of that. I just spent a couple of hours at a state park, I'll be outside most of the afternoon and evening, and I managed to get a bit of sunburn standing in the river behind my house yesterday, which I do not want to make any worse. So I'm good with being in a nice cool office for a bit, and I'll probably finish up sometime later today.

On June 7, I posted 

NSA overreach: is it actionable, or just random news? in which I intimated that there is little that is actionable from the perspective of a security practioner.

Since then, the revelations have continued, and NSA are about as FUBARed as it is possible to be. The popular press is where much of this is coming from, and this issue is not going away. I may talk about why in another post. I do keep up with this stuff, out of professional interest. Still, you might roughly classify news organizations into members of the

generic mass media,

generic IT media,

pop security media,

technical security media,

though the lines occasionally wander. I use many news sources, which I divide into tiers, based on accuracy, level of detail (these are not the same thing), frequency of update, political skew, etc. Efficiently keeping up with security news (and it must be done efficiently, lest it become a full-time job), is a difficult to do well. Note that this does not include research papers, or what corporate white papers have become over recent years. Those have to be read too, but I don't regard them as media, in the same sense as the above list.

Here is something you do not see every day.  Another piece from The Guardian, New NSA leaks show how US is bugging its European allies, the UK newspaper (generic mass media) which was one the first to take this whole thing public, reveals information on attacks against diplomatic embassies and  missions of the EU and member nations.

The generic IT media and pop security media have already begun to lump this entire thing into PRISM. It's easy to remember and search for, which is an advertising revenue win. Without which, they do not exist. Meanwhile, the generic mass media Guardian is more accurate than generic IT media, or pop security media  in their code names for attacks (NSA will have changed all these the moment they were revealed, but they may become useful search terms, if only for students of history).

BLACKFOOT: French diplomatic mission to the UN

WABASH: French embassy in Washington

BRUNEAU, HEMLOCK: Italian embassy in Washington

POWELL: Greek UN diplomatic mission

KLONDYKE: Greek embassy in Washington

PERDIDO: EU UN diplomatic mission

Crucially, they also provide a graphic related  to DROPMIRE, an attack against secure FAX. Specifically against Cryptofax, a product of the Swiss firm Crypto AG. As an aside, there have been rumors and accusations (since confirmed to my satisfaction) since the 1980s of collusion between Crypto AG and NSA. I am surprised that The Guardian didn't pick up on that.

This where things get actionable, in two areas. The first depends on how technically well-resourced your likely adversaries may be in a pure security context.

That image was enough for Dr. Markus Kuhn, of the Computer Laboratory, University of Cambridge to go on. In a convincing post to Light Blue Touchpaper (technical security media published by the Computer Lab) he has convinced me that this was a TEMPEST attack. Another codename, referring most commonly to radio emanations. Specifically, in this case, to monitoring the radio-frequency energy emitted as the FAX machines laser was switched on and off. That may be very much actionable.

TEMPEST attacks have a long history. I am actually a bit disappointed in the EU, and EU member states for allowing a TEMPEST attack to succeed; note that the home of the University of Cambridge is the UK, an EU member state.

The second area that might be actionable depends on whether you are in the midst of, for instance, sensitive negotiations with German counterparts.

In the real world, even friendly or allied governments spy on one another. Despite the public expressions of shock and dismay that you can expect to hear from members of EU governments, they almost have to. At the nation-state level, even friendly or allied governments do not have completely aligned interests (it's almost as if they are different countries or something), and you need to know if a friendly or allied government is about to stop being friendly or allied. Even if it is limited to a single issue, if that issue is important enough. An intelligence agency that gets this wrong will be said to have suffered an intelligence failure (Google that), and will be barbequed. 

In this case, the other guys have a bit of egg on their faces, as the expertise to prevent this was available to them, but wasn't effectively used. Of course, politicians being much the same in any Western nation, they will hope it blows over, and to attempt to cover with indignation if it does not. Or cover with the 'hackers on steroids' defense that was used by so many US organizations who were hacked to the bone by the script-kiddies of Anonymous. This is entirely predictable.

Make no mistake: this is not going away soon, even if the leaks stopped immediately. The politicians, and NSA, will be disappointed. It is not going to blow over, and will feel the need to be perceived as Doing Something, even if it is The Wrong Something. Repercussions seem likely to be large and long-lived -- consider that a federal election will occur September 22 in Germany. Germany is a NATO ally, the leading economic power of the EU, and a justifiably privacy-sensitive nation. Particularly given what came out about the East German Ministry for State Security (Stasi) before reunification. It has been revealed that NSA collected against Germany, and has classified them as a valid target.

Nor is Germany the only trouble spot that lies ahead.

On a final note, I am not defending all that NSA have done; their surveillance of US citizens, and lying to all and sundry to cover it up are heinous. NSA have a history of doing things that are either dubious, or simply illegal, and they need to be reigned in periodically. I am not defending the politicians who failed to do what they were elected to do, though at least Senators Ron Wyden and Mark Udall of the Senate Intelligence Oversight Committee tried.

There is a far greater likelihood of a whistleblower going to prison than an NSA official who breaks the law. We need to fix that if we intend to become a more just society. Possibly some of the more damaging leaks may have been intended to find a sympathetic ear; to find a safe haven after the hue and cry went up from the US government. If that is the case, a trustworthy whistleblower program would have prevented the majority of the damage to our foreign relations that has so far occurred. 

Whether the service that he has undoubtedly rendered to his fellow citizens by revealing the latest NSA overreach event is outweighed by the damage that he has done to foreign relations is for history, and more practically, a jury of his peers to determine.

Wednesday, August 28, 2013 Update

And the NSA has indeed become an election issue in Germany, according to Der Spiegel, Peer Steinbrück, Chancellor Angela Merkel's challenger in Germany's September general election, called for a suspension of trans-Atlantic free trade Peer Steinbrück, Chancellor Angela Merkel's challenger in Germany's September general election, called for a suspension of trans-Atlantic free trade talks.

Does Cloud Backup Meet CIA Requirements?

For most purposes, I am concerned with the CIA definition of security. Confidentiality, Integrity, and Availability. One implication of this is that if you don't have good  backups, you are FUBAR. And, since Murphy is alive and well, you will discover this at the worst possible moment.

I don't generally like to get into the Big Data and Cloud discussions on public fora, as there is entirely too much marketing noise. But sometimes I know of poor decisions being made, and I just can't resist.

It should be obvious by now that we have to bring analytics code to the data, not the obverse. But there are still organizations that want to sell cloud backups, and couch their 'solution' in big data terms. These are all really fashionable terms and all, but please price a OC-192 connection. That's also called a SONET 10G connection, because it very nearly is 10G. It is amazingly easy to saturate a 10G connection. Really. The server consolidation via virtualization shakeup has not yet played out, and you can ask your down-in-the-trenches network people, and you will likely get a couple of horrifying stories.

I'd be happy to hear what you are paying, if you have one. Perhaps prices have crashed, but it was recently in six figures. That's just the connection, not the storage on the other end. It may be very expensive to move Big Data. Things you might consider

• rate of data change
• required storage period
• required access speeds across tiers (on-line, near-line, archival)
• any compliance or regulatory issues (PII, PCI, etc.)
• if the data must be encrypted, how well do you trust key management

In some cases there should be discussions with the legal team and/or the auditors. In more cases it should involve discussions with the bean counters.

If you can do reliable backups into a cloudy infrastructure provider, the financial numbers work, and you feel as if you can trust their service, good for you. But if it were me, I'd have to be very, very convinced before I would forego having a local backup of important data on local tape. If you feel the same way, how does that affect the value proposition of cloudy backup?

Saturday security humor

Because

1. Trust can fail in unanticipated ways
2. It is Saturday, and I am looking at a computer

Valerie Hopkins turned this one up.

As a professional security consultant, I recommend that you have a nice day.

NSA overreach: is it actionable, or just random news?

It has been a busy day. The PRISM furor has the popular press in an uproar. I don't find it surprising at all. This sort of thing has a rather long history, and there were indicators that the status remains quo. That doesn't mean that I am unconcerned; once upon a time this sort of thing was made expressly illegal, and secret law is not a path to success in running a representative democracy.

The net effect on my day is that various people are pinging on me for either an opinion, or sympathy for their sense of outrage. IOW, I am busy, and it's an interruption. Ask yourself if this actionable, in the sense that you should immediately change your behavior or policies? In almost every case, the answer to that question will be no. Write your favorite politician (who are most of the problem, not part of the solution, with notable exceptions), or otherwise do what your conscience demands. But please, do not expect security professionals to be in a white-hot frenzy over this. Unless it's a marketing thing.

Many security practitioners are privacy advocates; it's security on a personal scale, and the same principles apply. Others may see it more from the perspective of a vendor selling tools or services. And some are purely pragmatic, wandering back and forth across that line, as circumstances dictate.

The people being quoted in the media at the moment are not practitioners; they are either managers with a large political bent, or purely politicians. What you are reading is about political agendas. It is of little practical interest to practitioners, because it is not actionable. Anything actionable will come later. Probably months or years from now.

In the meantime, there are more interesting things to think about. PKI is still broken, and possible solutions have been brought forward. Java stills hangs heavily around our necks (and I need to write Part 2 of that post) and Adobe Web products are pretty much as bad. The ability of government and industry to share information is still FUBAR, but there things we can do, today. We have no good handle on the problem of incident response (despite what you may read). We don't even handle code re-use particularly well.

That is the sort of thing that is actionable.

So, back into the trenches of real-world security.

Java Security Revisited--Part 1

Java Security Revisited--Part 1

I have already thrashed on Oracle Java security failings once, but it doesn't hurt to go on with it. There is a human cost to this, in stolen identities, banking details, etc.

At
https://blogs.oracle.com/security/entry/maintaining_the_security_worthiness_of
there is a post titled Maintaining the security-worthiness of Java is Oracle’s
priority.

This is as FUBAR as anything needs to be.

It is perhaps more appropriate to subsitute 'Establishing' for 'Maintaining' in
that title. Java has a long history of problems. You might want to go read that
5/17/13 post first. The one where I claimed that they had more or less fallen
on their sword, and admitted complete security FAIL.

Back now? Great. Let us start with a quote from that blog post. "Hi my name is
Nandini Ramani, I lead the software development team building the Java
platform. My responsibilities span across the entire Java platform and include
platform security."

This is The Man. The guy with the ultimate responsibility (save Larry Ellison,
who will probably fire him if results are not forthcoming, pretty damned
quickly).

"Over the past year, there have been several reports of security
vulnerabilities in Java, primarily affecting Java running in Web browsers."

Over the past year? I don't want to accuse Mr. Ramani of being dissengenuos on
a corporate blog; perhaps he has not been keeping up with even not-so-current
events. But let us look at open source intelligence.

I regard Brian Krebs as one of the foremost researchers of the business
mechanisms behind underground fora, exploit pack proliferation, and botnets,
and related matters. He is focused, talented, and has sources I will never
have. Read
http://krebsonsecurity.com/2010/10/java-a-gift-to-exploit-pack-makers/ in which
Mr. Krebs says, "I also found Java flaws to be the leading exploit vectors for
both the Crimepack and Eleonore exploit packs." This was in October, 2010, not
"Over the past year" and the rabbit-hole goes deeper than that.

At this point, I have no good reason to trust Mr. Ramani. I do, however, have
good reason to question his integrity, competence, or both.

Further down in the post, you will find that Java will be updated four times per
year, in sync with other Oracle Critical Patch Updates. This is obviously going
to increase the testing load for those who have to deploy this stuff. But my
focus is on security, and my take is that this should already be built into
your deployment workflow. If hasn't been, in the past, take it up-stream. Your
workload has certainly increased, and you need an increased budget. It's just
part of the ever-increasing cost of doing business with Oracle.

Mr. Ramani even provides helpful reminders. Summarizing:

February 2012 Critical Patch Update for Java SE provided 14 security fixes
June 2012 release 14
October 2012 release 30
(thus the total number of new security fixes provided through Critical Patch
Updates for Java in 2012 was 58)

February 2013 security releases provided 55 new security fixes
April 2013 Critical Patch Update for Java SE provided 42 new security fixes
(bringing the total number of security fixes released through the Critical
Patch Update for Java in the first half of 2013 to 97.)

The way Mr. Ramani probably does not want you to interpret these data is that
these numbers are a measure of how horribly broken Oracle Java security has
been, and how exposed you have been. Increased pressure on Oracle is indicated.
What else is out there, that you do not yet know about?

"The Java team has engaged with Oracle’s primary source code analysis provider
to enhance the ability of the tool to work in the Java environment. The team
has also developed sophisticated analysis tools to weed out certain types of
vulnerabilities (e.g., fuzzing tools)."

Sounds great, and while fuzzing is hugely useful, that usefulness has only one
domain. Gary McGraw, a recognized authority, pointed out in
Software Security: Building Security In, published by Addison-Wesley, that less
than 50% of vulnerabilities come from implementation flaws. Code analysis tools
do not find architecture flaws.

This is getting way too long, and I am not even remotely done. I am going to
have to continue this in a second part.

The current Java is update 7u21. There will never be a 7u22.

Write once, run everywhere. It's secure because it runs in a sandbox.

Eventually, even the largest IT marketing machines cannot overcome IT reality--social issues are a whole different thing, but (mostly) out of scope for this post.

"Write once, debug everywhere," is commonly heard. And we really don't understand how to reliably and easily build secure sandboxes, from chroot jails on up.

But I believe a page titled  Java SE - Change in Version Numbering Scheme to be a landmark. So far as I know, this is the first time that a major vendor of 'Enterprise' software has pretty much admitted complete defeat on the security front. Microsoft came close, several years ago (they have gotten much better), but this is an admission of  abject defeat.

Oracle has never had a good rep, on several fronts. Hardball business tactics and vendor lock-in don't really concern me here. That is an easy Web search away, if you aren't already affected by it. Being widely regarded as the least secure of the 'Enterprise' DB vendors does, but only insofar as it is symptomatic of the current problem. Which is that Oracle has not only consistently produced horrible products, but they have been very slow to fix their problems.

The way Java updates worked in the past was

1. Limited Update patches (feature adds and and non-security bugfixes) got even update numbers.
2. Critical Patch Updates (CPUs) got odd update numbers.

They are done with that. There have been so many security flaws that they have had to renumber releases, which causes huge problems for users in risk analysis, patch scheduling, etc. The majority of the problems involved Java running in Web browsers, not server-side Java.

That was the one ray of sunshine, but security teams had to take very careful looks at exactly what was being run, and where. Multiple versions, installed on the same PC, were a problem. Common exploit code can call the vulnerable version if it is available. And some organizations did not prevent Java installation, of any version, regardless of whether it was really necessary on that employee machine. To make matters worse, a common installation scenario involved developer and QA machines. These users could often mount a specious argument that they needed it to do their jobs, and would be given blanket permissions. 

It was, and is, completely fubar. It is a really great way to make an adversary's job easier.

So, Oracle has at last bagged it. How have they admitted complete defeat (though not in so many words, of course)? Like 1970 BASIC programmers, leaving space between line numbers. 

Limited Update patch numbers are assigned in increments of 20. That leaves fill-in-the-blanks room for CPUs that will be issued as odd numbers that are increments of 5. Leaving still more fill-in-the-blanks room toad more fixes, and/or fix their previous fixes. And they reserve the right to use even numbers in those CPU intervals. I would guess that they are expecting more than a few.

Buy Oracle products. Because Larry Ellison needs a bigger yacht, more CA beachfront property, or another Hawaiian island. It's the right thing to do, because Larry is a caring kind of guy.

6/4/13 Update. This just gets gets better and better. See Java Security Revisited--Part 1.