Commentary::Internals
Audience::All
UUID: 1af6f74e-015a-4cc6-a668-181a083b1850
Earlier today, I published #101, since 2013-03-17. A bit of a milestone, I guess, though I don't pay much attention to that sort of thing; I totally missed #100.
It does bring up a bit of a question, though. Some time ago, I mentioned that I wanted the date of publication right up top, where viewers would immediately see it. Because information gets stale rapidly. Arriving on a blog post from a search engine, reading some lengthy post, and then discovering that it is five years out of date (if you can discover it at all) is FUBAR.
It is even more FUBAR if you consider how many servers may have been incorrectly configured because due to dated information, etc. This one of the several reasons that blogs, particularly security blogs, and most definitely this one, suck. They are little, if at all, better than security trade press.
Here's the thing. At 100 or so posts, I can still maintain a mental image of what I have written in the past. I can go back to previous posts and and post an update.
This will not scale. What's more, I have an idea of posting about common themes (things that the security community might do better) that might conceivably have a greater impact. If I were to become successful at that, the specific content of individual posts on a given topic (log analysis comes to mind -- I could go on about that) is going to blur together. Success at one goal seams likely to lead to failure at another.
But I can't really set aside a block of time each month month, and delete the old stuff. First off, time is scarce. Second, I will break links from more-recent posts to what has become background information.
A blog seems to not be an appropriate tool. A wiki, or a document series on GitHub, might be more useful. Or perhaps using this blog to announce revisions to either. The thing is, there is a critical mass at which a community forms, feedback is received and acted upon, etc. A rule of thumb seems to be that perhaps one in a thousand blog viewers will comment. This blog gets a few hundred visitors per month, so it seems unlikely that a critical mass will ever be reached.
Perhaps I am wrong about this, and I just needed to announce an open thread. OK. Done and dusted. I have my doubts, but the idea has to be given a chance, if only to give potential community members a voice in describing something that might better fit their needs.
Showing posts with label Internals. Show all posts
Showing posts with label Internals. Show all posts
Wednesday, May 13, 2015
Thursday, May 7, 2015
Sharing is Complicated
Commentary::Internals
Audience::All
UUID: bd74c00b-02cd-42b4-8d62-514dfab4b217
There are a lot of
things I want to share, from images to code. Roadblocks are often
unexpected, and can be weird as hell e.g. file-naming issues with my
camera that began at the same time that I modified the copyright
information that is stamped into EXIF data. The solution to that
probably involves adopting something like the UC Berkeley calphotos
system http://calphotos.berkeley.edu/,
and writing a bit of code to support a new pipeline. Also known as a
workflow, and which term is used is suggestive of many things. But I
digress. Most popular articles (and at least some software) related
to how to image storage and retrieval are overly simplistic. Duh. In
other exciting news, the Web has been found to be in dire need of an
editor.
Sharing documents
(specifically including code) is also an issue, and one that is a bit
more important to me at the moment.
I don't want to get
into the version control Holy Wars. Use git, mercurial, subversion,
or even one of the proprietary systems. Whatever. If I had to guess,
it would be that how well you can use the tool will in most cases
outweigh the power (and idiosyncrasies) of the tool.
That said, this is
about github, because this post is about sharing.
Github suffers,
periodically, from DDoS attacks, which seem to originate from China.
I say 'seem to' because attribution is a hard problem, and because
US/China cyber-whatever is increasingly politicized, and this trend
is not going to end any time soon.
Points to Ponder
a) Copying of device
contents as border crossings are made. There have been State
Department warnings on the US side of the issue, but at least one
security actor, justly famous for defeating TLS encryption with the
number 3 (that is not a joke, search on Moxie Marlinspike), has been
a victim as well. There is some question as to whether my systems
could be searched without a warrant, due to my proximity to an
international border. Nation-states act in bizarre ways, the concepts
of 'truth' and 'transparency' seem to be a mystery to national
governments, and I do not regard it as impossible that the US would
mount a DDoS on GitHub, if a department of the US government thought
it both expedient and deniable.
b) Is China a
unitary rational actor? On occasion, acts of the Chinese government
seem to indicate a less than firm grasp of what other parts of the
government are doing. A culture of corruption is one issue, but there
are others, such as seeming amazement at adverse western reactions to
an anti-satellite (ASAT) missile test back in 2007. Which was
apparently quite the surprise to western governments, and makes me
question what all of this NSA domestic surveillance effort really
accomplishes. I won't digress into that can of worms, other than to
note that there is much evidence suggesting that the US may not be a
unitary rational actor, either.
Circling Back to
GitHub
The entire point of
a distributed version control system, of whatever flavor, is
availability. Yet there are trade press stories dating back a couple
of years, at least, about widespread outages due to DDoS attacks.
The most recent one that I am aware was in April of this year. In
every case, much panic and flapping of hands ensued. Developers
couldn't work. Oh noes!
That rather blows
the whole point of GitHub out of the water, doesn't it? The attacking
distributed system beat up on your distributed system. Welcome to the
Internet Age, cyber-whatever, yada yada yada. Somewhat paradoxically,
a good defense involves more distribution, and not allowing GitHub to
be a sole point of failure.
The problem is
pipelines. Or, again, workflows. A truly resilient system needs more
than something that has demonstrably had accessibility issues for
years, and the problem is two-fold.
1) There is no
fail-over.
2) The scripts that
drive it all tend to be fragile.
It is entirely
possible to build a robust system, hosted in the DMZ or in the cloud,
as a backup to GitHub. Most of this is just bolting widely available
Linux packages together, and doing the behind-the-scenes work. With
an added component of writing great doc; the system will only be
exercised when things have gone to hell, and everyone is stressed. If
there were ever a time where great doc were a gift from $DEITY, this
would pretty much be it. Because Murphy is alive and well, so some
periodic fail-over test (you do that, right?) probably got skipped
for some reason.
At this point I am
going to be polite and just mention that the DevOps community might
do a bit more work in getting some best practices information to the
public. If GitHub is more important than just free hosting (and it
may not be, for completely valid reasons) please build an adequate
system. It will save you from having to publicly whine about how your
distributed system did not turn out to be resilient.
Friday, April 17, 2015
If Leave the ACM, Some of Their e-Mail Becomes SPAM
Really, people. If I tell my rep that I will not be renewing, most renew-now messages stop. This is not the case with with the list servers. The ACM Bulletin, TechNews, and whatever else you may be subscribed to will continue on their merry way.
At a certain point, this becomes spam. If, after all, I regarded those lists as being extremely valuable, I would likely never have left ACM in the first place.
Just saying.
There's a lot going on right now. which is how it comes to be that my first post of the month is on the 24th. Over a month since my last post. So I have to regard ACM mail as something that should have vanished at the end of last month. Just random stuff that I have to send unsubscribe messages to.
At a certain point, this becomes spam. If, after all, I regarded those lists as being extremely valuable, I would likely never have left ACM in the first place.
Just saying.
There's a lot going on right now. which is how it comes to be that my first post of the month is on the 24th. Over a month since my last post. So I have to regard ACM mail as something that should have vanished at the end of last month. Just random stuff that I have to send unsubscribe messages to.
Wednesday, March 18, 2015
(Some) Books That Seem Important To Me
Commentary::Personal
Audience::All
UUID: 796fa48c-2be5-4e09-a181-a3a3c00bc4a0
Have an image of a stack of books. They are all worth writing about, in contexts that may be surprising. That one on the bottom? SPAM NATION was a recommended purchase in Just Buy Spam Nation. It became a best-seller, which had nothing to do with my efforts (this site only gets a few hundred hits per month), but because Brian Krebs rocks, in terms of consumer security. Which is why I recommend his site. Pardon me, but I seem unable to force my fingers to type 'Blogroll'. It ranks right up there with 'Blogosphere', in terms of suckage.
I mentioned some of these in Four Books on Order, back on March 9.
Followers (both of you) may note that I am now including an Audience identifier, and a UUID. More on that in a future post.
Audience::All
UUID: 796fa48c-2be5-4e09-a181-a3a3c00bc4a0
Have an image of a stack of books. They are all worth writing about, in contexts that may be surprising. That one on the bottom? SPAM NATION was a recommended purchase in Just Buy Spam Nation. It became a best-seller, which had nothing to do with my efforts (this site only gets a few hundred hits per month), but because Brian Krebs rocks, in terms of consumer security. Which is why I recommend his site. Pardon me, but I seem unable to force my fingers to type 'Blogroll'. It ranks right up there with 'Blogosphere', in terms of suckage.
I mentioned some of these in Four Books on Order, back on March 9.
Followers (both of you) may note that I am now including an Audience identifier, and a UUID. More on that in a future post.
Monday, March 9, 2015
Four Books on Order
Commentary::Personal
Now and then you have to blow a hundred bucks or so on books. A Safari subscription at O'Reilly subscription can save you quite a bit on professional expenses, but at the end of the day, you often have to cough up some additional cash.
Today, the total was 4. One does not count: The Hydrogen Sonata, by Ian Banks. Pure entertainment.
So what does count? The following three.
Now and then you have to blow a hundred bucks or so on books. A Safari subscription at O'Reilly subscription can save you quite a bit on professional expenses, but at the end of the day, you often have to cough up some additional cash.
Today, the total was 4. One does not count: The Hydrogen Sonata, by Ian Banks. Pure entertainment.
So what does count? The following three.
- Hackers - Steven Levy. I am looking for support for my argument that the crypto wars never ended. The NSA would then be a continuing chapter in that game, as described very well by any Bamford work you would ever care to read. _Hackers_ is on my Safari bookshelf, but that is not the same thing as being able to refer to page numbers in the original edition.
- How Learning Works: Seven Research-Based Principles for Smart Teaching - Susan A. Ambrose. Widely acclaimed, and we damned sure need better methods of teaching security. Or any other subject, for that matter.
- Capital in the Twenty-First Century - Thomas Piketty. This book has already had enormous press, so I won't write much about it here. I will mention that I regard economics as a highly-politicizied proto-science, at best. But without bringing economics, in whatever state, into the mix, neither security practioners or researchers can really have much much effect.
Tuesday, February 17, 2015
An Update is Long Overdue
Commentary::Personal
It is difficult to believe that my last post was over a month ago. Time flies. Particularly when there is not much of it that could be classified as 'spare'. I am still entranced with the idea of a Semantic Web, but that has been the case for a very long time now. The practical pitfalls are many, and often seem insurmountable.
Lately, the guiding wisdom has been to not let the perfect be the enemy of the good. If you wait until you create some Enormous Perfect Thing to ship anything, you will never ship. In the case of a blog, you will also fail to push possibly useful information out. This is old wisdom, and long predates blogs, but it is still an easy trap to fall into. This has become a mantra that I chant to myself multiple times per day.
It also has implications for practical software development in a modern distributed environment. This is still old news, but it bears mentioning again. Huge and sudden changes are far less likely to be accepted than slower incremental ones, particularly in the absence of a certain degree of coordination. A game plan, if you will. I am more or less operating in the limit, where that might be defined as a situation where I am hesitant to accept my own changes. In other words, self-doubt.
As I indicated in New Year, and Changes On the Way, I expect to fail at this. But, as I said in that post, "This a case where not failing horribly would be a win."
The major thing I am contending with is, of course, name spaces. In my (even more) ignorant youth, I used to be annoyed at, say, antivirus companies all having a unique name for the same bit of malware. It reminded me of financial services classifying the same company in differing ways. One service would declare Foo Corp a member of the Services sector, while another would call it a member of the Energy sector. It smacked of marketing, and an attempt at locking customers into a proprietary classification scheme, which of course promised enormous benefits for the investor.
What, then, would be made of a corporation that provides financial transaction processing services, but is also a supplier of systems (hardware and software) for those that want to do it in-house? Is this a vendor of hardware, software, or services? Is it dependent on sales into those market segments? If so, how are changes over time accounted for?
With the best corporate will in the world, presuming a completely altruistic (yeah, right) outlook, with no intent of lock-in, this is a difficult problem. Nor can you consider, for the most part, governmental data sources as above the commercial fray, hence reliable data sources. The United States Department of Labor, Bureau of Labor Statistics, is incapable of providing data regarding even Systems Administrator statistics, much less the security specialties. Namespace issues are once again at the root of the problem. We have no reliable sources of data on whether security resources are increasing to meet increasing threats. What we have is marketing.
The problem continues in how one might evaluate the changing reliability of security news sources, including private and academic security researchers, the corporate publishers of white papers and press releases, the security trade press, etc.
So, am I working on some Impossible Shining Thing? Yes, if it is considered in absolute terms; some standard that is widely accepted, the evolution of which can be tracked over time, and continually evaluated by consumers as to efficacy.
So not letting the perfect be the enemy of the good returns to center stage. I might possibly succeed at creating something that is at least some sort of improvement over the current state of things. Given the huge and ongoing waste of resources, that would be a huge win. I would gladly settle for at least stimulating a bit more thought and discussion about the nature and extent of the problem, which seems sadly absent.
OTOH, this is a very low-traffic blog. Even that is a long shot.
It is difficult to believe that my last post was over a month ago. Time flies. Particularly when there is not much of it that could be classified as 'spare'. I am still entranced with the idea of a Semantic Web, but that has been the case for a very long time now. The practical pitfalls are many, and often seem insurmountable.
Lately, the guiding wisdom has been to not let the perfect be the enemy of the good. If you wait until you create some Enormous Perfect Thing to ship anything, you will never ship. In the case of a blog, you will also fail to push possibly useful information out. This is old wisdom, and long predates blogs, but it is still an easy trap to fall into. This has become a mantra that I chant to myself multiple times per day.
It also has implications for practical software development in a modern distributed environment. This is still old news, but it bears mentioning again. Huge and sudden changes are far less likely to be accepted than slower incremental ones, particularly in the absence of a certain degree of coordination. A game plan, if you will. I am more or less operating in the limit, where that might be defined as a situation where I am hesitant to accept my own changes. In other words, self-doubt.
As I indicated in New Year, and Changes On the Way, I expect to fail at this. But, as I said in that post, "This a case where not failing horribly would be a win."
The major thing I am contending with is, of course, name spaces. In my (even more) ignorant youth, I used to be annoyed at, say, antivirus companies all having a unique name for the same bit of malware. It reminded me of financial services classifying the same company in differing ways. One service would declare Foo Corp a member of the Services sector, while another would call it a member of the Energy sector. It smacked of marketing, and an attempt at locking customers into a proprietary classification scheme, which of course promised enormous benefits for the investor.
What, then, would be made of a corporation that provides financial transaction processing services, but is also a supplier of systems (hardware and software) for those that want to do it in-house? Is this a vendor of hardware, software, or services? Is it dependent on sales into those market segments? If so, how are changes over time accounted for?
With the best corporate will in the world, presuming a completely altruistic (yeah, right) outlook, with no intent of lock-in, this is a difficult problem. Nor can you consider, for the most part, governmental data sources as above the commercial fray, hence reliable data sources. The United States Department of Labor, Bureau of Labor Statistics, is incapable of providing data regarding even Systems Administrator statistics, much less the security specialties. Namespace issues are once again at the root of the problem. We have no reliable sources of data on whether security resources are increasing to meet increasing threats. What we have is marketing.
The problem continues in how one might evaluate the changing reliability of security news sources, including private and academic security researchers, the corporate publishers of white papers and press releases, the security trade press, etc.
So, am I working on some Impossible Shining Thing? Yes, if it is considered in absolute terms; some standard that is widely accepted, the evolution of which can be tracked over time, and continually evaluated by consumers as to efficacy.
So not letting the perfect be the enemy of the good returns to center stage. I might possibly succeed at creating something that is at least some sort of improvement over the current state of things. Given the huge and ongoing waste of resources, that would be a huge win. I would gladly settle for at least stimulating a bit more thought and discussion about the nature and extent of the problem, which seems sadly absent.
OTOH, this is a very low-traffic blog. Even that is a long shot.
Thursday, January 1, 2015
New Year, and Changes On the Way
This is nothing to do with new year
resolutions, as I discovered years ago that those simply do not work
for me. It is more about looking at various note files which will
become future posts. The good news (for me, anyway) is that there is
material for something like 30 posts in various states of completion.
Some of it would take as little as half an hour of work, if I were to
continue as I have in the past.
I don't want to do that. I don't know
that it has been particularly effective, and behind the scenes I have
been organizing the work into themes that I want explore in some
depth, and thinking about how to make the intended audience for any
given post explicit. Do I need a simple classification scheme, or do
I need to think in terms of an ontology? How is what I want to do
constrained by the Blogger platform?
The simplest case was in doing the
initial blog layout. I wanted the date of publication right up top.
There is a lot stale information on the Web, and I think that part of
that is purposeful. There is a business incentive (so-called Long
Tail economics of the Web) for making publication dates harder to
find. It is also a complete pain in the ass for many members of my
tribe (security workers) who may have to digest dozens of news
articles, blog posts, etc., every day.
I would also like to make life a bit
easier for the tool builders who are creating the next generation of
automated ingestion systems. So I would like to keep things as stable
as possible. Meaning don't change the current date format, and put
some thought into the forthcoming 'Intended Audience', etc.
I expect to fail at this. The design
space begins to resemble API design (difficult to get right) and also
involves some Semantic Web thought. The Semantic Web has, rather
famously, never taken off. This a case where not failing horribly
would be a win.
Do not expect this stuff immediately.
It has been a couple of weeks since I posted, and it is going to be
difficult to post again until sometime around mid-January. Sorry
about that, but life intrudes.
Monday, December 15, 2014
Today Was an Infrastructure Day
Sometimes we are all under the hammer
of time, and things have to happen Right Now. But now and then most
of us get lucky, and slack time. I treasure those days, because I can
drag out that mental, digital, or physical TODO list of things that
need to be done for the future. And I pound on it, because that it
some of the most interesting work that I do, and there is always a
reward of some sort.
Most times, it involves working on
infrastructure; the scaffolding that we have to have in order to keep
doing what we do, only better. So slack time doesn't mean go sit on a
beach. Which is just as well, in my case. This is December, and
Oregon almost perfectly fails to resemble Hawaii. I touched on this back in August, in Optimize for the Exploration of Ideas.
So, What Did I Do?
I organized some stuff. I have a
directory named 'REDACTED' (naming specific directories and what they
contain is a huge information leak, if you care about security) that
accumulates ideas, design notes, TODOs, etc., on where various
internal projects need to go. It can accumulate a lot of cruft, and
lose value as a planning tool. It needs all the care I can give it,
and I measure success in this at least partially in how much useless
crap I deleted.
I rebuilt some stuff. Because sometimes
it only takes a few changes in a tool (either physical or software)
to radically improve your capabilities. The bad guys are innovating
at a tremendous pace, with the value and speed benefits that
innovation always provides. If we cannot more than match that, we are
hosed. Economics does not much care whether your hat is black or
white.
I wrote some stuff. I never intended to
become a writer of any description, but I have always admired
talented writers. It was writers that turned me into a technologist
at an early age, and the power to steer a life is something to be
respected. But I follow a couple of writer blogs, and they tend to
advocate things like writing a minimum number of words per day, which
my life completely fails to allow. I wrote a lot of fragmentary, but
hopefully clear notes to myself. I wrote this post, and hopefully
moved a few others closer to publication. I also wrote final versions
of template files that will change my software documentation workflow
completely. Perhaps that should have fallen under 'rebuilt'.
Thursday, December 11, 2014
Good News: Power Failure
Rather expected it, actually. There are high winds blowing in from the Oregon coast, the weather news is full of it, etc.
So there I was, minding my own business, coincidentally thinking about data QA and reaction times, for an entirely unrelated. Which makes for a very sweet coincidence, as now I've pulled data from a couple of scripts I wrote to check the APC UPS. There is frequently a PostGreSQL db server running on this machine, and the combination of databases and unreliable power always ends badly.
That usually happens sooner rather than later, but like most people I tend to put off characterizing what things actually look like as systems fall down. I advocate doing this all the time, to the extent of of periodically killing test systems at the power distribution panel. "Really. Now and then, just replicate into a test environment, and flip the breakers".
That can be a huge pain in the ass, but there is really no other way to be absolutely certain. Cloud is not the answer to this issue. Or, at best, it can only be part of the solution. There are many examples of cloud failure.
Today, I got some great data on UPS drain and recovery, and found a problem with time-stamping of notifications. Discovering that bug in my code is a win. As is jogging me to post on the topic of a bug in the Linux APC UPS monitor daemon. Which I (obviously) have had no control over, and served as an example of why greater care should be taken than previously before turning on SELinux enforcement.
Things Are Going to Go Wrong
As long Murphy is alive and well, and Murphy seems to be immortal, things will continue to go all pear-shaped, at the worst possible times. I almost wrote 'periodically pear-shaped', but we don't always have the benefit of periodicity. Aside from the Big Three of periodic FUBAR announcements (Microsoft, Adobe, and Oracle), anyway. I might justifiably add OpenSSL and other Open Source projects, but the data to back that up is a whole new post. That is not going to happen today. Which is just as well, because the ongoing incompetence of Sony beggars the imagination. I don't even want to think about it, beyond being very happy that I am not on their security team.
Today, As An Example
From a security perspective, we are most concerned with the CIA triad of Confidentiality, Integrity, and Availability of data. Power problems on database systems will cause issues with integrity and availability, as mentioned above. Confidentiality only becomes a factor if disparate systems with responsibility for authentication/authorization fail open if a remote system is not available. That is rare, these days. Possibly because it is an easy test. So run it, just to be certain. Really. It's just a temporary firewall rule. And, as always, make it a test, so that pass/fail is always recorded.
But, we can never miss an opportunity to get better. Particularly under circumstances so benign as a power outage. Which, for people focused on security rather than pure availability, really is benign.
So, We Are Back to Logs
I first mentioned logs in We Still Fail at Log Analysis back in July of 2013. Nothing much has happened since then to change my opinion. 18 months, and little or no progress on an operational problem that has been with us for time out of mind. That is a bit discouraging, so I feel the need to visit this issue again, and probably not for the last time.
Please look at log policy again. Logging takes many forms, of course. System and application state and performance data are both vital. Were those recorded? Was it possible for an adversary, possibly internal, to avoid detection by shutting down a remote log host, or a network path to that host? In a virtualized environment, do you have records of what machines were spun up or migrated, and the security posture of those systems? If so, are those records amenable to analysis, or are they just data for the sake of data?
That last question is not meant to imply that you may be doing anything obviously wrong, BTW. Effective means, which will stand the test of time, have yet to evolve. I regard this as an open research question. Which is a bit sad, considering how badly the failures have been in legacy environments.
Possibly, for some environments, the future lies solely in data exfiltration detection.
So there I was, minding my own business, coincidentally thinking about data QA and reaction times, for an entirely unrelated. Which makes for a very sweet coincidence, as now I've pulled data from a couple of scripts I wrote to check the APC UPS. There is frequently a PostGreSQL db server running on this machine, and the combination of databases and unreliable power always ends badly.
That usually happens sooner rather than later, but like most people I tend to put off characterizing what things actually look like as systems fall down. I advocate doing this all the time, to the extent of of periodically killing test systems at the power distribution panel. "Really. Now and then, just replicate into a test environment, and flip the breakers".
That can be a huge pain in the ass, but there is really no other way to be absolutely certain. Cloud is not the answer to this issue. Or, at best, it can only be part of the solution. There are many examples of cloud failure.
Today, I got some great data on UPS drain and recovery, and found a problem with time-stamping of notifications. Discovering that bug in my code is a win. As is jogging me to post on the topic of a bug in the Linux APC UPS monitor daemon. Which I (obviously) have had no control over, and served as an example of why greater care should be taken than previously before turning on SELinux enforcement.
Things Are Going to Go Wrong
As long Murphy is alive and well, and Murphy seems to be immortal, things will continue to go all pear-shaped, at the worst possible times. I almost wrote 'periodically pear-shaped', but we don't always have the benefit of periodicity. Aside from the Big Three of periodic FUBAR announcements (Microsoft, Adobe, and Oracle), anyway. I might justifiably add OpenSSL and other Open Source projects, but the data to back that up is a whole new post. That is not going to happen today. Which is just as well, because the ongoing incompetence of Sony beggars the imagination. I don't even want to think about it, beyond being very happy that I am not on their security team.
Today, As An Example
From a security perspective, we are most concerned with the CIA triad of Confidentiality, Integrity, and Availability of data. Power problems on database systems will cause issues with integrity and availability, as mentioned above. Confidentiality only becomes a factor if disparate systems with responsibility for authentication/authorization fail open if a remote system is not available. That is rare, these days. Possibly because it is an easy test. So run it, just to be certain. Really. It's just a temporary firewall rule. And, as always, make it a test, so that pass/fail is always recorded.
But, we can never miss an opportunity to get better. Particularly under circumstances so benign as a power outage. Which, for people focused on security rather than pure availability, really is benign.
So, We Are Back to Logs
I first mentioned logs in We Still Fail at Log Analysis back in July of 2013. Nothing much has happened since then to change my opinion. 18 months, and little or no progress on an operational problem that has been with us for time out of mind. That is a bit discouraging, so I feel the need to visit this issue again, and probably not for the last time.
Please look at log policy again. Logging takes many forms, of course. System and application state and performance data are both vital. Were those recorded? Was it possible for an adversary, possibly internal, to avoid detection by shutting down a remote log host, or a network path to that host? In a virtualized environment, do you have records of what machines were spun up or migrated, and the security posture of those systems? If so, are those records amenable to analysis, or are they just data for the sake of data?
That last question is not meant to imply that you may be doing anything obviously wrong, BTW. Effective means, which will stand the test of time, have yet to evolve. I regard this as an open research question. Which is a bit sad, considering how badly the failures have been in legacy environments.
Possibly, for some environments, the future lies solely in data exfiltration detection.
Friday, April 18, 2014
Well, I Thought I Had a Spare Drive
But it turns out that my spares
collection is FUBAR. Here's the thing. I was planning on rebuilding a
lab system this weekend. I have always enjoyed messing with hardware
of one sort or another, I've built many systems over the years, and I
have a bit of an ingrained work flow optimized for minimal SOHO
office disruption, etc. Move that pile of books out of the corner and
re-shelve, fill in the space with the parts collection, etc.
The parts collection was where I ran
into a bit of snag.
I get a bit uncomfortable when I am
down to a single spare general-purpose SATA drive. For RAID, things
are a bit more complex, as I like matching firmware versions, and I
also like to keep a pristine portable drive around. What I have is a
new factor, in that I can't use a basic 1 TB Caviar Black.
It's been sitting for several years, still in original packing. The
only thing I've ever done regarding this drive was to print the
output of uuidgen(1) and purchase details, and stick it into the bag.
Due to the nature of my work, I need to unambiguously track which
hard drives are used by which systems. Which is a topic for another
post.
Now, this drive has to be sequestered,
because I seem to have built up a collection of performance notes
which involved I/O to an identical drive. I may want to revisit those
numbers, well after the normal life of a drive, and it is worth it,
to me, to remove a variable from any comparison I might care to make.
An opportunity to reduce the complexity of a future data analysis,
for the cost of a consumer hard drive, is a no-brainer.
So I'm down to one consumer hard drive.
If I use it, I will have no spare until a replacement arrives. That
isn't something I would do, except in an emergency, which this isn't.
I could just drive into town, buy one, and get on with the job. But
it is probably wiser to buy three online; the odds of getting
identical firmware are good, so I get more down-the-road flexibility.
I could make a reasonable case for buying six, and may do that.
That's a bit of a bummer, as I did want
to get that hardware upgrade done this weekend. If, for whatever
reason, it can't happen next weekend, then it is no longer a casual,
do it ahead of the need thing. It rises to the level of Important.
Murphy is alive and well, and all it will take is a call from a
client.
This sort of thing is best avoided, and
that it happened in this case is my own FUBAR. I failed to allow for
all the factors that might influence the effective size of the
spares pool. That informal work flow bit me, and better systems
administrators than I can now commence laughing. Lesson learned, and
I hope that others do not make the same mistake.
Friday, August 2, 2013
Things are really busy right now.
I have a new project: documenting what I did, and the rationale for the choices I made, for one of the recent data analysis projects. I always write docs, but this is more in the spirit of a HOWTO for people that need some basic instruction on how data analysis pipelines (or workflows, if you will) are commonly constructed on Linux, and does not depend on humans clicking around, enduring the horrors of statistics in Excel, etc.
It's mostly about pre-processing data, feeding only what you need into a sane database (why give up three orders of magnitude in speed for the bits that will not have relational queries run), when to do matrix math, when to fire a decision-support plot because a threshold has been exceeded, etc.
Somewhat at variance with physics pipelines, it was written in bash, Python, R, and Go. I should do a post about that. But like I said, things are busy right now.
Fedora 17 reached EOL on 7/30/13
Why does that matter? Well, Fedora is regarded as upstream of Red Hat Enterprise Linux. RHEL, and derivatives such as CentOS, ScientificLinux, and Oracle Linux (though Oracle will never admit that). What that means is that Red Hat chooses a moment to grab the current Fedora distribution, and make some of the various bits more robust. Meaning supportable, at sane cost structures. That forms the basis of the forthcoming Red Hat Enterprise Linux. Running a current, or near-current, Fedora provides insight into the oncoming RHEL, which will be RHEL7, by the end of the year.l
This is useful, particularly as this will be the most powerful RHEL ever, by a wide margin. Mondo cloudy stuff is going to be in there. That should be another post; it is by no means all marketing.
But right now, I have to rebuild some lab machines. This isn't a huge deal--that's what labs are for. But it will keep me busy for a bit, because I have to characterize what I am doing.
How do you secure this stuff?
As Frank Zappa once wrote, "The crux of the biscuit is the apostrophe." Secure what? Against what threat? At what cost?
I have never been a fan of PCI-DSS. The standard cannot change rapidly enough to reflect changes in the threat envelope. Compliance costs are out of control, and it is not clear to me that there is any rational means of choosing any particular solution to a PCI-DSS line-item. Sometimes I hate to even talk about PCI-DSS; there are other requirements in other industries that are more interesting (medical record security comes to mind), and some things (design flaws in cryptographic protocols, etc.) apply to any industry.
The basics apply in any environment. Control access, authentication, and authorization, and the majority of your risk goes out the window. This is doable, even via bash scripting. From a Director Information Security at Fiserv (Acumen platform)
Write a master script that calls subscripts by control number. The downside is that it adds complexity; you will be touching some configuration files more than once. It works, and assessors love it. You do however, need a capable and auditable version control and build system. Git works fine, if you bolt on some additional tooling.
The point is that RHEL7 will offer more controls--you will have more power to meet any standardization, legal, or regulatory challenge.
It's mostly about pre-processing data, feeding only what you need into a sane database (why give up three orders of magnitude in speed for the bits that will not have relational queries run), when to do matrix math, when to fire a decision-support plot because a threshold has been exceeded, etc.
Somewhat at variance with physics pipelines, it was written in bash, Python, R, and Go. I should do a post about that. But like I said, things are busy right now.
Fedora 17 reached EOL on 7/30/13
Why does that matter? Well, Fedora is regarded as upstream of Red Hat Enterprise Linux. RHEL, and derivatives such as CentOS, ScientificLinux, and Oracle Linux (though Oracle will never admit that). What that means is that Red Hat chooses a moment to grab the current Fedora distribution, and make some of the various bits more robust. Meaning supportable, at sane cost structures. That forms the basis of the forthcoming Red Hat Enterprise Linux. Running a current, or near-current, Fedora provides insight into the oncoming RHEL, which will be RHEL7, by the end of the year.l
This is useful, particularly as this will be the most powerful RHEL ever, by a wide margin. Mondo cloudy stuff is going to be in there. That should be another post; it is by no means all marketing.
But right now, I have to rebuild some lab machines. This isn't a huge deal--that's what labs are for. But it will keep me busy for a bit, because I have to characterize what I am doing.
How do you secure this stuff?
As Frank Zappa once wrote, "The crux of the biscuit is the apostrophe." Secure what? Against what threat? At what cost?
I have never been a fan of PCI-DSS. The standard cannot change rapidly enough to reflect changes in the threat envelope. Compliance costs are out of control, and it is not clear to me that there is any rational means of choosing any particular solution to a PCI-DSS line-item. Sometimes I hate to even talk about PCI-DSS; there are other requirements in other industries that are more interesting (medical record security comes to mind), and some things (design flaws in cryptographic protocols, etc.) apply to any industry.
The basics apply in any environment. Control access, authentication, and authorization, and the majority of your risk goes out the window. This is doable, even via bash scripting. From a Director Information Security at Fiserv (Acumen platform)
"we did get our PCI-DSS ROC and the assessors loved the hardening scripts and the way you listed the hardening steps by control number."
Write a master script that calls subscripts by control number. The downside is that it adds complexity; you will be touching some configuration files more than once. It works, and assessors love it. You do however, need a capable and auditable version control and build system. Git works fine, if you bolt on some additional tooling.
The point is that RHEL7 will offer more controls--you will have more power to meet any standardization, legal, or regulatory challenge.
Saturday, June 8, 2013
Saturday security humor
Because
- Trust can fail in unanticipated ways
- It is Saturday, and I am looking at a computer
As a professional security consultant, I recommend that you have a nice day.
Sunday, May 5, 2013
I Really Need a Document Management System
[gregm@feynman ~]$ du -hs $HOME
52G /home/gregm
[gregm@feynman ~]$ find $HOME -type f -name *pdf | wc -l
5056
[gregm@feynman ~]$
That 'working set' is nothing, in terms of storage requirements; it's a speck on Terrabyte drives. The requirements are so low because there are very few media files. I don't 'do' media, as a rule. I want my favorite entertainment producers to get paid, so that they can keep entertaining me. I buy DVDs, have only ripped MP3s from CDs that I bought (years ago, and still have) etc.
But there are also large numbers of text files (notes and code), email, spreadsheets, etc. That stuff, combined with the PDFs, is important, in the sense that having this stuff on tap, efficiently accessible, is directly related to whether I can make a living at protecting people and the things that are important to them. I'd obviously like to continue to do that, as it's one of the more worthwhile things I can do with my life. And life is all too short.
Currently, I am most concerned with the PDFs. Some small percentage are chapter-by-chapter downloads of books from my Safari account. Those can be recreated, and there are other examples of PDFs that I am not very concerned about. But a large number, were they lost, would be difficult to replace, for a variety of reasons.
- An academic has changed positions
- A commercial entity has ceased operations, deleted old files, etc.
- A security industry private researcher has lost interest and allowed their site to lapse
- I do not have data on why that missing file is important, and/or the source
- Other. And 'Other' is large.
I'm very old-school about having a well-organized file system; I know how my directories are organized, and I'm far from reliant the file indexing systems that bog so many systems down. Nor am I fan of various 'tagging' systems; their usefulness seems ephemeral in that it's mostly useful in the scope of a single project, or a small number of related projects. It is perhaps likely that these would also be ephemeral, while I am also interested in the broad sweep of history, and how these things evolve over time.
The notion of keeping a good mental reference breaks down at 5000 files. Is something filed under privacy, breaking anonymized data, or what?
Monday, March 18, 2013
Attribution: On the Shoulders of Giants
If I have seen further it is by standing on the shoulders of giants.
-- Isaac Newton, 1676
Attribution is a vital thing. One of the more fubar things about 'security researchers' is that they do not always rigorously credit people that laid the foundation that they are building upon.
If we have attribution, we can trace the evolution of thought in all of science and mathematics. The knowledge that we would impart to others gains context. This isn't a corporate, marketing thing, or the current round of patent fights; it's more about how your children will lead better lives. Attribution is important. because it is the scaffold upon which we build, and can trace, the most important advances the human race has ever achieved.
No, I am not writing about the development of brewing. Yes, beer is important. But I'm not going there right now. It's more important that I say something that should probably determine whether this thing is worth the time it takes you to read it.
I have a couple of rules about how I will post, the first of which is that I don't name people or clients without permission and attribution, and I don't believe in failing in either.
I have permission to admit I worked a gig at Fiserv. I negotiated that going in, as it was obviously going to be a fairly long-term thing, and I was once an employee there, doing a lot of security-related things related to Linux, HP-UX, cloud-facing servers, etc.
Where I can do proper attribution, I will. In some cases, I have permission from Fiserv. In other cases, I was on other gigs, and absolutely do not have permission to do any traceable attribution. That was also understood from the start, and anything related to that work can only be discussed in theoretical terms. I wish it were otherwise, but this is the fubar world we live in.
Bye for now. There is yard work to be done, this is the Northwest, and the weather forecast (fubar that it is) says rain for the next few days.
-- Isaac Newton, 1676
Attribution is a vital thing. One of the more fubar things about 'security researchers' is that they do not always rigorously credit people that laid the foundation that they are building upon.
If we have attribution, we can trace the evolution of thought in all of science and mathematics. The knowledge that we would impart to others gains context. This isn't a corporate, marketing thing, or the current round of patent fights; it's more about how your children will lead better lives. Attribution is important. because it is the scaffold upon which we build, and can trace, the most important advances the human race has ever achieved.
No, I am not writing about the development of brewing. Yes, beer is important. But I'm not going there right now. It's more important that I say something that should probably determine whether this thing is worth the time it takes you to read it.
I have a couple of rules about how I will post, the first of which is that I don't name people or clients without permission and attribution, and I don't believe in failing in either.
I have permission to admit I worked a gig at Fiserv. I negotiated that going in, as it was obviously going to be a fairly long-term thing, and I was once an employee there, doing a lot of security-related things related to Linux, HP-UX, cloud-facing servers, etc.
Where I can do proper attribution, I will. In some cases, I have permission from Fiserv. In other cases, I was on other gigs, and absolutely do not have permission to do any traceable attribution. That was also understood from the start, and anything related to that work can only be discussed in theoretical terms. I wish it were otherwise, but this is the fubar world we live in.
Bye for now. There is yard work to be done, this is the Northwest, and the weather forecast (fubar that it is) says rain for the next few days.
Sunday, March 17, 2013
Hello, and welcome to fubarnorthwest
Hello, and welcome to fubarnorthwest. If you don't get the title, that's because it's fubar; a term anyone who has any business here already understands. The northwest bit got tacked on because that's where I live, I love it, and all the cool names were already taken.
I want to talk about a lot of things, mostly related to how we should fix the huge problems in computer-related security. I don't know if this venue will meet my needs. In one sense, I know it won't: personal privacy is security personalized, and Google is dedicated to invading privacy. It's a big piece of their business model.
I am a big fan (as much as I am a fan of anything in a fubar world) of irony.
If we lay aside the marketing, and attempt to get to the truth of things, there is little in this world that isn't fubar. Not operating systems, the smartphone you favor, your best-loved camera or programming language, your software repository or your system of government.
It's all fubar to most of us, because we all have different priorities.
I want to talk about a lot of things, mostly related to how we should fix the huge problems in computer-related security. I don't know if this venue will meet my needs. In one sense, I know it won't: personal privacy is security personalized, and Google is dedicated to invading privacy. It's a big piece of their business model.
I am a big fan (as much as I am a fan of anything in a fubar world) of irony.
If we lay aside the marketing, and attempt to get to the truth of things, there is little in this world that isn't fubar. Not operating systems, the smartphone you favor, your best-loved camera or programming language, your software repository or your system of government.
It's all fubar to most of us, because we all have different priorities.
Subscribe to:
Posts (Atom)