Human error is something to be engineered around, not lamented

Random thought of the day, apropos of this comment by Jon Callas:

We know that the attack against EMC/RSA and SecureID was done with a vuln in a Flash attachment embedded in an Excel spreadsheet. According to the best news I have heard, the Patient Zero of that attack had had the infected file identified as bad! They pulled it out of the spam folder and opened it anyway. That attack happened because of a security failure on the device that sits between the keyboard and chair, not for any technology of any sort.

Quite frankly, if this is what qualifies as human error in a security system, then we’re all in deep trouble. We’re stuck with it. We’re born to it.

I’ll assume one of two things happened here:

  1. An AV scanning system identified a known signature inside of an attachment, recognized that this could be an exploit, and responded to this very serious issue by moving the file into the SPAM folder, where it joined many other legitimate messages that were improperly marked as spam.
  2. A Spam filter noticed something funny about a header, and moved the file into the SPAM folder, something it probably does eight times per week for no reason at all.

Unless your users are superhuman, the problem here is not the user. It’s the system. If the file legitimately contained a vulnerability, it shouldn’t have been moved into the SPAM filter where it could easily be mistaken for a random false positive.

If, on the other hand, the problem was just something to do with the headers, then maybe the user was just doing what was normal — pulling a probable false positive out of their spam folder, just like they did every day.

People are not superhuman. They react to the inputs you give them: GIGO applies. If security systems give people crap inputs, then they’ll make crap decisions. Fixing this problem is our job. We don’t get to complain every time a user does something perfectly understandable in response to bad data that we (security system designers) give them.

And of course, this leaves aside the basic fact that the master seed was available to this attack in the first place, something that boggles the mind… But I guess that’s all been said.

3 thoughts on “Human error is something to be engineered around, not lamented

  1. I agree with you 100%. I'm an well-known advocate of making systems that don't leave the users with decisions, because we know they will make the wrong one.

    The major reason I know this is that I know that *I* often make the wrong one when presented with a security decision.


  2. Hello, this is a really fas­ci­nat­ing web blog and I have loved read­ing sev­eral of the arti­cles and posts con­tained upon the site, sus­tain the great work and hope to read a lot more excit­ing arti­cles in the time to come. Thank you so much.
    do follow blog

Comments are closed.