Performance Art: Why Secure E-mail Never Went Mainstream…

…or “Why Johnny doesn’t want to encrypt.”

There’s been a been a thread on Perry Metzger’s “CRYPTOGRAPHY” maillist, about “Why the poor uptake of encrypted email?”.

There was the usual citation of “Why Johnny Can’t Encrypt” (summary: cryptography is hard for people to grasp and harder for programmers to make user-friendly) – but because of my work on Adriana’s Mine Project I see another reason which is not really covered in the paper.

To try drive the point home I submitted my response encrypted under rot13, and bless him Perry was kind enough to post it on verbatim without querying it.

So what I have done by “protecting” that message with cryptography is deny it indexing by Google (cf: deny it integrated search like Spotlight) – make it harder to find, read, quote… all the same side-effects that “secure” e-mail imposes upon a secure e-mail’s recipients in pursuit of solving a transport security problem.

I attach the (unencrypted) body of the e-mail, below; the final few paragraphs are the key ones for the next decade of computing.

I believe there are some fundamental axioms, or postulates of computing that are flat-out wrong – or will be, soon – but which strongly inform our notions of “how computing should be”; to me these are like the the “parallel postulate” of Euclidean geometry – when they fall, when the world suddenly sees their “self-evident” nature is false, then exciting things will happen.

What are these computational parallel-postulates? Stuff like:

  1. users can’t be online 24×7 (cf: ADSL, hosting providers)
  2. (also phrased as) connection time is prohibitively expensive (ie: dial-up)
  3. users are not able / not allowed to serve information to third-parties (cf: blogs, feeds, apache)
  4. users can’t afford to host a server to act upon their behalf (cf: hosting, iPhone)
  5. users can’t store heaps of data (cf: terabyte hard disks)
  6. secure end-to-end communication is not possible between servers / across the internet (cf: im, skype)
  7. the client-server programming model means we should implement hierarchies, not peer-to-peer meshes (iTunes Store vs: Bittorrent) [updated 1]
  8. the web is a series of hierarchies, not a mesh (also known as the “deep linking is bad” argument) [updated 2]
  9. computers need to be rebooted daily/weekly [updated 3]

[ed: I am adding extra entries to the above list, denoted with square brackets]

…and a host of others; I could spend a day enumerating these fallacies and defunct limitations. Almost the entire identity-provider industry is based upon numbers 1, 3, 4 and 6 – whilst people gape at the extraordinary power of Bittorrent and the shocking proposition of OpenID without seeing them as logical consequences of overturning numbers [1,2,3,5,7] and [3,4,6] respectively.

This is one of the reasons I like The Mine Project – it kicks over a bunch of those rules – most, perhaps all of them – and in such a new space very exciting things may happen.

Anyway – herewith the e-mail:

Beyond the “Why Johnny” paper – focusing upon usability – I think there is a higher problem of interoperability and information-access at play here.

There can be no access to your mail without use of a client if you are using cryptography – even ROT13 – and this alone is a big problem, because mediated access to your e-mail is *really* painful.

For some 15 years I used mh/nmh/exmh (latterly with fetchmail), then moved to Mail.app, recently tried Thunderbird for a few months, and am re-considering nmh for long-term archiving of e-mail. I also use my iPod, three laptops with varying species of Unix, and a 3G phone to access e-mail. Occasionally I still copy stuff out of /var/mail/.

I would have suffered immensely were I required to use a particular crypto-enabled client to deal with my e-mail at each stage, or were I required to use historical crypto-clients to access older mails.

Anyone whose college thesis is in WordPerfect on a 5.25″ floppy at the back of a closet somewhere, should understand this problem.

To this day Project Gutenberg uses flat ASCII as a lowest common denominator format, and similarly I need my e-mail in the simplest form so that I can grep it, perl it, quote it and search it.

So “why has encrypted e-mail failed?” I suspect that static data encryption revolts against the nature of personal communication and the needs of personal information re-use.

For comparison, consider the convergence of instant messaging and e-mail – they are becoming ever more alike, but the former mostly relies upon end to end transport security, often assuming that the privacy of logs at either end are at the whim of *that* user.

For some reason this works rather well; as security geeks we complain about it, but there have been many times when Skype has bailed me out of trouble with its ability to drill through almost anything and provide me with messaging and file-transfer.

Similarly AIM, Jabber, GChat – all of which I happily run with OTR – give me necessary mostly-secure communication.

In the world of e-mail the problem is that the end-user inherits a blob of data which was encrypted in order to defend the message as it passes hop by hop over the store-and-forward SMTP-relay (or UUCP?) e- mail network… but the user is left to deal with the effects of solving the *transport* security problem.

The model is old. It is busted. It is (today) wrong.

It’s like ordering lobster bisque, and having a live lobster turn up at your table; what you want is in there – heavily armored – and yes you can render what you receive into what you actually desire; BUT it’s messy and you’re really stuck unless you have a mouli, a saucepan and a small PGP hotplate at hand.

And of course you have to archive copies of the lobster, not the soup.

S/MIME and its bretheren exist to simultaneously address the security of [both] data in motion and data at rest – but people don’t want the latter in the form that it provides, because it inhibits interoperability and usability at a level above the “this software sucks” matter…

And if the “data in motion”/”end to end” security issue is being addressed by things like IM/OTR and Skype, then perhaps “secure” e-mail will soon go the way of Telnet and FTP?

11 Replies to “Performance Art: Why Secure E-mail Never Went Mainstream…”

  1. Actually, I can see two problems which cause user-to-user encrypted e-mail to have not been broadly taken up:

    1) Secure, reliable, ubiquitous, trustworthy, cheap, simple public key storage.
    2) The energy barrier to make it simple and obvious. (Users are focused on the task of sending the information, anything else is a hindrance and noise.)

    On another topic further down your item you talk about the power of bittorrent. It is indeed a powerful tool. Highly useful for penny pinching businesses to offload the costs of distribution onto their customers (and the people they’re sharing their contention with on ADSL/cable TV connections) and the ISPs, which have a very shaky business models at the best of times. It (and other peer-to-peer applications) also have a habit of targeting and swamping the fastest network pipes on the mesh. Basically, it’s a highly anti-social technology in its current form. And this is before the problems of the new version of BT which use UDP to overcome throttling for QoS reasons.

  2. >Highly useful for penny pinching businesses to offload the costs of distribution onto their customers

    …but I’ll bet that’s not a big proportion of the data out there. πŸ™‚

    To me, BitTorrent is true “cloud storage” – the data is out there, in aggregate, and it can be accessed fragmentally (sp?) and makes greater sense when seen from a distance.

    All this “Amazon S3” stuff – that’s just window dressing atop NAS.

    Your point is taken, though – time will tell how well the networks scale to accomodate the loads put upon them.

  3. I see two reasons and only two reasons why encrypted emails are uncommon today:

    1) Email is client-independent but encryption is client-dependent. If I email people who use Outlook, Outlook Express, Thunderbird, and Gmail, then I must use an encryption package which supports all of those email clients. I could use PGP but then people have to spend money and many people resist. If even one of my recipients does not support (my) encryption package, then I must either send unencrypted to all or skip that one person. Neither is a good solution.

    2) Key management is a pain.

    I do not see people considering the long-term storage issue of encryption. After all, if you encrypt in transit and store unencrypted, then this is not an issue.

    For those few people who do use encryption, each uses his own package. If I really want to communicate with everyone, then I must support all packages, which means I must use all packages. Yuck!

    A better solution (this might sound like marketing) is to switch from email to an email replacement, like TrulyMail (TrulyMail.com). Then you would have a new communication system, not dealing with email, which easily supports encryption. There is no special encryption software to install (this is needed to get the masses encrypting their messages).

    The bottom line is that if the masses are to implement encryption, then it must be easy for them. That is not the case with email but there are other solutions than sticking with email.

  4. There’s a much larger elephant in this room; ASCII. ASCII _alone_ is /the/ failure of the platform as a whole; in that we still can’t manage persistent mappings and transformations for the data itself as it evolves over time. By data I also mean the machine code all the way to the application level we interact with. Email, IM, end-to-end, they’re all symptoms of the elephantine memory of which elders remember where they’ve been in order to know where to head back to in cases of emergency: The ASCII elephant. If you stop long enough, you can feel the breeze from his tail (the left side of the A) swatting the bit rot, and hear his trunk (I) slurping new bits away as he takes in every sound structure (C) for miles around.

    The problem is that his environment is changing so rapidly, that upon returning to what were once considered safe havens, he finds that they no longer are. That information gathered from the past is no longer compatible with this ASCII elephants future, and he either has to evolve to match that changing environment in a certain amount of time, using the tools he carries, or he dies off. One elephant can only carry so much and either way the elephant acts, some part that once was the richness of that elephant changes and is lost as transformation occurs.

    Unless we take great care of the environments in which the ASCII elephant can continue to adapt and evolve, we’ll always end up with ASCII end of files like S/MIME.

    The issue is not the data, not the structure of the data, nor the security; the ASCII elephant can handle that. The issue is mobility of the tools the ASCII elephant keeps in his tool belt. He’s not able to take all of those with him to new surroundings. Why it helps to have the experience and adaptive skills of a herd *around* you, to carry some of your load.

    The ASCII elephants on our desks and in our hands have yet to herd together coherently. For the most part their ASCII is immobile, and until they learn to move together, each ASCII elephant has to take care of his own load; exponentially losing things along the way as the going gets tougher, and having to evolve ever faster in order to stay alive. All while swatting the bit rot in his wake, and losing lessons learned, lessons that only history can teach.

    History that can be invaluable when it comes to a rainy (steaming) day.

  5. Things are changing. Email encryption, encryption in general, and secure communications SHOULD be simple for endusers to use, and ubiquitous for all. Well defined standards such as X.509, RSA, S/MIME exist that provide interop. Simple to use and manage systems are hard to find. I invite you to check out http://www.echoworx.com/products/. We believe we have solved many of the traditionally overlooked issues with encryption such as public key distribution, key management, and most importantly EASE of USE, without any compromise on security, and adherance to recognized cryptographic standards and internet protocols. And distribution partnerships with the trusted service providers such as AT&T, Verizon, TELUS, BT, and IBM.

  6. @Michael Roberts

    >We believe we have solved many of the traditionally overlooked issues with encryption such as public key distribution, key management, and most importantly EASE of USE

    You ship solutions for the Nokia E61, iPhone, OpenSolaris and Mac?

  7. @craig:

    >The issue is mobility of the tools the ASCII elephant keeps in his tool belt. He’s not able to take all of those with him to new surroundings. Why it helps to have the experience and adaptive skills of a herd *around* you, to carry some of your load.

    Could you elucidate, please? This bit loses me but it seems to be key…

  8. The explanation that it can’t be indexed/searched, don’t apply to using cryptographic signatures to prove identity of plain text authorship which would seem a useful subset of cryptography for current email problems. I note that only one of the financial institutions I do business with uses such technology routinely, the rest don’t.

    As such, whilst I agree with the sentiment somewhat, it is not a sufficient explanation for what we see. I think large vendors failing to implement common standards is probably a larger issue.

    However S/Mime is widely implemented and supported in email clients, and the lack of use of S/Mime for financial institutions raises interesting questions.

    Surely Skype is only secure if your copy hasn’t been knobbled in advance by your local Skype partner. Isn’t freenet a better example πŸ˜‰

  9. @Simon:

    Actually, indexing, at least, is feasible as the e-mail client has all the information required to decrypt the messages (otherwise the person couldn’t read them). The client could, therefore, create its own metadata database and search that. Indeed, it could store a database of all words found within the e-mails with a reference to the place in the decrypted e-mail(s) that the word exists. To do a search for a phrase would be a data selection problem, e.g.

    “Select all e-mails which contain the words ‘eat’ ‘my’ ‘shorts’ in successive positions within the body.”

    This database itself could be encrypted using the private key of the user if you’re feeling that paranoid.

Leave a Reply

Your email address will not be published. Required fields are marked *