OpenSolaris, Pluggable Crypt, and the SunMD5 Password Hash Algorithm

Several years ago now, Darren Moffat, Casper Dik and I started swapping e-mail about how pathetic it was to still be using the traditional 8-character-password unix crypt() routine in Solaris, and how we could architect something to be much better.

The result was the Solaris Pluggable Crypt Framework, as was announced:

  1. A project exists within Solaris engineering, to integrate a pluggable crypt() routine into Solaris, which will allow use of arbitrary password-hashing algorithms, of arbitrary lengths, etc, in Solaris.

  2. Interoperability with Linux/*BSD MD5- and Blowfish-based hash algorithms is a goal of the project

  3. If I remember right – and I may well be incorrect, as I am not responsible for this aspect of the project – the release is scheduled for Solaris9.

    I am consulting with the team/doing development work with them, on account of my (erm) extensive experience with crypt() implementations…

  4. PAM was considered as a solution for this, and it was decided to not be the appropriate vehicle for delivery of an alternative crypt() routine, because (in summary) PAM is essentially an API for user-interfaces (/bin/login, ftpd, etc) – as opposed to an API for interfacing to the directory-services within which the password entries reside; consider “getpwent()” and family.

Pluggable Crypt was rolled-out in Solaris 9 (Update 2, if my memory is correct?) and we were quietly pleased that our users suddenly reported being able to transparently use a mixture of Traditional, Linux-MD5 and Blowfish hash algorithms. GNU Autoconfig (aka: "./configure") spotted it immediately, next time Darren built Apache. All was cool.

Incidentally, we didn’t get any awards or anything, but that’s management for you. It was a nice example of co-operation inside the company because I was at that point actually working for Sun Professional Services, and not really an engineer.

The hook which permitted us to implement “Pluggable Crypt” was the presence of a dollar-sign (‘$’) as a field separator in the latter two kinds of hashes. We took that as a style cue, and designed the new crypt() shim so that lack of a ‘$’ prefix causes a fall-through to the “traditional” algorithm.

After several weeks of considerable argument (three extremely opinionated and notoriously frank security programmers designing a API, it must have been funny to watch from a distance behind smoked glass) we decided that for a given ciphertext:


…that “foo” – the token being terminated by a dollar or comma – would be used to look up and load a shared library in crypt.conf(4), and that the latter would be at liberty to re-interpret the ciphertext as it willed.

We argued over the design using the following two ciphertexts as examples of desirable syntax:



…which are essentially the same algorithm and might possibly be implemented by the same shared library, but which would have different output syntaxes. The mechanism we created could permit this.

Provided with this wonderful power I mused what I could do in order to highlight the new features it provided, and so we came up with the SunMD5 Hash Algorithm that would be written exclusively with the purpose of annoying people who write brute-force or dictionary-based password guessing engines.

In short: for annoying people like me.

After some beer consideration of the problem, I concluded that password-cracker writers:

  • really hate hash-algorithms which allow large, near-infinite, clash-free numbers of salts.

  • really hate hash-algorithms with large, variable, possibly near-infinite round-counts.

  • really hate hash-algorithms which make precomputation/table-lookup impractical.

…and I further reasoned that:

  • writing a wholly new hash algorithm would be a really really unutterably silly thing to do, but…

  • aha! pumping vast and variable quantities of information through an existing hash algorithm would be reasonably computationally expensive, and the process may be tweakable to make it demonstrate the above desired qualities.

So, after some more beer coffee considerable thought, I hit upon the following process:

  1. choose a PLAINTEXT, and a SALTSTRING

  2. push PLAINTEXTSALTSTRING through MD5, generating HASH0; let n=0

  3. condense HASHn into a single BIT, using MuffettCoinTossAlgorithm

  4. where “NNN” is a decimal ASCII representation of “n”; and where “HAMLET” is the complete 1517 bytes of “To Be Or Not To Be” soliloquy (Hamlet act 3 scene 2, copyright-free source) …

  5. if BIT=0, push HASHnNNN through MD5 generating HASH(n+1)

  6. if BIT=1, push HASHnHAMLETNNN through MD5 generating HASH(n+1)

  7. let n=n+1

  8. repeat steps 3 thru 7 for (4096+X) iterations, where X is an arbitrary positive integer 32-bit number specified in the input salt/ciphertext/metainformation.

  9. result:


    – using base64 encoding where needed to encode binary data.

That’s a quick rundown and I think it’s approximately correct. I’ve not been drinking coffee this week so I may be a little off in the fine details, so check the source if you want verification. In terms of the blame we all argued the overall design, I did the SunMD5 algorithm, Darren the glue and Casper and Darren both the larger shared library interfaces, docs, integration and so forth.

The only blackbox in the above is MuffettCoinTossAlgorithm which I truly cannot be arsed to explain (or, more pointedly, why should I grep my mailbox archives for the design docs when you can read the source and figure it out for yourself?) – but the essence was to create something which was somewhat a function of the roundcount (strictly, there are 128 different ways it can go, since it’s f(roundcount modulo hashbits)) and which self-referentially recombines bits of the previous round’s hash both as dynamic-lookup-table and as data-looked-up, eventually extracting two bits from the hash, which then get XOR’ed to create a truth value. It was designed with painful memories of trying to speed-optimise DES S-boxes, plus a few hints of “let’s try to align a few fetches unevenly bridging word boundaries, in case someone tries to inline it with longword fetches.”.

You could try speculative computation, but then you’re a masochist.


<DIG TYPE=’amusing’> Aside: Reviewing the code nowadays, I’m sure it read a lot more clearly before Darren put it through cstyle. </DIG>

The benefits of the SunMD5 design:

  1. The saltstrings are of near-arbitrary length and can contain all sorts of stuff, including (for instance) the username as a substring, which could thereby guarantee the uniqueness of a salt within a given environment.

  2. The roundcount can be enormous; 1 round = (1517 * 50%) + 8 + 4 = 770 bytes pushed through MD5, so setting “rounds=904” (making 904 + 4096 builtin = 5000 rounds total) pushes 3.8Mb of data through MD5. If you want it to take longer, set “rounds=68000” and pump 50Mb through it instead.

As ever, much of the power of an algorithm is not merely how it works, but also in how you use it. The above should supply you with some idea of how it’s meant to be used. In terms of the actual implementation, there’s a feature which I don’t like and for which there is a RFE somewhere; it’s demonstrated by the following code:

# – test the sunmd5 implementation on solaris9u2 or above
# roundcount: 904 (user specified) + 4096 (builtin) = 5000 rounds total
# syntax examples
$exsalt1 = ‘$md5$rounds=904$saltstring$dummy’;
# this one is strictly correct
$exsalt2 = ‘$md5$rounds=904$saltstring$’;
# this one ought to be the same
$exsalt3 = ‘$md5$rounds=904$saltstring’;
# as should this one
# test plaintexts
@words = qw( sesame abcdefgh abcdefghijklmnop );
foreach $word (@words) {
$ct1 = crypt($word, $exsalt1);
$ct2 = crypt($word, $ct1);
printf “with exsalt1: %-20s => %s\n”, $word, $ct1;
printf “sanity check: %-20s => %s\n”, $word, $ct2;
printf “with exsalt2: %-20s => %s\n”, “”, crypt($word, $exsalt2);
printf “with exsalt3: %-20s => %s\n”, “”, crypt($word, $exsalt3);
print “\n”;
exit 0;

Basically: in order to produce correct outputs, the current parser for SunMD5 algorithm requires a dummy (or real!) ciphertext at the end of the salt-input; this is not a problem in operational usage since /bin/passwd and the other user-admin tools are meant to do the right thing, however it’s not really a nice user experience for Joe Shmoe who’s hacking around in perl as above.

So: when the examples with exsalt2/exsalt3 show the same output as the preceding, then it means that my RFE’s gone through and the patch has been installed. Until then, be aware.

All this has been written in response to a bunch of questions from David Magda, who (over over the course of several e-mails) posed the following questions, the answers to which I shall use to wrap-up this posting:

  • What advantages does the SunMD5-based algorithm have over the BSD one?

    See the above; the algorithm is meant to demonstrate interesting features, innovative ideas in password hashing / what constitutes a ciphertext, and to highlight the Pluggable-Crypt framework which is now (of course) published as part of OpenSolaris.

    Also: it exposes more programmers to Shakespeare, which has got to be a good thing.

  • With the recent attacks on MD5 (and SHA-1), is there any thought to moving to the Blowfish-based hash as a default, or is MD5 still “good enough” for this purpose?

    I reckon that given our modus operandi it’s more than good enough, but for interest’s sake Darren and I have long been planning an uprated version and we’re considering SHA-512 as the underlying hash algorithm.

  • Why another hash?

    “Because People Always Want The Latest iPod” 😎

    Plus, well, why not? Why can’t we live in a world with a dozen different hash algorithms and support the lot of them? Pluggable Crypt is a really good idea:

    WarStory: I gather that there is at least one site where old Ultrix / OpenVMS / OSF-1 ciphertexts have been imported into a Solaris environment by installing a suitable Pluggable-Crypt module, then prefixing the ciphertexts with $bc$ (bigcrypt) or $c16$ (crypt16) and performing migration from these old, weak algorithms by immediately deprecating $bc/$c16 by using the controls in policy.conf(4).

    Insanely neat, huh?

    Further: it allows the spooks to use their own password hashes. They really like that sort of thing. Makes them feel safe.

  • Given that the Blowfish-based one is extendable by increasing the rounds, why not use it?

    Because I thought it insufficiently evil, and evidently the progeny of a cryppie, rather than a cracker-programmer, mindset. 😎

Solais manpages you need to read: passwd(1), crypt(3C), crypt.conf(4), crypt_genhash_impl(3C), crypt_gensalt(3C), crypt_gensalt_impl(3C), getpassphrase(3C), passwd(4), crypt_unix(5)

Manpage browser at []

i just had my chest shaved by an attractive blonde in a nurse’s outfit…

…but then, she was a nurse, and I was having an ECG done.

I have a rest heartbeat rate of between 56 and 60, offset by blood pressure at the high end of the normal range. I’m basically normal.


However I now have several crop-circles on my chest. Might as well take the rest off, even it out a bit…

Meatloaf’s Surname

Via [] and [] I learned that Meatloaf‘s surname is Aday

It may be that this really is his given surname, but that didn’t stop me reflexively wondering whether his sister might be Drinka Pintamilk.

BellSouth: Let Us Blackmail Web Sites

Is this for real? I could imagine quite a few lawsuits hitting any ISP who tried this and managed to block access to virus updates and suchlike services…

BellSouth: Let Us Blackmail Web Sites

BellSouth has come up with a plan that would even make Tony Soprano blush: It wants to charge Web sites extortion money so they’ll load faster than their rivals. And it wants to do the same to any competitors that want to use its network for VoIP.

William L. Smith, BellSouth’s chief technology officer, told reporters and analysts that his company should be able to charge a site like Yahoo an extra fee so that it loads faster than a rival’s, such as Google’s.

Left unsaid was that it could also make any Web site load slower, which would let BellSouth then ask that site for money so that it doesn’t load quite so slow.

This is no different than the old Mafia business of hitting up all businesses in a neighborhood for protection money, saying that unless they paid up, bad things might happen to them.

Unfortunately, BellSouth isn’t the only telecom giant that believes that extortion is a good business plan. SBC chief Edward Whitacre has also laid out plans to extort money from big Web sites, saying he would like to block subscriber access to them unless the sites paid him protection money.

In the real world this behavior is called extortion or blackmail. In the telecom world, it’s called competition. Let’s hope Congress steps in to stop it…although considering the extortion ring that they’re running, they make the telecom industry look like pikers.

Cite: []
Links: [] []

‘Gay weddings’ become law in UK

Hundreds of gay couples are preparing to form civil partnerships in the coming weeks as the law changes after decades of campaigning. At least 1,200 ceremonies are confirmed as being scheduled already, according to figures from councils compiled by the BBC News website.

Councils are preparing for the first ceremonies, with couples permitted to register from Monday morning.

Campaigners says the law ends inequalities for same-sex couples.

The first ceremonies under the Civil Partnerships Act can take place in Northern Ireland on 19 December, followed by Scotland the next day and England and Wales on 21 December.

Under the law, couples who want to form a partnership must register their intentions with local councils. Unlike marriages, the signing of the legal partnership papers does not need to happen in public.

There’s going to be a lot of happiness around, this winter. Given the number of rocky “for the sake of the children” marriages that I know are ongoing, I’m quietly happy to be living in a society where suddenly a whole new vein of people can get hitched because they want to.

A Cross, a Crescent and a Crystal

So, belief brings people together?

In Geneva diplomats from the 192 countries which have signed the Geneva conventions are meeting to discuss a third emblem for the international Red Cross.

Red Cross officials hope approval of a third emblem will finally put an end to decades of controversy over the issue.

At the moment, the only two emblems recognised under the Geneva conventions are the red cross and the red crescent; relief workers and ambulances bearing these symbols are protected under international law.

In war zones or disaster regions, they must be granted free access to people in need of help.

Over the years, a number of countries have applied to have their own distinctive emblems recognised, but all have been refused. Francois Bugnion, director of international law with the International Committee of the Red Cross (ICRC), believes too many emblems could compromise protection.

“Hundreds of emblems would mean they would not be recognised, and so they would not be respected”, he explains.

“They would be identified with different countries, so instead of conveying the message ‘this is an ambulance, this is neutral and must be respected’, it would be a sign of identification which would run against the objective of the protective symbol.”

One country in particular, however, refuses to use either the red cross or the red crescent.

Israel’s Magen David Adom Society (MDA) uses the unrecognised red star of David as its emblem. As a result, the society is still not a member of the international Red Cross movement, something which many Israelis see as unjust.

Israel’s ambassador to the UN in Geneva, Itzhak Levanon, shares that opinion.

“MDA is active all over the world,” he points out. “In the tsunami, for example, or following earthquakes, we are often first on the scene, and we deserve to be part of the international movement.”

The problem is, the star of David is primarily Israel’s national symbol, rather than an emblem of humanitarian relief.

Arab states have made it clear they will never accept the red star being recognised under the Geneva Conventions.

At the same time, the conventions stipulate that national relief societies must use only recognised symbols outside their internationally accepted borders.

That means Israel should probably not be using the red star in the occupied territories.

The wrangling over the issue has gone on for decades, and it is proving costly for the International Red Cross.

For the last five years, the American Red Cross has withheld its subscriptions to the international movement in protest at Israel’s exclusion, creating a shortfall of over $30m.


How Swearing Works

How Swearing Works

We all know what “bad words” are. Unlike most other language rules, we learn about swearwords and how to use them without any real study or classroom instruction. Even very young children know which words are naughty, although they don’t always know exactly what those words mean.

But swearwords aren’t quite as simple as they seem. They’re paradoxical — saying them is taboo in nearly every culture, but instead of avoiding them as with other taboos, people use them. Most associate swearing with being angry or frustrated, but people swear for a number of reasons and in a variety of situations. Swearing also serves multiple purposes in social interactions. Not only that, your brain treats swear words differently than it treats other words.

I just love the URL.