Re: weird crypt-sha* in DragonFly BSD

[Solar Designer <solar () openwall com>]

DragonFly BSD committers -

magnum has prepared a patch to address this issue in DragonFly BSD:

http://www.openwall.com/lists/john-dev/2012/01/19/1

This reverts the default to FreeBSD's MD5-crypt _and_ it takes care of
the magic strings in your crypt-sha* stuff to make those strings
constant (whatever they happened to be in your release in practice -
including the extra 4 bytes).

Please review and commit.

Alexander

On Mon, Jan 16, 2012 at 09:12:04PM +0400, Solar Designer wrote:
> Matt -
>
> magnum proceeded to implement support for DragonFly's SHA-2 based hashes
> in John the Ripper - to hopefully make you reconsider sooner rather than
> later.  While doing so, he ended up finding a nasty bug that I
> previously did not notice: the code uses sizeof(magic) instead of
> strlen(magic), where "magic" is a pointer.  Thus, the resulting hashes
> are non-portable between 32-bit and 64-bit systems, and additionally
> they may be non-portable between different 64-bit versions/builds of
> DragonFly (let alone to/from other systems).  While this lack of
> portability might make some attacks on stolen/leaked hashes more
> difficult (it certainly is an issue that we have to consider when adding
> support for these hashes to JtR), I doubt that this is what you want.
>
> I strongly recommend that you revert to FreeBSD's MD5-crypt ASAP.
>
> More detail here:
>
> http://www.openwall.com/lists/john-dev/2012/01/16/1
> http://www.openwall.com/lists/john-dev/2012/01/16/4
>
> For now, we'll support only the 32-bit flavor of these hashes in JtR.
> If you keep them in DragonFly for much longer, we'll likely do something
> about supporting the 64-bit flavors as well.
>
> The speeds on one CPU core (in a E5420):
>
> Reference (heavily optimized and parallelized FreeBSD MD5-crypt, 12
> hashes computed in parallel):
>
> Benchmarking: FreeBSD MD5 [SSE2i 12x]... DONE
> Raw:    25320 c/s real, 25320 c/s virtual
>
> DragonFly's alternatives:
>
> Benchmarking: DragonFly BSD SHA-256 w/ bug (32-bit) [OpenSSL 32/64]...  DONE
> Many salts:     1663K c/s real, 1646K c/s virtual
> Only one salt:  1479K c/s real, 1494K c/s virtual
>
> Benchmarking: DragonFly BSD SHA-512 w/ bugs (32-bit) [OpenSSL 64/64]...  DONE
> Many salts:     1377K c/s real, 1377K c/s virtual
> Only one salt:  1257K c/s real, 1257K c/s virtual
>
> That's 65 times faster cracking - before we even started optimizing.
>
> 8-way OpenMP on 2xE5420 (8 cores), reference:
>
> Benchmarking: FreeBSD MD5 [SSE2i 12x]... (8xOMP) DONE
> Raw:    202368 c/s real, 25264 c/s virtual
>
> (215k c/s is possible with Intel's compiler, but I did not bother here.)
>
> DragonFly's alternatives:
>
> Benchmarking: DragonFly BSD SHA-256 w/ bug (32-bit) [OpenSSL 32/64]... (8xOMP) DONE
> Many salts:     10870K c/s real, 1370K c/s virtual
> Only one salt:  6119K c/s real, 763973 c/s virtual
>
> Benchmarking: DragonFly BSD SHA-512 w/ bugs (32-bit) [OpenSSL 64/64]... (8xOMP) DONE
> Many salts:     8509K c/s real, 1065K c/s virtual
> Only one salt:  5207K c/s real, 656587 c/s virtual
>
> That's roughly a 50x speedup - again, for unoptimized DragonFly hashing
> vs. optimized FreeBSD hashing.
>
> With full optimizations, the difference will be more like 500x for the
> SHA-256 flavor.
>
> Please let us know if you're going to do anything about these issues.
>
> Thanks,
>
> Alexander
>
> On Tue, Nov 15, 2011 at 06:35:02AM +0400, Solar Designer wrote:
> > Hi,
> >
> > Matthew - when I read that DragonFly moved to using SHA-256 for
> > passwords by default, I thought this was referring to the SHA-256 based
> > flavor of Ulrich Drepper's SHA-crypt.  This would not be the best choice
> > to make, in my opinion, but it would not be that bad.  However, I just
> > found this:
> >
> > http://gitweb.dragonflybsd.org/dragonfly.git/tree/HEAD:/lib/libcrypt
> >
> > Are these crypt-sha256.c and/or crypt-sha512.c files actually in use?
> > I hope not...  They do not include any password stretching, resulting in
> > password hashes that are much quicker to crack than MD5-crypt's.
> >
> > There's also minor weirdness in the code - such as two local pointer
> > variables being declared static seemingly for no reason, and only
> > "final" but not "ctx" being zeroized in the end.  But even this lack of
> > proper cleanup is very minor compared to the lack of stretching.
> >
> > Oh, also the "$3$" prefix was apparently previously used for NTLM:
> >
> > http://en.wikipedia.org/wiki/Crypt_(Unix)#NT_Hash_Scheme
> >
> > "FreeBSD used the $3$ prefix for this."
> >
> > http://search.cpan.org/~zefram/Authen-Passphrase/lib/Authen/Passphrase/NTHash.pm
> >
> > "... crypt string must consist of "$3$$" (note the extra "$") followed
> > by the hash in lowercase hexadecimal."
> >
> > BTW, I looked at DragonFly's code while analyzing a more subtle issue
> > with Ulrich's SHA-crypt:
> >
> > http://www.openwall.com/lists/oss-security/2011/11/15/1
> >
> > I thought that maybe you reimplemented it in a better fashion avoiding
> > that issue, but I found this... %-)
> >
> > Alexander