Posted: 2012-11-03 Tags: Programming | SHA-3 | MD5 | Lisp

As atonement for the recent MD5 release, and since we seem to perversely enjoy bumming numeric Common Lisp code, we have put together an implementation of the newly announced SHA-3, i.e. the algorithm formerly known as Keccak.

The package should be portable across nearly¹ all ANSI compliant CL implementations with specialized versions tuned for implementations that offer unboxed 64bit arithmetic, unboxed 32bit arithmetic and for implementations with efficient fixnum arithmetic (requiring fixnums that can represent (unsigned-byte 16)). Especially the 64 and 32bit implementations have been mostly optimized for SBCL and CMU CL. For those implementations, digests with a 1024 bit-rate (and 288 bit digest output) can be generated in between 30 (64bit SBCL) to around 100 (32bit CMU CL) cycles/byte on an i7-640M; whereas optimized C/assembler implementations reach around 12 to 50 cycles/byte on 64/32 bit Intel hardware. The reason for the discrepancy probably lies in missing peephole and dependency optimizations in the SBCL/CMU CL compiler backend, which we might look at in a future release.

The release is available from its PMSF page, and its GitHub home.

Many thanks go to the Keccak Team (Guido Bertoni, Joan Daemen, Michaël Peeters and Gilles Van Assche, cf. http://keccak.noekeon.org) for their algorithm and excellent documentation and reference implementations.

Enjoy.

---

1. Currently there is no version of the code that would work for CL implementations that have the smallest allowable fixnum size, which only covers (signed-byte 16), not (unsigned-byte 16) as required by the 16bit version in the package. I'm not aware of such a CL implementation, and in any case this could be easily fixed, at the cost of some code clarity and/or performance.

Posted: 2012-10-23 Tags: Programming | MD5 | Lisp

The MD5 implementation for Common Lisp implemented by Pierre R. Mai a decade ago has been floating around in various tweaked versions for some time now. We have now produced a new consolidated release (release 2.0.0) of this with proper packaging, all known fixes and tweaks and implementation optimizations. The new release is still available from its PMSF page, and its new GitHub home.

Many thanks go to Christophe Rhodes, Alexey Dejneka, Nathan Froyd, Andreas Fuchs, John Desoi, Dmitriy Ivanov, and Kevin M. Rosenberg who improved, tweaked and maintained versions of MD5 over the years.

Note that MD5 has been cryptographically broken, and should therefore not be used in situations where a cryptographically secure hash function is needed. However due to its wide historical use and its remaining suitability in non-cryptographically secure settings, this implementation is still of some interest.