Fork me on GitHub

I have a left field question about spec. There are a number of libraries in clojure for describing binary formats (buffy, octet, etc). Essentially you create a "binary spec" which describes the byte layout of your binary format and then ask the library to parse a byte array, byte buffer etc and back come strings, numbers etc parsed data types. The wish list for the language where you create these binary specs is that they be composable, support references from one spec to another (string length stored in another field), be terse etc. It strikes me that this wish list has a lot of overlap with spec. Would it be sane to try to use spec as the language for describing these kinds of binary formats or is it just a bad fit? You would get a fair amount of power for free and in years to come you would be using a language people are already familiar it seems there are some upsides here. One example of a good fit is multi-specs which in spec parse maps of different types depending on a 'type' field. There is a very close correspondence in a lot of binary formats with a type int16 or int32 followed by some binary data conforming to the type specified. Collections also match well. Anyway, somebody please call me crazy and tell me why this doesn't work : )


It would be great if you would have a higher-level format describing the data, which could be serialized to bytes, and deserialized to language-specific data-structures of different languages. However it is not easy, I had some experience with avro, but even for bigintegers it doesn’t work for java out of the box..

Alex Miller (Clojure team)12:09:51

I do think there is some overlap in goals and you could build custom spec types for this similar to the regex ops part of spec. Byte level stuff has some special considerations so I don't think you'd want to use what's in spec now for that.


@alexmiller when/if you have time could you explain how my hack ruins s/form?

Alex Miller (Clojure team)16:09:46

if you have doc, then I guess it’s fine. really I’d just rather actually fix this.