That's a bit of a long answer They all let you do validation Where they differ is semantics, performance, and additional capabilities Support: Spec is built in with Clojure Syntax: Schema's syntax results in schemas looking a lot like the data they reflect Spec is very macro/predicate heavy and looks close to bnf in s expressions Malli: hiccup style data DSL, but added support for forms more similar to Schema Semantics: Notable exception for malli where schemas are data and can be persisted and manipulated Registry semantics: Schema: n/a Spec: global Malli: flexible, can have it whichever way you want Performance: Malli is fastest if that's important for you Coercion: Built in for malli Besides that, malli has plenty of additional modules and capabilities built in

Probably the phrasing of the question is a bit misleading. After re-watching the "Spec-ulation" talk, I surmise that the design decisions of Spec and Spec-2 serve the purpose of a wider problem than just "data validation", the problem of communicating dependencies for the code ecosystem.

Does https://corfield.org/blog/2019/09/13/using-spec/ help? It talks about how we use Spec in production (and dev/test) at World Singles Networks.

@U04V70XH6 That's helpful, thanks for sharing, helps to look at examples 🙂 Would you be able to compare it to the Malli library, in case you've had the chance to use it?

I haven't used Malli. I used Schema a bit, before Spec existed. I generally prefer to use things the core team provide and maintain, where I can.

Out of curiosity, what library are you using for data coercion? I like clojure.spec a lot for data validation, but the core team makes it very clear that it is not intended to be used for data coercion whatsoever, so libraries like spec-tools are technically not something one should use (or if one does use it, don’t expect stability). For this reason, I’ve been slowly phasing out clojure.spec with malli in my codebase, since it offers both validation and coercion (but still maintains them as separate processes)

We have actually used conformers in Spec for some data coercion in the past but we're slowly moving away from that and have been experimenting with https://github.com/exoscale/coax and I like that approach.

FWIW, I use spec, mostly because I try to avoid external dependencies as much as I can, though I have to admit after several years that my expectations for spec development were slightly higher. For coercion, I use a rather simple piece of code that I got from someone on the internet (no longer remember where). I adapted it to use multiple registries, so that I can coerce things to model form and to database form. This works very well for me.

In the past, I made the mistake of thinking that conformers are intended to be used for data coercion. They are definitely not. In general, I only use the validation part of spec.

for CLJ and JSON, a naive technique would be to convert to string and hash the string

but I need this to work in CLJ and CLJS and produce the same hash

same for byte arrays

any suggestions much appreciated

Hashp rings a bell as a library option

A hash can't be unique by definition. How exactly to tackle it depends on what you need that hash for. Also, just in case - byte arrays and JSON data are mutable. But if you hash something like that, and then mutate it, it won't correspond to the hash anymore.

true, I can tolerate some (rare) collisions

I can guarantee the JSON is immutable

I cannot use identical? for clj because values are (de)serialized from earlier copies and that would only work for clj anyway

while looking for the hashp lib I found this https://github.com/replikativ/hasch which might work

That might be what I'm thinking of sorry, it's used in datahike I think

yeah, they mention that. thanks. I’ll try it

it looks like exactly what I need (with expected cljs perf issues) so I can benchmark. much appreciated

• If performance is a significant issue, and these cached functions need this entire JSON object that might vary slightly at leaves, one alternative is to generate a unique id per object on the creation side? Then you get false disequality for re-generated identical json objects but much improved speed, that's what I think clj objects do under the hood and equivalent to python id() function

I could see some kind of Merkle tree down the road for this when seeking perf

Immutable.js and clojure data structures must both have this implemented efficiently on production side somehow

agreed, must be there for any hashed data structure but I doubt the impls for jvm vs node produce the same hash values

https://github.com/DotFox/jsonista.jcs i recently made an extension for clojure jsonista library that produces json the same as JavaScript.

Perhaps something like rewrite-clj can be used to rename symbols in a macroexpansion result to make them more readable.

An example would help understanding the issue in context

A plain (macroexpand '(clojure.core.async/go)) will have a lot of symbols with 4-digit numbers in them. old-frame__6905__auto__ is a bit hard to grasp with a quick look.

Generally, I think the stream you get back is based on the API's network connection. Your aws library probably has a timeout, but even if it didn't, the API endpoint almost certainly has a timeout.

ohh, right I guess streams are lazy? and if the socket gets closed the stream is gone?

I think the reality is that there is no "stream". There are packets and buffers. If you don't read from the buffer (ie. "stream"), then the packets are either never sent or will stop being sent.

TIL, thanks @U7RJTCH6J

You also might get this error if you read the stream (for example by using slurp or depending on your setup just printing it in the REPL) and try to read it again

I believe streams are read ounce, similar to an iterator. And also, once they're closed by calling .close on them, they cannot be used anymore.

You could try doing the API call in a with-open form, which should force you to handle the streams correctly.