Fork me on GitHub
#malli
<
2021-03-10
>
Adam Helins11:03:10

I have written a WebAssembly decompiler/compiler and I would like to "spec" the WASM intermediary representation I am using. After a long while of avoiding clojure.spec, I've tried using it for that purpose but it's just too hard, too limited. So I am trying out Malli. Given how flexible it is, I have high hopes. But I am a Malli noob so here I am. My problem is actually a common one: a data schema were parts of the schema are references ("pointers") to other parts of the schema. Validation is easy but generation is hard. Spec makes that extra hard by forcing a global registry. Let's stick to this WASM example to have something concrete to work with. There exists a type section (map of type index to type) and a function section (map of function index to function where a function holds an existing type index). Generating a valid representation in pseudo-code would look like: 1. Generate a type section where type index is a nat? 2. Mutate registry so that type index becomes an enum of generated type index 3. Generate a function section , now assured that generated type index exist Solution A. I guess one way would be to do exactly that. Generating things in the right order, step by step, and keep on doing that while successively building a local registry based on what has been previously generated. I am unsure how practical that would be for a more complex example. Solution B. Another way would be to have one schema with a "local mutable registry" and declaring custom generators along the way which would mutate this "local mutable registry" based on what they generate. However I understand that local registries must be concrete maps and do not allow this. On the longer term, I am not sure this would be more pratical than Solution A and it certainly is not very functional. Any better, idiomatic way?

Adam Helins11:03:20

Note regarding Solution B: Unless I am doing it wrong, I cannot manage to use a mutable registry in a local way (ie. using it as an explicit :registry argument throws :malli.core/invalid-schema {:schema :map})

ikitommi13:03:23

@adam678 Sounds really interesting! I can’t recall why the local registries only support maps. Would be most likely a small change it to support the Registry

ikitommi13:03:22

one option woud be to just collect stuff to a mutable registry or an custom atom, use it via (m/schema x {:registry registry}). when everything is collected and if the schemas are serializable, you could writen them into one local (immutable) registry.

jjttjj14:03:13

I'm wondering what the "limits" of the malli value transformations should be philosophically. If I'm working with an external api that represents dates as strings, but I want to represent them as Instants in my app, is that a valid use for transformers? What about if the external api provides a nested "address" map that I want to eventually turn into an Address record type? It wouldn't necessarily be used as a two way transformation very much, I wouldn't necessarily be sending my addresses back to their api and need to encode them again as strings. Is this just a separate problem, and I should just use a function to get things into my domain records/types?

juhoteperi14:03:51

Coercing stings to proper types at least is fine use. Similar to coercing json or path or query string parameters to booleans, dates etc.

nilern14:03:19

We do those sorts of things all the time, also with Schema and Spec

jjttjj14:03:46

So it's good to use malli to get from string all the way to our full on domain objects?

ikitommi14:03:59

sure. If it like looks complex, then move the transformation out. I haven't seen a always valid limit. string->map map->record both perfect cases.

jjttjj14:03:50

by "move the transformation out" you mean separate it into a different transformer?

ikitommi14:03:06

wrote a while back this generic nonsense (on spec): > Domain-specific data-macros are cool, but add some complexity due to the inversion of control. For just this reason, we have pulled out Schema-based domain coercions from some of our client projects. Use them wisely.

ikitommi14:03:23

by moving out I meant into a separate function outside of schemas, e.g. (external->internal data) thing.

jjttjj14:03:17

And then do you call that from a transformer still? or do you mean the data goes from

json-string-> clojure data -> malli transformers -> external->internal

ikitommi15:03:48

• optimal: json-string + malli -> internal • current good practise: json-string -> EDN -> malli transformers -> internal • if the internal->external is complex. uses external data etc: json-string -> EDN -> malli transformers -> external->internal

ikitommi15:03:31

(did a spike on deriving jackson-decoder from malli schema, but nothing production grade atm, in theory, shoud be much faster)

jjttjj15:03:55

that makes sense, thanks again!

Adam Helins15:03:14

@ikitommi All right, thanks, I got a sense of where to start. As a beginner I was mostly troubled by the fact that local registries can be only map-based. Since you are not sure this is intended, I took the liberty of opening an issue: https://github.com/metosin/malli/issues/389

👍 3
Ed18:03:28

Hi. If I have a :dispatch multi-schema, is there an easy way to put a catch-all else clause in the matches?

ikitommi19:03:28

@l0st3d not atm, but could, does [:or [:multi …] :default]work for you?

ikitommi19:03:42

maybe there could be a :malli/default branch?

Ed19:03:57

I think I've worked out how to write what I was trying to write in a different way (using :or, but I needed an exclusionary condition instead of :default), but it might be a useful feature to add ... I quite like the idea of :malli/default. Partly because of the parallel to multimethods. Maybe there's a clean way of overriding the keyword?

Ed19:03:26

my exclusion clause in the second branch of the or is a bit big, but cos it's all data it's ok to write a function to produce the branches 😉