Fork me on GitHub
#clojure-spec
<
2020-04-03
>
Ben Grabow16:04:16

So far I've been using spec mostly for validating data structures "in-place", meaning I have an x, at the front door of my API I check if it's a valid format, and if it is I continue using x inside my API. There's another way of using spec that I don't really understand yet so I'm looking for good resources on it: I have an x, and at the front door of my API I check if it's a valid format and destructure/reformat it to the format expected internally in my API, y. y might have all the same data as x but perhaps arranged in a different nested structure or with different key names. Are there any good guides to using spec for this kind of thing? I guess this is using spec as a parser, rather than just a validator. This use case is hinted at (https://clojure.org/about/spec#_conform) but I haven't found a good set of examples yet.

Alex Miller (Clojure team)16:04:30

That’s what conform is for

Alex Miller (Clojure team)16:04:48

“Is this valid, and if so, why?”

Alex Miller (Clojure team)16:04:39

With the caveat that it is NOT for “perform arbitrary transformation to other structure”

Ben Grabow16:04:05

I'm trying to understand the situations it's good for. e.g. https://juxt.pro/blog/posts/parsing-with-clojure-spec.html

Alex Miller (Clojure team)16:04:28

If you want that, use conform + standard Clojure data transformation

Ben Grabow16:04:29

In that example, a sequence of characters is being parsed into a map that describes the components.

Ben Grabow16:04:48

I have one map with a nested structure that I want to transform to another map with a flat structure.

Alex Miller (Clojure team)16:04:11

(That applies to the string case)

Alex Miller (Clojure team)16:04:35

If you want to transform maps, use Clojure stuff to do that

Alex Miller (Clojure team)16:04:59

If you want to validate, use valid?

Alex Miller (Clojure team)16:04:31

If you want to know why something is valid wrt options and alternatives, use conform

Ben Grabow16:04:55

It's that last part I'm looking for more content on.

Ben Grabow16:04:25

I don't quite understand when I'm dealing with a situation that's good for conform and when I'm not.

Ben Grabow16:04:57

I remember in one of Rich's talks he mentioned David Nolen rewriting part of the cljs parser in spec. Is that code publicly available somewhere? It sounds like a powerful use case.

Alex Miller (Clojure team)16:04:35

Using it to parse the input to a complex macro is a good fit

Alex Miller (Clojure team)16:04:57

It’s particularly good for “syntax” stuff

Alex Miller (Clojure team)16:04:26

If your input is mostly maps and stuff, probably not as useful

Ben Grabow16:04:12

The difference in the way my input map is structured vs how my output map is structured feels like syntax to me so I wonder where the distinction lies

Ben Grabow16:04:32

There's the idea that a map is just a seq of kv pairs. What if the spec is an s/cat of kv-pairs instead of an s/keys?

Alex Miller (Clojure team)16:04:37

Syntax is positional (this is kind of present in the Greek roots of the word even)

Alex Miller (Clojure team)16:04:57

Maps are not positional

Alex Miller (Clojure team)16:04:28

Position in syntax is implicit

Alex Miller (Clojure team)16:04:52

Spec regex ops describe that, and the conformed values tell you how it was interpreted

Alex Miller (Clojure team)16:04:26

Must of the map specs essentially return the map if it’s valid - there was no thing to figure out and tell you about

Alex Miller (Clojure team)16:04:27

Specing a map as a seq of kv pairs is not the same thing as maps are inherently unordered

Alex Miller (Clojure team)16:04:45

You can’t rely on position to describe the structure of the map

Ben Grabow16:04:04

hmm, that's a good point

Alex Miller (Clojure team)16:04:09

The regex specs like cat will even error if you try to do this, for this reason

Ben Grabow16:04:27

I had in mind using a big s/or to describe all the possible keys. Which brings up an interesting question, how does the positionality of syntax impact s/or? I usually reach for s/or when I have a couple different possibilities for my input, like a union type, and I'm usually frustrated that I'm pushed towards destructuring and naming the options instead of conforming to just the value.

Ben Grabow16:04:38

(s/or :even even? :small #(< % 42)) from the docstring as an example.

Ben Grabow16:04:19

I wish the API were (s/or even? #(< % 42)) for the use I have in mind

Ben Grabow16:04:49

I feel like there's something I don't "get" in the reasons why s/or asks for keywords to name the options.

Alex Miller (Clojure team)16:04:09

or gives you back a map entry of tag (key) and value

Ben Grabow17:04:17

Sure, I get that. I'm wondering why it does that when it doesn't seem to be about processing the position of a thing. Instead it seems to be about processing the alternatives of a single thing.

Alex Miller (Clojure team)17:04:59

well the idea is the same - tell me what choice you made when there was an alternative

Alex Miller (Clojure team)17:04:39

in practice, it is often a source of issues in deeply nested data where that's the only conformed thing so I kind of would like to have some more options in spec 2

Alex Miller (Clojure team)17:04:57

whether that's an or variant, or a s/nonconforming, or something else

Alex Miller (Clojure team)17:04:07

but that's down the list a bit and we haven't talked about it

Ben Grabow17:04:57

Glad to hear the sentiment is shared.

Ben Grabow17:04:38

I often run into situations where a map is mostly just a set of specific things so s/keys works, until we add uuids into the mix and they could be strings or uuid objects. I'm using a custom conformer to get around the s/or destructuring for that case but it would be nice to have a more general solution.

Alex Miller (Clojure team)17:04:00

one workaround right now is to wrap those s/or's in s/nonconforming

Alex Miller (Clojure team)17:04:07

which is not documented, but exists

Ben Grabow17:04:50

(comment
  (s/conform (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)) (UUID/randomUUID))
  (s/conform (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)) (str (UUID/randomUUID)))
  (gen/generate (s/gen (s/nonconforming (s/or :x uuid? :x ::sc/uuid-string)))))
All these work as expected. The first two produce just the value, without the keyword tag. The third generates both strings and uuids.

Ben Grabow17:04:26

My ::uuid-string spec is still a little complicated to make the generator work but I think it could be simplified.