Fork me on GitHub
Sam H10:09:01

Basic question about spec, but I would expect clojure to complain if a spec doesn’t exist but seem it just seems to ignore the fact:

(s/def ::test-spec
  (s/keys :req-un [::req-does-not-exist]
          :opt-un [::opt-does-not-exist]))

(s/valid? ::test-spec {:req-does-not-exist :foo})
;; => true

(s/valid? ::test-spec {:req-does-not-exist :foo
                       :opt-does-not-exist :bar})
;; => true
Any idea if there’s a reason for this or how you usually handle being aware of missing specs?


I'm guessing that's because spec is an open system. When you say I need ::a to be present in the map, s/keys will verify that, and if you have specified what ::a should look like, then it will verify (and generate) that too correctly. This is what allows us to spec as little or as much as we want about our domain.


Also comes in handy when doing interactive development, where spec doesn't become this thing that is constantly bothering you about missing spec definitions. You get to decide how much you want to spec, and accordingly you get back as much value and precision in validation and generation. Hope that answers your question.


Code above looks like a bug to me.


by specifying :req-un, you asked spec to check that :req-does-not-exist key should be checked against ::req-does-not-exist spec during a call to valid?


it's one thing to have forward declarations in spec where you don't check if referenced spec is declared during "definition phase", and another to ignore required keys during "validation phase"

Sam H10:09:40

^^ yep, that’s what I was thinking


Yeah. That does seem like a bug. Didn't read the whole thing. My bad.


Actually, s/valid is behaving correctly here. (s/valid? ::test-spec {}) correctly returns false.

Sam H10:09:00

it seems to mean you could add a missing spec to :req/opt-un and add it to the data but this wouldn’t actually be checked or give any indication the spec doesn’t exist


Stuart H wrote a gist to check missing keys a couple of years ago

Sam H23:09:02

Cheers dude. You’re always a good source for clojure knowledge 🙌

👍 4

This seems fine to me. You're ensuring that a map contains a specific key, but making no claims about the value of that key. I do this frequently when receiving required but complex data from external systems - i want to check that the data is there, but i don't know enough to write a comprehensive spec for it.


Here is my situation: I have a top-level spec that allows for two situations - cluster updated and cluster removed. This is handled by two multispecs as follows:

(defmulti cluster-updated :cluster-type)
(defmulti cluster-removed :cluster-type)
(s/def ::cluster-updated (s/multi-spec cluster-updated :cluster-type))
(s/def ::cluster-removed (s/multi-spec cluster-removed :cluster-type))

(s/def ::cluster (s/or :cluster-removed ::cluster-removed
                       :cluster-updated ::cluster-updated))


Each of those multispecs has two implementations. The implementations generate nested maps, where some of the nested maps have generators that generate matching input, but conforming strips away extra data as specified.


Now, I would like to call (s/conform ::cluster (g/generate (s/gen ::cluster))) and get either a :cluster-removed or :cluster-updated data, but conformed recursively all the way through. However, conformance does not seem to propagate to the level of the multispecs, it simply stops at the level of ::cluster.


I can work around it by examining the tag I get from ::cluster, then conforming again against either ::cluster-updated or ::cluster-removed, which does the correct thing against the correct implementation. But ideally, I would like to conform only once and have the correct thing happen. Is there a way to do that?


FYI - I have two multispecs there as unfortunately multi-spec seems to expect a single field as a dispatch tag, even though the multimethods themselves can dispatch on more than one field (through the use of juxt).


@alexmiller ^ Will definitely appreciate some thoughts on this situation! 🙇

Alex Miller (Clojure team)17:09:21

well on the last point first, you can use an arbitrary function for the multi-spec, doesn't have to be a keyword

Alex Miller (Clojure team)17:09:41

so you could for example use juxt to produce a vector and dispatch from a single multimethod

Alex Miller (Clojure team)17:09:47

maybe that solves your whole problem, not sure


What would I use for a retagging function though?

Alex Miller (Clojure team)17:09:48

it will pick a random multimethod, gen from that, then "retag" to something passed to the multimethod

Alex Miller (Clojure team)17:09:47

here it would need to go from the dispatch value (the vector output of juxt) to modify the generated defmethod map to get back to your original expected input

Alex Miller (Clojure team)17:09:17

depending on your method generators, there may be nothing to do and you could just use identity

Alex Miller (Clojure team)17:09:27

or you could assoc back in the keys and the values from the juxt vector


With juxt I'd be ideally looking at something as follows:

(defmulti cluster (juxt :cluster-type :action))
(s/def ::cluster (s/multi-spec cluster ?))
I'm just not sure what to put in instead of the ?.

Alex Miller (Clojure team)17:09:34

as above, identity might work, or you could use (fn [[type action]] (assoc % :cluster-type type :action action))


Ah, identity might just do the trick. Let me give that a try!

Alex Miller (Clojure team)17:09:01

or maybe I've got the signature wrong there - does retag fn take 2 args?

Alex Miller (Clojure team)17:09:33

yeah, it's value and dispatch value


I think it only takes one? It can also be a keyword, which would support that argument.

Alex Miller (Clojure team)17:09:58

so should have been (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))


Ah no, it actually takes two. But apparently one can supply a keyword as well.

Alex Miller (Clojure team)17:09:01

I think that's right. I haven't done one of these in a while.


Why is the retagging even necessary, when the multimethods return a spec that is narrow enough to always generate the right data? I guess that's one thing that's confusing me.


Hm, this did not really solve the problem - but thanks anyway. I'll post here if I manage to solve this in a satisfactory fashion.


Actually, I take that back - I had a dangling redef on the multispec elsewhere. Doing it as per Alex's suggestion:

(defmulti cluster (juxt :cluster-type :action))
(s/def ::cluster (s/multi-spec cluster (fn [gen-val [type action]] (assoc gen-val :cluster-type type :action action))))
does the trick. Thanks for the help, @alexmiller!