Hi, can someone help me understand the difference between spec/and and spec/merge? The example of merge given in the Clojure Spec guide https://clojure.org/guides/spec (using :animal/common and :animal/dog) seems to work even if we use and instead of merge.

@rahulr92 s/merge will merge two s/keys specs (and, if I recall correctly, does not flow the conformed value); s/and is for combining a general spec with a general predicate (or spec) -- and it does flow the conformed value.

Also, if you look at the docstrings, the differences should be reasonably clear https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/merge

https://clojure.github.io/spec.alpha/clojure.spec.alpha-api.html#clojure.spec.alpha/and -- that specifies that conformed values are propagated.

In the merge docs, you'll see another difference: it generates maps that conform to all the merged specs. With and, the first spec is used as the generator, and then the other specs are used as predicates to filter the generated values (which may not satisfy those predicates and so overall generation of values can fail).

Hope that helps @rahulr92?

@seancorfield Thanks for the help. I understood the difference in generated value. However I still do not fully understand the difference while using these for validation. Could you please elaborated on what you mean by the ‘flow’ of conformed value? > ’Unlike ‘and’, merge can generate maps satisfying the union of the predicates. Doesn’t and also perform union i.e. it passes validation only if all the specs in it pass?

@rahulr92 Here's an example showing a difference

boot.user=> (require '[clojure.spec.alpha :as s])
nil
boot.user=> (s/def ::a (s/or :i int? :s string?))
:boot.user/a
boot.user=> (s/def ::b (s/or :b boolean? :s string?))
:boot.user/b
boot.user=> (s/def ::merged (s/merge (s/keys :req [::a]) (s/keys :req [::b])))
:boot.user/merged
boot.user=> (s/def ::anded (s/and (s/keys :req [::a]) (s/keys :req [::b])))
:boot.user/anded
boot.user=> (s/conform ::merged {::a 1 ::b "x"})
#:boot.user{:a [:i 1], :b [:s "x"]}
boot.user=> (s/conform ::anded {::a 1 ::b "x"})
:clojure.spec.alpha/invalid
boot.user=>

If you look at the result of conforming that map against (s/keys :req [::a]) you'll see why it fails with s/and but works with s/merge:

boot.user=> (s/conform (s/keys :req [::a]) {::a 1 ::b "x"})
#:boot.user{:a [:i 1], :b [:s "x"]}
boot.user=>

and finally, regarding generation:

boot.user=> (s/exercise ::merged)
([#:boot.user{:a 0, :b false} #:boot.user{:a [:i 0], :b [:b false]}] [#:boot.user{:a "", :b "N"} #:boot.user{:a [:s ""], :b [:s "N"]}] [#:boot.user{:a "u", :b "N"} #:boot.user{:a [:s "u"], :b [:s "N"]}] [#:boot.user{:a "", :b false} #:boot.user{:a [:s ""], :b [:b false]}] [#:boot.user{:a "8", :b true} #:boot.user{:a [:s "8"], :b [:b true]}] [#:boot.user{:a "u", :b "k"} #:boot.user{:a [:s "u"], :b [:s "k"]}] [#:boot.user{:a 2, :b "IzO"} #:boot.user{:a [:i 2], :b [:s "IzO"]}] [#:boot.user{:a 1, :b "8s0"} #:boot.user{:a [:i 1], :b [:s "8s0"]}] [#:boot.user{:a 0, :b "QsU1J"} #:boot.user{:a [:i 0], :b [:s "QsU1J"]}] [#:boot.user{:a 11, :b true} #:boot.user{:a [:i 11], :b [:b true]}])
boot.user=> (s/exercise ::anded)

clojure.lang.ExceptionInfo: Couldn't satisfy such-that predicate after 100 tries.

(which is no surprise because the ::anded spec won't accept anything)

Does that help @rahulr92?

@seancorfield Great explanation. This really helps. Thanks a lot!

I'm not sure if anyone else has had this issue before, but I have a really strange spec problem that has shown up twice: I have a spec that defines a or:

(s/def ::val (s/or :int integer? :symbol :simple-symbol?))

And then I put that inside something else: (s/def ::coll-of-stuff (s/coll-of ::val))

But when I try to validate some data, explain fails and tells me: (s/explain ::coll-of-stuff '[x 1 2]) "value [:int 1] failed to conform to integer? or symbol?"

The problem is that something is trying to validate the conformed value? What's strange is I can't seem to reproduce this in a smaller situation. But so far it's always been with s/or and every time it tries to validate the conformed value

how are you doing the symbols out of interest?

doing the symbols?

(s/explain-data ::vals ['s 1])

like that?

that was part of the example here, I'm actually using keywords

The symbols in your data you are trying i mean sorry

Sorry i’m a little confused

So here's the actual error message:

In: [:ast/select-attrs 0] val: [:attr :bar] fails spec: :ast/select-attr at: [:ast/select :ast/select-attrs :attr] predicate: simple-keyword?
In: [:ast/select-attrs 0] val: [:attr :bar] fails spec: :ast/select-attr at: [:ast/select :ast/select-attrs :child] predicate: map?
nil

(namespace names changed for security reasons

And the relevant specs:

(s/def ::select-attr (s/or
                       :attr simple-keyword?
                       :child (s/keys :req [::from ::select-attrs ::logical-attr]
                                      :opt [::where])))
(s/def ::select-attrs (s/coll-of ::select-attr))

So what I don't understand is why it's saying [:attr :bar] is failing to conform to simple-keyword?

It's like or is trying to validate after it conforms the value or something.

Wouldn’t [:attr :bar] be a (s/coll-of simple-keyword?), rather than a simple-keyword??

this is interesting, at the top level, there's a multi-spec. If I remove that it seems to work fine.

Oh, I see, you’re saying that [:attr :bar] is what you’d get after conforming (s/conform ::select-attr :bar)

@jgdavey that's what's strange, the data I'm passing in looks like this [:foo :bar]. Hence the path in the error being [:ast/select-attrs 0] so it gets into the select-attrs path, and then the first item doesn't validate because [:attr :bar] is not a simple-keyword.

right

Does using s/every instead of s/coll-of yield the same results?

wow, yep, I changed my ::select-op spec to this:

(s/def ::select-op (s/and
                     (fn [x]
                       (println "VALIDATE " x)
                       x)
                     (s/keys :req [::select-attrs ::from ::where])))

That's the top level spec. If I run it through multi-spec the values it tries to conform are [:attr :bar]. If I run it without multi-spec, the values are :bar

So multi-spec is doing something

let's see if I can get a minimal case now

@alexmiller Does https://dev.clojure.org/jira/browse/CLJ-2054 need to be marked as "ready for testing?"

wait...does:

(s/and ::foo
            ::bar)

guess so: " Returns a spec that returns the conformed value. Successive conformed values propagate through rest of predicates."

boom that's it, @jgdavey I used s/and and that flows conformed values through the predicates (rather mis-leading name). Replaced it with s/merge and it works fine

Interesting. That’s probably a subtly worth calling out in docs or something.

yeah, funny thing is that the docs are fairly clear, both s/and and s/merge mention it. I wonder if the behavior there changed somewhat recently

lol, nope, it's been like that since the start

I guess both could have options to control how/if the flow of conformed values is done

I tripped on the behavior of s/merge when I started with spec, so the other way around. I am not alone either

Yeah, I think part of it is that names are a tad overloaded in English. Merge makes sense to me since I'd expect (s/merge ...) to and things together. But it seems like a better name for s/and would be s/flow or something like that.

Maybe we should rename s/merge to s/smoosh 😛

we considered having s/and (non-flowing) and s/and-> (flowing) at one point

s/& works the same way

unfortunately the current s/and does the latter :(

but in the end, I didn't read the docs, so I'll slap my own hand this time

yes, s/and and s/& flow. s/merge does not

s/merge is conceptually pushing the value through all specs “in parallel”

so flowing doesn’t make sense (except conforming returns the conformed result of the last spec which is hard to reconcile with that view)

So what's the common use-case for flowing specs? I think I've only ever used them with s/conformer, which I tend to avoid these days

if you flow through a regex you’ll get structure which you can then easily apply predicates to

naming vars & is not the best idea

it was originally called andre :)

flowing is handy when you have s/keys spec checking map's keys and vals independently, and then you apply predicate checking inter-kv-dependencies.

although, thinking about it a moment longer – this use case is sort of orthogonal to flowing :D

'Is it pronounced "an-dray" or "and-re"?'

Would be the new "a-sosh" or "a-sock". 😄

the former