Fork me on GitHub
#clojure-spec
<
2021-04-09
>
Matti Uusitalo10:04:36

Background: I want to create a spec which can validate values in different branches of a document in a manner that what I get out of spec/explain-data points to the right value. For example:

{:min 0 :max 10 :values [0 1 2 3 11 2]}
I want the spec to be able to say that because 11 is not between the values given in :min and :max, it is in error. I’m trying to implement the Spec protocol to add a wrapper for a spec which works like this
(defmacro value-binding [pred bind]
    `(let [wrapped# (delay (#'spec/specize ~pred))]
       (reify
         spec/Specize
         (specize* [s#] s#)
         (specize* [s# _] s#)

         spec/Spec
         (conform* [spec x#]
           (binding [~bind x#]
             (spec/conform* @wrapped# x#)))
         (unform* [spec y#]
           (binding [~bind y#]
             (spec/unform* @wrapped# y#)))
         (explain* [spec# path# via# in# x#]
           (binding [~bind x#]
             (spec/explain*
              @wrapped#
              path# via# in# x#)))
         (gen* [spec overrides# path# rmap#]
           (spec/gen* @wrapped# overrides# path# rmap#))
         (with-gen* [spec gfn#]
           (spec/with-gen* @wrapped# gfn#))
         (describe* [spec]
           (spec/describe* @wrapped#)))))
The idea is that when the validation passes this spec, the validated value is bound to the given dynamic variable. The wrapped spec can then use the value in that dynamic variable. I got this working with spec/valid? but not with spec/explain-data, which always returns an empty collection. Any ideas why? I made a test spec which prints if the dynamic variable has been bound to a value or not. Inside conform* it is bound, but inside explain* it is not.

Matti Uusitalo11:04:07

Some further info… I can make a minimal spec in unit tests, but in a larger context this breaks down

ikitommi11:04:58

for that given example:

(s/explain (s/coll-of (s/int-in 0 10)) [0 1 2 3 11 2])
; 11 - failed: (int-in-range? 0 10 %) in: [4]

Matti Uusitalo11:04:29

The issue here is that values in the collection should be able to be valid or invalid depending on values in a different part of the document. The example I wrote is a minimal example that tries to convey the idea

Matti Uusitalo11:04:56

I would use my spec like this

(def ^:dynamic *document*)
(spec/def ::min int?)
(spec/def ::max int?)

(spec/def ::value (fn [v]
                      ; compares that v is between min and max
                   ))
(spec/def ::values (spec/coll-of ::values))

(spec/valid?
 (value-binding
  (spec/keys :req-un [::min ::max ::values])
  *document*)
{:min 0 :max 10 :values [0 1 2 3 4 5 11]})

ikitommi11:04:42

if spec supported maps with inlined entry definitions, that would be easy to do - you could create a new spec with new ::values subspec based on the whole document, but now it would require going through the global registry. I believe spec2, plumatic & malli all make this easy to do. there is also spec-tools, with a working(?) dynamic var for stuff like this, but don’t recommend it.

benoit11:04:20

I would usually put the constraint on the map itself since it is a constraint between its elements.

benoit11:04:10

(s/and (s/keys ...)
       (fn [{:keys [min max values]}] (every? #(< min % (inc max)) values)))

benoit11:04:57

But you want explain-data to return the value ... I see. Good luck with that 🙂

benoit12:04:02

Could you just redefine the spec for every document?

benoit12:04:17

(defn data-spec
  [{:keys [min max values]}]
  (s/def ::min int?)
  (s/def ::max int?)
  (s/def ::values (s/coll-of (s/int-in min max)))
  (s/def ::data (s/keys :req-un [::min ::max ::values])))

(let [data {:min 0 :max 10 :values [0 1 2 3 11 2]}]
  (data-spec data)
  (s/explain-data ::data data))

Matti Uusitalo03:04:43

The most annoying thing is when I make a simple example to see where it breaks down it works fine.

ikitommi06:04:35

the data-spec is not safe as it mutates the global registry. If you have two documents with different min & max, the last one overrides the first.

ikitommi06:04:49

How did you make it work @USGKE8RS7?

benoit20:04:20

Yes, you have to be careful to not use data-spec for multiple documents at once. Sometimes it is an acceptable trade-off.

benoit20:04:17

I'm not seeing a way around it if you want to use s/keys and the global registry. It does not make sense to me to define a spec on the global keyword ::values that is specific to a given map. The contract that the integers in ::values must be between :min and :max is a property of the map, not the global ::values keyword. If you still want to benefit from the s/explain infrastructure, you can always write a "local spec" like this:

(defn validate-map
  [{:keys [min max values] :as data}]
  (let [s (s/coll-of (s/int-in min max))]
    (when-not (s/valid? s values)
      (throw (ex-info "Invalid map."
                      {:explain (s/explain-data s values)})))))

Matti Uusitalo04:04:22

So I finally figured out what was the problem. I had to wrap the nested spec/explain* call to a doall, because apparently explain returns a lazy sequence which can’t access the bound value if it is returned out of the binding block before realizing the sequence

Matti Uusitalo04:04:12

@U055NJ5CC i have now

(defmacro value-binding [pred bind]
  (let [pf #?(:clj (#'spec/res pred)
              :cljs (res &env pred))]
    `(let [wrapped# (delay (#'spec/specize ~pred ~pf))]
       (reify
         spec/Specize
         (specize* [s#] s#)
         (specize* [s# _] s#)

         spec/Spec
         (conform* [spec x#]
           (binding [~bind x#]
             (spec/conform* @wrapped# x#)))
         (unform* [spec y#]
           (binding [~bind y#]
             (spec/unform* @wrapped# y#)))
         (explain* [spec# path# via# in# x#]
           (binding [~bind x#]
             (doall (spec/explain*
                     @wrapped#
                     path# via# in# x#))))
         (gen* [spec overrides# path# rmap#]
           (spec/gen* @wrapped# overrides# path# rmap#))
         (with-gen* [spec gfn#]
           (spec/with-gen* @wrapped# gfn#))
         (describe* [spec]
           (spec/describe* @wrapped#))))))
and then for example
(testing "Bound values can be referred to in specs"
    (let [test-spec
          (sut/value-binding
           (fn [v]
             (= *test-binding* v))
           *test-binding*)]
      (is (spec/valid? test-spec 123))))
at some point that breaks down without that doall because of the lazyness & bindings

Matti Uusitalo04:04:51

that cljs stuff is there because clojurescript has a different implementation of clojure.spec.alpha

Michael Stokley19:04:07

given that all map keys are validated against the spec registry, what is the significance of the :opt argument to s/keys? documentation?

jjttjj19:04:39

I think it adds the generator stuff for the optional key

jjttjj19:04:52

(so it will sometimes be generated)

🙌 3
Alex Miller (Clojure team)20:04:35

also ends up in the doc output

🙏 3