Fork me on GitHub
#clojure-spec
<
2019-05-20
>
Jakub Holý (HolyJak)05:05:39

I see I should have used https://clojure.github.io/spec.alpha/clojure.spec.test.alpha-api.html#clojure.spec.test.alpha/check instead. Still, how to best integrate with clojure.test? Something like

(deftest xyz 
  (if-let [f (:failure (st/check ::my-spec)]
    (throw f) 
    (is true)))
?

conan10:05:06

I use this namespace:

clojure
(ns ic.test-util
  (:require [clojure.spec.test.alpha :as stest]
            [clojure.test :refer :all]
            [expound.alpha :as expound]))

(defn check
  "Passes sym to stest/check with a :max-size option of 3 (i.e. generated sequences will have no more than 3 elements,
   returning true if the test passes or the explained error if not"
  [sym]
  (let [check-result (stest/check sym {:clojure.spec.test.check/opts {:max-size 3}})
        result (-> check-result
                   first ;; stest/check accepts a variable number of syms, this does not
                   :clojure.spec.test.check/ret
                   :result)]
    (when-not (true? result)
      (expound/explain-results check-result))
    result))

conan10:05:31

then my tests look like this:

(ns ic.date-test
  (:require
   [clojure.test :refer :all]
   [ic.date :as date]
   [ic.test-util :as tu]))

(deftest inst->local-date-time-test
  (is (true? (tu/check `date/inst->local-date-time))))

conan10:05:30

this will either pass, or give an expound-formatted error showing where the :args, :ret and :fn specs went wrong for the generative tests run for my function (in this case, inst->local-date-time)

borkdude11:05:51

so in the context of this question: https://github.com/borkdude/respeced#successful

❤️ 4
borkdude11:05:31

that function also checks if the sequence of results is not empty

boyanb16:05:59

Does anybody have experience they would be willing to share in regard of human readable messages at API boundaries via spec? We've looked at expound(which doesn't fit) and phrase(which could do the job, but I am personally not convinced by the predicate focused approach). It feels that a simpler solution focused around explain-data and a message registry(similar to the one found in expound) would produce a better result. Has anybody implemented/is currently using spec for this purpose?

jeroenvandijk16:05:08

@boyanb Can you share how you feel expound is not a fit?

boyanb16:05:14

It's not really designed with the idea of message formatting for "users". Expound could very easily be enhanced or parts/ideas of it lifted into a library that could fit it. AFAIK, while looking at the code, there were several places where we needed paramtrization/additional control that is currently not available by the public facade of the library,

boyanb16:05:24

(had to do with custom printers and expected outputs)

boyanb16:05:00

I could probably dig in a little further and find the concrete examples. Are you using it with user facing messages in any way?

jeroenvandijk16:05:06

I've used expound like this

(defn validate-data [spec data]
  (if (s/valid? spec data)
    :ok
    (if-let [explain-data (s/explain-data spec data)]
      (let [expound-state
            (try
              (expound/expound spec data)
              :ok
              (catch Exception e
                (println "Expound had difficulties using " explain-data)
                :error))]
        (throw (ex-info (if (= expound-state :ok)
                          "Spec error (see stdout)"
                          "Spec error (see explain data)") {:explain explain-data})))
      (throw (ex-info "Specs are in a weird state, as we can't explain why data is invalid" {})))))

jeroenvandijk16:05:17

But I agree it is not perfect

jeroenvandijk16:05:13

I've actually used this in a clojurescript environment (using with-out-str). Worked pretty will when you do a validation on every key change

kenny16:05:25

@boyanb We had a similar problem. We wanted to spec our API using Clojure Spec but we didn't want to expose Spec's error messages to our users (most people are not used to seeing error messages in that format). This immediately meant Expound was out of the question (too similar to Spec). I looked at Phrase but it seemed like you'd need to duplicate some code to get error messages. I didn't really like that. The overall approach Phrase took made sense -- take the output of explain-data, match it, and convert it to error messages. The matching part seemed like a perfect fit for core.match. So I took that direction and wrote https://github.com/Provisdom/user-explain. I didn't have time to write docs for it 😞 IMO the library is much more general than Phrase due to all the features of core.match. It's also pretty simple -- only 75 LOC 🙂

👍 8
ericstewart21:05:20

thank you for sharing this! Going to take a look as I have been on the same path as you and you are further ahead it seems.

kenny21:05:52

Of course! LMK if it works out for you.

boyanb16:05:28

our implementation was with-out-str exactly.

boyanb16:05:30

(when with expound). In the end, we still didn't have the formatting we wanted. Main point is, underlying users are really not familiar with anything clojure and shouldn't care about implemenmtation and in the end we needed fine grained control to explain-data to be able to format path within spec + spec message as we needed.

boyanb16:05:11

@kenny - what you describe matches exactly our needs. I'll take a look.

kenny16:05:43

The tests may be helpful for documentation. I'd recommend just reading the 75 line implementation though. There's a lot of areas I want to improve. As you start diving into the data produced by explain-data, you realize that there's a ton of "core" predicates that need to be handled. A common case is #(contains? % kw) which is used with s/keys to validate keywords exist on the map. I'd like to provide a way get automatic nice error messages for all the "core" predicates.

boyanb16:05:04

Oooh, that's great. @kenny. Looking at the tests it's almost exactly what we are looking for. I'll play with it and let you know if we decide to ship.

boyanb16:05:17

Yes, the tests are where I started ;o)

kenny16:05:11

There's a couple weird things about the implementation that I really need to write down before I forget: 1. Since defexplainer does not have a name associated with it, the only way to uniquely identify "explainers" is via the matching map. If you change the matching map (i.e. add or remove keys), the old matcher will still be def'ed. You'll need to run clear! to reset everything. This is really annoying and I don't have a great solution atm. The most obvious solution would be to name every defexplainer. 2. Order of defexplainers does matter. If you def the most general explainer first, it will always get matched first. Ensure more specific matches are def'ed before general ones. Technically this could be fixed by sorting based on some sort of heuristic but I didn't have the time to work out what that should be.

kenny16:05:40

Sure. LMK if it ends up working out or if you guys take another approach.