clojure-spec

Jonas Östlund 2024-07-09T11:02:33.754609Z

Given some spec ::my-spec , is there some validator function that lets me pre-compile a predicate function with good performance like (def my-spec? (validator ::my-spec)) so that the expressions (my-spec? x) and (clojure.spec.alpha/valid? ::my-spec x) are equivalent for any value x? Background: Using VisualVM, I noticed that calling clojure.spec.alpha/valid? took a lot of time and thought that there must be a way to speed it up. I use version 1.11.3 of Clojure and the clojure.spec.alpha that comes with it. My attempt: Using the library https://github.com/metosin/spec-tools , I wrote the following incomplete and buggy implementation of such a validator function and achieved a speed-up of 45 times for my usecase:

(require '[spec-tools.core :as st]
          '[spec-tools.parse :as st-parse])

(declare validator)

(defn key-validator [k on-missing]
  (let [p (validator k)]
    (fn [x]
      (let [y (get x k ::missing)]
        (if (= y ::missing)
          on-missing
          (p y))))))

(defn normalize-spec [x]
  (cond
    (map? x) x
    (st/spec? x) x
    (keyword? x) (recur (st/get-spec x))
    :else (st/create-spec {:spec x})))

(defn validator
  "Given argument `src`, returns a function `f` such that `((f src) x)` and `(clojure.spec.alpha/valid? src x)` are equivalent for any `x`."
  [src]
  (let [{:keys [type spec leaf?] :as x} (normalize-spec src)]
    (case (st-parse/type-dispatch-value type)
      :vector (let [item-pred (validator (::st-parse/item x))]
                (fn [x]
                  (and (sequential? x)
                       (every? item-pred x))))
      :map (let [preds (vec (for [[on-missing ks] [[false (::st-parse/keys-req x)]
                                                   [true (::st-parse/keys-opt x)]]
                                  k ks]
                              (key-validator k on-missing)))]
             (fn [x]
               (and (map? x)
                    (every? #(% x) preds))))
      :string string?
      :long int?
      :boolean boolean?
      (if (and leaf? (fn? spec))
        spec
        (throw (AssertionError. "Cannot make predicate"))))))
But maybe there is a library that does this already? I have the feeling that I may be reinventing the wheel...

ikitommi 2024-07-09T11:29:15.407819Z

Hi Jonas, long time 🙂 Not sure if there are existing libraries for this but as author of spec-tools: if you want to add fast-path validator into spec-tools, happy to review / help with the changes. Can also give Maintainer-access. If we could change spec internals, I would do the following: 1. conform* -> conformer* (return a function to do the conform) 2. validator* -> fast path for just validation, .e.g. returning boolean

ikitommi 2024-07-09T11:33:29.493009Z

I recall spec hasn’t had performance as a goal, so it doesn’t have this built-in.

Jonas Östlund 2024-07-09T11:36:33.696469Z

Hi Tommi 🙂 Thanks! OK, I may submit a PR to the spec-tools library if I have time with such a validator function. I am a bit unsure about the normalize-spec part in the code above, but I guess that can be reviewed in a PR if I decide to submit one. I don't think it would be so much work to make the validator function complete so that it covers all types of specs that there are.

👍 1
Alex Miller (Clojure team) 2024-07-09T12:09:48.912849Z

We took some steps in this direction in spec 2 (adding a non-conforming validation path)

👍 1