This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-03-09
Channels
- # aleph (1)
- # announcements (4)
- # asami (6)
- # babashka (45)
- # beginners (19)
- # biff (3)
- # calva (35)
- # cider (4)
- # clojars (5)
- # clojure (117)
- # clojure-art (3)
- # clojure-denmark (2)
- # clojure-europe (89)
- # clojure-gamedev (5)
- # clojure-nl (4)
- # clojure-norway (17)
- # clojure-spec (3)
- # clojure-uk (5)
- # clojurescript (84)
- # conjure (13)
- # datomic (11)
- # emacs (2)
- # figwheel (2)
- # fulcro (16)
- # graphql (5)
- # honeysql (7)
- # introduce-yourself (1)
- # lsp (86)
- # malli (16)
- # music (1)
- # off-topic (2)
- # pathom (14)
- # polylith (28)
- # re-frame (11)
- # reagent (23)
- # releases (1)
- # reveal (19)
- # shadow-cljs (72)
- # spacemacs (13)
- # sql (1)
- # test-check (3)
- # timbre (4)
- # tools-deps (45)
- # vim (18)
Hello, Malli looks very cool.
Is it possible in Malli to retain values generated during validation like spec/conform
and spec/conformer
? Or if not maybe you can think of some Malli-ish way to solve this:
I have some expensive validations, for example I accept an xpath as input and validate by compiling it. I don't want to throw the resulting value away and have to re-generate it later. So I transform (with s/conformer
) the xpath string to a map with a compiled version (object) of it during validation and this map is the result of validation (with s/conform
):
(require '[clj-xpath.core :as xpath])
(defn expr-or-nil
[maybe-expr]
(try
(xpath/xp:compile maybe-expr)
(catch org.xml.sax.SAXException e)
(catch javax.xml.xpath.XPathException e)))
(s/def ::xpath (s/and string?
(s/conformer (fn [value] {:value value
:expr (expr-or-nil value)})
:value)
#(:expr %)))
(s/conform ::xpath "//a/@href")
;; {:value "//a/@href",
;; :expr
;; #object[org.apache.xpath.jaxp.XPathExpressionImpl 0x1aa56eab "org.apache.xpath.jaxp.XPathExpressionImpl@1aa56eab"]} <---DONT HAVE TO RE-PARSE LATER, YAY! :-)
(s/conform ::xpath "%#^@")
;; :clojure.spec.alpha/invalid
I do similar things when I accept regex strings as input - I validate by compiling them with re-pattern
in a conformer then I save the output from s/conform
for use later.
So is it possible to do something like this with Malli? The :decode/math
examples in the Value Transformation section of the README.md (https://github.com/metosin/malli#value-transformation) seem possibly close? But it's not clear to me if decode
validates.
Maybe Malli is trying to keep validation in a different stream from transformation?
(I admit I don't understand the transformation section of the docs very well, particularly when it comes to doing custom functions as opposed to the pre-canned transformers for json and so forth.)
Kiitos/thanks for any help@ryantate you have two models there: a string and a parsed map. With malli, you should describe either the string (source) or the target (a map). If the map presentation is what you expect, describe that and add a decoder from string->it. A naive impl:
(defn expr-or-nil [maybe-expr]
(when (= maybe-expr "//a/@href") identity))
(defn decoder [value]
{:value value
:expr (expr-or-nil value)})
(def Xpath
(m/schema
[:map {:decode/string decoder}
[:value :string]
[:expr fn?]]))
(defn coerce [value]
(let [decoded (m/decode Xpath value (mt/string-transformer))]
(if-not (m/validate Xpath decoded)
::invalid
decoded)))
(coerce "//a/@href")
; => {:value "//a/@href", :expr #object[clojure.core$identity 0x2c879e65 "clojure.core$identity@2c879e65"]}
(coerce "invalid")
;; => :user/invalid
but yes, the transform and validation are separated. There might be use cases where this is not good, but for most cases, itโs easy to compose the two using public apis. If/when someone finds out a need for one-pass-do-it-all &/ stateful transformers, would like to hear those ๐
@ikitommi Ah thanks for the comprehensive reply!! ๐
string -> compiled-xpath is the goal, the map is just there to allow s/unform
which is not essential.
In my case I would probably just accept the cost of two passes because I need the xpath compilation in order to validate the string) and also the xpath string needs to be composed into a more complicated validation so I can just do one m/validate
call on the larger structure (which can have many xpath strings). So I think the coerce
function would be tricky to use. I could just validate all xpath as string in the larger value and then m/validate
and m/parse
the larger value and then walk it and call coerce
on each xpath item but I would have to do that also for regex and anything else that gets compiled - becomes basically building a new validation layer which defeats the point a bit.
I did a benchmark and compiling xpath is only about 2 ฮผs and regex is even less so I will just put those checks in the schema so it is part of a compostable validation and then do it again in m/decode
. It is a performance hit but a small one, hopefully not too bad ๐
oh, if you want to validate the big thing, you could do schema for string, and then m/encode
it to the parsed format. there is no easy way to customize m/parse
per schema atm.
for performance, you should use m/validator
& m/decoder
/ m/encoder
for MUCH better perf.
;; pure functions, can be cached
(def validate (m/validator Xpath))
(def decode (m/decoder Xpath (mt/string-transformer)))
(defn coercer [value]
(let [decoded (decode value)]
(if-not (validate decoded)
::invalid
decoded)))
Issue with :orn
schema inside decode
.
Any idea why this works:
(m/decode [:string {:decode/string (fn [s] (try (re-pattern s) (catch java.util.regex.PatternSyntaxException e)))}] "foo" mt/string-transformer)
;;#"foo"
But this does not?:
(m/decode [:orn [:v vector?] [:re-string [:string {:decode/string (fn [s] (try (re-pattern s) (catch java.util.regex.PatternSyntaxException e)))}]]] "foo" mt/string-transformer)
;;"foo"
Is decode
meant to work with :orn
, :catn
, :or
, :and
etc?Actually I think I have mixed up encode
and decode
.
I found this which has helpful definition of differences (maybe later I can make a PR to merge this table into README.md?):
https://cljdoc.org/d/metosin/malli/0.8.4/doc/value-transformation
decode
is for making values valid, encode
is to transform valid values into something else.
decode
will revert value if it does not match the schema per https://clojurians.slack.com/archives/CLDK6MFMK/p1615225531254000
I rewrote this way which works:
(m/encode [:orn [:v vector?] [:re-string [:string {:encode/string (fn [s] (try (re-pattern s) (catch java.util.regex.PatternSyntaxException e)))}]]] "foo" mt/string-transformer)
;;#"foo"
Also, a transformation debugger would be nice (and not hard to implement?): would emit which steps are run, in which order and how they change / not the value.
Can I combine parse
and encode
functionality using a single schema?
I can parse:
(def re-schema [:string {:encode/string (fn [s] (try (re-pattern s) (catch java.util.regex.PatternSyntaxException e)))}])
(def re-w-opts-schema [:and vector? [:catn [:pattern re-schema] [:opts [:* [:enum :i :s :u]]]]])
(def re-maybe-w-opts [:orn [:pattern re-schema] [:pattern-w-opts re-w-opts-schema]])
(m/parse re-maybe-w-opts ["foo" :i])
;;[:pattern-w-opts {:pattern "foo", :opts [:i]}]
I can encode:
(m/encode re-maybe-w-opts ["foo" :i] mt/string-transformer)
;;[#"foo" :i]
How to do both and get
[:pattern-w-opts {:pattern #"foo", :opts [:i]}]
?currently, you canโt combine those and there is no schema property-based extension for parsers, so you could plug in custom logic to parsing. I think you could cal m/parse
in a custom encoder :leave
yourself?
oh, the branching happens at parse, would not work that way. should add hook to parsing to allow custom steps or somehow mix the two (encode & parse). ideas welcome
Thanks for the reply, going to work on this for a bit. Parsing is more important for my use case than other transforms. Will come back if I have API extension ideas. Kiitos ๐
I came to ask exactly the same thing. @ryantate can you let me know if you come up with something?