Fork me on GitHub
#malli
<
2020-10-25
>
jeroenvandijk10:10:18

I did, yes it is more involved 🙂

jeroenvandijk10:10:15

To be more specific, this project is a (conceptual) port from a clojure.spec project https://github.com/jeroenvandijk/aws.cloudformation.malli

jeroenvandijk10:10:43

I was generating the clojure.spec definitions from the AWS Cloudformation spec. This generation part was much easier with Malli

jeroenvandijk10:10:05

There are enough differences that I cannot consider it as a 1 on 1 port

borkdude10:10:13

Feel free to respond on Reddit as this was not my own question, just forwarded it here

jeroenvandijk10:10:34

yeah thanks. I’m not using reddit. I hope the reddit user finds it here 😅

👌 1
borkdude16:10:42

Is there any guidance on how to write your own transformer?

ikitommi16:10:43

@borkdude there is no guide yet, but lot of tests and some samples in the README.

ikitommi16:10:20

happy to help if you need any.

borkdude16:10:33

@ikitommi I think I asked this before, but I can't find any docs on this, nor can I find the conversation on Zulip. So here it goes:

(def Schema
  [:map [:x int?]])

(defrecord Wrapper [obj loc])

parsed 
;;=> {#script.Wrapper{:obj :x, :loc {:row 1, :col 2, :end-row 1, :end-col 4}} #script.Wrapper{:obj 1, :loc {:row 1, :col 5, :end-row 1, :end-col 6}}}

(defn wrapper-transformer []
  (mt/transformer
   {:name :wrapper
    :default-decoder :obj
    :default-encoder (fn [obj]
                       (->Wrapper obj nil))}))

(prn (m/decode Schema parsed wrapper-transformer)) ;;=> nil ... ???

borkdude16:10:14

so I want to validate/transform the wrapped value according to the schema

borkdude16:10:28

any value that is not Wrapped should just be transformed as is

borkdude16:10:42

but a value that is wrapped should be unpacked using :obj

borkdude16:10:05

and I want to give an error message based on :loc

ikitommi16:10:11

oh, i recall. need to dig in my histories too if there was/is a good answer.

borkdude16:10:40

I tried this:

(defn unwrap [x]
  (if (instance? Wrapper x)
    (:obj x)
    x))

(defn wrapper-transformer []
  (mt/transformer
   {:name :wrapper
    :default-decoder unwrap
    :default-encoder (fn [obj]
                       (->Wrapper obj nil))}))
but this just returns the entire object itself

borkdude16:10:19

it seems it doesn't handle the value recursively

ikitommi16:10:40

the problem is that the decoder is first given a map, it calls :obj on it, which return nil.

ikitommi16:10:05

I would use clojure.walk/pre-walk for first sweep of decoding.

ikitommi16:10:26

the decoding here walks: 1. the :map 2. the keys (`:x`)

ikitommi16:10:58

unless the Wrappedis already decoded on 1, there is no :x and the decoding stops.

ikitommi16:10:33

so, with malli, you would need to decode all the keys & values on :map step also. doable, but extra noise 😞

ikitommi16:10:40

(ns user
  (:require [malli.core :as m]
            [malli.transform :as mt]))

(def Schema
  [:map [:x int?]])

(defrecord Wrapper [obj loc])

(def parsed {(map->Wrapper {:obj :x, :loc {:row 1, :col 2, :end-row 1, :end-col 4}})
             (map->Wrapper {:obj 1, :loc {:row 1, :col 5, :end-row 1, :end-col 6}})})

(defn unwrap [x]
  (if (instance? Wrapper x) (:obj x) x))

(defn wrapper-transformer []
  (mt/transformer
    {:name :wrapper
     :decoders {:map (fn [x] (reduce-kv (fn [acc k v] (assoc acc (unwrap k) (unwrap v))) {} x))}
     :default-decoder unwrap
     :default-encoder (fn [obj] (->Wrapper obj nil))}))

(m/decode Schema parsed wrapper-transformer) ;;=> nil ... ???
; => {:x 1}

ikitommi16:10:49

but, this is much simpler:

(clojure.walk/prewalk unwrap parsed)
; => {:x 1}

borkdude16:10:51

Yes, but how do I get validation errors if I first call prewalk on this thing?

borkdude16:10:06

based on :loc

ikitommi16:10:10

I’ll check more of this later.

borkdude17:10:46

Thanks. Me too, cooking dinner :)

borkdude17:10:52

Hmm, in spec I would maybe have to spec a key as ::wrapped-int and then coerce it after it was checked?

borkdude17:10:11

which is not ideal

ikitommi17:10:40

can the wrapped by anywhere? Wrapped map/vector/set? All values are wrapped (any nested edn value)? Or just keys and values in the map?

ikitommi17:10:07

I have an idea, but would like to understand the case first.

borkdude17:10:13

@ikitommi The use case is preserving location information for non-iobjs and using that for error messages while validating malli schemas

borkdude17:10:36

e.g. you want to validate an EDN file and you get an error: this should be an int, on line 5, row 12

👌 1
borkdude18:10:38

@ikitommi This is the complete code: deps.edn:

{:deps {metosin/malli {:mvn/version "0.2.1"}
        borkdude/edamame {:git/url ""
                          :sha "ba93fcfca1a0fff1f68d5137b98606b82797a17a"}}}
(ns script
  (:require [edamame.core :as e]
            [malli.core :as m]
            [malli.transform :as mt]))

(def Schema
  [:map [:x int?]])

(defrecord Wrapper [obj loc])

(defn iobj? [x]
  (instance? clojure.lang.IObj x))

(def parsed
  (e/parse-string "{:x 1}"
                  {:postprocess
                   (fn [{:keys [:obj :loc]}]
                     (if (iobj? obj)
                       (vary-meta obj merge loc)
                       (->Wrapper obj loc)))}))

(defn unwrap [x]
  (if (instance? Wrapper x)
    (:obj x)
    x))

(defn wrapper-transformer []
  (mt/transformer
   {:name :wrapper
    :default-decoder unwrap
    :default-encoder (fn [obj]
                       (->Wrapper obj nil))}))

;; (prn parsed)
(prn (m/decode Schema parsed wrapper-transformer))

borkdude19:10:17

@ikitommi So the way it can work is:

(defn fail! [schema {:keys [:obj :loc]}]
  (throw (ex-info (str obj " did not satisfy " schema
                       " [at " (str (:col loc)":"(:row loc)) "]") {})))

(def <42 [:and 'int? [:< 42]])

(defn lift-non-iobj-schema [schema]
  [:map {:encode/success :obj,
         :encode/failure (partial fail! schema)} [:obj schema]])

(def Schema [:map [:a (lift-non-iobj-schema <42)]])

(def valid? (m/validator Schema))
(def success (m/encoder Schema (mt/transformer {:name :success})))
(def failure (m/encoder Schema (mt/transformer {:name :failure})))

borkdude19:10:33

E.g. this will print:

(prn (parse-validate-and-transform "{:a 42}"))

borkdude19:10:45

Syntax error (ExceptionInfo) compiling at (script.clj:32:1).
42 did not satisfy [:and int? [:< 42]] [at 5:1]

borkdude19:10:49

The downsize of this is that I have to wrap schemas myself in case I want to check something non-iobj-ish (strings, keywords, numbers)

borkdude19:10:25

And this doesn't seem to work for keywords for example:

(def <42 [:and 'int? [:< 42]])

(defn lift-non-iobj-schema [schema]
  [:map {:encode/success :obj,
         :encode/failure (partial fail! schema)}
   [:obj schema]])

(def Schema [:map [(lift-non-iobj-schema :a) (lift-non-iobj-schema <42)]])

borkdude19:10:09

anyway, maybe this is a too niche use case

ikitommi20:10:21

have an idea, need 30min to test & play

borkdude20:10:34

take yer time

borkdude20:10:36

if this can be fixed, potentially it would also work for rewrite-cljc structures: normal malli specs, but they run over the rewrite-cljc nodes using a simple function that looks at the actual value

borkdude20:10:43

it's basically a projection from wrapped thing to the essential value while the wrapped thing has more info to produce useful output in case of failure to parse/validate