Fork me on GitHub
#malli
<
2021-11-23
>
Yehonathan Sharvit08:11:28

Malli decoders are very cool. But as far as I understand, in order to use a schema that relies on decoders, clients have to decode data before validating it. It means that a schema that relies on decoders cannot be used as-is for validating data. Am I correct?

ikitommi09:11:07

For now, that’s true. There was a discussion some time ago to add :parsed schema element. That would just run m/-parse in before validate, explain, transform etc. like s/conform on spec. Could also be :decoded, so that this would work:

(m/validate [:decoded {:decode :string} schema] "ip/127.0.0.1") ; => true

ikitommi09:11:18

you can do that “easily” in the user space too. Just a wrapper-schema basically.

Yehonathan Sharvit12:11:46

@ikitommi Here is my attempt to write a parser in user space:

(defn my-parse
  [schema data]
  (let [data' (decode schema data (transformer string-transformer default-value-transformer))]
    (if (validate schema data')
      (encode schema data' (transformer string-transformer default-value-transformer))
      (humanize (explain schema data')))))
It does the job with asset ids but the problem is that it transform integer to strings.
(def schema 
  [:map
   [:count :int]
   [:asset-id [:multi {:dispatch first
                     :decode/string #(str/split % #"/")
                     :encode/string #(str/join "/" %)}
             ["domain" [:tuple  [:= "domain"] domain]]
             ["ip" [:tuple {:error/message "Invalid IP"} [:= "ip"] ipv4]]]]])

(my-parse schema {:count 12
                  :asset-id "ip/127.0.0.1"})
;; {:count "12", :asset-id "ip/127.0.0.1"}
;; Oops "12" instead of 12

ikitommi12:11:31

just return the original instead of encode the whole value into string-domain?

ikitommi12:11:45

because:

(m/encode :int 1 mt/string-transformer) ; => "1"

ikitommi12:11:09

… or create a custom name for the parsing transformer, e.g. :parse - it’s it a name you have defined, it doesn’t transform anything else.

ikitommi12:11:11

(let [schema [:map
              [:x :int]
              [:y [:int {:decode/parse (partial + 10)
                         :encode/parse (partial * 2)}]]]
      t (mt/transformer {:name :parse})]
  (as-> (m/decode schema {:x 0, :y 0} t) $
        (m/encode schema $ t)))
;; => {:x 0, :y 20}

Yehonathan Sharvit13:11:49

I cannot return the original as I would like the default values to be added by the parser. For instance, with a schema that assigns 42 as a default value to :count

(def schema 
  [:map
   [:count [:int {:default 42}]]
   [:asset-id [:multi {:dispatch first
                     :decode/string #(str/split % #"/")
                     :encode/string #(str/join "/" %)}
             ["domain" [:tuple  [:= "domain"] domain]]
             ["ip" [:tuple {:error/message "Invalid IP"} [:= "ip"] ipv4]]]]])

(my-parse schema {:asset-id "ip/127.0.0.1"})
;; {:asset-id "ip/127.0.0.1", :count "42"}

Yehonathan Sharvit13:11:41

However, I really like the idea of a custom name

Yehonathan Sharvit13:11:19

We need that as our plan is to have a company wide schema repository to be consumed by all apps

Nikolas Pafitis21:11:28

Is there an option for MapSchemas to skip generating specific field/s?

ikitommi21:11:54

not atm, but could be. Ideas welcome

ikitommi21:11:48

here’s two:

;; a) a top-level property?
[:map {:gen/fields [:x :y]}
 [:x :int]
 [:y :int]
 [:z :int]]

;; b) entry-level property?
[:map
 [:x :int]
 [:y :int]
 [:z {:gen/gen nil} :int]]

ikitommi21:11:37

b would be 1 line change to the current impl.

Nikolas Pafitis22:11:28

i guess mu/select-keys could be used

Nikolas Pafitis22:11:08

I might have encountered a bug:

{:example/user   [:map {:fully/entity?   true
                        :fully.entity/id :user/id}
                  [:user/id :uuid]
                  [:user/username :string]
                  [:user/password :string]
                  [:user/email :string]]}
I have this map and i merge it with default-schemas and schemas to make my registry. Validate and generate work when I use them against
(m/schema type {:registry registry})
but the following returns just nil
(m/properties (m/schema type {:registry registry}))

Nikolas Pafitis22:11:10

It seems to work if i do deref before properties

Nikolas Pafitis22:11:40

Is this expected behaviour?

ikitommi20:11:29

It's the current behavior and surprises me every time. References are also schemas, and usually without properties. Think like:

(def foo (with-meta {} {:a 1}))
(meta #'foo) ;=> .. no :a here

ikitommi20:11:33

not 100% happy, but not sure would anything more right than the current behavior

Nikolas Pafitis21:11:26

I guess when passed to malli functions like m/properties the different schema types should be dereffed if required. Like in nil punning where nil "can take the correct shape" depending on context

ikitommi18:11:59

we would assume the references don't have properties, but they could. We could enforce them not to have, but not sure if that would be a good design decision :thinking_face: