Fork me on GitHub
#malli
<
2023-09-22
>
David G00:09:34

I'm getting some weird errors with my schema with the decoder/transformers and :+ and would love if someone can help me figure this out (commented inside the code block):

(require '[malli.core :as m]
         '[malli.transform :as mt])

;; Malli doesn't seem to have a type that can be decoded for bigdec hence this more elaborate schema
(def bigdec-s (malli/-simple-schema
               {:type :core/bigdec
                :pred (partial instance? BigDecimal)
                :type-properties
                {:decode/json (fn [x]
                                (try (bigdec x)
                                     (catch Exception _ x)))}}))

;; :+ doesn't work
(m/decode [:+ [:and bigdec-s pos?]]
          ["0"]
          (mt/transformer
           (mt/key-transformer {:decode keyword})
           mt/json-transformer))
;; => ["0"]

;; :vector does work
(m/decode [:vector {:min 1} [:and bigdec-s pos?]]
          ["0"]
          (mt/transformer
           (mt/key-transformer {:decode keyword})
           mt/json-transformer))
;; => [0M]
What's concerning is that m/explain just throws an exception (also does m/coerce):
(m/explain [:+ [:and bigdec-s pos?]]
           (m/decode [:+ [:and bigdec-s pos?]]
                     ["0"]
                     (mt/transformer
                      (mt/key-transformer {:decode keyword})
                      mt/json-transformer)))
;; => Execution error (ClassCastException) at malli.core/-simple-schema$reify$reify$explain (core.cljc:658).
;;    class java.lang.String cannot be cast to class java.lang.Number (java.lang.String and java.lang.Number are in module java.base of loader 'bootstrap')

(m/explain [:vector {:min 1} [:and bigdec-s pos?]]
           (m/decode [:vector {:min 1} [:and bigdec-s pos?]]
                     ["0"]
                     (mt/transformer
                      (mt/key-transformer {:decode keyword})
                      mt/json-transformer)))
;; => {:schema [:vector [:and :core/bigdec pos?]],
;;     :value [0M],
;;     :errors ({:path [0 1], :in [0], :schema pos?, :value 0M})}
I can paste this into the Github repo as an issue if that's preferable

steveb8n06:09:26

instrumentation is checking value but the errors thrown don’t include/display the explanation

steveb8n06:09:55

I guess it’s either not possible or need a specific setup

steveb8n06:09:36

I can workaround by capturing input values in REPL and using m/explain but the jvm DX is so nice I’d love to have it in the node env as well

ikitommi11:09:05

not using in node, so just all 👂s with this one.

dvingo15:09:40

I just tried it out with a small :node-script target and just need to change the default report argument from thrower to reporter: https://github.com/metosin/malli/blob/2a5fc9a6bbf1360881104df2974dc319d3d17431/src/malli/dev/cljs.cljc#L29 in shadow-cljs.edn:

:instrument-node {:target :node-script
                  :output-to "out/instrument_node.js"
                  :main malli.instrument-node/main }
the ns:
(ns malli.instrument-node
  (:require
    [malli.core :as m]
    [malli.dev.pretty]
    [malli.dev.cljs :as dev]))

(defn plus [x] (inc x))
(m/=> plus [:=> [:cat :int] [:int {:max 6}]])

(defn minus
  {:malli/schema [:=> [:cat :int] [:int {:min 6}]]}
  [x] (dec x))

(defn main [& args]
  (println "in main"))

(dev/start! {:report (malli.dev.pretty/reporter) :skip-instrumented? true})

(comment
  (m/function-schemas :cljs)
  (plus "hi")
  (minus 5)
  )
in the terminal where node is running you'll get printed output now instead of exceptions e.g.:
-- Schema Error ------------------------------------------------------------------------------------

Invalid function return value:

  4

Function Var:

  malli.instrument-node/minus

Function arguments:

  [5]

Output Schema:

  [:int {:min 6}]

Errors:

  {:in [], :message "should be at least 6", :path [], :schema [:int {:min 6}], :value 4}

More information:

  

----------------------------------------------------------------------------------------------------

dvingo15:09:02

not sure about "instrumentation is checking value but the errors thrown don’t include/display the explanation" I'm seeing even with the thrower I get the same report in the repl output it just doesn't show in the terminal

steveb8n23:09:51

thanks @U051V5LLP you are right. I need to apologise because I just realised it was my own logging config that was not displaying the explanation in the node env

steveb8n00:09:12

it is working as you demonstrated for me too

steveb8n00:09:44

that said, I have uncovered a challenge in this scenario. I have a try/catch block that is catching the exception thrown by malli.core/-fail

steveb8n00:09:02

in the catch block I’m trying to log the explain. it’s not in the exception data. I’ve tried using m/explain using the :schema and :inputs data from the exception but it returns nil. can’t figure out why

steveb8n00:09:26

here’s a repro….

steveb8n00:09:34

(mx/defn ^:malli/always combine :- :string
  [{:keys [prefix suffix]} :- [:map
                               [:prefix :int]
                               [:suffix :string]]]
  (str prefix " -> " suffix))

(md/start!)

(try
  (combine {:prefix "1" :suffix "a"})
  (catch :default e
    (let [{:keys [args schema]} (:data (ex-data e))]
      (malli.core/explain schema (first args)))))

steveb8n00:09:53

the explain in the catch block returns nil in the node env

steveb8n00:09:48

@ikitommi @U051V5LLP any suggestions on this? I now suspect either mx/defn or ^:malli/always as the problem because they were necessary to reproduce this

dvingo11:09:13

no worries @U0510KXTU was going to say if you have a repro.., so thanks : ) yea I think the issue would be in mx/defn implementation. From a quick scan for any differences vs what dev/start! is doing it looks like the mx/defn does not pass the :report option to -instrument https://github.com/metosin/malli/blob/30a2176f1893602d8a948ef3103fde91ec5fd638/src/malli/experimental.cljc#L56C39-L57C84 and those reporters are what add the :data and :type https://github.com/metosin/malli/blob/30a2176f1893602d8a948ef3103fde91ec5fd638/src/malli/dev/pretty.cljc#L88 to the ex-info map so could try adding that to the mx/defn implementation

dvingo11:09:13

also looks like both code paths in mx/defn (with and without malli/always) don't add :report

steveb8n08:09:10

Brilliant. I might try fixing this. Will PR if I do

steveb8n23:09:44

@ikitommi I just tried to fix mx/defn but I get lost in the connections between mx/defn calling -instrument and how that data flows to pretty. could you give me a nudge in the right direction about what needs to change for pretty to have the data it needs from mx/defn? I’ll test the fix locally once it is working and will PR

steveb8n23:09:16

tbh this shows my lack of macro-foo. I tried requiring pretty into the experimental ns and then creating a pretty/thrower to use in the macro body. but fails at runtime because I haven’t correctly spliced in the thrower fn.

steveb8n23:09:01

gpt4 tells me that cljs macros can’t use fns because they aren’t constants. at this point I’ll be guessing so I’ll wait for some cljs macro Jedi master instruction

markbastian13:09:35

Is there a way to specify a protocol using malli? Something like this:

(defprotocol MyProtocol)
(mc/validate MyProtocol (reify MyProtocol)) ;:malli.core/invalid-schema

skynet14:09:35

don't know if there's a better way, but you could do

[:fn #(satisfies? MyProtocol %)]

markbastian14:09:10

Cool! Works for me. Thanks!

👍 1
Craig Brozefsky14:09:18

I asked a similiar question a few months ago, this is what I have in a common library now:

(defn protocol-schema
  "Returns a malli schema that ensures the value satisfies the given protocol."
  [prot]
  (m/-simple-schema
   {:type            (:on-interface prot)
    :pred            #(satisfies? prot %)
    :type-properties {:error/fn (fn [error _] (str "Expected value to satisfy " (:on-interface prot) " protocol."))}}))

Craig Brozefsky14:09:41

It MAY be worth considering what happens with protocols being redefined, and change that to take a symbol that is resolved in the pred....

Craig Brozefsky14:09:31

I saw may because I have not confirmed my hunch that it makes my schema a little delicate in the face of protocol redefs

Noah Bogart15:09:06

We have a validation middleware set up in our ring app that has historically used json-schema, and i'm introducing malli to it. everything works for the most part, but I'm running into slight awkwardness. I have a POST endpoint that expects a payload with a string uuid, and in the compojure handler I'm decoding the provided json to clojure/edn. When the endpoint validation was using json-schema, this worked nicely because json-schema expects a string and would verify that it could be parsed to uuid and then the malli decoding transformed the validated string to uuid. and I could use the same malli schema for both instrumentation and for validation because m.js/transform converted my :uuid malli schema to a {"type": "string", "format": "uuid'} json schema. Now I'd like to use the same :uuid schema for both payload validation and instrumentation. is my best bet just to write a (def payload-coercer (m/coercer PayloadSchema (mt/transformer mt/json-transformer))) and use that in the payload instead of using the schema itself? has anyone else run into this situation?

ikitommi18:09:10

I'm not sure I understand what is the question here.

Noah Bogart18:09:21

lol i'm sorry

Noah Bogart18:09:20

i think the question is, what's the best way to have a given malli schema represent both input to validate and internal model?

Noah Bogart18:09:35

is that possible (or desirable)?

ikitommi18:09:19

sure, m/coerce , m/coercer , m/assert or just function schemas:

(mx/defn no-op :- PayloadSchema [p :- PayloadSchema] p)

ikitommi19:09:04

do you want just dev-time validation or also runtime validation?

Noah Bogart14:09:39

both. i wired up a ring middleware which calls m/validate on a provided malli schema. i could call m/coerce, but that's not cached and i'm hoping to avoid writing manual (def ObjCoercer (m/coercer ObjSchema)) for every schema.

Noah Bogart14:09:59

maybe i could just write a coercer cache myself

ikitommi15:09:28

wait what, which routing library are you using? Reitit at least has built-in mw for request and response coercion with cached coercer.

Noah Bogart15:09:51

We're using Compojure and we have a homespun url-specs system that's very reminiscent of Reitit but predates it by a while

Noah Bogart15:09:56

i've suggested moving to reitit, but we have roughly 350 endpoints and switching is hard lol

ikitommi15:09:14

Cached now resolution in compojure is really hard, things like context bodies are resolved for each request at runtime

ikitommi15:09:27

compojure or compojure-api?

Noah Bogart15:09:06

just compojure

Noah Bogart15:09:04

we're not caching anything in the compojure routes at all, we have a separate map that is filled with entries like this:

[:get "/v0.1/public-invite-codes/:public-invite-code"] {:response #'invites/public-get-response}
and when resolving the request, the validation middleware calls (get url-specs (:compojure/route request)) and then for each of the keys in the spec, it calls the related validation function

Noah Bogart15:09:07

(when payload-schema (validate-payload route payload-schema payload)) etc

Noah Bogart15:09:22

it's pretty simple but does the trick