Fork me on GitHub
#malli
<
2021-03-31
>
valerauko05:03:24

After seeing the #Performance section of malli's readme I'm quite interested in trying it out. One thing I was wondering is how it fares at compile-time? I'm just assuming that it achieves high runtime speed by offloading as much as possible to macros. Are there any numbers how this affects eg lein startup time?

ikitommi05:03:12

@vale good question. There are no macros in malli (ok, one mandatory). the “schema compile time” is at runtime, you can have a separate step to create an immutable & optimized validator/transformer/parser and reuse that.

ikitommi05:03:41

in many cases, even without reuse, it’s still faster than with alternatives, e.g. with spec / spec-tools - one-shot “gimme a validator, use it once and forget it”.

valerauko08:03:39

awesome, time to dive in then! thanks for the great work!

ikitommi05:03:36

malli.core (without sci) should load under 1sec from source and 0.1sec from classes.

ikitommi05:03:57

(= int? (m/validator [:and int? int? [:and int? [:and int?]]]))
; => true

ikitommi05:03:40

(= identity
   (m/decoder
     [:map
      [:id :int]
      [:name :string]
      [:tags [:vector :string]]]
     (mt/json-transformer)))
; => true

robert-stuttaford07:03:16

was hoping that humanize would use language like 'at least one string? is required' here, but instead, i get "unknown error". am i doing it wrong, or is this a TODO for humanize?

(m/explain [:sequential {:min 1} string?] ())

  ;; {:schema [:sequential {:min 1} string?]
  ;;  :value  ()
  ;;  :errors (#Error{:path   []
  ;;                  :in     []
  ;;                  :schema [:sequential {:min 1} string?]
  ;;                  :value  ()
  ;;                  :type   :malli.core/limits})}

  (-> (m/explain [:sequential {:min 1} string?] ())
      me/humanize)

  ;; ["unknown error"]

ikitommi07:03:12

@robert-stuttaford just missing an error mapping for :malli.core/limits. PR welcome to malli.error.

ikitommi07:03:48

you can also play with the mappings by passing them into me/humanize:

(-> (m/explain [:sequential {:min 1} string?] ())
    (me/humanize {:errors {::m/limits {:error/message {:en "there's no limits"}}}}))
; => ["there's no limits"]

ikitommi07:03:45

uncomplete, but close:

(defn min-max-sequence-error-message [error options]
  (str ((me/-pred-min-max-error-fn {:pred identity}) error options) " elements"))

(-> (m/explain [:sequential {:min 1} string?] ())
    (me/humanize {:errors {::m/limits {:error/fn {:en min-max-sequence-error-message}}}}))
; => ["should be at least 1 elements"]

Adam Helins08:03:45

The following scheme systematically produces a stack overflow. It is a recursive definition where one item (`:E`) contains another item (which might be :E as well, or not). I am surprised this stack overflow is that systematic. There must be something wrong somewhere unless I am missing the obvious. Tried with :or instead of :multi, same thing.

(def registry
     {:A    [:tuple [:= :A]]
      :B    [:tuple [:= :B]]
      :C    [:tuple [:= :C]]
      :D    [:tuple [:= :D]]
      :E    [:tuple
             [:= :E]
             :item]
      :item [:multi
             {:dispatch first}
             [:A :A]
             [:B :B]
             [:C :C]
             [:D :D]
             [:E :E]]})


(malli.gen/generate [:schema
                     {:registry registry}
                     :item])

Adam Helins08:03:05

Execution error (StackOverflowError) at malli.core/-schema (core.cljc:242).

Adam Helins08:03:19

Validation results in the same, I probably should open an issue actually

ikitommi08:03:11

@adam678 would [:ref :item] work? It's a lazy reference type

Adam Helins08:03:01

In the :E definition? Such as:

[:tuple
 [:= :E]
 [:ref :item]]
I've tried it different ways but get an exception: :malli.core/invalid-ref {:ref :item} Sorry, seems I didn't drink enough coffee today 😅

Adam Helins08:03:59

Right, refs must be namespaced...

Adam Helins09:03:21

@ikitommi However, there is still something wrong with generation. Adding another recursive item makes things very slow (eg. 2-3 seconds for generating something like [:E [:E [:A]]]. Adding a third recursive item makes things so slow that my fans are churning and after a couple of minutes there still isn't any result.

Adam Helins09:03:34

I did: https://github.com/metosin/malli/issues/408 I hope I am not spamming. It's just I don't quite remember having that sort of problems with Spec so I am having a bit of a hard time working with that. Not complaining though, so far using Malli has been very comfortable!

ikitommi08:03:21

there is no recursion detection for eager references.

ikitommi08:03:57

I Think StackOverflow is a correct way to fail here

nilern08:04:14

It makes sense but does leave something to be desired

fugbix08:03:16

Greetings everyone!! Thank so much for this super neat lib, really cool and useful. I have a bit of an issue, whereby requiring malli.core triggers a ClassNotFoundException:

nREPL server started on port 55188 on host 127.0.0.1 - 
REPL-y 0.4.4, nREPL 0.8.3
Clojure 1.10.3
Java HotSpot(TM) 64-Bit Server VM 11.0.9+7-LTS
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (require '[malli.core :as m])
Execution error (ClassNotFoundException) at java.net.URLClassLoader/findClass (URLClassLoader.java:471).
malli.impl.util.SchemaError

user=>
my project is leiningen and includes many dependencies (including borkdude/sci because I am using multi dispatching, Datomic, Kafka etc). I am using the latest version of malli (`metosin/malli "0.3.1"`), Of course using malli from a toy lean leiningen project works perfectly fine. Although not a Clojure expert, I’ve never seen something similar. And I really can’t imagine why malli.impl.util doesn’t load? Any help would me much appreciated, I’ll keep digging!

nilern08:04:30

No idea, I don't see anything nonstandard going on there

fugbix12:04:15

Thank you @U4MB6UKDL I have removed my target directory and everything’s fine now. Of course I would love to understand what brought it in this state, but I’ll carry on with work for now. Sorry for the fuss, and thanks again for that wicked malli lib, absolutely loving it.

dharrigan09:03:14

A little while ago, there was a conversation about stripping nulls from the return back to the client

dharrigan09:03:34

hjmmm...let me think a bit more...

dharrigan09:03:32

So, okay, I'm doing this in the setup of the ring/router

dharrigan09:03:35

:muuntaja (m/create
                      (assoc-in m/default-options
                                [:formats "application/json" :opts]
                                {:mapper (-> (j/object-mapper)
                                             (.setSerializationInclusion JsonInclude$Include/NON_EMPTY))}))

dharrigan09:03:07

Following throught the code, I believe I'm handed back an instance of the ObjectMapper from jsonista.core when I then set that option

dharrigan09:03:28

the thing is, if I do this (now!), then all of my coercion fails (using malli)

dharrigan09:03:54

"coercion": "malli",
    "errors": [
        {
            "in": [
                "repId"
            ],
            "message": "missing required key",
            "path": [
                "repId"
            ],
....
....

dharrigan09:03:17

if I put it back to :muuntaja m/instance, then all is well with the world

dharrigan09:03:48

I'm a bit lost as to why it fails

dharrigan09:03:24

it feels like it's not able to parse the incoming JSON data from the request

dharrigan10:03:45

I realise this is in the wrong channel (maybe I should put into #reitit)

ikitommi13:03:41

pushed out [metosin/malli "0.4.0"]. It’s a new MINOR as the :multi parse is changed (fixed) and leaning on the old (broken) code can break someone. https://cljdoc.org/d/metosin/malli/0.4.0/doc/changelog