Fork me on GitHub
#malli
<
2021-08-25
>
ikitommi06:08:14

@zikajk looks weird. md/start! btw calls the clj-kondo too, so just md/start! and you are done

ikitommi06:08:27

if you can repro, I can take a look

Akiz07:08:57

@ikitommi I will try, I experienced this after long running repl session so there could be a lot of reasons. After restarting - everything is working great.

ikitommi06:08:00

works here:

(def valid?
  (m/validator
    [:map
     [:export? boolean?]
     [:import? boolean?]
     [:gcs-name string?]
     [:extra string?]]))

(valid? 
  {:export? true
   :import? true
   :gcs-name "kikka"
   :extra "foo"})
; => true

Karol Wójcik06:08:10

I'm trying to use reitit-malli coercion with malli 0.6.1. When evaling [reitit.coercion.malli :as rss] I'm getting

------ WARNING - :fn-arity -----------------------------------------------------
 Resource: <eval>:2007:43
 Wrong number of args (3) passed to malli.core/-fail!
--------------------------------------------------------------------------------
Can someone reproduce?

ikitommi07:08:20

@karol.wojcik that’s a bug, the 3-arity was removed. please write an issue, need to fix reitit (or revert the change if it’s really needed)

Karol Wójcik07:08:25

Thank you @ikitommi for blazing fast response. I will write an issue! No worries I can wait 🙂

ikitommi08:08:46

Everyone: malli master has massive performance improvements related to Schema creation and transformation performance. If you are using malli especially with cljs, you might see order of magnitude improvements to startup-times. Please try and report both gains and bugs. Big thanks for @ben.sless for much of the hard work.

(def ?schema
  [:map
   [:x boolean?]
   [:y {:optional true} int?]
   [:z [:map
        [:x boolean?]
        [:y {:optional true} int?]]]])
        
;; 44µs => 8.5µs (5x)
(def schema (m/schema ?schema))

;; 26µs => 1.3µs (20x)
(m/walk schema (m/schema-walker identity))

;; 51µs => 6.5µs (8x)
(mu/closed-schema schema)
… there is still a lot of room for improvements, but would like to release these soon. Issue tracking, alternative ways to build large schema systems described and comment welcome on https://github.com/metosin/malli/issues/513.

🚀 18
Ben Sless10:08:57

@ikitommi Here's a riddle I can't find a good answer to, how come this is significantly faster than a PersistentHashMap backed registry

(defn fast-registry
  [m]
  (let [pred (comp keyword? key)
        im (doto (new IdentityHashMap 1000) (.putAll (into {} (filter pred) m)))
        -m (doto (new HashMap 1000 0.25) (.putAll (into {} (remove pred) m)))]
    (reify mr/Registry
      (-schema [_ k]
        (let [^Map m (if (instance? Keyword k) im (if (instance? Var k) im -m))]
          (.get ^Map m k)))
      (-schemas [_] m))))
By about 35 ns for keyword lookup compare
(cc/quick-bench (m/schema :int {:registry r}))
  (cc/quick-bench (m/schema :int))
But the registry lookup isn't that much faster, only ~7 ns
(cc/quick-bench (mr/-schema r :int))
  (cc/quick-bench (mr/-schema m/default-registry :int))

ikitommi10:08:56

🤷 micro benchmarks are hard. I would ask JMH how they really differ. Just swapping order of the tests, you might get different results. Also, more code, JIT behaves differently.

Ben Sless12:08:42

Interesting, JMH gives opposite results

👂 3
lread11:08:07

The malli readme might be a good candidate for https://github.com/lread/test-doc-blocks. I could take a whirl at it if there is any interest.

👍 3
lread20:08:54

@ikitommi I won’t proceed unless you indicate an interest. Might be your cup of tea and might not! Happy to do the PR for you to evaluate then decide if you like - or not.

martinklepsch11:08:11

I’m getting an fn-arity warning when requiring malli.core in a cljs repl:

; eval (current-form): (require 'malli.core)
; (err) ------ WARNING - :fn-arity -----------------------------------------------------
; (err)  Resource: <eval>:2007:43
; (err)  Wrong number of args (3) passed to malli.core/-fail!
; (err) --------------------------------------------------------------------------------

Karol Wójcik16:08:28

Had the same issue. It's fixed in master

🙌 3
martinklepsch17:08:19

thanks Karol!

😄 3
Karol Wójcik20:08:36

You welcome Martin! 🙂

martinklepsch13:08:38

@ikitommi do you think it’s worth documenting this in the changelog for 0.7?

ikitommi14:08:25

PR welcome @U050TNB9F, but there is a a already this line: > * fixed arity error in `m/function-schema`

ikitommi14:08:56

does not pinpoint the problem very well :thinking_face:

Akiz12:08:42

Hi, I am getting Expected: regular expression, received: string. in clj-kondo lsp / Emacs combo with instrumentation for : (def gcs-uri "") (gcs/get-blob-moddate gcs-uri) and these are my schemas:

(m/=> get-blob-moddate [:=>
                         [:cat Gcs-uri]
                         [:or
                          [:fn #(= (type %) org.joda.time.DateTime)]
                          nil?]])
(def Gcs-uri
  [:re #"gs:\/\/.*"])

Karol Wójcik16:08:58

Is it possible to instrument function via metadata in Clojurescript?

ikitommi17:08:44

someone needs to port the tooling to support cljs. I suck at macro+interop, so not going to do that any time soon. help most welcome.

👍 3
Karol Wójcik17:08:24

I will take a look into it when I have some time 🙂

lread17:08:54

Am having a great time experimenting with malli to validate cljdoc’s cljdoc.edn . The docs and demos are great in helping me to understand how things work. Thank you for all that effort. As a newb, I was wondering about the following: 1. The https://malli.io/?value=%5B1%20%5B2%20%5B3%20%5B4%20nil%5D%5D%5D%5D&amp;schema=%5B%3Aschema%0A%20%7B%3Aregistry%20%7B%22ConsCell%22%20%5B%3Amaybe%20%5B%3Atuple%20%3Aint%20%5B%3Aref%20%22ConsCell%22%5D%5D%5D%7D%7D%0A%20%22ConsCell%22%5D uses [:ref "ConsCell"] but the https://malli.io/?value=%5B%3Adiv%20%7B%3Aclass%20%5B%3Afoo%20%3Abar%5D%7D%20%5B%3Ap%20%22Hello%2C%20world%20of%20data%22%5D%5D&amp;schema=%5B%3Aschema%0A%20%7B%3Aregistry%20%7B%22hiccup%22%20%5B%3Aorn%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Anode%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Acatn%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Aname%20keyword%3F%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Aprops%20%5B%3A%3F%20%5B%3Amap-of%20keyword%3F%20any%3F%5D%5D%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Achildren%20%5B%3A*%20%5B%3Aschema%20%5B%3Aref%20%22hiccup%22%5D%5D%5D%5D%5D%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Aprimitive%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Aorn%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Anil%20nil%3F%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Aboolean%20boolean%3F%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Anumber%20number%3F%5D%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%5B%3Atext%20string%3F%5D%5D%5D%5D%7D%7D%0A%20%22hiccup%22%5D uses [:schema [:ref "hiccup"]]. Are [:ref x] and [:schema [:ref x]] equivalent? 2. I was unfamiliar with :maybe and was not confident what it did. From the examples it seems that [:maybe x] validates for x or nil? 3. I was unfamiliar with :tuple and now think I understand it just fine. But maybe a sentence contrasting it with :vector (if that makes sense) would be helpful to newcomers. 4. Very subjective, but I would put the motivation section before examples. As a newbie, this kind of info would be helpful to me up front. If updates to docs and demos for any the above would be helpful I am happy to make those wee changes. Also once I’ve finalized my cljdoc.edn validation I would be happy to add that as a, perhaps familiar, example to docs, demos or both.

👏 6
Ben Sless17:08:48

I think schema of ref cannot be derefed. This lets you write things like recursive sequence schemas

lread18:08:38

Thanks @ben.sless! - Are you suggesting maybe the hiccup demo might be a bit off?

Ben Sless18:08:57

Can't say without giving it a deeper look. I bumped by head against it separately trying to port the datalog BNF to malli

Ben Sless18:08:45

Schema schemas create a "hard" boundary when schemas are compiled and you don't get deref-ed recursively ad inifinitum

Ben Sless18:08:59

your understanding of :maybe is correct

3
Ben Sless18:08:38

with touple, just making sure, it is a heterogeneous fixed length product type

lread18:08:12

is a tuple necessarily a vector?

lread18:08:16

Coolio, so I think I get it and could maybe add a sentence in the README.

Ben Sless18:08:44

I'm working on a getting started guide 🙃

lread18:08:11

Oh cool! Maybe I don’t need to update README then.

Ben Sless18:08:35

It's a good idea to do so anyway, it's a small change you can do now, the guide will take some time

lread18:08:50

Ping me if you want a reviewer. I’m a newb!

Ben Sless18:08:27

I have a bunch of guinea pigs at work 😛

Ben Sless18:08:03

But once it's sorta baked I'll go hunting for external perspectives

Ben Sless18:08:13

so expect a ping in your future!

👍 3
Ben Sless18:08:55

Just, you know, don't deref it or anything

lread18:08:30

When working on your datalog BNF were you able to generate and sample without sporadic StackOverflowError exceptions?

Ben Sless18:08:11

Yes, let me check if it's public I'll send it

Ben Sless18:08:50

Note the abundance of [:schema [:ref ,,,]]

lread18:08:41

Yeah! So maybe that tells us something about my first question(?). You don’t seem to use :ref without it wrapped in :schema.

Ben Sless18:08:31

That's because my schemas are mostly recursive but I probably don't need to do it everywhere

Ben Sless18:08:20

I can try by process of elimination to knock it out and see what's up but it's pedantic and didn't really feel like fighting with it

Ben Sless18:08:54

besides, I consider it a bit of an anti-feature

lread18:08:06

the having to do something special for the recursive case?

Ben Sless18:08:22

Yeah, I'd expect a ref to be enough

👍 3
lread21:08:52

Hiya mallians! I’m making some good progress with my cljdoc.edn malli schema. I am, though, getting some StackOverflowError exceptions when generating samples. This might be normal, I’m new, what do I know? simple_smile Rather than share and diagnose my cljdoc.edn schema, it might be more fruitful to work from the familiar. The Hiccup schema from the malli README seems to have-ish the pattern of recursion I want for cljdoc.edn. If I take the Hiccup schema unchanged from the README I do not seem to suffer StackOverflowError exceptions.

(def Hiccup
  [:schema {:registry {"hiccup" [:orn
                                 [:node [:catn
                                         [:name keyword?]
                                         [:props [:? [:map-of keyword? any?]]]
                                         [:children [:* [:schema [:ref "hiccup"]]]]]]
                                 [:primitive [:orn
                                              [:nil nil?]
                                              [:boolean boolean?]
                                              [:number number?]
                                              [:text string?]]]]}}
   "hiccup"])

(mg/sample Hiccup)
But if I yank out some stuff and make the schema look a little more like my cljdoc.edn schema usage, I can easily trigger StackOverflowError exceptions by re-evaluating (mg/sample Hiccup2) a handful of times:
(def Hiccup2
  [:schema {:registry {"hiccup" [:catn
                                 [:name keyword?]
                                 [:props [:? [:map-of keyword? any?]]]
                                 [:children [:* [:schema [:ref "hiccup"]]]]]}}
   "hiccup"])

(mg/sample Hiccup2)
I am likely quite naive in my understanding here. I’m guessing if I somehow limited the maximum tree depth (which would be reasonable thing to do anyway for my use case) that would help but I am not sure how to express that. I am using malli 0.6.1.

jeroenvandijk11:08:10

@UE21H2HHD Without saying i’m an authority here, it seems that your schema doesn’t have a stop condition? So endless recursion? Whereas the hiccup one has an alternative which stops the recursion. Makes sense?

jeroenvandijk11:08:58

The only stop condition you have is when there are 0 children for one node

lread14:08:53

Ah right… thanks @U0FT7SRLP! That makes perfect sense. For the generator, the :orn is probably a 50/50 coin toss for termination whereas hitting 0 children for termination is much less likely.

👍 3
jeroenvandijk14:08:26

Yeah and then there is the branching of the children 😅 1 child still ok, but 2 or more and there you go

lread14:08:36

So constructing a schema for validation is one thing and then adapting it to to support sensible generation is another consideration.

👍 3
lread14:08:37

Or maybe folks write custom generators?

lread15:08:31

Oh I see :gen/* stuff can influence generation, I’ll poke around more.

jeroenvandijk16:08:13

With clojure.spec https://clojure.org/guides/spec#_custom_generators, so I’m guessing this would make sense for Malli too. (I haven’t done it yet with Malli)