malli 2020-08-10 | Slack Archive

How can I express a schema of "map of string keys to integer values?" in malli?

@linshuai2012 [:map-of string? int?]

Thanks @ikitommi, just found it's there in README, I read through the README and missed it 😄

btw thanks for creating malli, it's much data friendly and intuitive than spec ! I'm most looking forward to see the fdef equivalent https://github.com/metosin/malli/issues/125

ikitommi12:08:42

me too 😉

borkdude12:08:36

I'm also looking forward to a s/conformer equivalent

zclj13:08:32

@ikitommi I have a schema where some maps have a lot of optional keys, up to 50-ish. When I generate from this schema I got a, to me, surprising result. What happens is that even for very small sizes I get very large generated maps (and since they are recursive this is magnified). Looking into how malli generates maps with optional keys, it seems that each key will have a 50% change of being included, so there is no notion of 'size' for the complete map. This also effect shrinking. So in practice my first generated map can be empty, while the second sample contains 20 keys. Intuitively I would expect the generation of optional keys to start small and then grow larger with size. Is this something you think malli should take care of or is it out of scope for malli and something I need to take care of with custom generators?

ikitommi13:08:05

@zclj if you have a simple improved algorithm for generating with optional keys, please PR, the code is kinda naive atm: https://github.com/metosin/malli/blob/master/src/malli/generator.cljc#L84-L87

zclj08:08:57

@ikitommi I have tried out some different solutions to this and though about it. My conclusion so far is that there are really no perfect default solution. As is usally the case in software testing (where I use the generators) the proper distribution of values are dependent on both the system under test and the outcome you want (positive tests, negative tests etc.) With this in mind, would you be open to a solution where malli could allow for the options to contain a :malli.generators/map-gen-optional-fn, where the user can provide their own implementaiton of the value selection and keep the default as is? In general, if malli provided such options it would be an awesome way to experiment with different distributions and allow for libraries to provide commonly used variants.

ikitommi08:08:03

with the current api, you can override the whole :map generator with:

(defmethod malli.generator/-schema-generator :map [schema options] (-my-map-gen schema options))

ikitommi08:08:13

would that be enough?

ikitommi08:08:29

(not happy that one can globally override the generators, I concider this as mutable evil, but is like that today)

zclj08:08:16

I experimented some with that option but did not pursue it further due to me still wanting the -recur funtionallity and then having to do that in my own code. I could of course re-use mallis -recur but it felt that I started to depend to much on malli internals for this to be a clean method of extension, but maybe I was to defensive in that decision?

ikitommi08:08:36

#Also sent to the channel

need to publish a malli internal naming convention guide. my idea was that all public - starting functions are public for library extensions / advanced use, tracked separately in CHANGELOG from the public end user api. Using those for basic use cases is a sign of invalid usage. Those will break more than the user-facing public api, but so that it’s easy for advanced users adapt to changes, e.g. no silent breaking changes after initial public release.

ikitommi08:08:35

but, on second thought, if that one new option is good / you can provide alternative example impl (into README/docs) that might be of value to someone else, open to PR for adding that.

ikitommi08:08:26

as options are passed in to -map-gen already, it’s most likely <4 lines of code + tests + docs.

ikitommi08:08:51

(and doesn’t add much to cljs bundle size)

eskos13:08:38

Is there a reason the private stuff simply isn’t defn- ? Are the private parts split across multiple namespaces in such a way it would make this complicated?

ikitommi13:08:28

real privates are ^:private. Vars like malli.core/-map-schema are public but only needed if you build your own registry, which is for advanced users only. Another is malli.core/-parse-entries which is a helper for building your own Schema instance, which wants to use the map-syntax

ikitommi13:08:59

spec is mostly closed for extensions, malli is built to be extended.

zclj13:08:53

@ikitommi I will take a look and see if I can figure something out 🙂

Shuai Lin14:08:46

another question, can I add some meta information to each field? Like this:

(def Person
  [:map
   [:name string? {:doc "The name of the person"}]
   [:age string?]])

ikitommi14:08:58

@linshuai2012 map entries can have optional property map:

(def Person
  [:map
   [:name {:doc "The name of the person"} string?]
   [:age string?]])

ikitommi14:08:55

just polishing the generic traversal of child-parents:

(defn parent-properties [schema path]
  (loop [i (count path), acc []]
    (if (>= i 0)
      (let [p (subvec path 0 i)
            s (mu/get-in schema p)
            ++ #(conj % (m/properties s))]
        (if (and (m/entries s) (< i (count path)))
          (recur (dec i) (++ (conj acc (m/properties (mu/get-in schema (conj p [::m/entry (path i)]))))))
          (recur (dec i) (++ acc))))
      acc)))

(parent-properties
  [:map {:error/message "1"}
   [:y {:error/message "2"}
    [:and {:error/message "3"}
     [:map {:error/message "4"}
      [:x {:error/message "5"}
       [:and {:error/message "6"}
        int? [:> {:error/message "7"} 18]]]]]]]
  [:y 0 :x 1])
;[#:error{:message "7"}
; #:error{:message "6"}
; #:error{:message "5"}
; #:error{:message "4"}
; #:error{:message "3"}
; #:error{:message "2"}
; #:error{:message "1"}]

ikitommi16:08:58

@borkdude collected some thoughts on the s/conform here: https://github.com/metosin/malli/issues/241. Comments and ideas welcome.

borkdude17:08:59

Cool! I was asking about s/conforming specifically which was a spelling error, it should be https://clojuredocs.org/clojure.spec.alpha/conformer

borkdude17:08:32

This was the "transform while conforming" issue

ikitommi08:08:36

replied to a thread:@zclj if you have a simple improved algorithm for generating with optional keys, please PR, the code is kinda naive atm: https://github.com/metosin/malli/blob/master/src/malli/generator.cljc#L84-L87

2020-08-10

Channels