Fork me on GitHub
#malli
<
2023-03-10
>
ikitommi07:03:14

Figuring out best way to https://github.com/metosin/malli/issues/264. While doing, about to break “content-dependent schema creation” using a new :compile prop instead of a top-level callback function. Look like this:

(def Between
  (m/-simple-schema
   {:type :user/between
    :compile (fn [_properties [min max] _options]
               (when-not (and (int? min) (int? max))
                 (m/-fail! ::invalid-children {:min min, :max max}))
               {:pred #(and (int? %) (>= min % max))
                :min 2 ;; at least 1 child
                :max 2 ;; at most 1 child
                :type-properties {:error/fn (fn [error _] (str "should be betweeb " min " and " max ", was " (:value error)))
                                  :decode/string mt/-string->long
                                  :json-schema {:type "integer"
                                                :format "int64"
                                                :minimum min
                                                :maximum max}
                                  :gen/gen (gen/large-integer* {:min (inc min), :max max})}})}))

(m/form [Between 10 20])
; => [:user/between 10 20]

(-> [Between 10 20]
    (m/explain 8)
    (me/humanize))
; => ["should be betweeb 10 and 20, was 8"]

(mg/sample [Between -10 10])
; => (-1 0 -2 -4 -4 0 -2 7 1 0)

Casey08:03:27

I have an annoying input to a value in my system. It is a vector of items. But if there is only a single item, I receive the one item, not wrapped in a vector. Is there a built in way with malli to handle this? I want the single item to be plopped in a vector. i.e.,

[:map [:items [:vector :string]]]

;; the input I receive is
{:items  ["a" "b" ...]}

;; or, sometimes, when there is only one item
{:items "a"}

;; my desired output from m/decode is that :items should always be a vector
{:items ["a"]}

Bingen Galartza Iparragirre08:03:15

You can use :tuple instead of :vector The docs say: A :tuple describes a fixed length Clojure vector of heterogeneous elements

Casey08:03:15

Yes, but :items isn't fixed length it can have length >=1

Bingen Galartza Iparragirre08:03:07

Ups, I didn't read the example properly 😅

Casey08:03:15

Some sort of transformer might work, but I don't want to apply this to every instance of string/:vector in a schema

ikitommi08:03:07

I would:

(def vectorify-transformer
  (let [coders {:vector #(cond-> % (not (vector? %)) (vector))}]
    (mt/transformer
     {:decoders coders
      :encoders coders})))

(m/decode [:map [:items [:vector :string]]] {:items ["a"]} vectorify-transformer)
; => {:items ["a"]}

(m/decode [:map [:items [:vector :string]]] {:items "a"} vectorify-transformer)
; => {:items ["a"]}

ikitommi08:03:05

there is mt/collection-transformer which ensures the correct type, but doesn’t create collections of non-collection items. There could be a similar built-in for this too.

Casey08:03:10

Is there a way to apply a transformer to just a specific key in the schema?

ikitommi08:03:59

sure, many ways, simplest being:

[:vector {:encode/string vectorize} ...]

Casey08:03:20

Ah.. in the past i've always defined my own simple schema

Casey08:03:35

m/-simple-schema that is

ikitommi08:03:22

… or you could tag the keys and read those in the transformer:

[:map [:items [:vector {:vectorize true} :string]]]
or
[:map [:items {:vectorize true} [:vector :string]]]

ikitommi08:03:01

non need for simple-schema here, if instance properties are good enough. or a new property for more declarative transformation

Casey08:03:03

aha, that's a nice solution.. it keeps the schemas as data

☝️ 2
Casey08:03:41

Can the transformer get access to the original schema?

ikitommi08:03:02

transformers can read the schemas (and properties) at transfromer creation time, so it’s also super efficient, e.g. creating a decoder from this would mount the transforming function:

[:map [:items [:vector {:vectorize true} :string]]]
, but for this:
[:map [:items [:vector :string]]]
… the transformer known there is nothing to do and doesn’t emit any code

ikitommi08:03:51

see the mt/default-value-transformer which reads the :default key from schema. there is a :compile option to access the schema.

ikitommi08:03:33

returning nil from :compile as “nothing to do here”

Casey08:03:13

:compile should return nil in the nop case, or the encoder/decoder fn itself?

ikitommi08:03:19

from :compile.

Casey08:03:37

brilliant!

(def vectorize-transformer
    (let [coders {:vector {:compile
                           (fn [schema _]
                             (when (some-> schema m/properties (find :vectorize))
                               #(cond-> % (not (vector? %)) (vector))))}}]
      (mt/transformer
       {:decoders coders
        :encoders coders})))

Casey09:03:42

many thanks @U055NJ5CC malli is great 🙂 sometimes i find it difficult to find the extension points like that. diving into the core code is a great tip

Casey10:03:21

@U055NJ5CC given schema [:vector {:vectorize true} :string] , (m/properties schema) returns {:vectorize true} . What is the equivelent malli.core function to get the :string part? of which there may be more than one in the case of :tuple or :enum

Casey10:03:00

Usually I use (m/form schema) to get the plain data and then things like (second form) or (nth schema 2)

ikitommi11:03:48

There is m/type, m/properties and m/children for this

niwinz10:03:21

hello o/ I think I found an inconsistent behavior between clj and cljs, in cljs, the uuid is properly validated using regex, but on CLJ the validation is relied to UUID/fromString, it turns out that fromString accept incomplete UUID's as valid uuids.

niwinz10:03:27

I think a better approach would be to perform regex validation always, what do you think?

ikitommi11:03:49

Thanks for reporting, PR to fix this (I think regex is good) would be most welcome.

niwinz12:03:13

ok, I will do it

👍 2
stathissideris13:03:01

if I have

(def schema1
    {::id      int?
     ::country string?})

  (def schema3
    {::secondary-id ::DOES-NOT-EXIST
     ::age          int?})

  (def merged (merge
               (malli/default-schemas)
               schema1
               schema3))
Is there any way to get an error because of trying to refer to the ::DOES-NOT-EXIST recipe? I remember figuring this out before but I didn’t make a note at the time 😕

ikitommi13:03:36

Currently, the registries do not check the references eagerly to see if everything is linked correctly. m/schema does check those eagerly:

(def schema1
  {::id      int?
   ::country string?})

(def schema3
  {::secondary-id ::DOES-NOT-EXIST
   ::age          int?})

(mr/set-default-registry!
 (merge
  (m/default-schemas)
  schema1
  schema3))

(m/schema [:map ::id ::age])
; => [:map :user/id :user/age]

(m/schema [:map ::id ::secondary-id])
; =throws=> {:type :malli.core/invalid-schema, :message :malli.core/invalid-schema, :data {:schema :user/DOES-NOT-EXIST}}

ikitommi13:03:56

there could be a helper to verify the correctness of a registry, but is not atm.

stathissideris13:03:10

thanks, I did try this, but I think I’m using incorrectly because it doesn’t work with valid references:

(def schema1
    {::id      int?
     ::country string?})

  (def schema2
    {::secondary-id ::id
     ::age          int?})

  (malli/schema
   (merge
    (malli/default-schemas)
    schema1
    schema2))
I get an exception

ikitommi13:03:12

malli/schema expects one schema, you have a full registry there.

stathissideris13:03:12

oh, so I’d have to check every schema one by one, right?

ikitommi13:03:40

(m/schema
 ;; schema syntax
 [:map ::id ::secondary-id]
 ;; options
 {:registry (merge
             (m/default-schemas)
             schema1
             schema2)})
; => [:map :user/id :user/secondary-id]

ikitommi13:03:00

that’s correct syntax for m/schema

ikitommi13:03:57

.. but checks just what is in the schema syntax. to verify all schemas in registry, you have to pull the schemas (via malli.registry/schemas) and check them 1 by 1, I guess.

stathissideris13:03:10

my use case is to merge registries and check whether they have any broken links, let me see if I can come up with a solution

stathissideris13:03:40

kinda works (will make less imperative):

(defn check-registry [registry]
  (let [schemas (-> registry
                    mr/registry
                    mr/schemas)]
    (doseq [[k _] schemas]
      (when (or (string? k) (and (qualified-keyword? k)
                                 (-> k namespace (str/starts-with? "malli") not)))
        (malli/schema [:map k] {:registry registry})))))

ikitommi14:03:57

you could check just your own custom registry, no need to crawl the default schemas I guess. btw, there is m/-reference? to check if the key is a valid reference type (string or qualified kw).

stathissideris14:03:56

both great tips, thank you!

stathissideris14:03:24

I filtered out the malli schemas because the check fails when I pass it one of them

ikitommi14:03:31

yes, there is a bunch of schemas that can’t be created without children, e.g. (m/schema :schema) fails, it requires 1 child, (m/schema [:schema :int]). Same with :map-of, :enum , :or, :vector etc.

ikitommi14:03:49

there is ::m/schema and ::m/val as internal schema types, marking eager references and entry values, both require a child.

stathissideris14:03:53

hmm simply merging my own schemas into a registry (and not the default malli schemas) and checking the result with my function doesn’t work, presumably because they use int? and string? which only exist in the default schemas

ikitommi15:03:33

Yes, you have the full registry when validating (via options or global registry), but just need to loop the own ones

👍 2
ikitommi13:03:56

Idea: a mr/requiring-resolve-registry that can be used to create var-schemas without needing to register those:

;; a simple custom Var-schema
(def Over
  (m/-simple-schema
   {:type `Over ;; here: use qualified symbol so we reconstruct the schema from form!
    :compile (fn [_ [value] _]
               {:pred #(and (int? %) (> % value))
                :min 1
                :max 1})}))

;; normal usage
(m/validate [Over 6] 7)
; => true

;; defauls schema + auto-resolve qualified symbols
(mr/set-default-registry!
 (mr/composite-registry
  (m/default-schemas)
  (mr/requiring-resolve-registry)))

;; as symbol
(m/validate [`Over 6] 7)
; => true

;; as symbol
(m/validate ['user/Over 6] 7)
; => true

;; durable
(-> [Over 6]
    (edn/write-string)
    (edn/read-string)
    (m/validate 7))
;=> true
… schema-like simplicity with durability 💪

ikitommi13:03:02

implementation for the new registry would be ~7 loc.

jasonjckn06:03:46

that’s pretty cool , its not for everyone but very legit and simple approach i like it