Fork me on GitHub
#malli
<
2021-03-21
>
Yevgeni Tsodikov11:03:10

Hey, This might be a silly question. Is it possible to validate a map while ignoring optional fields if they are invalid? for example it would return false here:

(m/validate [:map 
             [:a string?]
             [:b {:optional true} int?]]
            {:a "Hey" :b "Nope"})
=> false
But there would be a way to conform a map with invalid optional fields and “sanitize” the map
; This made-up function would return {:a "Hey"}
(m/conform [:map 
            [:a string?] 
            [:b {:optional true} int?]]
           {:a "Hey" :b "Nope"})

ikitommi11:03:27

Do you mean the first should return true?

Hankstenberg12:03:10

Hm, shouldn't a validation with an open map that just containts the schema for :a do the trick? Or let b be any? Under what circumstances should b be an int?

ikitommi12:03:21

> Is it possible to validate a map while ignoring optional fields if they are invalid? for that, yes, you can transform the schema before validation, just convert optional values to :any. but if you would like to strip away invalid values, you could: 1. run m/explain on data 2. recursively remove all values in :errors :in path. the invalid paths are part of the explain result:

(m/explain
  [:map
   [:a string?]
   [:b {:optional true} int?]]
  {:a "Hey" :b "Nope"})
;{:schema [:map [:a string?] [:b {:optional true} int?]],
; :value {:a "Hey", :b "Nope"},
; :errors (#Error{:path [:b], :in [:b], :schema int?, :value "Nope"})}

Hankstenberg13:03:46

@U055NJ5CC That's something I'd like to be able to do too. Is there an idiomatic way to do that?

Yevgeni Tsodikov13:03:39

> Do you mean the first should return `true`? Yes 🙂 > recursively remove all values in `:errors` `:in` path. the invalid paths are part of the explain result: The :optional data is missing from the :schema field, how can I know which field is safe to remove and which field makes the map actually invalid?

Yevgeni Tsodikov13:03:55

My scenario is an API with some optional fields. If the client doesn’t send then -> no worries. If the client sent some invalid optional fields, in some cases I’d like to pass the request and not fail it. (Such cases may be that the client is a mobile device with a crappy sdk, which might report invalid data for optional fields)

Yevgeni Tsodikov13:03:49

Also - > Do you mean the first should return `true`? Not without some hints or options.

; This is correct
(m/validate [:map 
             [:a string?]
             [:b {:optional true} int?]]
            {:a "Hey" :b "Nope"})
=> false

; Additional options
(m/validate [:map 
             [:a string?]
             [:b {:optional true} int?]]
            {:a "Hey" :b "Nope"}
            {:fail-on-optional? false)
=> true

ikitommi12:03:21

> Is it possible to validate a map while ignoring optional fields if they are invalid? for that, yes, you can transform the schema before validation, just convert optional values to :any. but if you would like to strip away invalid values, you could: 1. run m/explain on data 2. recursively remove all values in :errors :in path. the invalid paths are part of the explain result:

(m/explain
  [:map
   [:a string?]
   [:b {:optional true} int?]]
  {:a "Hey" :b "Nope"})
;{:schema [:map [:a string?] [:b {:optional true} int?]],
; :value {:a "Hey", :b "Nope"},
; :errors (#Error{:path [:b], :in [:b], :schema int?, :value "Nope"})}

Yevgeni Tsodikov14:03:35

That’s great, thanks @U055NJ5CC!

Yevgeni Tsodikov14:03:11

Is there a way to conform/remove the invalid optional fields?

ikitommi14:03:25

yes. Many ways to do that : 1) attach a custom transformer to the new :any fields to strip those away 2) in case of error, call m/explain and remove all values from paths that have error

ikitommi14:03:56

don't have time to write an example, but definitely doable 😉

Yevgeni Tsodikov16:03:02

What’s your opinion on something like:

(defn allow-invalid-optional-values [schema]
  (m/walk
    schema
    (m/schema-walker
      (fn [s]
        (cond-> s
                (m/entries s)
                (mu/transform-entries
                  (partial map (fn [[k {:keys [optional] :as p} s]]
                                 (if optional
                                   [k
                                    (assoc p :original-spec s
                                             :decode/remove-invalid-fields {:compile (fn [schema _]
                                                                                       (fn [x]
                                                                                         (if-let [original-spec (:original-spec (m/properties schema))]
                                                                                           (when (m/validate original-spec x)
                                                                                             x)
                                                                                           x)))})
                                    :any]
                                   [k p s])))
                  (m/options s)))))))

(-> [:map
     [:a string?]
     [:b {:optional true} int?]]
    allow-invalid-optional-values
    (m/decode
      {:a "Hey" :b "Nope"}
      (mt/transformer {:name :remove-invalid-fields})))
=> {:a "Hey", :b nil}
I don’t like having multiple transformers, though. How can I unify them? Similarly to what malli offers with mt/strip-extra-keys-transformer ?

Yevgeni Tsodikov15:03:07

The allow-invalid-optional-values function is missing the (m/options s) arg of mu/transform-entries

respatialized16:03:17

hi! I've been really impressed with malli so far. really appreciate all the work you've put into it. I'm trying to generate sample values from a schema, and I'm encountering a case where :orn fails and :altn succeeds:

(mg/generate [:altn [:bool boolean?] [:num int?]] {:seed 20})
=> [true]
(mg/generate [:orn [:bool boolean?] [:num int?]] {:seed 20})
=> Execution error (ExceptionInfo) at malli.impl.util/-fail! (util.cljc:16).
:malli.generator/no-generator {:schema [:orn [:bool boolean?] [:num int?]], :options {:seed 20}}
Is this expected at this stage? Is there additional implementation for :orn generators that still needs to be done, or is this a bug?

respatialized16:03:32

simple :or also succeeds on this case:

(mg/generate [:or boolean?  int?] {:seed 20})
=> true

ikitommi16:03:33

merged in master

🎉 6
respatialized17:03:57

wow, that was fast! thanks!

borkdude17:03:13

it's weekend, the time where magic OSS happens

🙌 3
ikitommi18:03:51

pushed out [metosin/malli "0.3.1"], finally with a working cljdoc - https://cljdoc.org/d/metosin/malli/0.3.1/doc/changelog.

😍 24