Fork me on GitHub
#malli
<
2023-03-21
>
cap10morgan20:03:00

I keep running into a need to define something like a combination of :map and :map-of where I can assign specific key schema to specific value schema, but several different ones for the same map. The exact key values aren't known ahead of time, but when they conform to a certain schema, then the value should also conform to its schema. Is there any way to do that?

cap10morgan20:03:40

or maybe [:map [::key-schema ::val-schema]] can do that? I may have tricked myself into thinking it needed exact key values on the left. trying it now...

cap10morgan20:03:39

hmm... no. looks like it doesn't work. or I'm doing it wrong.

cap10morgan13:03:38

Yeah that was helpful for some of these use cases, but not all. I don’t know any of the exact map keys ahead of time, just can write schemas for them. And then I need to map several of those to their respective value schemas. So :map is too rigid with keys but :map-of only allows one key schema and one value schema.

cap10morgan13:03:43

I need multiple key schemas each mapping to their own value schema

cap10morgan13:03:30

Something like [:map-of [:or …]]

ikitommi14:03:19

I would use :or like you suggested, either:

[:map-of [:or ...] ...]

[:or [:map-of ... ...] [:map-of ... ...]]

cap10morgan15:03:04

would the former work? the latter seems like it would say "either the entire map must conform to this :map-of or this :map-of" etc. rather than what I want which is each key-value pair getting its own key-schema-value-schema pair.

cap10morgan15:03:43

and [:map-of [:or ::key1-schema ::key2-schema ...] ...] doesn't work b/c I need to map specific value schemas to specific key schemas. rather than "any of these key schemas can apply and then any of these value schemas can apply"

ikitommi15:03:41

so, this is not what you are looking for?

(def schema
  [:map
   [:x :int]
   [:y :int]
   [::m/default [:or
                 [:map-of :string :string]
                 [:map-of :boolean :boolean]]]])

(mg/sample schema)
;({:x 0, :y 0}
; {:x -1, :y 0, "G" "6"}
; {:x -1, :y -1}
; {:x -3, :y -3}
; {:x 1, :y -3}
; {:x 2, :y 5}
; {:x 2, :y 1, true true, false true}
; {:x -62, :y -1, "EB6wR" "oF", "i3m7" "5Qv", "18" "", "GgV3" "te10p", "57dU2r3" "", "J4Db" "X1z35"}
; {:y 1,
;  "4F" "6L",
;  "DZ" "2T0o",
;  "N75OTk9" "U82v",
;  "65jV3HS0" "pEtxQwqb",
;  "tM" "DrWy568x",
;  "a1ZmzBji" "o4TS",
;  "Meh" "3cjN197",
;  :x 1}
; {:x 39, :y -34, true false, false false})

(m/validate schema {:x 1, :y 2, "a" "a"})
; => true

(m/validate schema {:x 1, :y 2, false true})
; => true

(m/validate schema {:x 1, :y 2, false true, "a" "a"})
; => false

cap10morgan15:03:43

hmm... that might work. the one wrinkle is I don't always have a static key -> value schema to put into the :map schema. but it might be OK to just have [:map [::m/default [:or [:map-of ::foo ::bar] [:map-of ::baz ::qux]]]]?

cap10morgan15:03:53

but that's a good new thread to pull on, thanks!

cap10morgan15:03:09

oh wait, no it isn't what I want. the key thing is your last validate call that mixes the two :map-of defaults. I want to be able to get validate => true for that.

cap10morgan15:03:28

I'm trying to say "this map will have keys of schema a that map to values of schema a' and keys of schema b that map to values of schema b' and keys of schema c that map to values of schema c'"

cap10morgan15:03:47

I think I just have to use a :fn schema for that for now

cap10morgan16:03:11

If I were to work on implementing this, would this syntax be reasonable?

[:map-of [::a ::a'] [::b ::b'] [::c ::c']]

ikitommi17:03:39

That starts to be close to :multi, don't want to add more syntax into :`map-of`. There is also opportunities like https://github.com/metosin/malli/blob/master/src/malli/destructure.cljc#L19-L27

cap10morgan17:03:35

OK maybe a whole new schema type then? not sure what to call it. it really feels like a natural extension of :map-of

cap10morgan17:03:51

not sure I follow the destructure stuff?

cap10morgan17:03:21

could also be an extension of :map that takes schemas for keys, but also not sure how to disambiguate that

cap10morgan17:03:12

maybe... like... :map-tuples b/c it kind of consumes the map as key-value pairs?

cap10morgan17:03:20

speaking of... can you already consume a map like that? something like [:or [:tuple ::a ::a'] [:tuple ::b ::b'] [:tuple ::c ::c']]?

ikitommi06:03:07

see the MapLike from the link above, bit similar.

ikitommi06:03:37

pasting here too:

[MapLike
 [:or
  [:tuple [:= :keys] [:vector ident?]]
  [:tuple [:= :strs] [:vector ident?]]
  [:tuple [:= :syms] [:vector ident?]]
  [:tuple [:= :or] [:map-of simple-symbol? any?]]
  [:tuple [:= :as] "Symbol"]
  [:tuple [:fn -qualified-key?] [:vector ident?]]
  [:tuple [:ref "ArgType"] any?]]]

ikitommi06:03:03

Implementing a new type of Schema into malli core means at least the following: 1. design (discussion on the need for it + syntax) 2. implementation (`malli.core` or other ns) 3. error messages (`malli.error`) 4. transforming (`malli.transform`) 5. generator (`malli.generator`) 6. json-schema mapping (`malli.json-schema`) 7. clj-kondo mappings (`malli.clj-kondo`) 8. optionally working with utils (`malli.util`) like merge, union etc. 9. optionally providers (`malli.provider`) 10. other: a. tests b. documentation c. DCE & code size on CLJS d. performance

ikitommi06:03:48

the current MapLike is just for validating + parsing, to make it full Schema would require a lot of work, so haven’t done that. If you would like to implement such, I propose you’ll write an Issue of this, let’s discuss there the best option for the 1. on the list, ok?

cap10morgan13:03:23

Looks like this is pretty much it, if a little implementation-specific: https://github.com/metosin/malli/issues/881

cap10morgan13:03:36

Opened 2 hours ago 😂

cap10morgan13:03:51

Although I wonder if that’s calling for and semantics or or

delaguardo15:03:12

this should be something like a merge of two and more homogenous maps.

👍 2
cap10morgan15:03:43

cool, yeah. that's exactly what I'm looking for as well.

cap10morgan15:03:05

I also think it would be a very natural extension of :map-of (perhaps activated by using vector tuples inside instead of the current syntax of just [:map-of :key-schema :value-schema]). But it sounds like @U055NJ5CC would prefer a new schema type altogether.

Rob Haisfield21:03:33

Anyone have a macro for converting types and interfaces from TypeScript into Malli schemas? I want to use TypeScript libraries through the JavaScript interop, but I also want to benefit from the TS safety.

Ben Sless04:03:00

Convert from from ts to json schema and then it's a short distance

Ben Sless04:03:09

There are packages for the former

Rob Haisfield14:04:30

What is the best path from a TS library to JSON? Specifically I’d love to use the AT Protocol api to do some experiments outside of what Bluesky provides (happy to give invites to anyone helping me get to malls schemas on this, btw) https://www.npmjs.com/package/@atproto/api

Rob Haisfield14:04:07

Okay seems like this library will do it. I have to ask though, if it’s just running TS—>JSON Schema and then using Malli’s decode functionality, why hasn’t it been built yet? Are there any gotchas? https://github.com/YousefED/typescript-json-schema

Ben Sless14:04:43

From personal experience I can share that typsescript to json schema works poorly and doesn't cover all possible forms

Ben Sless14:04:28

had better luck with ts-json-schema-generator but it was still iffy

Ben Sless14:04:58

If anyone has any experience with TS and can just dump the AST as JSON I'd rather process it in Clojure

Rob Haisfield14:04:08

Do you remember in what ways it was iffy?

Ben Sless14:04:40

Because it couldn't emit certain type declarations

Ben Sless14:04:44

so I just got "object"

Rob Haisfield14:04:58

I’d also be interested in the AST JSON approach you described but that sounds less straightforward for a relative beginner to both TS and CLJS like me to make

Ben Sless15:04:29

Once you do that CLJS is irrelevant. In theory we just need to pop the hood of one of the existing emitters and start hacking

Rob Haisfield16:04:42

@U04V15CAJ I’m curious your thoughts on this? Seems like you were asked about this at Dutch Clojure Days

borkdude16:04:47

Can you summarize your question so I don't have to read this whole thread?

Rob Haisfield18:04:22

What do you think about some sort of automatic conversion from TypeScript libraries to Malli schemas? I was thinking that TS->JSON Schema->Malli Schema could work, but seems like there are some gotchas at the TS->JSON Schema step

borkdude18:04:59

No special thoughts on that, I haven't done much TS so far

ikitommi06:03:03

Implementing a new type of Schema into malli core means at least the following: 1. design (discussion on the need for it + syntax) 2. implementation (`malli.core` or other ns) 3. error messages (`malli.error`) 4. transforming (`malli.transform`) 5. generator (`malli.generator`) 6. json-schema mapping (`malli.json-schema`) 7. clj-kondo mappings (`malli.clj-kondo`) 8. optionally working with utils (`malli.util`) like merge, union etc. 9. optionally providers (`malli.provider`) 10. other: a. tests b. documentation c. DCE & code size on CLJS d. performance