malli

2025-03-20T16:01:49.484659Z

This malli PR improves the caching of validators, explainers, and parsers in some subtle (but I think common) cases. If anyone can try this on a large system I'd be curious if it has impact on startup time and memory use, or any other feedback. https://github.com/metosin/malli/pull/1180

Colin P. Hill 2025-03-20T20:26:49.194049Z

It appears some optional schemas can't be used in local registries, for reasons I don't quite understand. (Something to do with them being opaque objects and not pure data?)

(m/validate [:schema {:registry (mu/schemas)}
             [:union
              :string :int]]
            "foo")
;=> #error {:data {:type :malli.core/child-error, :message :malli.core/child-error, :data {:type :select-keys, :properties nil, :children nil, :min 2, :max 2}}
I've been resisting the temptation to change the default registry, but I'm trying to use some of these optional schemas with m/=>, which doesn't take an options argument. Is there any workaround here, or do I have to bite the bullet and make a global mutation to the default registry?

2025-03-20T21:15:51.809339Z

You can pass the constructors directly to malli instead of referring to them by keyword.

2025-03-20T21:16:37.824349Z

m/schema [(m/-any-schema)])
:any

Colin P. Hill 2025-03-20T21:17:50.981979Z

Hmm, good idea. I'll have to mull over how much I value the clean look of using the keywords, but I think I'll probably go with that.

2025-03-20T21:20:45.936309Z

It's certainly in the category of last-ditch workaround. You might just be kicking the can further down the road to a different problem.

2025-03-20T21:23:26.185309Z

related, I'm working on an extensible default registry here to try and establish something more idiomatic for this common issue https://github.com/frenchy64/malli/pull/28/files

(m/reg-ctor ::any-alias (m/-any-schema))

(m/doc ::any-alias)
;TODO
; Schema constructor
; {:file "/Users/ambrose/Projects/malli-local-dev/register-macro/test/malli/doc_test.cljc", :ns malli.doc-test, :line 15, :column 1, :form (m/reg-ctor :malli.doc-test/any-alias (m/-any-schema)), :schema-form (m/-any-schema), :into-schema true}

Colin P. Hill 2025-03-20T21:24:56.199989Z

Ah, I like that! No reason not to just borrow spec's mutable-but-in-a-principled-way approach.

2025-03-20T21:25:46.631789Z

thanks! here's my new s/def for regular schemas 🙂

(m/reg ::another-schema
       "A documentation string"
       {:a static-meta-map}
       :boolean)

(m/doc ::another-schema)
; -------------------------
; Named Schema
; 
; A documentation string
; 
; :bool
; 
; Source code:
; (m/reg ::another-schema
;        "A documentation string"
;        {:a static-meta-map}
;        :bool)

Colin P. Hill 2025-03-20T21:26:36.631139Z

Haha, also learning from the "would really love to have metadata on these" complaint about spec, I see.

2025-03-20T21:27:25.234749Z

yes, and answering the question of "where do we put this stuff" as "only during macro-time" with a malli spin of "you can then export it all to a file if you need it later"

👍 1
Colin P. Hill 2025-03-20T21:29:06.856829Z

The use of namespaced keys in a global registry is one of my favorite things about spec. Would love to see this land.

2025-03-20T21:29:53.954139Z

yes the only downside to me is that they print terribly.

💯 1
Colin P. Hill 2025-03-20T21:30:13.311549Z

And the more robust your namespacing, the uglier they are

2025-03-20T21:31:10.185529Z

yes. for example, Typed Clojure types print differently based on the current namespace. partially because it uses symbols for names, but also because the parser is namespace-sensitive.

2025-03-20T21:31:55.229679Z

I don't think this is something that would work for malli, the latter point being the real problem. spec has the short form printing, but you can't do anything with the result except look at it.

Colin P. Hill 2025-03-20T21:34:01.366029Z

It kinda feels like printing and serializing ought to be distinguished. But that would interfere with a lot of stuff in the REPL.

2025-03-20T21:35:22.359179Z

hmm actually here's an idea. we attach the namespace that the schema is relative to in the output of m/form

2025-03-20T21:35:49.825459Z

(m/form [::foo]) => ^{:relative-to *ns*} [:foo}

🤔 1
2025-03-20T21:35:56.811259Z

then you can roundtrip

Colin P. Hill 2025-03-20T21:36:35.046639Z

Could work. Though you'd need to opt into printing metadata.

Colin P. Hill 2025-03-20T21:36:48.167719Z

Why not in the ordinary properties map?

2025-03-20T21:37:16.144359Z

The meta is just for the impl, let me walk through a use case

2025-03-20T21:37:50.609809Z

I don't think we can use m/form, maybe a new op m/pretty-form

2025-03-20T21:38:20.285489Z

but the idea is we override the print-method of the schemas to m/pretty-form relative to the current ns

2025-03-20T21:38:52.551159Z

(prn (m/schema ::foo))
[:_/foo]

2025-03-20T21:39:12.074539Z

then you can always capture the output if you really want to roundtrip

2025-03-20T21:39:19.297279Z

but most of the time this is just for humans

2025-03-20T21:40:45.056529Z

oh, well you can't capture the stdout. but what I mean is that you should be able to roundtrip with (-> ::foo m/schema m/pretty-form m/schema)

Colin P. Hill 2025-03-20T21:40:55.506499Z

Yeah, I got what you mean

2025-03-20T21:41:02.902379Z

so things like explain errors are pretty

Colin P. Hill 2025-03-20T21:41:07.387459Z

You can round trip in memory, but not through serialization

2025-03-20T21:41:13.823389Z

yeah

2025-03-20T21:41:26.977459Z

if you keep the same ns, you can roundtrip

Colin P. Hill 2025-03-20T21:41:28.525719Z

Potentially clj serialization with print-meta, but never through edn

2025-03-20T21:41:32.760519Z

I think that's a normal clojure assumption

👍 1
2025-03-20T21:42:09.413769Z

then you can go a step further and use ns-aliases to pretty print everything else.

2025-03-20T21:42:28.867889Z

which you then consult to parse it again

Colin P. Hill 2025-03-20T21:43:47.658549Z

If the ns aliases were captured explicitly in the top-level properties map, you could have a compromise which can also round trip through edn. Ugly, but all the ugly is in one place and kept away from what matters.

Colin P. Hill 2025-03-20T21:44:17.662559Z

This may also be useful for authoring directly in EDN.

2025-03-20T21:44:30.693109Z

yes that very much has an xml or json schema flavor

Colin P. Hill 2025-03-20T21:44:53.346299Z

True. A mark in its favor, imo! XML is annoying but its namespacing is excellent.

👍 1
💯 1
2025-03-20T21:44:54.654779Z

as in, we're following tech that take namespacing very seriously

2025-03-20T21:45:55.866589Z

I like all these ideas.

Colin P. Hill 2025-03-20T21:46:53.279449Z

It would also mean Malli solves declarative aliasing before EDN itself does 😂

2025-03-20T21:47:01.437409Z

omg

2025-03-20T21:47:58.825699Z

m/pretty-form and m/compact-form?

2025-03-20T21:48:44.100289Z

related, it's always bothered me that malli's core schemas aren't namespaced.

Colin P. Hill 2025-03-20T21:50:40.158369Z

I generally feel that the authors of any given library are entitled to use un-namespaced keywords, and it's incumbent on the lib users to avoid defining their own. A little like how Clojure (and EDN) reserves un-namespaced reader tags for its own future extensions.

Colin P. Hill 2025-03-20T21:51:25.399379Z

In fact I think all of Malli's examples of schema-local registries use namespaced keywords.

2025-03-20T21:51:50.032729Z

but would xml have stood for that?

2025-03-20T21:51:58.215189Z

clojure.core is not in the global namespace

2025-03-20T21:52:18.168159Z

I get what you're saying though, just being fussy

Colin P. Hill 2025-03-20T21:52:22.417619Z

XML didn't define a core vocabulary. It's a little different.

Colin P. Hill 2025-03-20T21:53:28.270659Z

But I do in general feel like namespaced keywords are underused. They're a killer feature and make a lot of otherwise unthinkable operations trivially safe, if you're disciplined about using them.

2025-03-20T21:54:19.490639Z

yes. it's the "discipline required" part that bothers me. spec forces specs to be namespaced because they're macros and hence tied to a namespace.

Colin P. Hill 2025-03-20T21:54:49.617679Z

I always felt like it was a little bit of a nudge from Rich to encourage people to use nsed keywords more, too.

Colin P. Hill 2025-03-20T21:55:48.617529Z

He talks about maps flowing through the system, containing keys from all over the place. That's only possible if you're heavily using namespaces.

Colin P. Hill 2025-03-20T21:56:45.118829Z

I wonder sometimes if a small ergonomic tweak might have led to different habits: swap the meaning of : and ::

💯 1
2025-03-20T21:57:13.865719Z

yes. what I love about vars is that it's impossible to not namespace them. you don't need to know about the benefits of namespacing to use them properly. they come with huge baggage of namespace loading which nsed keywords fixes, but the extra discipline needed is a conundrum.

💯 1
Colin P. Hill 2025-03-20T21:58:45.590099Z

This was in fact the feature that got me to dig deep into Clojure in the first place. "Just namespace every identifier" was huge after working in OO systems where method name collision could be a big headache.

Colin P. Hill 2025-03-20T21:59:19.189389Z

And I was downright confused when I explored the ecosystem and saw such little use of that superpower.

2025-03-20T21:59:49.492229Z

vars?

Colin P. Hill 2025-03-20T22:01:33.952279Z

Namespacing in general, symbols and keywords both. The way protocols improve on Java interfaces by adding namespacing was a big revelation for me, but the way this strength generalized to data in maps was pretty immediately clear to me.

👍 1
2025-03-20T22:03:13.212179Z

I didn't decomplect vars into identifier+value until way after spec existed so it wasn't obvious to me. but it's clearly sunk it.

2025-03-20T22:03:59.858369Z

which I guess makes my point about discipline very personal 🙂

Colin P. Hill 2025-03-20T22:04:11.338369Z

Vars proper took me a while to grok. But the basic idea that the "method" had a qualified name is what jumped at out me.

2025-03-20T22:05:17.151509Z

ah yes. I took a long time (rightly so in hindsight) to fully understand the implications of the interface method being unqualified.

Colin P. Hill 2025-03-20T22:05:54.658139Z

I had the benefit(?) of just recently, at the time I picked up Brave and True, finding an idea of mine impossible because of a method name collision.

Colin P. Hill 2025-03-20T22:06:40.556429Z

in fact it was a method name collision – the method was called name

😁 1
2025-03-20T22:07:17.234519Z

yes it makes protocols need to be slightly leaky abstractions to fully utilize them

2025-03-20T22:07:30.282899Z

to know you needed to use extend

Colin P. Hill 2025-03-20T22:08:30.495859Z

This was a problem I encountered in a pure Java system (where I wasn't going to get away with introducing Clojure), to be clear – so I was seeing how much better things could be, if only I were using Clojure

👍 1
Colin P. Hill 2025-03-20T22:09:55.459339Z

Anyway, it's been genuinely delightful chatting, but I meant to log off 40 minutes ago and really need to get going now 😂 Really looking forward to your PR landing!

❤️ 1
2025-03-20T22:10:10.944499Z

see ya!

2025-03-20T23:59:22.596409Z

First step towards reversibly compacting qualified schema forms https://github.com/frenchy64/malli/pull/30

(is (= [:tuple
            {:aliases {:malli.core-test :_,
                       :malli.generator :mg}
             :registry {:_/a [:ref :_/b],
                        :_/b :mg/gen,
                        :mg/gen :_/a}}
            :_/a :_/b]
           (m/compact-form [:tuple
                            {:registry {::a [:ref ::b]
                                        ::b ::mg/gen
                                        ::mg/gen ::a}}
                            ::a ::b])))

❤️ 1
🎉 1
2025-03-21T04:38:40.847749Z

After exploring this a bit, it's much better to just compact the edn itself. I didn't find any benefits to supporting the syntax directly in the schemas.