Fork me on GitHub
#malli
<
2024-02-04
>
Joel02:02:31

(defn x
  ([a] 7)
  ([a b] 42))

(def v [:cat :string [:+ [:alt keyword? string?]]]) ; or :vector?
(m/=> x
      [:function
       [:=> [:cat v] :int]
       [:=> [:cat :int v] :int]])

(x ["string" :keyword])
(x 1 ["string" :keyword])
I suspect the error has something to do with not being able to determine the arity, but I can’t figure out how to fix this.

Joel02:02:41

[{:type clojure.lang.ExceptionInfo
   :message ":malli.core/multiple-varargs"
   :data {:type :malli.core/multiple-varargs, :message :malli.core/multiple-varargs, :data {:infos [{:min 2, :arity :varargs, :input [:cat [:cat :string [:+ [:alt keyword? string?]]]], :output :int} {:min 3, :arity :varargs, :input [:cat :int [:cat :string [:+ [:alt keyword? string?]]]], :output :int}]}}
   :at [malli.core$_exception invokeStatic "core.cljc" 136]}]

ikitommi07:02:58

try [:vector {:min 1} [:or :string :keyword]] as second arg instead. Reson: sequence schema flattening.

Joel02:02:05

That worked, but then I can’t enforce a :string as the first item in the vector: [:vector {:min 1} :string [:or :string :keyword]]

Joel02:02:42

It sounds like you are saying that a :cat within another :cat is problematic?

Joel21:02:13

@U055NJ5CC Is this a temporary limitation?

ikitommi08:02:50

Want to use Classes as Schemas? Easy to add optional ClassRegistry into malli, like there is DynamicRegistry, definetely NOT as a default thing. https://github.com/metosin/malli/issues/1007.

❤️ 2
ikitommi08:02:02

malli, the library for making schema libraries - just add your own opinions awesome🙈

ikitommi08:02:21

would it be technically easy (or hard) to add support for optional type syntax for Clojure language itself? e.g. having :- or : as a marker for “next element is a type, to be omitted from code, but available for type/schema tooling? JS land has a proposal for this (https://tc39.es/proposal-type-annotations/). See https://github.com/metosin/malli/issues/1003#issuecomment-1925637260

ikitommi08:02:02

same with imaginary ide/lsp support (fade out the type-docs part so it’s more pleasant to use)

❤️ 1
Ben Sless10:02:23

How about something similar to the new method type hint syntax?

Karol Wójcik13:02:40

Optional type system! Yes please 🙏

Noah Bogart13:02:27

Technically? Probably easy yes. Likely to be merged? No lol

truestory 1
delaguardo10:02:26

isn't that kinda already there? you can add metadata to almost any symbol

☝️ 2
tony.kay23:02:35

Also, have you checked out how we’re doing it in Guardrails? It isn’t aimed at being Clojure Typed (optional type system) but the notation of gspec is way better than inline. Metadata is great for hard-core types (like Java interop or even local loop optimizations on primitives), but if we’re talking about checking “Did I hook this function to that one correctly?” then I claim that types are not the ultimate tool for a language like Clojure(script). https://github.com/fulcrologic/guardrails The gspec vector has the distinct advantage of: • Not cluttering what you’re already used to reading (not inline) • Is actually compatible with defn, in that a freestanding vector after the arglist is just “ignorable” I’ve also made decent inroads into making a type checker from there that can do things like show you the inferred “type-like” value of a symbol in things like a let. I don’t personally thing the “types” as a core Type System are very useful in Clojure. What is useful are data specifications ala spec/malli…and those are most definitely not formal types, but more like data contracts. https://github.com/fulcrologic/guardrails?tab=readme-ov-file#gspec-advantages

👆 3
Karol Wójcik04:02:07

Guardrails with malli. Day starts good! Thank you @U0CKQ19AQ for this beautiful piece of software!

tony.kay04:02:17

yeah, we haven’t announced the release yet…just finishing up some output enhancements…but I’ve been using it for a few days.

ikitommi06:02:59

Looking forward to announcement of 1.2.0 @U0CKQ19AQ! People have been asking for ghostwheel syntax for malli (there is https://github.com/teknql/aave, but not maintained) but as it’s sugar on top of the core, I think it should be a separate library. For same reason, the Plumatic/inline syntax is under malli.experimental, not sure where it should go. Plumatic doesn’t support inline types with key destructuring, asked many times, original reason for this thread, see https://github.com/metosin/malli/issues/1003#issuecomment-1925637260. Quick comments about Guardrails & malli: 1. you get 10-100x perf if you can cache the validator & explainer here: https://github.com/fulcrologic/guardrails/blob/main/src/main/com/fulcrologic/guardrails/malli/core.cljc#L92-L93. e.g. (m/validator schema) returns an optimized and pure function. m/validate re-creates (and forgets) that for each time it’s called (unless the first argument is a Schema object, in case it caches validator and explainer internally, but still slower than m/validator & friends 2. mutable registry. I know many people (me included) define their own registry at application/system-level, using malli.registry/set-default-registry!, with default schemas + a mutable registry backed by a project level atom. Maybe add a way to use this in guardrails? e.g. com.fulcrologic.guardrails.malli.registry/set-registry-atom! or similar 3. malli dev-mode, malli function schemas and emitting clj-kondo definitions, would be great if guardrails followed those, e.g. ! would enable this 4. happy to help with these

ikitommi06:02:44

e.g. this is a the way for spec-like registry today with malli:

(require '[malli.core :as m])
(require '[malli.registry :as mr])

(def registry (atom {}))

(defn register! [type schema]
  (swap! registry assoc type schema))

(mr/set-default-registry!
  ;; linear search
  (mr/composite-registry
    ;; core schemas
    (m/default-schemas)
    ;; to support Var references
    (mr/var-registry)
    ;; mutable (spec-like) registry
    (mr/mutable-registry registry)))
, registering schemas here doesn’t add them to scope of guardrails, so you need to register them twice.

ikitommi06:02:15

… how to fix that: 1. use m/default-registry var in guardrails, which points to whatever user has defined as default-registry, e.g. (mr/composite-registry m/default-regisrtry ..guardrails-atom-which-starts-empy..) -> anything you have registered elsewhere is visibile to guardrails, but not the other way around 2. add com.fulcrologic.guardrails.malli.registry/set-registry-atom! which allows user to override the atom in guardrails so that both ways (project-spesific register and guardrails def> register to same place 3. add a “shared mutable atom” + register-function into malli that everyone should us, instead of defining their own mutable store per application, guardrails could use that too

ikitommi06:02:22

about adding metadata to symbol, works for symbols, keywords, strings and maps only, e.g. this works:

(let [{:keys [^:int x ^:int y]} {:x 1, :y 2}]
  [x y])
but for a inlined vector schema, you need to wrap it into:
(let [{:keys [^:int x ^{:type [:tuple :int]} y]} {:x 1, :y 2}]
  [x y])
which is really bad imo, also, misusing existing features.

ikitommi06:02:08

mx/defn and ghostweel defn> are both good, haven’t made my mind which I prefer more 🙂

ikitommi06:02:40

IF there would be “optional types as docs” extension to Clojure syntax, it would open up problem: one could use all of plumatic, spec or malli schemas there, e.g.

;; plumatic
(defn kikka : Long [x : Long, y : Long] (+ x y))

;; spec
(defn kikka : int? [x : int?, y : int?] (+ x y))

;; malli
(defn kikka : :int [x : :int, y : :int] (+ x y))
, which would be really confusing for everyone.

tony.kay07:02:31

To me the two serve different purposes. Type annotations as metadata are info the compiler can make use of to write more optimal code (e.g. no use introspection or use primitives). The use of formal types in clojure IMO is otherwise useless, since most things are either primitive, maps, sets, vectors, or lists. That gives you a little insight, but not a lot. Putting schema on metadata is just too noisy. I want to see arity and names of args clearly. Now, it would be nice if the docstring showed the arglist AND under that the types of the args and the return type (the gspec). Then you’d have both.

tony.kay07:02:33

RE: the registry. Remember that library authors are going to use this registry, and they may load before user code. If the application code then overrides the registry, they have to comb through libs and figure the whole mess out. No, the GR registry is for GR, and you need to merge your schemas that you want to use in gspec into what GR has defined otherwise lib composition falls apart, doesn’t it? Thus, you’d instead merge your global default INTO GR’s registry. That way your global data reg doesn’t end up with a ton of library-related schemas, and you don’t accidentally muck up those library schemas. Of course, lib authors better ns their schemas well 🙂

tony.kay07:02:26

I really thought hard about the problem, and having the user be able to change out the registry seems a bad idea. Just merge the things in you need and then also benefit from what your library authors have also provided for use with their GR functions.

tony.kay07:02:09

As far as optimization: I did not know about the validator vs. validate. Yes, I’ll plan on making that optimization in a delta release soon.

tony.kay07:02:21

We did tinker with malli dev mode and emitting those, but decided not to do that for now. If someone wants to maintain such an add, we can ask @U9S6X97KQ if he still has what he wrote for that.

ikitommi07:02:05

> To me the two serve different purposes. Type annotations as metadata are info the compiler can make use of to write more optimal code (e.g. no use introspection or use primitives). The use of formal types in clojure IMO is otherwise useless, since most things are either primitive, maps, sets, vectors, or lists. That gives you a little insight, but not a lot. Putting schema on metadata is just too noisy. fully agree. I was not proposing to use meta-data just because of this, it was just proposed earlier on this thread. Having extra marker : and just ignore the next token (like TC39), that I would not object.

delaguardo09:02:50

> about adding metadata to symbol, works for symbols, keywords, strings and maps only I don't get it. I did not suggest using existing :type metadata but add something malli specific:

(defn plus
  ^{:malli/schema :int}
  [^{:malli/schema :int} a
   ^{:malli/schema :int} b]
  (+ a b))
it is already there, you don't need language support for that. and it is possible to simplify using a macro

gnl14:02:35

Couple of unstructured notes on the general discussion:

gnl14:02:39

• Regarding a possible syntax for spec/schema/type annotations – what I would like to see is a single, universal, widely adopted syntax that can serve as a basis for both function call validation at runtime, as well as any kind of possible static type analysis. To me, both metadata and interspersed : <type> annotations have various issues with regard to succinctness, noisyness, not having a good solution for such-that predicates, getting particularly messy with multi-arity functions, etc.

gnl14:02:45

From my – obviously rather biased – perspective only gspec https://github.com/fulcrologic/guardrails?tab=readme-ov-file#gspec-advantages I care about (most of them objective, some less so), and, as already mentioned, has the added bonus of being perfectly compatible with regular defn syntax – no need to worry about IDE/editor support, can easily flip >defn to defn and it just works, etc.

gnl14:02:57

• Currently Guardrails' >defn requires a gspec vector, but if gspec were to be used in core's defn or any other macro that has to support both specced and unspecced functions, it would require determining whether the first body form is just a regular one or a gspec vector, in which case it should be appropriately validated for possible syntax errors. The way I imagine this would work is that if the first body form in a multi-form function body is a vector, that's a gspec and treated accordingly, otherwise it's evaluated regularly. This should be reliable, as there's otherwise no legitimate reason to have a vector in there, because it's not returned and doesn't do anything, unless you use nested side-effecting calls inside of it, which – don't.

gnl14:02:17

• Guardrails already emits a native Malli function schema in the metadata of the generated defn (`{:malli/schema ...}`), so if anyone wants to use Malli instrumentation for the generation of clj-kondo type annotations (or really any kind of native or third-party tooling that works with Malli function schemas), they can easily do so. As @U0CKQ19AQ already mentioned we ultimately decided against emitting a clj-kondo config automatically, but the basic implementation of it is on https://github.com/gnl/guardrails/blob/gnl/type-experiments/src/main/com/fulcrologic/guardrails/malli/core.cljc#L85. It also still has the code for the generation of JSDoc Closure type annotations (left over from Ghostwheel), which we removed in the new Guardrails release.

gnl14:02:30

• Thanks @U055NJ5CC for the validator/performance hint – neither of us had much experience with Malli before we started implementing it in Guardrails, so this is very helpful.

👍 1
Noah Bogart14:02:48

is there a reason guardrails doesn't use the attr-map?

gnl14:02:36

Do you mean for defining function specs or in general? We use it for setting function-specific configuration options.

gnl14:02:43

There are multiple reasons we don't use it for function specs, mostly having to do with not being able to handle predicates + symbol references/refactoring as concisely and smoothly as we can with the gspec vector, and it being a huge mess when you're dealing with complex multi-arity functions. See the https://github.com/fulcrologic/guardrails?tab=readme-ov-file#gspec-advantages for more details, there are multiple things on there you can't do with the attr-map.

👍 1
gnl14:02:21

Probably the easiest way to illustrate this is to have a multi-arity function, specced-out with argument and return-value predicates, as well as differing return specs per arity, and macroexpand the >defn on that to see the monstrosity that is generated as a native clojure.spec fdef (we'd have to do something similar if we had to put the whole function spec in a single place like the attr-map). You could use some of the multi-arity https://github.com/gnl/ghostwheel.specs/blob/master/src/ghostwheel/specs/clojure/core.cljc as an example.

Noah Bogart14:02:52

thanks for the details, that's a compelling case

👍 1
Noah Bogart15:02:45

oh i said "attr-map", but i was actually thinking of the prepost-map, which goes inline after the argument vector of each fn body. i'm guessing the reasoning is the same, tho

👆 1
gnl15:02:00

I would also add that in multi-arity functions I like the spec/schema/type being in the same place as the arg vector, rather than having to track down the correct arity in a separate place (whether it's attr-map, s/fdef, m/=> or whatever).

👍 1
ambrosebs01:02:59

Here's how Typed Clojure achieves this. It combines the insights already mentioned that Clojure already supports this via metadata, and that you need namespaced keywords to distinguish different backends https://github.com/typedclojure/typedclojure/blob/main/example-projects/zero-deps/src/typed_example/zero_deps.cljc

👍 2