The subject looks fantastic! Sadly, the live presentation will at 1:30 in the morning here in Paris, France. I'll watch the replay.

Trimmed video is here https://www.youtube.com/watch?v=QE3PZLlefUE

Thanks for posting this! I was about to post it here now.

I am still watching intently, but I think, at around the 21 minute mark, I think you might have skipped over https://github.com/metosin/malli/blob/master/docs/function-schemas.md#function-guards

@U051GFP2V I'm aware of them, they're only really useful for non-higher-order functions. here's the identity specs I tested https://github.com/typedclojure/typedclojure/blob/main/typed/clj.spec/test/typed_test/clj/spec/identity.clj

But check out what it looks like to achieve something similar for comp https://github.com/typedclojure/typedclojure/blob/7ef50ae3db663a713960a06b804ca0927da71a92/typed/clj.spec/test/typed_test/clj/spec/comp.clj#L14-L27

Yeah, I should have mentioned once you got into the typeclass sort of stuff (typing comp escpecially), I can see how your work is a different level of abstraction. For more specific types it is possible, in Malli, to have a relation from inputs to outputs using function schemas, but it isn’t derivable from the specs themselves.

The relationships between inputs and outputs was a bit of a sleight of hand for presentational purposes. If I were to give the presentation to you, it would be very simple: 1. insight 1: a type variable can be replaced by an arbitrarily specific type and the type will still be inhabited by the same program (e.g., identity is both X->X can be 1->1) 2. insight 2: you can get a surprisingly accurate generative test by making specs that look like the instantiations (like 1->1, (1->2)[1]->[2]) 3. You can only write only spec per function, so let's make a syntax that works both for instrumentation and excercising a function that uses this trick "for free" instead of worrying about :fn and even surpassing it in some cases).

I didn't mention in the talk that you can get back the instrumentation spec in many cases by instantiating variables to :any.

Another thing to understand about the talk was I challenged myself to make it understandable to a 10 year old who hasn't programmed yet. 🙂

I'm very proud to say he summarized my talk as: "specs must be specific to be useful".

I appreciated the starting-slow style. Leaving off namespace aliases etc, makes things a lot clearer.

True, the syntax is really difficult for slides.

Watched - nice! Curious, what's are your thoughts on Spec function instrumentation :fn s? (or Malli https://github.com/metosin/malli/blob/fb467791f3748f8eb2795bbcc029e583c028875e/docs/function-schemas.md#function-guards) Frankly, I've very rarely wanted to reach for them. Although I'd be open that the idea of them (or their leveled-up counterparts) bringing in some power.

@U45T93RA6 thanks! about :fn I guess it's an escape hatch for dependent schemas. My main reservation is that it's code, not a spec, so there's less you can do with it besides just run the code. I tried to push :fn about as far as you can go with generative testing. https://github.com/typedclojure/typedclojure/blob/7ef50ae3db663a713960a06b804ca0927da71a92/typed/clj.spec/test/typed_test/clj/spec/comp.clj#L14-L27

But I think :fn fits with the rest of spec's design, which is mostly about hardening functions that pass domain-specific data around (and not super concerned with higher-order implementation-level things).

I don't have any concrete ideas on how to improve :fn but the spirit of what I hope is possible is more like:

(fspec :args (dcat :a int? {:keys [a]} :b (coll-of any? :min-count a)))

e.g., raises too many questions about nesting with sequence regex specs, and probably only works left-to-right. But maybe it's a little more useful to programmatically inspect?

But the main downside to :fn that comes to mind in practice is that it doesn't participate in generating arguments for the function. You just blindly generate the args, and then hope it conforms to the :fn later on https://github.com/clojure/spec.alpha/blob/c630a0b8f1f47275e1a476dcdf77507316bad5bc/src/main/clojure/clojure/spec/test/alpha.clj#L410

Thanks for the overview :) Would be curious to see where all this work goes. Is it ambitious nowadays, or something more research-y / side-projecty?

I think it's pretty grounded, I have prototypes for spec/malli/schema. Working on malli right now, seems the most flexible. But really a lack of community interest is the main roadblock. The responses to the talk have been encouraging.

Well, the other roadblock is perfectionism. Or at least, wanting to get it right so I don't need to make breaking changes. 🙂 For example, I want to write specs that work for all arities of map/comp/every-pred etc. It's very fun, but it's hard to tell whether it needs more hammock time.

It's been about 3-4 years, maybe it's enough time? 😉 The question I'm trying to answer right now is about the "kind" of specs (like kinds in type theory, or if you're not familiar, the question of "what is the spec for a spec"). So far I think regular non-regex specs are of kind "Spec". This is so you can write (all [x :- Spec] [:=> (cat x) x]) and disallow instantiating x with [:cat now identity takes fives args] . So the next question is, what's the kind of regex specs? I think you can just lift them to the kind level. like the spec for memoize is (all [x :- (* Spec), y] [:=> [:=> x... y] [:=> x... y]]) . That's what I'm wrestling with atm.

One of the joys in this work is that spec/malli/schema have given me a playground to work out these ideas. The same problems pop up in Typed Clojure, but it's so much more difficult to even formulate the problem. My hope is that this will teach me a bit about how to support proper sequence regex types in Typed Clojure.

But the other joy is that I've been really impressed with the utility of these extensions to spec/malli/schema, especially compared the implementation effort vs implementing a type system. It's got me reconsidering what a "gradual type system" might look like. Traditionally, it's static typing protected by instrumentation at the typed-untyped boundary. But, what if it was static typing protected by generative testing?

Concretely, perhaps if you annotate a type as "no-check" in Typed Clojure (don't statically check its definition), it instead runs generative tests at type-checking time?

Or even, apply that same idea to clj-kondo. Enhance clj-kondo's type syntax to include polymorphism, but verify it with generative testing instead. Then just use the simplified type during clj-kondo static checking.

Wow thanks for writing up your thoughts: novel and interesting!

> But really a lack of community interest is the main roadblock. Being optimistic, it might be simply some sort of chicken-and-egg problem, i.e. in absence of incredibly attractive features, adoption will be smaller, which in turn means less development. In my perspective, most people think in terms of 'features'. They might care less about the theory behind as long as there's some big ROI. Beside from good old type checking, I tend to want, for instance, whole-system generative testing (e.g. my distributed system behaves nicely even if X times out and Y fails). Results (as opposed to exceptions) seem a good fit. I wonder if, should I give that sort of task a shot, would 'naive specs' cut it, or would I end up needing higher level constructs 😄

Being optimistic, it might be simply some sort of chicken-and-egg problemYeah you hit the nail on the head. I haven't gotten to ride the adoption/interest wave yet to a stable release, which is both a blessing and curse. > I wonder if, should I give that sort of task a shot, would 'naive specs' cut it, or would I end up needing higher level constructs I think if it you needed high-level constructs to test systems end-to-end, they would already be supported by spec given how pragmatic it is. This work is definitely more niche. I tried to lay it out explicitly in my talk. It's hard to come up with relevant features like "this is how to fuzz test your transducer" because most people don't (get paid to) write transducers.

Developers of transducer libraries might be confident enough to not want to fuzz test them. I'm not exactly sure how to market these things.

Perhaps if the entire vision is realised of static checking + generative testing, then things will be more compelling.

> I tried to lay it out explicitly in my talk. Yeah I had gotten the impression that this stuff shines the most when writing/hacking clojure.core or so. At the same time I'm open to be surprised as for broader applications > Perhaps if the entire vision is realised of static checking + generative testing, then things will be more compelling. Sounding great, I'd be curious to see a clj-kondo + higher-level + generative thing, sounds like a good composition

Maybe even a pitch like "well, we all have a utils ns right? here's how to fuzz test that ns".

> Yeah I had gotten the impression that this stuff shines the most when writing/hacking clojure.core or so. Yes. And I guess utils namespaces look the most like clojure.core in most code bases. Maybe that's a good angle.