Clojurians Log v2

i think merge is defined only for maps so this is UB?

correct - this always was undefined

i feel like i've seen this discussed before

merge docstring talks explicitly about maps

"Look, my setup works for me. Just add an option to reenable spacebar heating vec merging by conj'ing."

note that the jank clojure test suite has many tests related to this - they mostly fail now, but should just be removed imo. haven't had a chance yet to file an issue about that

I'm fully on board with this, just need to know which edge cases to support in SCI ;)

the case I have found in a few places is more commonly this (which used to do something):

user=> (merge {1 2} [3 4])
Execution error (IllegalArgumentException) at user/eval3 (REPL:1).
Don't know how to create ISeq from: java.lang.Long

let it crash seems good to me

I have filed issues on a couple projects related to this

I can sharpen clj-kondo to detect these maybe

https://github.com/jank-lang/clojure-test-suite/blob/main/test/clojure/core_test/merge.cljc#L42-L63 if you want some test cases :)

https://github.com/jank-lang/clojure-test-suite/issues/955 just so I don't forget

Ah, finally that multi-vector concat I've always wanted 😄

(merge [1 2] [3 4] [5 6]) => [1 2 3 4 5 6]

you mean, a multi-vector concat that will break in 1.14? ;)

Don't ruin it, I just got it.

some people just like living on the edge!

this spec is still correct right? https://github.com/borkdude/speculative/blob/87bd332d39d154a8bd282c380b7dc0746f8fcf20/src/speculative/core.cljc#L333

or is seq of 2-vector now also allowed?

ow yeah...

ERROR in (swagger-spec-test) (/Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/test/malli/swagger_test.cljc:334)
generates swagger for ::parameters w/ basic schema + registry
expected: (= {:definitions {"malli.swagger-test.body" {:minLength 1, :type "string"}}, :parameters [{:description "", :in "body", :name "body", :required true, :schema {:$ref "#/definitions/malli.swagger-test.body"}}]} (swagger/swagger-spec {:malli.swagger/parameters {:body (m/schema :malli.swagger-test/body {:registry registry})}}))
  actual: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword
  clojure.core/merge                        - sci/impl/namespaces.cljc:64:1
  malli.swagger/dissoc-non-root-definitions - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:192:21
  malli.swagger/dissoc-non-root-definitions - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:156:1
  malli.swagger                             - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:191:17
  clojure.core/reduce-kv                    - clojure/core.clj:6993:1
  malli.swagger                             - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:180:10
  clojure.walk/postwalk                     - clojure/walk.clj:53:1
  malli.swagger/expand-qualified-keywords   - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:177:5
  malli.swagger/expand-qualified-keywords   - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:174:1
  malli.swagger/swagger-spec                - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:203:4
  malli.swagger/swagger-spec                - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:201:4
  malli.swagger/swagger-spec                - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/src/malli/swagger.cljc:199:1
  malli.swagger-test                        - /Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/test/malli/swagger_test.cljc:375:14
  clojure.core/list                         - clojure/core.clj:16:1

and:

n (request-parameter-definition-regression-test) (/Users/runner/.gitlibs/libs/metosin/malli/6b0862135ea58ef2ac92bda9ae605994a5fb20d6/test/malli/swagger_test.cljc:533)
collects :definitions for all parameters
expected: (= expected (fix (swagger/swagger-spec {:malli.swagger/parameters {:header [:map [:h :string]], :query [:map [:q :string]], :body (m/schema :malli.swagger-test/req-body {:registry registry})}})))
  actual: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.lang.Keyword
  clojure.core/merge                        - sci/impl/namespaces.cljc:64:1

the problem seems to be here: https://github.com/metosin/malli/blob/76603d587553c5a8d60fa64c9e631fa3addb4b2e/src/malli/swagger.cljc#L201-L202 cc @joel.kaasinen - repro:

clojure -Sdeps '{:deps {org.clojure/clojure {:mvn/version "1.13.0-alpha8"} io.github.metosin/malli {:git/sha "76603d587553c5a8d60fa64c9e631fa3addb4b2e"}}}' -M -e '
(require (quote [malli.core :as m]) (quote [malli.swagger :as swagger]))
(swagger/swagger-spec
 {::swagger/parameters
  {:body (m/schema ::body {:registry (merge (m/default-schemas) {::body [:string {:min 1}]})})}})'

thanks! I thought we were running tests on 1.13 but maybe this didn't get covered

oh heh https://github.com/metosin/malli/issues/1319 missed this one

malli fix in master, I'll have a point release out Soon(tm)

thanks for pinging me @borkdude, I wasn't on top of this at all

Thanks for fixing!

out in malli 0.20.3

not sure I need an example, this is a common issue

it is a good match for the things we are adding to Clojure 1.13 and thinking about right now with respect to destructuring. the way to think about this is that the data structure has a vocabulary of attributes and those attributes hopefully have a well-known shape (you may be in a nested variant of this). the key is to not try to spec the top-level wrt s/keys req/opt. instead, apply the Clojure 1.13 :keys! and/or :missing to specify the structural constraint in destructuring

Consider a system that receives orders, looks up prices, and annotates the order with these prices

;; incoming order
{:order/id 1234
 :order/items [{:item/id 1}
               {:item/id 2}
               {:item/id 3}]}

(s/def :item/id integer?)
(s/def :order/item (s/keys :req [:item/id]))
(s/def :order/items (s/coll-of :order/item))
(s/def :order/id integer?)

(s/def :incoming/order (s/keys :req [:order/id
                                     :order/items]))

;; order after looking up prices
{:order/id 1234
 :order/items [{:item/id 1
                :item/price 100}
               {:item/id 2
                :item/price 300}
               {:item/id 3
                :item/price 250}]}

{:order/id 1234
 :priced.order/items [{:item/id 1
                       :item/price 100}
                      {:item/id 2
                       :item/price 300}
                      {:item/id 3
                       :item/price 250}]}

> the key is to not try to spec the top-level wrt s/keys req/opt. instead, apply the Clojure 1.13 :keys! and/or :missing to specify the structural constraint in destructuring Hmm ... maybe I'm missing something, but for me :keys! and destructuring fill a different need than specs (which I use for things like spec.gen and OpenAPI specifications)

we will (eventually) be connecting these

I think generally if you have a key that means different things in different contexts, you should use a different name

but what you have here seems like they are the same thing but have different sub-vocabularies of attributes (in different contexts)

in general, I would say we are moving towards s/keys with :req and :opt being something you should stop doing, and instead focus on using destructuring to specify your structural constraints at specific points where you want to enforce them (with :s/keys!/:missing).

I see--thank you! that's very interesting and it's good to hear that this is something that is being thought about and worked on.

I think in my contrived example it would be easy to just say, make :item/price optional and move on. But when I've run into this in real systems, the relationships between sets of optional attributes are more complicated. Like: in stage one you have attributes A, B, and C, stage two adds D and E, and stage three adds F. And those relationships also cross levels of nesting.

I'm curious/excited to see how it gets connected in terms of how nested non map collections get represented (e.g. s/coll-of/`s/cat` within a map, like :order/items above). To me that's the missing link right now, everything else makes perfect sense. But besides that, I think the destructuring forms are good at representing stuff like your example: > in stage one you have attributes A, B, and C, stage two adds D and E, and stage three adds F. I think part of the idea is you're sometimes better off not always naming the intermediate steps and just checking in the destructuring sites.

yes, the relationships are contextual so you want to specify structural constraints at the point of use, not globally

you can nest destructuring, although I think in this case you'd have to do it with two levels of destructuring - one for orders, one for items

Chiming in with the obvious recommendation here: @enn have you seen Rich's Maybe Not talk? It touches on this exactly. And the recent In Context talk from last week (but that's not out yet). Although Alex effectively summarises things here and how the ideas have evolved.

importantly, the talk from last week is an update and change from Maybe Not (which led to spec 2, which is now dead)

hoping to get Rich's talk up soon

also if you missed it, this post from Rich was really useful for understanding his more recent updated thinking wrt the stuff in spec2/Maybe Not https://clojurians.slack.com/archives/C06E3HYPR/p1783538440018859

That post should come with a spoiler alert for the In Context talk 😄

Ooh, I have watched "Maybe Not" (in fact, I think that talk is what made me feel like I should be able to define attributes once and then come up with many variant combinations of those attributes in different contexts) but had not seen that post from Rich. He describes exactly what's been tripping me up: > clojure.spec follows RDF in supporting global definitions of attributes independent of containers, which can then be arbitrarily composed (e.g. via spec/keys). But spec/keys still requires a named spec for each context. That's not so terrible when maps are flat, but becomes a serious problem as things become nested. Every context might need a new set of variant s/keys specs. Thinking about it a bit more, I do start to see how destructuring addresses these problems. Very excited for the new talk to drop (and I'm sad I could not make the Conj this year).

@enn One of the things in Alpha 8 that can be very useful here is selector. This takes a destructuring map, and produces a function that will take input and produce the result of the destructuring (via :select, :all, :missing, :excess). This allows you to create structural predicates that you can use to check, in code, for the "shape" of the data, much like you might previously have tested record-like Specs, and then of course use Spec to ensure the individual, named (qualified) items conform to their Specs a la RDF, i.e., at the "information" level.

We have some test code that currently uses Meander for structural pattern matching, and I realized I can replace it all with a few selector calls.

The 1.13 stuff is making me rethink a lot of life choices as far as our code at work goes 🙂

I am a little curious how the new destructuring approach works for recursive data shapes

True, you're not going to be to specify a recursive structure. But you can check the relevant structure at the appropriate places in your code (which, I assume, is also recursive?).

I'm probably not getting something, so I'll wait till it's all as ready as can be, but I'm still confused about data generations with selectors. Or would using Spec+selectors, as Sean outlined, necessitate using custom generators?