a followup question to one I posted yesterday, if I have a conformer from string to seq of chars:

(s/def ::layout-string
  (s/and string?
         not-empty
         (s/conformer seq)

I take that back, turns out @dergutemoritz already answered this one

important info re 1.9/spec - https://groups.google.com/d/msg/clojure/10dbF7w2IQo/ec37TzP5AQAJ

spec will be versioned then?

yes

versions will be ala core.async: 0.1.<git-change-derived>

the specs for clojure.core functions/macros will stay with clojure?

no, that's what core.specs.alpha is for AFAICT

it'll be interesting to see how this impacts CIDER's support for spec. Now the tooling will have to make sure to match the version of spec that the application uses

@bronsa that's a namespace in the spec artifact?

it's a separate contrib lib

oh so you need three artifacts if you want to use clojure with the core specs?

I'm assuming clojure will depend on core.spec.alpha but dunno

I'm just trying to figure out if/how the specs can drift from the implementations

if the specs stay with clojure then that's not an issue

> Additionally, this is a first step towards increased support for leveraging dependencies within Clojure. I don't know what that means

@bronsa yes, clojure will depend on the two new libs

but you can also specify a newer version and override the one its using

@gfredericks towards demonolithing clojure

FWIW i have a similar dependency tree with t.a/t.a.jvm and I found that it's easier to just depend on the leaf dependency (t.a.jvm or in spec's case core.spec.alpha) and release concurrently when the root dependency needs a change

just curious: why not put the “alpha” bit in the version string instead of the ns?

The only problem I see with making libraries like spec (not part of lang) are “jar hell” dep issues

in JVM land libraries using deps always leads to trouble because you end up using 2 libs with 2 different version of the same dep and they are incompatible

(eventually)

However, I know Rich’s talk at the last Clojure/conj was about never making a breaking change essentially. So maybe clojure.spec will be robust enough to avoid the problem and you can always just “take the newest version” of the dep when multiple lib deps diagree on version.

I just know that I ran into issue with say, Plumatic schema when several libraries were using it and I was using those libraries and then schema came out with a set of breaking api changes

This libraries having dependencies conflict problem has been one of the greatest pains of mine on the JVM it seems. I don’t think I’m alone there though. It can get difficult to deal with. I think it often encourages libraries to not use any dependencies. Which I feel is unfortunate for code-reuse or using useful utilities for documentation like spec/schema.

Or in Java-land sometimes people do build-time repackaging of classes to some “internal” package name. I think it is harder in clj to confidently add a prefix to a set of namespaces though in a way that doesn’t miss anything.

My experience — six years of production Clojure with a code base of now 50K lines — is that API version conflicts are actually pretty rare in Clojure libraries in the wild.

They are more common with Java libraries, true, but Clojure people seem to be better about API changes.

(says the man who’s guided clojure.java.jdbc through several breaking API versions, however)

I think the danger comes mainly from Clojure libraries that themselves have a large number of transient dependencies where they rely on Swiss Army Knife utilities libraries for just one or two “cool functions” — which is an indictment of the questionable validity of such libraries, IMO. They should be broken up into much smaller libraries so you don’t have to pull in the whole thing for just one or two useful functions.

But this is a case of “do one thing and do it well” which such libraries inherently don’t follow 😞

Right, part of the issues with "jar hell" are really problems with non-additive changes to APIs.

Yeah, that’s sort of what I was thinking was the answer to this concern here. It’d be great to see that workout.

It’s the “secret weapon” 😛

the whole idea is to make all versions of a library compatible

but the important thing here is that alpha is a time before this applies and breaking stuff is ok (you as a user have to track the changes)

@alexmiller I asked on the list but I’ll ask here too: will the new org.clojure/spec.alpha artifact be available for a few days before Clojure master SNAPSHOT actually removes the namespaces?

it’s available now

Oh, cool!!

org.clojure/spec.alpha 0.1.94

as of just a few moments ago

Thank you! Much appreciated!!

but we are moving (slowly) towards an updated alpha release of clojure that uses it today

I’ll get our codebase switched over today then…

@alexmiller I'm a little curious about this shift. If you depend on "1.9.0", is it part of clojure's api to expose it's transitive dependency on spec? Or should I always explicitly depend on spec if I'm using it in my lib?

I would expect core 1.9 will conditionally use spec at this point… So if you have spec loaded, you’ll get better macro error messages?

@dominicm clojure will have a compile-time dependency on spec.alpha so you can always rely on it being available. So you don’t need to explicitly list the new library as a dependency. You may do so though in order to specify a newer version than Clojure is depending on.

@seancorfield it is not currently conditional in any way

Is there a mechanism in spec for namespacing keys as part of a conform? For example, a map parsed JSON will not have namespaced keys so I'm applying some functions that namespace them for me but this feels like a pretty common use case which must have a better solution

s/keys has :req-un and :opt-un options for this

So Clojure 1.9 will transitively bring in the spec library automatically? But you can bring in a different version yourself? How will that work if the spec library breaks the API?

@pdlug see the guide for an example https://clojure.org/guides/spec

@seancorfield yes. yes. then you’re broken (see “alpha = things may change”).

sorry, have to jump in the car but can pick up later

@alexmiller Any pointers as to where? I've been through that a few times, I see the examples on spec'ing unqualified keywords but not coercing to namespaced keywords as part of a conform

Ah, so for clojure.spec to break Clojure itself, you’d already have to have a new public release of Clojure available (even if it is 1.10 alpha perhaps at this point)?

I’m hoping you wouldn’t make a new release of clojure.spec that then required users of it to rely on SNAPSHOT builds of Clojure — you’d at least have a new alpha of Clojure available by that point?

(as an active user of clojure.spec in production, this is all a bit frustrating and a little worrying!)

@pdlug I think maybe Alex misread what you were trying to do? There's no automated way to produce qualified keys from unqualified ones.

@seancorfield Thanks, that's exactly what I was asking, that's unfortunate because this seems like a really common use case (JSON APIs, JSONB in PostgreSQL, etc.)

Since any namespace qualifier you would be adding is arbitrary, I don't see how spec plays into this: you control that arbitrary key qualification.

Somewhere you're converting string data to a Clojure representation of JSON. Why not introduce the qualifier at that point -- then spec with qualified keys directly?

That's why clojure.java.jdbc provides a :qualifier option, for example.

i do it by typing (on the keyboard).

Pardon @mobileink ?

for a json schema i'm working with i can come up with namespaces for everything, but automating it seems like a lot more work than just typing it in.

@seancorfield if you were asking about spec breaking Clojure, I don't see how we would do that. The integration points are pretty small.

@alexmiller I was considering a situation where you made a breaking API change in clojure.spec.alpha that could affect Clojure itself (since it depends on the earlier, non-breaking API)

e.g.json schema puts "di" in schema "foo.bar". in spec: :foo.bar/di.

I think this would be easier if there were more functions for manipulating maps to add namespaces to keywords. I keep copying the same keywordize-keys-ns function in every project to coerce incoming JSON into keywords w/ namespaces. Would be nice to also have a rename-keys that could do unqualified to qualified mappings en masse.

But if the integration surface is small, I guess that breakage is unlikely — and what I meant was that you’d already have to have a version of Clojure somewhere that used the new (changed) API in order to test/use it anyway. I just wanted to check that if you ever made such a breaking change to clojure.spec.*, you’d ensure there was a *non-SNAPSHOT* version of Clojure made available at the “same time”.

@seancorfield the only thing clojure depends on from spec atm is s macroexpand-check, I find it really hard to imagine a situation where that could be broken

I would consider that broken :)

Well, the World Singles’ main codebase now uses clojure.spec.alpha with an explicit version dependency so I’m “happy” 🙂

@alexmiller any particular reason why the core.specs.alpha is a separate package?

Vs?

keeping it in clojure.spec.alpha or clojure.core I guess

just wondering if there are extended plans for it

You mean why in a different ns or why in a different project/artifact?

different artifact

It's logically separate and can evolve at its own rate

ok I'm still confused how the whole fdef / macroexpand-check integration is going to work, guess I'll have to wait and see

We made the decision long enough ago that I've forgotten the details tbh

@thheller it's exactly the same?

The code is just in a different artifact

ok, I keep thinking there is a circular dependency somewhere

There is

spec.alpha depends on Clojure

Clojure depends on spec.alpha

There is a bootstrap aspect to it

ah right, forgot the current code already does a var lookup.

I find it's best not to think about it too hard :)

hehe yeah I'm thinking more about CLJS at the moment

I was using the core.specs for CLJS but I guess that is just as easy with them in a lib

It's easier

The specs are not the same though for things like ns

well, everything besides ns is though

but still trying to come up with a good way to intergrate them with CLJS properly. might need to copy them anyways for self-hosted.

Open to helpful ideas

still trying to solve the "helpful errors" issue, currently the spec explain for something wrong in a let doesn't look much more helpful than the actual compiler error

Well more to come on that

@thheller I think the benefit will come from familiarity — all spec-powered errors will have a similar output format and folks will quickly get used to them (both from let and other core constructs, as well as from general libraries that use spec). And any improvements to how explain works will improve all errors from code that uses spec.

I mean, yeah, it’s a lot of output, but it’s very regular and structured.

not too sure about that

CLJS error in demo/errors.cljs at 3:1
In: [1] val: 1 fails spec: :clojure.core.specs/arg-list at: [:args :bs :arity-1 :args] predicate: vector?
In: [1] val: 1 fails spec: :clojure.core.specs/args+body at: [:args :bs :arity-n :bodies] predicate: (cat :args :clojure.core.specs/arg-list :body (alt :prepost+body (cat :prepost map? :body (+ any?)) :body (* any?)))
:clojure.spec/args  (x 1)

the most helpful part is demo/errors.cljs at 3:1 😉

the problem in this case is that 1 is a number which doesn't have reader metadata, so you cannot narrow the cause down further

(defn x 1) is the actual source so 3:1 is the start of the (defn not the 1 which would be more accurate

I don't follow

that seems pretty clear

1 isn't a vector, 1 isn't that other big long thing

each case gives the value that failed, and the predicate it failed

and which spec the predicate came from

@hiredman the predicate can get cryptic as seen in the second message

the grammar is cryptic 🙂

I think it is alright as well, a beginner might not be so inclined

but my issue for not is more about more accurate source locations

for the most part they are accurate enough though

The lack of reader metadata is annoying but not an endpoint necessarily

yeah coupled with the spec errors you can usually figure out whats wrong pretty quickly

it kinda breaks apart if the printed val gets too large though

I read that first error as “arg-list is not vector?” which is pretty clear. Even the second one is pretty straightforward: “args+body” is not a sequence of “args followed by body” — and if you (source defn) that should triangulate to the thing after the symbol being defnd is not either of those.

... please don't get hung up on my toy example. This particular example is "good enough" with spec.

Parameter declaration "1" should be a vector is the default error message without spec ...

which IMHO is better than the spec error (but much less accurate)

@alexmiller will the new spec projects/artifacts get their own JIRA projects or should issues still go to CLJ?

CLJ