Fork me on GitHub
#clojure-spec
<
2019-10-03
>
noprompt00:10:09

Put the semantics with the source.

seancorfield00:10:15

Well, we're just going to disagree on how clear we think the Clojure/core team have been on that... ¯\(ツ)

alexmiller00:10:20

I've added a warning to the readme of both core.specs.alpha and spec.alpha

❤️ 1
seancorfield00:10:35

Problem solved! 🙂

alexmiller00:10:45

If you have another suggestion of how to signal this, please let me know

noprompt00:10:59

I literally have tears in my eyes. 😂

alexmiller00:10:02

sorry it wasn't clear

noprompt00:10:31

But they are now! Which is great!

noprompt00:10:57

You have literally changed my attitude.

alexmiller00:10:11

I think making alpha ok with breaking stuff is fine, with the caveat that having something in alpha for years and deps of official releases is admittedly weak sauce

alexmiller00:10:22

yet, that's where we are

alexmiller00:10:56

I have a burning desire to move these libs out of alpha and stabilize

noprompt00:10:37

Speaking from my experience with the greek letter style namespaces, its been incredibly liberating to not have to worry about breaking users.

alexmiller00:10:05

and as fair warning, core.specs.alpha is eventually going to get updated to depend on spec-alpha2 (or some future non-alpha variant) and will necessarily break again. But on the upside, these will be new names, not breaking versions of old names.

noprompt15:10:19

Is there an appropriate place to discuss this or get an explanation? It doesn’t seem like a necessary requirement for spec alpha to depend on any other version of spec.

noprompt15:10:28

Put another way: is there a path forward that doesn’t cause breakage? Breakage should be taken seriously and avoided at all costs. It takes real time and effort on behalf of users to deal with it.

alexmiller16:10:17

We have had some discussion about it but are deferring until later the full analysis of what we will do for migration. Most likely, there will be a new lib (probably core.specs) with new namespaces and new (spec 2) specs. There are a number of possible paths to continuing to support spec 1 specs (via some adapters possibly) and many choices about to what extent they are supported. Exploring this in full depth is more than I'm going to do in this slack thread.

alexmiller16:10:59

You're preaching to the minister here on breakage, but as an alpha library the plan was always to ultimately be replaced and we've been pretty vocal about that from the start.

noprompt16:10:03

Where does the full depth discussion take place?

alexmiller16:10:25

N months from now, Rich and I will go through a design process to try to flush out all the questions and answers around this

alexmiller16:10:41

how and what kind of feedback we'll need at that point I'll defer to that point

alexmiller16:10:45

what the world will look like then, I have no idea so I'm not going to worry about it now

noprompt00:10:27

Thats the win I’ve experienced.

seancorfield00:10:02

clojure.spec.alpha -> clojure.spec-alpha2 has some major breakage for users who choose to migrate. Most mainstream stuff is unchanged but there's a lot of "dragons" if you've ventured off that path 🙂

noprompt00:10:28

Its been incredibly liberating for me to be able to make new versions of my library without worrying about breaking users or, worse, stagnating.

noprompt00:10:07

Maybe I took what Rich said to the extreme but, in practice, what I’ve come to discover is that strive for a “stable version” is just a withdrawal symptom of breaking away from a bad versioning habit and way of thinking about a project.

noprompt00:10:20

Like, it no longer bothers me that the next version of Meander, when I get to it, will live at meander/zeta. I don’t care that I can’t (:require [meander.core :as m]). (:require [meander.zeta :as m]) is fine.

noprompt00:10:25

The whole idea of a “stable version” is odd to me when we all know the only thing we can rely on as software developers is that things are rarely, if ever, stable.

noprompt00:10:56

So, I would hope, that spec doesn’t devolve to SemVer at some point.

noprompt00:10:37

I think it would be a step back to one day to have [clojure/spec X.X.X].

noprompt00:10:01

That just puts you back in the soup.

alexmiller00:10:23

well, you will have that

alexmiller00:10:46

but at the point we go there, we will try to maintain only additive changes after that point

noprompt00:10:08

Right but the point is by not doing that you don’t have to do what you just said.

noprompt00:10:11

You don’t lock yourself in.

noprompt00:10:28

Once you go back to SemVer, you’re locked in.

alexmiller00:10:36

it's not sem ver

noprompt00:10:43

You have to have the discipline to do what you just said.

alexmiller00:10:52

it's an increasing set of numbers that make only additions

alexmiller00:10:07

I believe we have that discipline :)

alexmiller00:10:21

and will submit the rest of clojure as evidence :)

noprompt00:10:29

But why put yourself in a situation where you necessarily must have discipline.

alexmiller00:10:55

because supporting an ever changing set of namespaces across versions is irritating?

alexmiller00:10:11

there's a balance of tensions here

noprompt00:10:20

How would that be any different from just depending on a fork?

noprompt00:10:44

I can’t speak for others, but I don’t see that as an irritant.

noprompt00:10:39

The question is “how do you know you’re at a stable point where its possible to only sustain the software on discipline you can be sure you have?”

alexmiller00:10:51

in general, I would rather just bump versions than bump versions AND update the namespaces in all my code

noprompt00:10:21

In principal you can do that already with any artifact that uses SemVer in honestly and never has a major version bump.

alexmiller00:10:46

I'm going to just say experience. but changing to new namespaces or even new lib names is not an option that's lost - we can always do that if needed

alexmiller00:10:07

I'm suggesting a small thing for users while still retaining the option to do the big thing for users

alexmiller00:10:19

that seems better regardless

noprompt00:10:34

Well, Alex, I’m not going anywhere anytime soon as far as I can tell so I’m looking forward to seeing where this goes. 🙂

noprompt00:10:03

Though, I don’t mind a little typing. 😉

noprompt00:10:46

I gotta go be a dad but, again, thanks @alexmiller for making that change to the README.

alexmiller00:10:45

ditto on dad time

roklenarcic08:10:35

Current spec alpha has a function s/merge which works like s/and in that it will only conform the last spec. I am wondering if spec-tools coercion suffers from same problem?

ikitommi09:10:36

@roklenarcic there is spec-tools.core/merge

ikitommi09:10:19

> .. that selects only the specced keys from each conformed result, then merges those results onto the original input. This avoids overwriting conformed values with unconformed values while preserving all unspecced keys of the input. Fixes #90. By Arttu Kaipiainen.

roklenarcic09:10:43

does it work this way with s/conform or only with coerce?

ikitommi09:10:24

not sure, please test 🙂

roklenarcic20:10:37

Hm, ended up not using coerce, because it doesn't use any conforms defined in specs.

ikitommi04:10:37

there is also decode, which does both. It's quite messy to try to implement transforming from a 3rd party library. https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FCLJ-2251#issue/CLJ-2251 would make things better.

practicalli_john11:10:21

@alexmiller just like to say thanks for all the work you and the team have done and continue to do with Spec. It is much appreciated. Thank you.

💯 5
4
jonathan16:10:34

Hi all. I'm trying out spec alpha2. I noticed I accidentally forgot to include a key in my schema, but s/select does not seem to validate that the key exists in the schema.

(s/def ::first string?)
(s/def ::last string?)
(s/def ::user (s/schema [::first ::last]))
; Should the select blow up? email is not in spec
(s/def ::create-user (s/select ::user [::last ::email]))

ghadi16:10:15

it doesn't care in spec1 either

alexmiller16:10:41

s/schema and s/select both support open maps (extra keys are ok), just like s/keys

jonathan16:10:41

I thought (s/select ::user [::last ::email])) is specifically saying I'm choosing to require ::email from the ::user schema

alexmiller16:10:43

whether you should get some feedback if s/select requires things not in the schema is a reasonable question, and not something we've talked about

jonathan16:10:51

Without any validation it seems like I don't need the schema at all except for documentation purposes

alexmiller16:10:06

several things - 1) supports gen in both schema and select, 2) basis for closed spec checking if you want to do that, 3) yes, docs

jonathan18:10:09

Thanks. I appreciate your work on spec!

jonathan16:10:30

(s/explain ::create-user {::last "Smith"})
#:user{:last "Smith"} - failed: (fn [m] (contains? m :user/email)) spec: :user/create-user
^^ Validates email perfectly fine

seancorfield16:10:40

One question that came to my mind yesterday as I worked through getting our entire code base switched to the latest Spec 2 and our full test suite passing: how will folks deal with the world when they want to use Spec 2 but they rely on a chain of dependencies where one of the downstream libs still depends on Spec 1. https://github.com/cognitect-labs/anomalies/blob/master/src/cognitect/anomalies.cljc is a good example: very small, focused lib that defines Spec 1 specs and could be depended on by any number of libs in your tool chain.

seancorfield16:10:06

To get our code base passing all its tests, I had to essentially recreate Anomalies' specs in my code, I had to create Spec 2 compatible versions of expectations.clojure.test (one I maintain), and worldsingles/web-specs (work maintains), since those are downstream dependencies using Spec 1.

seancorfield16:10:04

@alexmiller If Spec 2 becomes clojure.spec (no alpha), how will folks deal with a mixed Spec 1 (alpha) / Spec 2 (contrib) code base?

seancorfield16:10:50

In particular, in our case, we create specs in our own code (Spec 1 on master, Spec 2 on a branch) that incorporate the specs from Anomalies -- which I suspect is going to be a fairly common situation that folks get themselves into...

alexmiller16:10:10

I think it's possible to build an adapter layer from spec 1 to spec 2, but we're going to push off answering these questions till we have something good enough to migrate to

seancorfield16:10:41

Fair enough 🙂 Is there a specific outline of work left to do on Spec 2 at this point? Or is even that aspect not "baked" yet?

alexmiller17:10:28

there's a rough list at the end of my clojutre talk

taylor17:10:53

v nice talk

2
alexmiller17:10:13

* More on map forms
* Function specs, ret/fn specs, defn
* Metadata / doc strings
* Open enumerated sets
* Spec traversal

alexmiller17:10:21

that doesn't all necessarily have to be done to be at a release point if we feel it's additive from that point

alexmiller17:10:17

so the actual list may be shorter (or longer) but certainly nailing down 1,2, and probably 4

seancorfield17:10:35

Ah, and I just watched that the other day and forgot about that list. Sorry.

seancorfield17:10:07

What do you mean by "Open enumerated sets" in that context?

seancorfield17:10:54

And "More on map forms" means the unqualified, inline s/schema definitions?

alexmiller17:10:41

re open enumerated sets - using a set as a predicate #{:red :green :blue} is a different form of "closed" predicate. Often you want to say "these constants", but also other things later

alexmiller17:10:03

"more on map forms" = more work on the "map" form of specs

alexmiller17:10:52

as the maps currently in there are a first pass and we will likely change the form of some of them and probably automate the form<->map definition somehow

seancorfield17:10:46

Ah, the "AST" stuff... OK.

seancorfield17:10:34

Re: enumerated sets. That sounds interesting. I'm not sure I can come up with a use case right away tho', but I can see how set literals are currently "closed".

alexmiller18:10:33

options are a common case

1
kenny21:10:31

Anyone know of a library that lets you strip extra keys of a multi-spec/parameterized map? For example, if a map with :type set to :a requires keys :a1 and :a2, any additional keys would be removed.

dmarjenburgh11:10:55

You can use select-keys, but the problem is that you define the key set in two places. I had a similar problem and defined the schema as data {:req [::a1 ::a2] :opt [::a3]} . From there, I use the data to generate the spec and the keyset.

kenny21:10:04

It needs to be a bit smarter than that because multi-spec is used. I’d only like to select the keys that the multi-spec for a particular dispatch defines.

kenny21:10:29

Ideally it’d also be recursive.

hmaurer21:10:20

Hello 🙂 Possibly a silly question but what is the recommended way of working with specs whose data they model I want to serialise and deserialise to JSON with unnamespaced keys?

hmaurer21:10:04

so, in clojure the keys should be namespaced (since, as far as I understand, that’s the only way to work with spec), but in JSON they should not, and when deserialising JSON into a clojure datastructure I want to automatically namespace the keys based on some spec

hmaurer21:10:18

I thought s/conform would do this if I specified :req-un, but it doesn’t appear to

seancorfield22:10:54

:req-un and :opt-un are for unqualified key names -- but let you associate qualifiers for the matching spec names.

seancorfield22:10:30

(s/def :foo/bar string?)
(s/def :foo/map (s/keys :req-un [:foo/bar]))
(s/def :quux/bar int?)
(s/def :quux/map (s/keys :req-un [:quux/bar]))
;; so you have have {:bar "s"} in one context and {:bar 42} in a different context
Does that help @hmaurer?

hmaurer22:10:43

@seancorfield thanks! I think it does; initially I was thinking I would want some sort of way to deserialise {"bar": 42} (json) into something like {:foo/bar 42} in a spec-driven way, but I could just use req-un and have unqualified keywords everywhere

seancorfield22:10:36

Yeah, I'm not sure what to recommend for switching between "JSON" style maps and "internal" (qualified) maps. I think a lot of that is context-dependent / application-dependent so I'm not sure there's a generic approach that suits everyone.

seancorfield22:10:04

Given the specs, you can always get the form of the spec and get the list(s) of keywords used in the spec and you could create a generic way to call clojure.set/rename-keys or simply process the (JSON) map to qualify the keys yourself. Much will depend on exactly what your application needs and why you need to perform that transformation.

hmaurer22:10:08

Alright; I'll look into that. Thank you