Fork me on GitHub
#clojure-spec
<
2019-01-14
>
devth06:01:30

i wonder why aren't specs first class things we can define literally and pass around, and why they need their own special registry when we already have namespaces and vars? :thinking_face: i'm playing with building up specs in an automated way (e.g. reducing a data structure into a spec representation). looks like i'm gonna have to s/def things along the way. isn't that some kind of PLOP? maybe this is something that will be improved in future version 😄

mpenet09:01:47

The latter it seems

mpenet09:01:49

Why another registry, I believe to not clutter vars/nses more and make their potential evolution separate?

devth15:01:33

then why not have a registry for all atoms? and another for all fns? and on and on. 😂

mpenet15:01:57

There are already issues with vars initialization at startup right now, I guess that's not to make it worse among other thing. Then it's a "private" thing, we never get exposed to the fact it's separate

devth15:01:57

i see. 😞

mpenet09:01:31

We ll see with alpha2 I guess. The posts from @alexmiller about the ongoing work on this are quite interesting

👍 5
mpenet09:01:48

they don't reveal much about these 2 questions in particular, but soon enough it might

borkdude09:01:34

@devth if you can give an example, we can see if it can be improved using the current version of spec?

misha11:01:55

@devth there is s/spec which gives you spec w/o being registered, which might be useful for you if you build specs dynamically, use, and throw away right away.

Alex Miller (Clojure team)13:01:31

there are no plans to change the registry aspect of spec

✅ 5
Alex Miller (Clojure team)13:01:42

all of the programmatic construction aspects will be different in spec 2

flyboarder15:01:03

@alexmiller could you elaborate more on that? Will existing dynamic specs break?

cjsauer16:01:03

Does there exist a flavor of s/merge for disjunctions, i.e. s/or? I have a base set of disjunctions (A or B) that I’d like to be able to extend in certain contexts (base or C or D).

borkdude16:01:27

@devth Isn’t this the flaw of current spec that Rich spoke about in his recent Clojure Conj talk?

devth16:01:05

i'm not sure i've seen his latest conj talk. i'll have to look it up

devth16:01:25

oh, i have seen part of that one.

devth17:01:01

still curious about the place vs value oriented nature of spec :thinking_face: any thoughts?

borkdude17:01:22

I recommend watching the talk.

devth17:01:41

k, i'm 30 min in. still watching 🙂

borkdude17:01:02

let it sink in on the hammock 😉

🤯 5
devth18:01:04

taking notes

devth00:01:14

Big ideas: 1. separating out schema (shape) and selection (optionality). cool stuff! - it's still not "just data". why the departure? everything in Clojure is just data, and for good reason. the entire core lib is built around manipulating data. it's the best part of clj. - it's still place oriented, at least from what i gather so far 😞 2. Better programmatic manipulation of specs. great! but again: why not make it data? then programmatic manipulation comes with. Imagine if the schema for:

{:weather {:weatherbitio {:key "xoxb" :default {:zip "98104"}}}
   :command {:prefix "!"}}
Was simply:
{:weather {:weatherbitio {:key string? :default {:zip string?}}}
   :command {:prefix string?}}
There's elegance in mirroring the data structure you're specifying, kinda like how Datomic Pull queries mirror the data you get get back. :thinking_face:

borkdude16:01:55

or do you mean “why do I have to gives names to each sub-spec, can’t you inline them?” ?

borkdude16:01:07

it may be related to each other

devth16:01:21

yeah, both i think. i want values

borkdude16:01:15

future spec will allow you (I think!) to spec the whole schema and then define selections on them, so you don’t have to name each selection.

borkdude16:01:27

I’m not entirely sure if this maps to your issue with spec, but just my 2cts

devth16:01:51

pretty sure that'd help!

devth16:01:13

any word on timeline for the next version of spec?

cjsauer16:01:58

Whenever I’ve seen this asked, the short answer is “when it’s ready” 🙂

devth16:01:18

haha. fair

Alex Miller (Clojure team)16:01:57

I’m working on it every day

Alex Miller (Clojure team)16:01:13

hard to say when the next point will be where it’s useful to look at but it won’t be “a” next version - it’s going to be a series of releases

devth17:01:31

so to be clear: currently there isn't necessarily a better way to achieve what i'm doing? i have to dynamically s/def a bunch of stuff?

borkdude17:01:10

@devth there is a lib called spec-tools which may help you accomplish this, but I expect breaking changes when new spec comes out

borkdude17:01:23

I haven’t used it in anger myself

devth17:01:18

looks interesting, thanks.

borkdude17:01:07

you may lose the ability to generate data this way, I’m not sure

borkdude17:01:13

@devth o wait, maybe it was this: https://github.com/metosin/spec-tools#data-specs (the README is so long ;))

devth17:01:39

> Just data, no macros 💯

devth17:01:50

long READMEs are nice 🙂

Alex Miller (Clojure team)17:01:11

most of things spec-tools is built on are going away, but there will be one or maybe even two alternative paths to this goal

✅ 5
borkdude17:01:19

I’m not sure if I agree. Do one thing well also has benefits 🙂

devth17:01:24

not familiar enough with spec-tools to have an opinion on whether it should have been multiple libs. but long searchable READMEs with lots of examples are always nice.

devth17:01:51

i think i'll play with spec-tools for now. if things change i can always port to the new way

borkdude17:01:09

glad I could help 😛

👍 5
cjsauer17:01:08

(create-ns 'my.really.long.ns)
(alias 'mrln 'my.really.long.ns)

::mrln/attribute
Is this trick for shorter namespaced keywords likely to get me scolded by experienced clojure spec devs? Is it better to formally create the ns file even if it’s largely empty?

borkdude17:01:43

@cjsauer there is something like this in spec itself, so I think it’s ok

borkdude17:01:55

@cjsauer if you’re writing portable code, be aware that this may not work on CLJS

borkdude17:01:27

@cjsauer I tried moving this to a proper ns, so CLJS could use the alias as a namespaced keyword as well, but that was rejected: https://dev.clojure.org/jira/browse/CLJ-2421

cjsauer17:01:24

Ah interesting, thank you. in-ns is a bit better. Which part of this does cljs not like? I am hoping to use these specs on both server and client.

borkdude17:01:51

@cjsauer in-ns simply doesn’t work in CLJS.

borkdude17:01:08

it does, but only in the REPL.

cjsauer17:01:29

Bummer…does the same go for create-ns?

borkdude17:01:39

(I think so, gonna check now)

borkdude17:01:54

(ns dude)

(in-ns 'foo)

(defn foo [x])

(in-ns 'dude)

(defn dude [x])

$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.439"} org.clojure/test.check {:mvn/version "RELEASE"}}}' -m cljs.main -t node -c dude
WARNING: dude is a single segment namespace at line 1 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 3 /private/tmp/repro/src/dude.cljs
WARNING: Use of undeclared Var dude/in-ns at line 7 /private/tmp/repro/src/dude.cljs

cjsauer17:01:46

Shoot…so then portable code will have to use the qualified keyword everywhere at the moment?

borkdude17:01:04

the fully qualified one yes

cjsauer17:01:37

I may actually fall back to formally creating the namespace file for now, even if its mostly empty. Keyword fatigue is real…

borkdude17:01:37

it’s not a huge pain. you could make a function that creates the keyword if you want to save characters

borkdude17:01:00

@cjsauer we also have that in the app I’m working one, several almost empty ns-es for this…

cjsauer17:01:03

Once CLJ-2123 is patched, I imagine it should be quick work to refactor

borkdude17:01:41

cool. hopefully CLJS gets this too

cjsauer18:01:49

What is the philosophy behind spec’ing attributes that are intended to be used as references to other entities? For example, I might have ::company/employees which models a one-to-many relationship. How would I spec this attribute? My intuition is to use something like (s/coll-of ::employee), but the issue here is that I now have to define some minimum key-set that is ::employee, which is a spec that I think is largely contextual. Further, there may be contexts where (s/coll-of (s/tuple #{::employee/uuid} ::employee/uuid)) are valid ::company/employees, e.g. datomic idents. Ultimately what I think I’m wondering is: can reference attributes only be spec’d within a given context, or at specific boundaries (e.g. fdef)?

cjsauer19:01:31

I think my confusion stems from the fact that a :db.valueType/ref attribute can take on lots of different forms throughout an application, and so it’s difficult to spec. For example, it could be a raw entity id, a {:db/id 000} map, a [:db/id 000] ident, some kind of custom [:employee/uuid #uuid "000"] ident, or it could be a fully formed employee map…all these can values live under the same ref attribute at one time or another.

favila20:01:15

I think only spec-ing the pull form (ie what you would get from a pull)

favila20:01:49

Those other variants are all part of the Tx map DSL

favila20:01:16

This just shifts the problem around though

favila20:01:53

I think the TX spec would only spec the coll not the keys individually

cjsauer20:01:23

>I think only spec-ing the pull form (ie what you would get from a pull) I’ve been playing with a similar strategy. This way you can still make good use of generators without the concept of a database getting involved.

cjsauer20:01:12

>I think the TX spec would only spec the coll not the keys individually @favila by this do you mean something like (s/coll-of (s/keys))?

favila20:01:05

Coll-of whatever a tx is

favila20:01:58

Coll of either Vecs with entity ref (= 2-tuple, long, or keyword) first item (Tx fns and raw add/retract) or tx maps; which are maps with entity ref keys and Tx map or col-of-txmap or entity ref or non-ref value or coll of non-ref value)

favila20:01:07

So pretty open...

cjsauer20:01:37

In my tx function fdefs I’ve been specifying the :ret much more tightly. I’m less interested in spec’ing the :ret of any tx function, and more interested in spec’ing this specific tx function.

cjsauer20:01:49

I’ve been staring at the code more, and I think what’s troubling is that when I want to specify the form of some ref attribute in general, it can’t really be done…the question of when is inescapable so to speak. Because if I specify the pull form for all of my ref attributes, now I’ve bound the keyword :company/employees to one specific context: the fully realized one. I can’t use the name :company/employees in other contexts in which it might make sense, because spec has already registered the “one true form”. Does this make sense, or am I complicating the issue? :thinking_face:

cjsauer20:01:20

For example, a tx function might return a collection that includes a map with the :company/employees keyword in it, but it may not conform to the pull form. It may be a collection of idents instead, meaning it references pre-existing entities.

cjsauer20:01:19

Because I’ve already spec’d it to conform to the pull form, I can’t use that keyword again in a new context…

lilactown20:01:47

Assuming it's a map using s/keys, you could make everything optional

cjsauer20:01:48

I had the same idea, and it works okay. I think the downside is that it forces you to resort to custom generators pretty much everywhere…

lilactown20:01:06

But this is the exact problem Rich stated in his last conj keynote Maybe Not

lilactown20:01:25

There's not a great solution for it in spec

cjsauer20:01:43

>But this is the exact problem Rich stated in his last conj keynote Maybe Not I think so as well. Pretty much anywhere that a map has to be specified, this problem will arrive. The concept of “entity” is still difficult to grasp in my opinion. “Entity” doesn’t seem to become concrete until some data arrives at a boundary/function.

favila21:01:57

@cjsauer you are exactly right about the problem

favila21:01:08

I don't think this is the same problem Rich has acknowledged

favila21:01:48

spec has a strong opinion that the keyword of a map is (essentially) describing the type of its contents

favila21:01:17

but there are cases where that is not true and the keyword is really a structure rather than a type tag

favila21:01:42

one is DSLs: e.g. tx map dsl, or even the pull expression dsl

favila21:01:14

in those cases the key of the map is a structural tag, not a type tag

favila21:01:59

and foreign system's data routinely make their map entries contextual to the type of thing they are in

favila21:01:20

both these use cases I have found are bad fits for spec; the only workaround is predicates

favila21:01:29

or more keys and lots of key renaming

favila21:01:15

"structural tag" is wrong

favila21:01:34

what I mean is they are like structurally-expressed arguments to some function call implied by the dsl

favila21:01:37

{:my/attr [:db/id :db/ident]} is not expressing a :my/attr value, it's telling you to do something to a :my/attr value in some different context

favila21:01:09

they can't be interpreted apart

favila21:01:24

so it's the type of the entry itself that matters rather than the type of the value

favila21:01:08

so, maybe multi-spec on the key as the dispatch value is getting towards the right answer?

cjsauer21:01:29

>I don’t think this is the same problem Rich has acknowledged @favila I’ve just rewatched Maybe Not, and I think you’re right, it’s not the same issue. It is exactly the type/structural conflation that you’ve identified.

favila21:01:12

if anything, rich is moving even further away from keywords having any contextual meaning

favila21:01:28

s/keys at least let you attach predicates to the whole

cjsauer21:01:42

He actually also mentions that :ret in function specs is starting to smell, and it’s in those :ret specs that I encountered this issue. So that may be telling…

cjsauer21:01:49

If I ease up on trying to “nail everything down” with specs, as Rich puts it, it may alleviate some of these limitations

borkdude21:01:43

What did he say about ret specs again?

cjsauer22:01:30

He touches on it at around this point in the video: https://youtu.be/YR5WdGrpoug?t=3174

borkdude22:01:04

what he’s saying there is that ret specs don’t tell you much, you almost always want fn specs. so he wants to refine fn specs (make them easier to use?)

cjsauer22:01:43

True…I think the root of it is definitely this “contextual keys” idea (keywords have different specs in different contexts). What might be cool is the ability to define a “disjunctive schema”:

(s/def ::widget (s/or* :A string? :B pos-int?))
and then use a function similar to Rich’s proposed s/select function in order to mask the contextually relevant predicates, e.g. (s/select* ::widget [:A]) would mean “in this context, only condition :A can match (anything other than a string would result in error)“.

cjsauer22:01:09

Almost reminds me of tools.deps’ alias flag clojure -A:a:b:c