Fork me on GitHub
Alex Miller (Clojure team)02:06:01

I’ve updated the guide for 1.9.0-alpha8 although there will be some additional updates


I have a question about spec'ing maps: lets say I have two keys that should follow the same spec, is the intended way to define proxy specs which I then use as map keys?

Alex Miller (Clojure team)11:06:03

You can register the map keys to point to either the same spec definition or to another registered spec. I think doing this will be common. One layer of registered base "types" and another layer of domain attributes.


- [changed] explain-data - output is now a vector of problems with a :path element, not a map keyed by path
Whats the reasoning behind this change? Could there be multiple problems referencing the same path?


@martinklepsch: yes, multiple problems at same path happens with collections (was already happening with ‘keys' which jammed them all into same pred, now doesn’t)


@martinklepsch: on your question from 11:20 today, do you mean like what i've done with ::pos-int here?


if you want to use a generic spec in keys without using it's key you need a proxy like this

Alex Miller (Clojure team)14:06:22

@robert-stuttaford: there is a pos-int? predicate now btw in core

Alex Miller (Clojure team)14:06:34

that ::types spec you have can now be accomplished with coll-of in alpha8 too: (s/coll-of ::type :kind set? :min-count 1)

Alex Miller (Clojure team)14:06:38

that version has the benefit of automatically gen’ing too

Alex Miller (Clojure team)14:06:07

And I think your ::spell, ::creature, ::planeswalker, etc can also be handled by s/merge now


how far, as a %, would you say the core.spec slated for 1.9 is?


it seems like you've still got a lot of meat on the bone, given all the stuff that gets added with every alpha


@robert-stuttaford: not a lot of big things - assertions, perf tweaks, reconciling regexes and vectors, some refinement of keys*, then details


then moving to infrastructure around test-running etc, the whole model implied by instrument+test which is different from current practice


On the Complexity and Performance of Parsing with Derivatives -- PLDI 2016

ghadi14:06:41 of matt might & co's work


fantastic, rich! very exciting set of tools you're building, here. fundamental, like the data-oriented approach and immutability are. going to take some time to soak in


i realise this may well be answered with 'maybe :-)', but i'm super curious how you plan to leverage spec in Datomic 🙂


@ghadi: thanks for the links


Datalog is an obvious one, as is transaction data. wondering how else it may improve the experience of programming with it


@robert-stuttaford: still TBD (but lots of obvious opportunities), first things first, and that’s a stable clojure.spec


I can say we are getting a lot out of using it in the development and testing of Datomic already


i can't think of a better way to dogfood it


the new stubbing support is something to check out - you can now take a version of your wire interface, spec it, and get a no-wire stub for free from spec


ah - if i understand you correctly, you mean inferring a spec from an existing data set?


if so, thanks .. i was planning to do this as a learning exercise!


no, let’s say you have a data-driven wire protocol that talks to a service - you spec the protocol, instrument its namespace saying :stub in the overrides, and then run tests of code that consumes the protocol. Now those tests don’t talk to a service at all but get checking of their payloads and generated returns


aside: I really like the syntax of (s/alt :foo x :bar y). That is something I didn't get quite right in pex, my vm-based PEG parser; I made capturing things too verbose. Definitely going to adopt that approach


you can selectively override gens to generate ‘stateful’ things like connections


@ghadi: yeah, the path system is something I think people resist initially, but it keeps paying dividends


man. i feel like i'm going to need a spec sabbatical.


so many new toys, but at such a fundamental place in the language.. it's going to touch everything!


curious, why do people resist the path system?


sometimes you don’t care about the labels or want to see them in the destructuring


but they let us talk about the structure of a spec all the way down, which is huge


not seeing anything about stub in there, yet


my plan for better dev-time experience in the PEG library was to provide a tracing mode on all the patterns attempted


since backtracking is likely in a failure, and most of the ways to handle it were through heuristics like longest match

Alex Miller (Clojure team)14:06:47

@robert-stuttaford: I think those are lagging an alpha or two, I will get them updated


oops, didn’t realize API docs were behind


alright, I'm running into an issue in alpha7

(s/def ::string  (s/or :string string?))
  (s/def ::myid ::string)
  (s/explain (s/and
              (s/keys :opt-un [::myid])
              (s/keys :req-un [::myid]))
             {:myid "asdf"})


Results in

In: [:myid] val: [:string "asdf"] fails spec: :strictly-specking.test-schema/myid at: [:myid :string] predicate: string?


it appears to be validating against the conformed value


I'll check against alpha8, in a bit, i have to port some stuff


and it's strange because this works

(s/def ::myid string?)
(s/explain (s/and
              (s/keys :opt-un [::myid])
              (s/keys :req-un [::myid]))
             {:myid "asdf"})


and that makes sense because the conformed value is just a string


‘and’ does flow conformed values - "Successive conformed values propagate through rest of predicates."


that may change for ‘merge’, which is what you should switch to for this application


there may also be and and and-> variants since sometimes you need this (e.g. when preceding pred is a regex) and sometimes not


great, looking at merge now


the above works with merge right now and inspection reveals that the conformed values aren't flowing right now.


@bhauman: merge does flow through conform, not explain (that’s broken)


but no reason for it to flow through conform, that will change. The big benefit of merge is that it will gen where ‘and’ cannot for this


my use case is taking an existing keys spec and constraining it by moving some keys into :req


I understand, and that will be common


also various intersections


just so i'm sure i understand what you're both talking about, are you talking about what i'm trying to do here?


so no flow is the right thing


where i want to gen (s/and (s/keys ...) (s/keys ...)) automatically?


seems like s/merge may be what i need to use instead


yes, that’s precisely the point of merge


it gens map unions


subject to the (temporary) flow problem @bhauman identifies above


i've just seen and it's going to be terribly tempting to rewrite everything!


i guess there'll be ways to add or remove namespaces from these maps?


this only affects reading, not the map


i.e. it’s not a lingering property of the map


ah, so if i wanted to add namespaces to a non-nsed map, i'm going to do so to the keys in the map myself


this is only reading/printing

seancorfield16:06:39 I hope that whatever comes out of this doesn’t make it harder to use other testing libraries (like Expectations) harder.


(but, yeah, great changes in Alpha 8… I updated java.jdbc to accommodate enough changes to get it running again but I haven’t dug deep into redoing the specs to leverage the new features)


An optional piece of data inside a tuple is not supported, I assume? (s/def ::my-tuple (s/tuple (s/? keyword?) symbol?)) claims ['foo] is invalid: [foo] fails spec: :workflo.macros.command/my-tuple predicate: (= (count %) 2)


My current "workaround" is to use (s/and vector? (s/cat :optional (s/? ...) :mandatory ...))


Which is ok, it does mean that you have to write a custom generator for it though.


Also, is there something like s/tuple for fixed-size non-vector sequences with a specific order of subspecs?

Alex Miller (Clojure team)16:06:05

You can use s/cat, s/? etc for things like this

Alex Miller (Clojure team)16:06:09

If you have optionality regex is definitely the right choice over tuple

Alex Miller (Clojure team)16:06:36

We are still looking at options to also enforce vector on regex in a good way

Alex Miller (Clojure team)17:06:25

s/and is sufficient as you saw but needs custom gen as you said


@alexmiller: Ah, s/cat didn't work because I forgot s/spec around it to make it a subspec and require the content of s/cat to be in a child vector or sequence.


my colleagues would like to create a single source of truth to generate both datomic schema and specs. Is this advisable? I was under the impression that perhaps spec would be written mostly by hand and not codegen

Alex Miller (Clojure team)18:06:31

seems low on the crazy scale to me


haha. thanks

Alex Miller (Clojure team)18:06:50

code is data and all that :)


Is clojure.spec.test/test meant to be used with clojure.test?


I guess more specifically, what is the idiomatic way to set up generative tests with clojure.spec is there an equivalent of defspec from test.check for clojure.spec?

Alex Miller (Clojure team)20:06:05

fully utilizing the instrumentation and test facilities in clojure.spec.test is a different kind of process from the existing clojure.test and I don’t think we will provide a linkage like defspec.

Alex Miller (Clojure team)20:06:57

we have more to say about this and I expect we will have more guidance to provide before 1.9 is released


Gotcha, is it expected to be a developer concern? Or will there be additional functionality added to integrate it with clojure.test? The return seems more data-centric than clojure.test.

Alex Miller (Clojure team)20:06:57

I don’t currently expect anything that integrates it with clojure.test. The returns are more similar to the results from test.check as that’s what we’re actually invoking.

Alex Miller (Clojure team)20:06:17

there are levels of question here - some around what’s provided in core, another level around runners, and another level around build tooling


I suppose the core of my question is around whether or not clojure.spec.test/test will integrate into the existing test tooling or is it expected that new test tooling will need to be built? instrument seems to fit into the current ecosystem but it seems like clojure.spec.test/test will require new tooling.


Is there a difference between (s/def ::type vector?) and (s/def ::type (coll-of ::s/any :kind vector?)) ?


When a required key is missing from a map the error looks like this:

{:pred [(contains? % :to) (contains? % :subject)],
   :val {},
   :via [],
   :in [],
   :path []}
Is there a way to get an easier to parse description that will allow me to (programatically) figure out which keys are missing from a map?


You can get that data structure with s/explain-data


@tyler: if you want to run generative tests and have report of it in clojure.test you can use the clojure.spec.test/test like this: (is (true? (-> (clojure.spec.test/test 'my.ns/some-fn) :result)))


Thanks @wilkerlucio. I had been playing around with that a little bit but the output isn’t very informative for failures for that. I think I’m going to go the route of experimenting with developing my own runner. Just wanted to make sure I wasn’t duplicating work/effort.


@angusiguess: right, what I pasted is part of what s/explain-data returns, was just wondering if there's a better way to get a list of missing keys


@tyler: try using something like this:


not super pretty, but might be a good start 🙂


Thanks I’ll give that a go


Is there an explanation somewhere of how the sampling of every works? I'd like to get a sense for the probability that an error is detected if I switch from coll-of to every.

Mathieu Corbin21:06:20

Hello, i play with clojure-spec and i don't understand this :

Mathieu Corbin21:06:09

Why explain returns Success ? The second condition in the "s/and" should return false with a string parameter


@mcorbin I believe the lambda pred is always invoked with a vector, i. e. [:string "Hello Clojurians"]


@mcorbin: theres a discussion of this from earlier today ...


see my posts above

Mathieu Corbin21:06:47

@leongrapenthin: you win, thanks @bhauman i will look, thx

Alex Miller (Clojure team)21:06:52

@puzzler: if you mean “how it samples” - no there intentionally is not, as it may change in undefined ways in the future

Alex Miller (Clojure team)21:06:11

all you should count on is that it will sample no more than *coll-check-limit* elements


@tyler: FYI, we use Expectations to drive test.check stuff and ended up with a pattern like this /cc @wilkerlucio

(expect (more-of {:keys [result num-tests]}
                 true result
                 100 num-tests)
        (tc/quick-check 100
                        (prop/for-all [s gen/string]
                                      (= 0 (ip->long (str/replace s #"[0-9]" "."))))))


Suggestion: Right now it looks like every and coll-of check for distinctness with (apply distinct? ...). Would recommend (or (set? ...) (apply distinct? ...)) to short-circuit when a set is passed in to a function that expects a collection of distinct items.