Fork me on GitHub
#clojure-spec
<
2016-07-20
>
bbloom06:07:34

those generated emails are amusing 🙂 i’ve noticed that generated tests tend to have pretty wacky looking data - i guess that’s good for tests, but i can’t help but wonder: what about “example data”? anybody taken a crack at that? would be cool to lorum ipsum up a full database based on specs!

rickmoynihan08:07:31

Sometimes specs seem to dereference themselves... e.g. (s/def ::k (s/keys :req [::foo ::bar])) dereferences ::foo ::bar but I've noticed that for example (s/merge ::k ,,,,) blows up... instead you need to bind :k to a real var... i.e. it won't dereference the keyword into the specs value. I'm just wondering if this is by design? And what the reason is... and how to know what to expect

rickmoynihan10:07:58

actually this does make a lot of sense... sorry being stupid... keys/`:req` requires knowledge of the keys themselves... and clearly giving keywords symbol semantics would be insane

rickmoynihan10:07:46

I guess you have to rely on the doc strings and specs specs... to know the difference between a spec and a keyword... I guess it's kinda like the distinction between quoted and unquoted symbols... I've only just got started with spec - so knowing when to use which isn't entirely clear to me

mpenet10:07:33

is there a version of #(partial instance? %) in core or spec?

mpenet10:07:17

apparently not

mpenet10:07:58

I guess this could even simply be another arity of instance?, given how common this is

rickmoynihan10:07:59

yeah I've written that one more than a few times

mpenet10:07:31

ex: (instance? Cluster), which would be a partial on instance?/2

mpenet10:07:11

@alexmiller: would you consider something like this, I'd gladly send a patch for it ?

Alex Miller (Clojure team)11:07:23

No thanks, has been considered by Rich

gfredericks12:07:57

are the lower-level details of clojure.spec, in particular the implementation of s/keys, meant to be public so that users/libraries can use them to write new kinds of specs?

gfredericks12:07:15

it might just amount to implementing a protocol, in which case the question is if the protocol is meant to be public

Alex Miller (Clojure team)13:07:58

I would say that you should consider all implementation details of spec to be just that - subject to change without notice

Alex Miller (Clojure team)13:07:22

spec is meant to provide a relatively complete set of compositional parts, extensible (at the unit level) via predicates

Alex Miller (Clojure team)13:07:10

which is not to say that extending via the protocol is necessarily wrong, just that we’re not committed to maintaining that api, so use at your own risk (like any other Clojure “internals")

Alex Miller (Clojure team)13:07:41

the public api is what’s in the api docs

Chris O’Donnell13:07:38

@alexmiller: is there a reason with-gen does not pass along the unformer function to its return value?

Alex Miller (Clojure team)13:07:08

don’t know - example?

mpenet13:07:43

are there improvement coming to prevent having to manually create non existing kw namespaces?

Chris O’Donnell13:07:02

;; conform works, unform fails, gen works
(def date-spec
  (s/with-gen
    (s/conformer date-conformer date-unformer)
    (fn [] (s/gen inst?))))

Alex Miller (Clojure team)13:07:19

@codonnell: thx, I’ll pass along to Rich

mpenet14:07:38

and next in my wish list of stuff that will not happen for sure is a partially applied version of satisfies? 😆. Probably an opportunity for a lib with helpers like these

Chris O’Donnell19:07:45

that would be a fantastic name

gfredericks19:07:12

well I'm doing it right now

soloist19:07:48

any idea how to write a spec to check an aggregate value of a collection? eg: (s/valid? (= 5 (partial reduce +))) or something like that.. nvm mybad: (s/conform #(= 5 (reduce + %)) [2 3])

gfredericks19:07:26

lvh: ^ I finally created schpec

lvh19:07:33

awesome!

lvh19:07:43

I just saw an email notification from the corner of my eye

lvh19:07:20

"whatever other feature you miss from plumatic/schema” 💯

lvh19:07:18

it’s distracting to see how many layers deep you can go — I was trying to do something useful with swagger specs and now I’m explaining to clojure.spec how json schema works so I can generate json schemata so I can generate specs from them so I can generate instances of those specs and then see if they match the original json schema

lvh19:07:23

it’s specs all the way down 🙂

gfredericks19:07:20

a shed to put bike sheds in

bhauman19:07:51

I've got some bikes for that shed ...

Alex Miller (Clojure team)20:07:27

@jjcomer: read the instrument docs carefully - "Instruments the vars named by sym-or-syms”, "Opts for symbols not included in sym-or-syms are ignored.”, etc - I think that right away rules out your -3 and -4 examples. I’ll look closer at -2. Some of this stuff really makes most sense when instrumenting a ns or all and passing an opts map that sets up the env you want.

Alex Miller (Clojure team)20:07:45

for -3, if you replace

`inc-it
with
[`inc-it `print-it]
I think that works

Alex Miller (Clojure team)20:07:27

for -2 and -4, interestingly if you do that instrument and check outside a defn, just at the repl, they work

Alex Miller (Clojure team)20:07:02

(that is, don’t print)

Alex Miller (Clojure team)20:07:27

which implies things about compilation and may be a problem

Alex Miller (Clojure team)20:07:29

yeah, when the compiled class is loaded, the var has not yet been altered by instrument and that’s what is loaded into the class constants and used at runtime - the instrumented version of the var is not used

Alex Miller (Clojure team)20:07:09

nah that doesn’t make sense - check takes the symbol, not the var and resolves at runtime

kendall.buchanan20:07:28

I suspect I’m doing something wrong, but I’m reliably getting errors like this:

kendall.buchanan20:07:31

java.util.concurrent.ExecutionException: java.lang.ClassCastException: app.business.klass_test$eval75458$fn__75459 cannot be cast to clojure.lang.MultiFn, compiling:(clojure/test/check/clojure_test.cljc:95:1)

kendall.buchanan20:07:41

From stest/check.

kendall.buchanan20:07:38

It’s typically when I restart the REPL and try to run check from my test files (without first navigating over and evaluating the functions it calls). Note I’m requiring the namespaces first.

kendall.buchanan20:07:35

There isn’t a danger to calling stest/check within deftest is there?

Alex Miller (Clojure team)20:07:39

is lein or other build tooling involved?

Alex Miller (Clojure team)20:07:52

lein hacks up a lot of the clojure.test stuff

kendall.buchanan21:07:18

It’s certainly possible – I’m using weavejester’s eftest.

Alex Miller (Clojure team)21:07:40

does it happen if you don’t?

kendall.buchanan21:07:07

It happens if I simply eval the form, yes...

kendall.buchanan21:07:41

But it’s a fairly large project – it’s possible something else is intruding.

Alex Miller (Clojure team)21:07:12

eftest is definitely also mucking with clojure.test/report

kendall.buchanan21:07:23

Yeah, it’s a touch thing – if I go to the test namespace, require everything, and run a single form, it works.

kendall.buchanan21:07:39

But if I restart the REPL, go to the namespace, eval everything all at once, it falls apart.

kendall.buchanan21:07:32

K, thanks. Helps to know it’s likely in my environment. I’ll report back when I find it.

kendall.buchanan21:07:43

But yeah, only happens on check.

Alex Miller (Clojure team)21:07:28

both eftest and check are bashing the same clojure.test hook for reporting and that seems to be in the realm of the error you’re seeing

Alex Miller (Clojure team)21:07:50

so I’m going to suspect that combination as the most likely probable cause

Alex Miller (Clojure team)21:07:51

I guess not “cause”, I don’t understand why the two together fails but I suspect that’s involved

kendall.buchanan21:07:17

Okay, great insights. I’ll start by disabling eftest.

Alex Miller (Clojure team)21:07:43

if you can get a reproducible case that fails with both, feel free to file a jira and maybe one or the other can be made more tolerant/harmonious

kendall.buchanan21:07:46

Disabling eftest altogether had no effect:

kendall.buchanan21:07:21

That’s after taking any reference to eftest out of project.clj, and running lein test on its own.

kendall.buchanan21:07:38

I’ll keep poking around.

Alex Miller (Clojure team)21:07:19

are you explicitly requiring clojure.test.check.clojure-test?

kendall.buchanan21:07:54

Made myself a couple helpers to simplify tests with check.

Alex Miller (Clojure team)21:07:21

unless you’re loading it, it’s not clear to me why clojure.test.check.clojure-test is getting called at all. nothing in core calls it. Your definition above won’t call it, yet you are compiling it. are you AOT’ing?

kendall.buchanan21:07:43

Seeing if I can create a reproducible lein project...

kendall.buchanan21:07:25

Cool, just reproduced it.

kendall.buchanan21:07:38

Very well could be lein, cause I have a barebones example.

kendall.buchanan21:07:05

I’m happy to post to JIRA too if you feel like it’s a legit reproduction.

Alex Miller (Clojure team)21:07:22

gimme a sec to look at it

Alex Miller (Clojure team)22:07:29

@kendall.buchanan: looks like this has nothing to do with spec per se

Alex Miller (Clojure team)22:07:39

clojure.test.check.clojure-test assumes that clojure.test/report is a multimethod (which it is in core) and modifies one of it’s cases - this is loaded as a side effect when clojure.test.check is loaded via gen via spec.test/check

Alex Miller (Clojure team)22:07:06

however, Leiningen monkeypatches core and replaces that clojure.test/report function with something that’s not a multimethod

kendall.buchanan22:07:42

I noticed the project threw off another error about gen being missing… until I added the test.check dependency.

Alex Miller (Clojure team)22:07:43

so lein test and test.check.clojure-test appear to be incompatible afaict

Alex Miller (Clojure team)22:07:58

you can turn off the monkey patching in lein though

Alex Miller (Clojure team)22:07:02

with :monkeypatch-clojure-test false in your project.clj

kendall.buchanan22:07:30

Interesting. I’m assuming lein has documented the trade-off of turning it off?

Alex Miller (Clojure team)22:07:03

I think you then lose “lein retest"

kendall.buchanan22:07:35

Not sure I’d ever have known to look to lein for this… is it appropriate to introduce this into spec docs as a “gotcha”? (just curious how you guys deal with things like this…)

Alex Miller (Clojure team)22:07:39

maybe, I was not aware of it before. the master version of test.check actually doesn’t load clojure-test anymore, so you wouldn’t even see this come up unless you included it explicitly

Alex Miller (Clojure team)22:07:11

I gotta go, but will think about it more tomorrow

kendall.buchanan22:07:24

Okay, I am loading explicitly in my project as well… good point.

kendall.buchanan22:07:35

@alexmiller: Thanks much for spending so much time on that.

Alex Miller (Clojure team)22:07:55

thx for the good and simple repro! that helped enormously

Alex Miller (Clojure team)22:07:46

@jjcomer I haven’t forgotten yours either - still looking at it

gfredericks22:07:09

alexmiller: I've been using lein-test with test.check.clojure-test for years, so I assume it's something subtler than a full incompatibility. I wasn't aware of this before, no, so I'll look into it

kendall.buchanan22:07:35

@alexmiller: One final note: turning off monkeypatching fixed it. Turning eftest back on broke it again.

kendall.buchanan22:07:01

It’s a great start, though, to at least know where it breaks down.

gfredericks22:07:44

kendall.buchanan: I'm trying to reproduce it now

gfredericks22:07:04

kendall.buchanan: any tips beyond "using test.check and lein test"?

kendall.buchanan22:07:36

gfredericks: Did you clone the repo above?

gfredericks22:07:53

oh no, didn't see it

kendall.buchanan22:07:57

I think the odd thing about stest/check, though, is it requires test.check, explicitly.

kendall.buchanan22:07:09

Otherwise you get errors saying generators are missing.

gfredericks22:07:24

what do you mean by "explicitly"?

kendall.buchanan22:07:33

Added to the list of dependencies.

kendall.buchanan22:07:48

It’s a function that can’t run without the user having to introduce another dependency.

kendall.buchanan22:07:03

@seancorfield seems to have identified the same thing in a slightly different context: http://dev.clojure.org/jira/browse/CLJ-1936

gfredericks22:07:20

stest/check is explicitly about generative testing, right?

kendall.buchanan22:07:39

Yes, it tests function specs.

gfredericks22:07:51

I'm not sure how it could make sense to run that without test.check available, given the underlying "test.check is optional" design decision

kendall.buchanan22:07:53

(It’s the only way to test the return values of functions, afaik.)

kendall.buchanan23:07:05

Okay, I figured as much.

gfredericks23:07:42

kendall.buchanan: I still don't quite understand how this happens (or rather, why it doesn't happen all the time), but I can at least tell you it's fixed on test.check master

gfredericks23:07:06

I'm hoping to make a new release soon

kendall.buchanan23:07:14

Wow, that’s fantastic.

kendall.buchanan23:07:46

That’s great news, thansk.

gfredericks23:07:11

oh actually I get it

gfredericks23:07:27

I think lein does the monkeypatching after all the requires get loaded

gfredericks23:07:49

so this is only a problem if you (or something) loads the test.check.clojure-test namespace later (at runtime)

kendall.buchanan23:07:57

@gfredericks: Just curious… if stest/check is all about generative testing, why is it in clojure.spec.test? Does it not make more sense to move it to test.check (making one aware of the other, but not in both directions)?

gfredericks23:07:37

kendall.buchanan: that might be possible; but spec might be too tied together with test.check to make that practical

gfredericks23:07:05

I'm not planning to investigate it anymore (I considered closing the issue immediately after creating it) until I hear that it's a problem for someone on test.check master somehow

gfredericks23:07:18

I figured I'd create the ticket to document the problem for future historians though