clojure-spec 2018-07-04 | Slack Archive

generative tests with specs take really long to run. what are some tips to try and reduce the testing time?

This can depend on your specs. Recursive specs can be especially costly. Can you give an example that’s taking a while to run?

ackerleytng23:07:56

it's not a recursive spec though

seancorfield17:07:27

@ackerleytng I don't feel generative tests should be run automatically as part of your (fast) unit test suite. They can/should be run separately or as part of CI for example. Because generative testing can definitely take a while.

dominicm17:07:11

Rich has mentioned a number of times that generative tests should only run when the relevant code changes. I'm sure he'd be very happy if someone built that for him.

Alex Miller (Clojure team)18:07:03

I have built the hard parts of it, just needs a lot of productization

👍 8

dominicm21:07:24

Do you mean to suggest that it's going to be a product that's sold or branded? Or just that it's rough? If it's rough, is it in such a state that the community might learn much from it by seeing it and seeing where they can take it?

Alex Miller (Clojure team)21:07:00

It will be a contrib library. It’s not at a state where I’m ready to publish it yet, both for design and impl

Alex Miller (Clojure team)21:07:11

We may end up sharing some code with codeq 2 also, which is similarly unfinished. Just a lot of stuff to shake out and none of us have time to work on either atm

gnl20:07:05

@dominicm Shameless plug – https://github.com/gnl/ghostwheel runs gen-tests per namespace on hot-reload which is effectively just that (or close to it), with the option to make re-rendering dependent on successful test completion.

dominicm21:07:43

I hadn't appreciated that's why it did those things. Very impressive!

Alex Miller (Clojure team)21:07:02

That’s great but what I’ve been working on is code analysis at the function dependency level

gnl21:07:09

Function-level granularity would be even better for this of course – looking forward to it. Is this going to be a core thing or a third-party library?

gnl21:07:36

@dominicm It's really just a fancy (run-tests) but it gets the job done. 🙂

gnl21:07:42

The important thing is that if you're working on layout/view stuff where quick hot-reloading is most valuable, it doesn't run any expensive event handler gen-tests, etc.

gnl21:07:19

@U064X3EF3 Oh, I just saw the other thread where you answered that, nevermind.

ackerleytng23:07:25

@seancorfield Thanks! I'm writing my own custom generators, could that contribute to slower testing?

ackerleytng23:07:26

even with overrides like {::foo (fn [] (gen/return <something pre-generated>} does not improve testing speeds much. why would this be?

ackerleytng23:07:42

gen/return probably doesn't run the generator at all, right?

gfredericks23:07:43

well it is a generator, but everything it does is trivial, so it by itself should never make anything slow

2018-07-04

Channels