Fork me on GitHub
#clojure-nl
<
2019-11-27
>
bbss03:11:28

@dmarjenburgh property based testing, cool. The usual suspects would be clojure.spec -> test.check -> core.logic

bbss03:11:48

But haven't looked into any specific tools. Might be a bunch more nowadays.

yannvahalewyn04:11:32

I’ve found that property based testing has a great value proposition, but comes at the cost that finding properties in real life systems that would add more value than their maintenance cost is an unsolved problem. It’s really shines sometimes, but most often it doesn’t without the appropriate time investment. Would love to be proven wrong.

yannvahalewyn04:11:10

When I watch talks by John Hughes I can see how incredibly powerful it is, but in every real life example he mentions it feels like he spent months developing the test model which makes me feel like this is more suitable for high budget and risk projects and not for rapid and agile development

yannvahalewyn05:11:49

Small segway into generative testing: Just an FYI: I once spent a couple of days on such a model and in retrospect I think I might’ve wasted my time a bit, it caught some bugs but in comparison those would’ve caught 5 minutes after a deploy because we have good monitoring. So the question really is are those couple of minutes in production bugs worth days of development. Answer might be yes depending on the project, but most oftenly no.

yannvahalewyn05:11:47

I might also be an idiot

yannvahalewyn05:11:24

which is statistically probable

dmarjenburgh08:11:30

I’m looking to use it for integration testing, which involves a lot of state. I guess a better term is simulation testing, which Michael Nygard calls it. There are a few tools like https://github.com/reifyhealth/specmonstah and https://github.com/candera/causatum which are useful.

erwinrooijakkers14:11:35

Also got simulant

erwinrooijakkers14:11:06

Used that to specify randomized test paths

dmarjenburgh08:11:15

I’ve watched the talks on simulation testing (https://www.youtube.com/watch?v=N5HyVUPuU0E), stateful property testing (https://www.youtube.com/watch?v=xw8ZFU8CGdA) and practical generative testing patterns (https://www.youtube.com/watch?v=zjbcayvTcKQ). They all have their own toolkit and it is indeed a lot of work. But they have roughly a similar pattern: • Generate actions using a state machine model (store these) • Apply the actions against your system (should be repeatable) • Apply the actions against your model (should be repeatable) • Check whether model and system are consistent If the actions are high level (they are actions users perform on your system), then they shouldn’t change much and the initial investment may be worth it.

yannvahalewyn10:11:28

@dmarjenburgh Thanks for the resources!

bbss14:11:11

@yannvahalewyn agreed, that not every piece of code would benefit from a lot of of in/output testing.

bbss14:11:41

I'm interested to one day do a kind of "user driven development" where user is a piece of code modeling all possible actions to take in a UI. Modeling what the user could input and in what order, I'm sure you could run into a lot of errors that way.

yannvahalewyn16:11:03

@bbss Do you mean like using markov chains to describe possible state transitions, and using generative testing to generate random possible chains and asserting properties on the system?