Fork me on GitHub
#testing
<
2016-01-29
>
paulspencerwilliams08:01:45

@donaldball: @nberger apologies for not replying, I had just left a meetup when you replied and so was enjoying a glass of wine in the pub. I’ve watched several videos, read tutorials etc but have problems identifying properties. Yesterday evening, I was trying to grok how to property test the convert function found at https://github.com/paulspencerwilliams/gormanic/blob/master/src/gormanic/core.clj

nberger11:01:39

@paulspencerwilliams: I think your property can check the round trip conversion using your convert function and the inverse of it, in the domain where the inverse can be defined. In this case the domain is "any date" right? Or is it intended to work with datetimes too? I'm any case, you'll need to write the inverse though. And if it supports datetimes, then the roundtripping is not "exact", your property would take that into account

paulspencerwilliams11:01:29

@nberger: it doesn’t have to deal with times; only date

nberger11:01:27

@paulspencerwilliams cool, easier then :). I'm not sure about the "Intermission" result, is that possible? Like month index could return something greater than 12?

paulspencerwilliams11:01:00

@nberger: oh, that couldn’t be inversed with intermission. Intermission in a leap year is ambiguous. it could be 30th or 31st December 😕

paulspencerwilliams12:01:36

@nberger: now, ignoring that detail, if I could unambiguously convert / invert, that would prove that for each value of input, there’s a unique value of output and the mapping is correct, but wouldn’t actually test for example, 1st January converts to 1st of March? Is that something that’s typically overlooked as the two way conversion is sufficient, or would property based tested be backed up by some simple examples?

nberger12:01:44

A paper that I recommend to read is "Property-Based Testing and Verification: a Catalog of Classroom Examples". In the introduction, they cite John Hughes saying > tests of functional programs most often fall into one of two categories: (1) comparing the results of computing a value in two different ways (function equality tests) (2) checking that an inverse function reproduces the original input when applied to the results of the forward computation (round-trip tests)

paulspencerwilliams12:01:05

okay, will have a look at that.

nberger12:01:01

@paulspencerwilliams you can back it with some simple examples of you feel it can help making things more clear. That's usually done

paulspencerwilliams12:01:43

I don’t yet appreciate the power of being able to go back and forth if that makes sense. I don’t trust it - YET

nberger12:01:10

About the leap year and the "intermission" (now that I understand it better), your property can take the ambiguity into account, and verify that it results in one of the two ambiguous values

paulspencerwilliams12:01:01

okay. so because the domain is ‘lossy’, the tests can mirror this.

nberger12:01:11

I think it's especially powerful when you actually have both functions. Think of a encoder/decoder pair

paulspencerwilliams12:01:13

Wow, I knew things would be significantly different to example based, and this will take a little time for me to process 😉

nberger12:01:14

I see your point in the difficulty of "trusting" it... Like what if both your function and the reverse are doing nothing, acting like the identity for example... The round trip would always hold :)

nberger12:01:55

Or what if the generator is not generating values over the entire domain...

paulspencerwilliams12:01:19

Yes! exactly that. It could just return and inverse returning the day of the year.

paulspencerwilliams12:01:54

Your second point is different, but another area of current untrust.

nberger12:01:10

I don't have much to say other than you can play with them in the REPL :). He he you can also do what you said: have specific examples as unit tests

paulspencerwilliams12:01:30

@nberger: yes, thanks so much for the help. I’ll read that paper, and will no doubt be back online afterwards.

nberger12:01:21

About the second point, I'm actually working on adding a statistics facility to test.check that would report the generated values distribution, based on a classification provided by the programmer. This feature is included in many QuickCheck implementations, and I hope test.check will include it soon :)

paulspencerwilliams12:01:08

Yes, so that it would provide a comforting feeling that all months, leap / non leap years are covered.

nberger12:01:35

Cool. Good luck with that, that would be great if you come back with your findings

paulspencerwilliams12:01:31

@nberger: I will. If I succeed in getting some tests around this, I will try to blog it too. I’ve been interested in this style of test for sometime, but never dipped a toe in.

nberger12:01:24

Yes, it could report what percentage of values are leap years (should be around 25%...), or you could classify the values by the month, so they should be distributed evenly across the 12 months... Evenly is a way to say, given that the sample is not too big, usually just 100 values, it won't be exactly even... But you get it :)

nberger12:01:04

Awesome. Yeah, it's very interesting. And powerful. I'll be waiting to read that blog post :)

nberger12:01:24

Just to correct my previous sentence, it's not that a bigger sample would guarantee the distribution to be 100% even. But there would be more chance it would be closer to there. And then there's the fact that the months distribution among dates is not even in nature...

paulspencerwilliams15:01:53

Hi @nberger, I’ve read the paper you suggested and it’s clarified a lot. Basically, typical example based tests are similar to ‘functional equality tests’; the SUT is functionally equivalent to the human test author’s ‘paper and pencil’ function. In my case, it would be way of establishing 1st January 2000 == 1st March 2000 in the Gormanic calendar.

paulspencerwilliams16:01:14

@nberger: the round trip tests could be used for a broader, more general testing.

paulspencerwilliams16:01:03

I could model a subset of the Gormanic / Gregorian ranges e.g. January / March of one year in the test, and maybe using gen/choose or similar, and use it form property to ensure basic functionality works. Then, I could implement a roundtrip test that has a bigger random set across years, leap years, months etc ensuring you can go forwards backwards?

nberger16:01:32

@paulspencerwilliams well, yes, if you have another function that makes the same calculation in a different way, you can use that. That's very common for example when you have an optimized version of an existing fn, you can compare the results for both

paulspencerwilliams16:01:25

yep, when I was originally doing this kata a couple of weeks ago, a different pair in the meetup actually modelled months with maps etc - more typing but much more ‘data oriented’, simple to understand, and significantly different to the real implementation to provide independence.

nberger16:01:25

One thing to note is that there's a big area not covered by "functional equality tests" and "round-tripping tests" which is checking invariants in a broader sense

nberger16:01:40

Or postconditions...

paulspencerwilliams16:01:16

@nberger: once again, thanks for the help and I will hopefully get to play with some tests over the weekend. Have a good weekend yourself.

nberger16:01:19

@paulspencerwilliams no problem, and thank you too

agile_geek16:01:17

I must read that paper too. Sounds like it will clarify my lack of understanding around the application of property based testing

nberger16:01:38

John Hughes talks are great too