Fork me on GitHub
#testing
<
2017-10-15
>
dottedmag18:10:00

I have a network protocol to implement, which is can be described as a complicated state machine (Petri net, actually). I can easily slap together something, but how do I test it?

dottedmag18:10:06

I have thought of inverting control, so that every step in a protocol is implemented as a pure function taking current state, event and producing new state and actions to perform (a-la Erlang gen_server), but then testing it would require crafting this internal state.

noisesmith18:10:25

if you do the protocol using code in terms of InputStream and OutputStream making test data for tests is straightforward

noisesmith18:10:50

oh, right, there's the back and forth thing

dottedmag18:10:52

@noisesmith I'm afraid this is network protocol, not application one. It's a client for layer3 VPN tunnel running over DNS.

noisesmith18:10:56

if I am testing a pure function, I always have to create the "state" it operates on

noisesmith18:10:19

@dottedmag right that's fine, what I meant was that it takes bytes as input, and outputs bytes, at some point

dottedmag18:10:25

That's why I hope there is another way, or a testsuite will be very brittle: any change in implementation would require changing all the tests.

noisesmith18:10:50

well, it's a protocol, right? if the bytes change meaning you need to change tests...

dottedmag18:10:02

Yes, but not the internal state.

noisesmith18:10:05

if you mean implementation as in internal states ... OK you aren't using pure functions

noisesmith18:10:22

you can make it pure by passing state as an argument (conventionally the first)

noisesmith18:10:31

attach that to a driver that gives it bytes

noisesmith18:10:40

could use the network, could use some data in a test

noisesmith18:10:30

and yes, if the state changes implementation, your functions need to change!

noisesmith18:10:40

and the tests of course

noisesmith18:10:18

but you should be able to define "reasonable-resting-state" or whatever in the implementing code

noisesmith18:10:25

it's part of the implementation clearly

noisesmith18:10:09

yeah - it would make sense to make the initial state at the very least

dottedmag18:10:20

They come from somewhere in the production — so maybe the same code should be exposed to the tests to produce the testing state.

noisesmith18:10:41

and if you express the state right it should be straightforward to express it as a literal in the test via some assoc calls or whatever

dottedmag18:10:30

E.g. something like (session-is-now-authenticated state) should be used both by a test which starts from an authenticated state, and by a protocol code itself once the handshake is complete.

noisesmith18:10:54

also, I have a library for dumping app state (args, return values, whatever) into a document (via transit) and load it as a one liner in my test

dottedmag18:10:15

That's a good way too. Though I wonder how readable the resulting tests are.

noisesmith18:10:16

@dottedmag (let [app (restore-from-disk "./test/data/overcommit.transit.json"] (is (ok? (recover app))))

dottedmag18:10:31

@noisesmith And how readable is overcommit.transit.json?

noisesmith18:10:48

it's a transit file - on disk it's a json, in a repl you can do anything with it

dottedmag18:10:02

I know, but can you really read it and understand the state the system in?

noisesmith18:10:04

but the name should be informative at the very least 😄

noisesmith18:10:31

so yeah, that is a concern to that approach, good point

dottedmag18:10:13

All right, I'll start simple (with a handcrafted protocol state), try extracting common code to be used both by implementation and test code and see how it goes.

dottedmag18:10:41

@noisesmith Thanks for thoughts.

noisesmith18:10:49

@dottedmag one thing I do to offset the transit files not being especially human readable is making test assertions where the fact that the assertion passes tells you something about the nature of the thing stored

noisesmith18:10:00

but that's not the same as being able to read the data of course