Fork me on GitHub

@j.m.frith Can you share the spec and the function that you're having problems with? (perhaps in a cut down version if it's a lot of code)


Thanks @U04V70XH6 I'm just working something up. In the actual code, the problem bit is fairly deep in a map, so I'm going to try to simplify


Back in a few minutes...


Hi everyone, I'm getting issues when running stest/check while a function is instrumented. It seems to be conforming the input before passing it to the function. I'll try and condense my code into a minimal example...


Nuts! My minimal example has no problems


I'll try expanding the code until I see an issue


The symptoms I'm seeing are: if I run (stest/unstrument) and then (stest/check 'my-fn) (not sure how to write a backtick in code but my-fun is quoted with a backtick not a single quote) I get tests all passing


If instead I run (stest/instrument) followed by (stest/check 'my-fn) then I get errors that the input has not conformed to spec.

Alex Miller (Clojure team)20:08:47

Could you just share the error?


I'm not sure that's quite as helpful as it might be, which is why I'm trying to scale down to get to the root of the problem


There's a whole bunch of data in there which isn't very helpful


Although right at the front of line 4 it's the part [:decision #:log{:decision {}}] this is definitely a conformed value - the spec has an (s/or ...) with :decision being one of the options


@j.m.frith Are you sure that function works with instrumentation on when called manually from the REPL?


One moment, will re-try now


It's working fine for me now


And it gives and returns a map as expected without the conform path


Have also just checked that (gen/generate ...) gives me a map that conforms fine to the spec.


I think it'll be easiest if I take a bit of time to whittle away all the nonsense so that things are a bit easier to follow.


One approach that I strongly recommend when working with Spec is to write and check each spec as you write each function -- that way you don't get a bunch of code for which you are trying to write a bunch of specs all at once and then testing all at once. The REPL allows you to take small steps and get feedback immediately on each piece of code as you write it.


For example, I have a (comment ,,,) block in each file where I'm working on code + spec and I have (s/exercise ,,,) calls in that comment that I eval to check each spec will generate (not all specs need to generate but I think it's important to check the ones that should as you write them).


That "Rich Comment Form" also includes instrument and check calls as needed.


That is definitely good advice, which I should have taken before getting to this point. I was too eager to learn the language first and built a working prototype and am now going back to learn how to use spec.


Also bear in mind that "nothing everything needs a spec" šŸ™‚ Spec is great for boundaries and for "key functions" that you really need to be sure of. Spec isn't a type system, so you don't need a spec on every function.


At work, we've used Spec since the first alpha builds in the 1.9 cycle and we mostly spec data, rather than functions. We use s/conform and s/invalid? in production code (and s/explain-data to help drive error message generation).


I guess I have a "base layer" of functions which directly interact with my state-map (in a functional way) and I'm currently only speccing those functions. The higher layers I may get to at some point


And the spec is mostly so that I can run generative tests on that portion of the code


We use instrument to support development work, mostly. We use check sparingly.


The problem is that one part of the system now has an (s/or in the spec and that is throwing off the earlier function which worked alright when I didn't have that in the spec...


Again, that sounds about right. I was really using check just to try and find edge cases. Maybe I should stick to either using instrument or check at a time. I'm getting no problems at all when using one but not the other


I'll be interested to see the cut down example that breaks because of s/or -- I've never run into that (in several years of working with Spec).


I'll work it out. It seems even stranger at the moment because the spec is actually saying it's failing on a function that shouldn't even be called. It's failing on game/player-count and I'm checking game/set-lead-player. set-lead-player has two lines, neither of which call player-count. It looks like I've got a bit of digging around ahead of me... I'll be back in the next few days once I've simplified things down


If you instrument a function that takes a function as an argument, and you have a fspec for that argument, it will be generatively tested when it is called.


(that sometimes surprises people)


I don't think that is the case here, but that is good to know. I have to head out now, but I'll be back šŸ™‚


Thank you for your time - I have to say that I'm impressed both by clojure and the community here

clj 4

I have got almost to the bottom of this! The issue was that I was calling a function within the spec definition, and not realising that spec passes a conformed value into whatever you give Code looks like this:

(s/fdef game/set-lead-player :args (s/and
                                     (s/cat :game :game/game
                                            :lead-player :player/player-no)
                                     #(<= (:lead-player %) (game/player-count (:game %))))
                             :ret :game/game)
I hadn't realised that (:game %) returns a conformed value of the argument passed in, rather than just passing on the value itself


Two questions would follow from this: 1. Is there any way to use a function in this way and pass in the original argument, rather than a conformed value 2. Should I in fact be avoiding this conflation of spec and function? i.e. any specs should depend purely on the data they are presented (and core clojure functions)


As to my question 1 above, the answer is s/unform. Worked perfectly with that in there. Yay! Answered my own question šŸ˜›


When you're writing compound predicates like that, it's expected that you take into account the shape of the conformed value -- calling s/unform is kind of a sledgehammer...


I'm a bit surprised you need a function to get the player count from a game tho'... isn't that just an element in the game's map? Or are you calculating it on every call from something inside the game map?


But your :game/game spec has s/or at the top-level? What are the alternatives there? (sorry if you posted it elsewhere, I'm only looking at this thread right now)


Thanks @U04V70XH6. Agreed that s/unform seems overkill, I just thought Iā€™d share that it does indeed achieve my (probably misguided) aim of being able to use the original argument rather than a conformed version.


The reason for the function is that, as I developed things, I kept changing up the structure of the map. Rather than having to change everywhere in the code which refers to this element, I made a function to pull it out and then only need to update that one method if/when I changed my mind again.


And no, the :game/game spec at the top level is just s/keys. One such key is :game/log which is a s/coll-of :log/log-entry :kind vector? and finally :log/log-entry is where the s/or happens to choose between various different types of log entry


Some log entries are for player actions, others are for rules being applied and each has a different structure.