Fork me on GitHub
#clojure
<
2019-12-28
>
craftybones06:12:33

Is there ever a need to mock anything that is not stateful when it comes to testing?

seancorfield06:12:56

@srijayanth That's a good question... If a function being called is pure -- with no side-effects at all -- then calling it can never cause anything observable so there's no reason to mock it that I can think of...

craftybones07:12:11

I can’t find an example, but this implies that mocking really is not necessary unless a function deals with state

seancorfield07:12:46

"state" includes any input, any output, any randomness...

craftybones07:12:58

Right. State/Effect

eskos07:12:59

If the side-effect free function takes a ridiculously long time to execute on every call, then mocking it with something which returns a dataset from eg. plain file is useful.

eskos07:12:52

Of course it's a trade-off, I can't really think of anything specific which would fall into that category 😛

craftybones07:12:35

The talk is good, but I think I’d like a more succinct way of putting this across to people

craftybones07:12:16

the TDD crowd doesn’t like hearing that one doesn’t quite need mocks or that mocks/stubs are a cottage industry based on stateful programming

😄 4
👍 4
craftybones07:12:34

frameworks around it rather

didibus07:12:28

In my opinion, everything that can be stand up with just a git clone doesn't need mocking

didibus07:12:56

Basically, if after a git clone, I can just run lein test and it works, you're good

seancorfield07:12:58

@srijayanth There are two schools of TDD: one is heavy on mocks, the other one is not.

craftybones07:12:42

Yeah. I mean, I’ve done that before in my life. I recently saw a mocked example in JS of an async call with the callbacks having assertions in them. I think that’s really taking it too far

craftybones07:12:03

Confuses the hell out of me. I’m deficient that way

craftybones07:12:38

It wasn’t wrong and the assertion was the right thing to do in the callback, but still, just looking at it sent my head spinning

craftybones07:12:39

What are gotchas around with-redefs and with-redefs-fn ? I am guessing running tests in parallel might cause issues….?

didibus07:12:41

Generally, inputs will be mocked in order to control them, so if you have side-effecting input, you'll need to mock those. And outputs are mocked in order to assert them. But if ins and outs are the arguments and return value, you don't need a mock. Just call the fn with the test inputs and assert its returned value.

craftybones07:12:11

@didibus - I think that’s perfectly fine to mock and stub when that is the use case

didibus07:12:00

In my opinion, there isn't really any other use case. When code is very imperative, the outputs are all over the place. So you need to mock a lot of things. And one test can affect the state of another. It gets quite hairy

didibus07:12:08

Now, the scope of what to test can be debated. Some people test only public things, and impl fns would be tested indirectly through its use from the public fns.

didibus07:12:14

Others test every one of them in isolation, that requires mocking the return of depended fns as well, so more mocking is needed for that style of testing

seancorfield07:12:41

I'll tell a little story about the "dangers" of mocking...

seancorfield07:12:13

Back when I worked at Vodafone in the UK -- a cell phone company for those who don't know -- I worked on the team that built the world's first pay-as-you-go system. We built it in two halves: the billing system and the actual cell connectivity part, and we built mocks for each as we went so we could develop and test each half completely independently. All good so far.

seancorfield07:12:55

Then we shipped the system into the QA team and soon they came back with the "all green" results. We were a bit surprised they found no bugs at all in testing.

seancorfield07:12:20

Until we realized that we had accidentally shipped the system with the mocks enabled...

seancorfield07:12:39

So they tested the mocks and of course the mocks passed all the tests -- by design.

craftybones07:12:50

Wow. The mocks were baked into source? Some sort of config toggle?

seancorfield07:12:18

It was a complex embedded system. We just packaged it incorrectly when handing it off to QA.

craftybones07:12:56

I’d guess the challenge was more to do with the packing than the mocking itself

craftybones07:12:33

But I see the point

seancorfield07:12:37

The lesson is: if you use mocks, be careful you don't end up just testing the mocks rather than your real code.

craftybones07:12:02

Yeah, absolutely

didibus08:12:43

Ya, I feel every mock is just one more thing you don't test. So I rather minimize what I mock

didibus08:12:01

Just found out about this: https://github.com/clojure-expectations/clojure-test that makes me much more likely to try and use the added conveniences of Expectations, where I never bothered before, because of how well integrated clojure.test is with everything.

vemv11:12:46

> The lesson is: if you use mocks, be careful you don't end up just testing the mocks rather than your real code. This. A similar principle I advocate is "don't test the compiler". Sometimes a given defn just contains a single if, so testing the defn would really test clojure.core/if For those cases, instrumenting the defn with spec can give a greater ROI

erik13:12:14

are mocks typically used with generative property testing as well? e.g. in a code base that predominantly relies on gen. prop. rather than unit tests

Alex Miller (Clojure team)14:12:34

Spec’s instrument supports automatic mocks based on specs

lmergen13:12:51

i think they are unrelated

pinkfrog14:12:01

why is there a =defn-=, but no =def-= ?

Alex Miller (Clojure team)14:12:03

There is some regret over even adding defn-, but we don't take things away... At one point all of the current metadata niceties didn't exist (used to be #^{:private true} some may recall) and defn- seemed worth doing I presume (pre-dates my involvement in core). But then that was all simplified down to just ^:private and it's preferred to compose the pieces rather than copy N things. There used to be a slew of these in the old clojure-contrib (https://github.com/clojure/clojure-contrib/blob/master/modules/def/src/main/clojure/clojure/contrib/def.clj - but no def- !).

Alex Miller (Clojure team)14:12:25

If you look at the frequency of need, private on def is far less common than private on defn

pinkfrog15:12:54

is instrument used in production or only in debugging?

vemv18:12:21

I'd use it in production maybe as a MVP. As soon as you have a serious production app it's more worthwhile to do custom instrumentation, e.g. logging instead of failing. Maybe asynchronously for minimizing latency impact ...Or double down on unit testing / QA so you don't have to rely on production tests

ghadi22:12:33

instrument is really a dev time tool

pinkfrog15:12:26

and how is it compared with the :pre and :post hook of a function? Both do sanity checks.

Joe Lane15:12:42

https://clojure.org/guides/spec#_instrumentation > It is not recommended to use instrumentation in production due to the overhead involved with checking args specs.

vemv18:12:00

Good question! * you cannot add :pre/:post to 3rd party defns. In practice I doubt people instrument such code, but eventually it can be very convenient * instrumentation is more flexible. With :pre/:post you can only choose whether the AssertionError iis thrown. With instrumentation it seems more trivial to build custom tooling (like logging instead of throwing, or only instrumenting a certain set of functions) * instrumentation has the burden of having to effectively activate it in all code that was intended to be speced. This is not a trivial problem, which is why currently it's only solved in e.g. Orchestra

vemv18:12:51

I do :pre/:post, but instrumentation seems a good choice provided you take the time to understand it and set it up correctly. There will be quirks. Also there are things you cannot fully spec, like anonymous fns (since those fns don't relate to a var that one can instrument)

didibus01:12:00

In prod its recommended you use explicit calls to valid? conform, assert, etc.

didibus01:12:30

So that you can Chery pick what needs runtime validation

didibus01:12:42

Generally, that's external input, and sometimes outputs as well.

didibus01:12:56

Everything else it is passed that at test time you have turned instrumentation on and performed generative testing on, and thus know that it all works

lmergen08:12:26

A pattern I've seen a lot is

(when-let [e (s/explain-data ...)]
   (throw (ex-info (s/explain-str e) e))

lmergen16:12:25

i can say that we are definitely using instrument (with orchestra + expound) in production

lmergen16:12:44

the only downside is the overhead, so we just don't use it on "hot" functions

lmergen16:12:32

in practice i would say that ~ 50% of our functions are running with instrumentation in production

🎉 4
Clypto22:12:53

is there a way to customize ring handlers based on lein targets, .eg. ssl on or off

vemv23:12:21

Make the ring handlers depend on an env var, or in a Java property Java properties can be nice because Lein can manage them: :profiles {:production {:jvm-opts ["-Dmyapp.ssl=true"]}} A config lib is likely cleaner. I like/use https://github.com/juxt/aero