testing

Ingy döt Net 2023-09-22T17:05:34.586609Z

I feel like I'm setting my tests up wrong because I get failure results like:

expected: (= (testfn test) (expected test))
  actual: (not (= "foo" "bar"))
when it feels like I should be seeing something like:
expected: ("bar")
  actual: ("foo")

Ingy döt Net 2023-09-22T17:06:28.552319Z

Everything feels fine when tests pass.

Ingy döt Net 2023-09-22T17:09:28.964389Z

btw, I put foo bar above but a real result is:

expected: (= (testfn test) (expected test))
  actual: (not (= "{:type \"Mapping\",\n :coll\n ({:type \"Scalar\", :style \":\", :value \"a\"}\n  {:type \"Scalar\", :style \":\", :value \"b c\"})}\n" "{:type \"Mapping\",\n :coll\n ({:type \"Scalar\", :style \":\", :value \"a\"}\n  {:type \"Scalar\", :style \":\", :value \"b c\"})}X\n"))
which is madness 🙂

respatialized 2023-09-22T17:14:55.589179Z

When I find there's too much output I just use a let binding before the test expression

seancorfield 2023-09-22T17:17:16.759889Z

You might like https://github.com/pjstadig/humane-test-output which improves the reporting in a lot of cases.

Ingy döt Net 2023-09-22T17:18:56.059639Z

@afoltzm not sure I follow. Example?

Ingy döt Net 2023-09-22T17:19:05.506829Z

@seancorfield I'll take a look

Ingy döt Net 2023-09-22T17:21:30.060559Z

I just don't get why it's putting the testing code in expected My testing function produces a string, so I want expected to be the string I'm declaring to be expected, and actual to be the output of my testing function.

pesterhazy 2023-09-22T17:22:34.144079Z

The actual expected and actual values are printed in the second line

seancorfield 2023-09-22T17:23:27.136389Z

clojure.test is pretty simplistic. If you say (is some-expression) then that is the "expected" expression, and the "actual" result is (not value-of-some-expression).

pesterhazy 2023-09-22T17:23:50.315189Z

The way it's printed is a bit unorthodox but you get used to it

Ingy döt Net 2023-09-22T17:24:07.138849Z

lol

seancorfield 2023-09-22T17:24:39.155439Z

That's why things like HTO got written and why there are also other test runners, like Kaocha, and other test libraries, like Expectations.

Ingy döt Net 2023-09-22T17:24:53.939859Z

HTO looks promising

pesterhazy 2023-09-22T17:25:29.278269Z

I use https://github.com/pesterhazy/elucidate when a failure is hard to read

👌 1
seancorfield 2023-09-22T17:25:40.265589Z

I liked Expectations enough that I took over maintenance of it 🙂 and I recommend HTO even with Expectations. But clojure.test's extension hooks are very primitive so there's a limit to what test runners can actually do.

Ingy döt Net 2023-09-22T17:26:05.907909Z

I made a really nice data driven testing lib over clojure.test, so I need something that works with clojure.test

Ingy döt Net 2023-09-22T17:26:21.760639Z

HTO does, right?

seancorfield 2023-09-22T17:26:42.354239Z

Expectations has two versions: the original standalone version and the clojure.test-compatible version. And, yes, HTO is an extension for clojure.test.

mpenet 2023-09-22T17:28:16.393789Z

I like eftest nowadays. The defaults are quite nice, it’s simple and writing a custom runner or reporter is trivial

👍 1
mpenet 2023-09-22T17:29:19.072379Z

Not to mention it can run tests in parallel (per var or ns)

Ingy döt Net 2023-09-22T18:08:56.191399Z

HTO is exactly what I wanted here! And trivial to add. No changes to my setup.

Testing yamlscript.composer-test
* test-1: Example Test

FAIL in (test-1) (test.clj:114)
expected: {:type "Mapping",
           :coll
           ({:type "Scalar", :style ":", :value "a"}
            {:type "Scalar", :style ":", :value "b c"})}
  actual: {:type "Mapping",
           :coll
           ({:type "Scalar", :style ":", :value "a"}
            {:type "Scalar", :style ":", :value "b C"})}
    diff: - {:coll [nil {:value "b c"}]}
          + {:coll [nil {:value "b C"}]}

Ran 1 tests containing 1 assertions.
1 failures, 0 errors.
{:test 1, :pass 0, :fail 1, :error 0, :type :summary}

Ingy döt Net 2023-09-22T18:09:49.551479Z

Thanks all. I'll check out the others someday 🙂

lread 2023-09-22T20:03:15.101689Z

https://github.com/lambdaisland/kaocha does nice reporting too. It is more of a complete replacement runner, so some folks find that too much, but I tend to like it!

lread 2023-09-22T20:04:34.889359Z

And when you get into diffing more complex structures, I think that like me, you'll find https://github.com/nubank/matcher-combinators a complete joy.

mauricio.szabo 2023-09-23T06:27:01.249999Z

I really, really recommend matcher-combinators, even for simple things

➕ 1
👌 1
mpenet 2023-09-23T06:27:35.826569Z

I quite like it as well