lazytest

seancorfield 2024-11-15T20:33:07.962119Z

And here's LazyTest and clojure.test (and Expectations) all running together in our BitBucket Pipeline at work:

Testing ws.seo.geo.core-test

Ran 6 tests containing 27 assertions.
0 failures, 0 errors.
  ws.seo.geo.core-test
    keyword-url
      ? should return the correct URL

Ran 1 test cases in 0.00326 seconds.
0 failures.


Test results: 28 passes, 0 failures, 0 errors.
I assume the ? is because the checkmark character doesn't show up properly on the web for BB?

2024-11-15T20:35:36.572999Z

That's interesting. What's the docstring?

seancorfield 2024-11-15T21:08:18.994289Z

This is the entire test:

(defdescribe keyword-url-lazy-test "keyword-url"
  (expect-it "should return the correct URL"
             (= "/greek-match-making"
                (keyword-url {:nationality "Greek" :defaultURL ""}
                             {:keyword "Match Making"}))))
When I run it locally, I see the green checkmark as expected. And the ? character in the BB log is green so at least to color part is working 🙂

👍 1
2024-11-15T21:08:48.922519Z

oooh the checkmark, yes, my apologies

seancorfield 2024-11-15T21:49:11.342919Z

https://github.com/NoahTheDuke/lazytest/commit/2223d91933dc837bf39e940a019e1ec8621931e2 which defines a hook defdescribe to behave as def produces this lint warning:

😄 1
seancorfield 2024-11-15T21:49:33.262499Z

Whatever Kondo does for deftest suppresses references and this warning.

2024-11-15T21:49:42.521689Z

ugh

seancorfield 2024-11-15T21:49:56.381969Z

If you lint as deftest you get no warnings 🙂

seancorfield 2024-11-15T21:50:18.099799Z

But I assume Kondo is doing something specific for deftest regarding references?

2024-11-15T21:50:18.476359Z

that relies on a clj-kondo bug: https://github.com/clj-kondo/clj-kondo/issues/2428

2024-11-15T21:51:04.510769Z

If I switch to deftest and this bug is fixed, then the docstrings will be marked as "unused value"

2024-11-15T21:51:47.284399Z

Hmm I have an idea

seancorfield 2024-11-15T21:54:13.980489Z

Heh, fun.

2024-11-15T22:05:05.685269Z

okay, i think i figured it out, due to #lsp’s tireless work

2024-11-15T22:05:25.305149Z

:exclude-when-defined-by #{my-ns/defflow}
will merge correctly with user configs

2024-11-15T22:05:55.355939Z

so i've switched from the hook to excluding vars defined with defdescribe from :clojure-lsp/unused-public-vars checks

2024-11-15T22:06:30.430489Z

i pushed the change, let me know if that works for you

2024-11-15T22:06:42.223239Z

i'm offline now, but i can help out more once my kids are in bed later lol

seancorfield 2024-11-15T22:19:34.938269Z

Yup, that addresses it. Nice!

🎉 1
seancorfield 2024-11-15T00:24:14.874699Z

Here's Polylith running both clojure.test/Expectations tests and LazyTest tests:

Testing ws.seo.geo.core-test

Ran 6 tests containing 27 assertions.
0 failures, 0 errors.
Ran 1 test cases in 0.00220 seconds.
0 failures.


Test results: 28 passes, 0 failures, 0 errors.
That's an existing ns at work, where I took one of the existing Expectations tests and added a copy of it rewritten for LazyTest. poly test is scanning 67 bricks for tests -- this is the only one test file containing one LazyTest (so far) but this is proof I can mix'n'match, to adopt LazyTest incrementally. This is using the latest SHA from the main branch in both LazyTest and in the polylith-external-test-runner. When the next release of LazyTest appears, I'll add docs to p.e.t.r and cut a new release of that too -- and then "advertise" LazyTest to the #polylith folks 🙂

🎉 2
dorab 2024-11-15T01:05:31.609989Z

Awesome. Would it make sense to modify the lazytest test runner to follow your lead and also run both kinds of tests? That way, anyone using lazytest would get the incremental adoption capability without any other dependencies. Alternatively, add a combined test runner to the lazytest repo in addition to the lazytest-only test runner.

seancorfield 2024-11-15T01:13:14.336199Z

I would advise against that. A clojure.test runner needs to duplicate all of the filter/focus/include/exclude stuff -- that's already in Cognitect's test-runner and in my Polylith external test runner. LazyTest has that built in for its own tests/suites -- but would have to duplicate all that stuff from test-runner in order to also run clojure.test tests.

dorab 2024-11-15T01:19:49.959559Z

Okay. Would non-Polylith users be able to use the p.e.t.r to run both types of tests?

seancorfield 2024-11-15T02:15:52.958719Z

No. It's a Polylith test runner. It wraps other test runners. Like there's a Polylith test runner that wraps Kaocha.

seancorfield 2024-11-15T02:19:58.440169Z

The issue is that every test runner has its own machinery for discovering tests, figuring out which tests to skip or to run, executing the tests and collecting all the results from them, and then reporting those results. In my p.e.t.r case, Polylith figures out what namespaces need to be tested. My test runner only has to detect whether a given namespace has any tests and, if so, run them and get a hash map summary. Then Polylith reports the success/failures at a namespace level (and whatever default reporter each test library uses prints details).

seancorfield 2024-11-15T02:25:08.088639Z

Cognitect's test-runner has a whole bunch of machinery for figuring out what tests it will run -- and a whole bunch of options to control that. LazyTest has a whole bunch of machinery for figuring out what tests it will run -- and a whole bunch of options to control that. Kaocha is similar. So, for the most part, you might as well just run the two command-lines. In the Polylith context, my test runner does very little -- it exists primarily to provide process isolation between the Polylith program and the project tests (the default test runner executes tests in-process, so you can run into library conflicts, memory leaks, and all sorts of other stuff in large projects). Because my test runner does so little, adding a call to lazytest.repl/run-tests was easy -- and really none of the filtering options are directly exposed (there's a Polylith-specific way of passing configuration all the way through, so you don't need to deal with command-line options etc), and none of that filtering is even wired up for LazyTest in p.e.t.r.

seancorfield 2024-11-15T04:00:09.682089Z

A quick fork of Cognitect's test-runner that will run both clojure.test and LazyTest stuff https://github.com/seancorfield/test-runner -- you'll need the head SHA of the master branch of that and you'll need to depend on the head SHA of the main branch of lazytest (and run -X:deps prep) in order to try it out. It's "dumb" integration in that no options are passed to LazyTest, so all the include/exclude stuff only applies to clojure.test tests. I'll wire up options if anyone cares enough to use this. And maybe I'll even publish releases to Clojars 🙂 Assuming Cognitect don't want a PR for it.

👍 1
logbot 2024-11-15T04:33:45.345279Z

@logbot has joined the channel

2024-11-15T04:33:50.930359Z

@zulip-mirror-bot has joined the channel

seancorfield 2024-11-15T04:34:36.868889Z

I added logbot and zulip-mirror-bot so this channel's discussions will start showing up on the Clojureverse logs and in the Clojurians Zulipchat (slack-archive).

👍 2