Hi. I have been planning to add that functionality to the sequences. You are welcome to file a feature request about it. Good to have more use cases than the ones I know of so far. For now, you'll have to run it before Jack-in, as you say. Or copy the jack-in command-line (there is a command for that in Calva) and make a script that you run, then use connect instead of jack-in. (This would create a bit of maintenance for you to keep up with any bumps of the nrepl dependencies we do. But it also lets you reload the VS Code window without killing your app.)

At Nubank, we use https://github.com/nubank/state-flow, and it seems Calva doesn't understand the defflow macro that is similar to deftest when trying to run a test

Cider has this as a variable cider-test-defining-forms

I wonder what happens with that variable...

I think cider just use the specified forms as test macros: https://github.com/clojure-emacs/cider/blob/65a23e54227e3575f5a909dcfbdbbaef70d1438b/cider-test.el#L61-L67

Probably calva checks for deftest right? then we could just move that to a variable where user can add more

Calva uses cider-nrepl for this. I don't think we specify deftest anywhere. I'll check.

I just checked, but couldn’t find any hardcoded deftest anywhere other then the testtree code

hum, but cider also uses cider-nrepl right? probably something each client needs to do

yeah, I’ll try to find the code that asks nrepl for the tests

We just use test-var-query, either with :exactly [ns] or :`test? true` Maybe we need to send along custom macros there somehow. Around line 425, @U02EMBDU2JU https://github.com/BetterThanTomorrow/calva/blob/dev/src/nrepl/index.ts

ah interesting, that gets forwarded to orchard and it returns all the tests in the var. I’ll check how thats done there

@UKFSJSM38 orchard returns all vars that have a :test meta. does your defflow set that?

nope, because the :test metadata needs a function with the body to run, but state-flow is a little more complex than that to set a body to run I think: https://github.com/nubank/state-flow/blob/master/src/state_flow/cljtest.clj#L45-L60

That's why I think supporting custom macros would solve the issue for almost any test lib

@U02EMBDU2JU actually we use deftest inside defflow, which already add the :test, but during macro expansion

mhh, then it’s weird that orchard does not find them. orchard/query.clj:

(filter #(:test (m/meta+ %)))

Yes, I sometimes write my tests in the :test entry inline in the functions. And I can test those...

https://github.com/clojure-emacs/cider-nrepl/blob/029dea5414aeeb215b76e275b84b5fce69ebbb68/src/cider/nrepl.clj#L442 is the nrepl middleware used, which does into the orchard query file below for the var-query part

When I run an inline test it looks like so:

Maybe the custom macro thing rather is about making cider evaluate the forms so that the tests end up in the metadata?

Pinging in @U0K592YDP

Would be hilarious if this ends up being something we can solve with clojure-lsp. 😃

I’ve looked at cider-emacs and it seems that cider-test-defining-forms is still alive from a time before orchard returned tests

we can’t use LSP for that, it has clojure.test/deftest cljs.test/deftest hardcoded for the tree 😅

yes, that would not work with clojure-lsp ATM, and it does seems like something that needs REPL information

although it should work if theres a kondo hook that tells clj-kondo that the defflow is using deftest under the hood.

But it would't work for tests that are not like deftest or use it under the hood

Sounds fragile to me having that on a tool that relies only on static analysis

true. you’d need a running repl anyway to run the tests, so might as well ask the repl for the info

I'm thinking that since orchard looks in the metadata, the code that puts it there needs to have been run.

That's for the REPL, of course. But maybe static analysis can help us figure out what to feed to the REPL? As a load step.

So, if you evaluate one of those defflow forms, manually, @UKFSJSM38. Does Calva still not find them when running tests?

Also, if this works in cider, we should be able to see how it does it. That's not always easy, but often I have found the cider code pretty nice to dig around in.

(defmacro defflow [sym & body]
  `(deftest ~sym
     [email protected]))

(defflow flow
  (is (= 1 1)))

@U0ETXRFEW it works in cider because you can add your test macros to its config and its going to search for them. thats the cider-test-defining-forms variable @UKFSJSM38 mentioned

Yes, we have had some general flakyness in the test discovery, but I think those should be fixed now.

So, cider is going to search for them and then evaluate them, right?

I think so

hum, evaluating the file and then running the test does seems to work for me @U02EMBDU2JU

I think this is enough, odd I thought that was not working, I'll double check and let you know, thanks both for the help!

sure, was interesting to dive into it 🙂 now I’m back to fixing camel case conversion for the snippets… less fun

@U0ETXRFEW here’s the commit that introduced the config into cider: https://github.com/clojure-emacs/cider/commit/a4d85b603e12a8edd4acd41671e3bec799244e1e

camel-snake-kebab comes to mind...

Yeah, just tested with a nubank service, it works like a charm, thanks and sorry for the false alarm!

Happy you started the discussion, @UKFSJSM38! It gave food for thought.