Fork me on GitHub
#reveal
<
2021-11-21
>
kenny16:11:09

Curious about how people think individual test assertions should be displayed. Test var results are shown in the first screenshot. If any assertion in the test var fails, it is marked with a yellow fail marker. Each test var runs many test assertions though. Each test assertion can succeed or fail. The test assertions do not have an identity like the test var they reside in. Cursive's test integration will show the output in a text like view (screenshot 2). Each test assertion failure is separated by some space. I can do that too. I could see that layout getting a bit overwhelming for cases where one test var has lots of test assertions. I was thinking an alternative view would be to add 1 more layer to the tree view. Under each test var, I show the assert-expr dispatch value (e.g., =) and line number. These would not be named since there's no identifier for them. For example, tree view would look like this: • example.test-ns ◦ unit-test1 ▪︎ 1. =:17 ▪︎ 2. =:19 ▪︎ 3. pos?:21 ▪︎ ... Thoughts?

kenny17:11:55

I might even be able to open files in IntelliJ by clicking a link in. a stacktrace or line number! https://www.jetbrains.com/help/idea/opening-files-from-command-line.html

vlaaad20:11:53

re idea: great find, that might be useful to integrate in reveal!

kenny20:11:39

Agreed! Thoughts on how someone might configure something like that in the case they're not using Cursive?

vlaaad20:11:51

re showing assertions: I'd go with clojure.test reporting semantics, e.g. not showing the assert-expr, but by default showing counters instead.

vlaaad20:11:37

maybe it makes sense to add more layers in the tree view for testing contexts?

vlaaad20:11:34

i.e. the ones available with clojure.test/*testing-contexts*

kenny20:11:54

Ah yeah, that could be nice!

kenny20:11:44

The thing I worry about with the layout Cursive shows is near impossible usability when testing output gets large. That text area used would be absolutely massive.

kenny20:11:22

I do agree that it departs from native reporting semantics, which I don't particularly like. Maybe there's some middle ground like collapsible sections in the text area thing.

vlaaad20:11:17

does it happen that outputs are large even per testing context in a single test?

kenny20:11:23

Yeah. Often a testing context will wrap many individual assertions.

vlaaad20:11:57

but is it an often case that individual assertions have big logs around them?

vlaaad20:11:53

my worry here is that this tree might also get pretty huge if we have something like this:

(dotimes [_ 1000]
  (is ...)))

vlaaad20:11:39

re open in text editor/IDE — I imagine a preference key that allows specifying some well-known editors, maybe with automatic inference to see if vscode/intellij is installed, maybe with option to provide a command string with substitutable places. Maybe it can be available as an action on vars/nses and things resolvable to vars/nses.

1
kenny20:11:18

In our case, yes. We deal with very large data structures. That's a great point. We also have some test vars with 10s of thousands of asserts 😅 (generated from data, akin to gen tests).

kenny20:11:37

What about Cursive's output style (test tree left & assertion view right) where the assertion view has collapsible sections?

vlaaad20:11:19

in that case I think it makes sense to make collapsible/expandable outputs

1
kenny20:11:15

What component do you suggest I use? I see the accordion but only being able to open 1 section seems super limiting...

vlaaad20:11:58

re Cursive's output style — I think it won't work for reveal, mainly because reveal is usually in "portrait mode", e.g. bigger in vertical direction

vlaaad20:11:41

so generally it's "cheaper" to align stuff vertically in reveal instead of horizontally

✔️ 1
kenny20:11:55

re testing context: Cursive does not include them in the test results tree view. I can easily include it, but I wonder if there was a reason he chose not to :thinking_face:

vlaaad20:11:45

I don't know

vlaaad20:11:03

btw there is also a stack of testing vars in addition to testing contexts

vlaaad20:11:18

clojure.test/*testing-vars*

vlaaad20:11:45

e.g. if you do this:

vlaaad20:11:09

(deftest foo
  (is ...))

(deftest bar
  (foo))

vlaaad20:11:58

the is block will be in stack of bar and foo vars

vlaaad20:11:12

not sure what to do with that 🤷

kenny20:11:59

Oh wow. TIL. Cursive ignores that nesting. Here's what it looks like when I run a test like that.

kenny20:11:40

Oh wait. It looks like it adds it as a space delimited list in the test var name.

kenny20:11:57

Yep. I can do that too. I wonder if people use that grouping mechanism in practice.

vlaaad21:11:13

they might, at least it's properly documented

kenny02:11:53

Man, that testing-vars thing really messes stuff up. Gotta support it, but it adds some constraints. For example, before I was able to render all the tests that would run upfront. But now, I don't have a guarantee that the vars passed in represent the actual tests that will run. Basically, everything that was keyed by var can no longer be keyed by var 😢

kenny02:11:48

(Cursive does this too -- dynamically adding tree items as they get discovered in test execution)

kenny19:11:59

Is there a way to adjust the size ratio of the output and result panels? I tried to drag the dividing line, but it doesn't seem to work like that.

vlaaad20:11:56

there is none!

kenny20:11:17

Got it. Is it possible to add or is there some limitation?

vlaaad20:11:34

No technical limitations.

1
vlaaad20:11:05

There is some intentionality behind not allowing changing the size of open panels

kenny20:11:37

Ah. What's the thinking?

vlaaad20:11:47

I wanted to make default sizes as good as possible so there will be no need for tinkering with sizes

vlaaad20:11:26

so you get a reasonably good view without doing anything

kenny20:11:37

Makes sense. Perhaps there's some special conditions that would be better suited with specialized sizes?

vlaaad20:11:04

There are some more things in this area I'm working on right now, e.g. maximizing the window with a shortcut (`F11`), so when you need to really focus on some view, it can be as big as possible

vlaaad20:11:44

And other thing I want to do is opening views in new windows instead of result panels. This, combined with F11 to maximize allows you to make some view take basically the whole screen.

kenny20:11:52

Cool! How would you control if a view opened in a new window or not? Something like combining shift+enter?

vlaaad20:11:08

exactly this!