Fork me on GitHub
#clojure
<
2023-06-08
>
pesterhazy09:06:51

Has anyone written a tool for deciphering test assertion failures? 🧵

pesterhazy09:06:59

Here's what I mean. Clojure assertions print out something like this

user=> (is (= {:a 1 :b 2 :c 3} {:a 1 :b -2 :c 3}))

FAIL in () (NO_SOURCE_FILE:1)
expected: (= {:a 1, :b 2, :c 3} {:a 1, :b -2, :c 3})
  actual: (not (= {:a 1, :b 2, :c 3} {:a 1, :b -2, :c 3}))
That's pretty hard to read. One option is to add logic to the test runner to pretty-print failures, but that has downsides: • Complexity: additional dependencies (to diffing tools, color output, etc) • Loss of information: sometimes you don't want to see the colorized/diffed/pretty-printed output, but you can't get to the original information Has anyone explored pretty-printing the test-failure after the fact, as a separate tool (maybe even a web-based tool)?

Ben Sless09:06:14

Rethinking my answer a bit - are you interested in just printing, or doing general processing with the test output? You can capture the test report as data and do whatever you want with it

Ben Sless09:06:26

Then you can throw plenty of tools at it, including humane-test-output, Portal's test report code, or even Clerk

vemv10:06:36

If you use https://github.com/nubank/matcher-combinators, failures will be displayed nicely. CIDER (and presumably others) are integrated with its format. It took me a while to really appreciate it, but once you do, you'll see how many common use cases are elegantly captured under a concise API.

6
mpenet11:06:55

one of the many reasons why I like eftest as a runner: by default it will show a decent error output

mpenet11:06:28

expected ... actual .. diff ...

mpenet11:06:51

+ it's very simple, easy to modify and integrate with t.deps

Ed11:06:44

I've done something like this in the past

(require '[clojure.data :as data])
    (let [x {:a 1 :b 2 :c 3}
          y {:a 1 :b -2 :c 3}]
      (is (= x y) (zipmap ['x 'y :both] (data/diff x y))))
but I've also ended up regretting doing a big = in test. I more often end up comparing only the keys that I care about, which you can do with something like
(let [x {:a 1 :b 2 :c 3}
          y {:a 1 :b -2 :c 3}]
      (are k (= (get x k) (get y k))
        :a
        :b
        :c))
or some kind of select-keys or whatever.

Ben Sless11:06:53

Still not sure why care about print forms at all? we have data

Ben Sless11:06:45

(let [res (atom [])]
  (binding [t/report #(swap! res conj %)]
    ,,,))

potetm12:06:17

Cursive automatically diffs equals assertions.

potetm12:06:15

It's not exactly what you're asking after, but it doesn't mess with the REPL output at all. It's just a convenience of the editor.

lread13:06:21

Like @U45T93RA6, I am https://github.com/nubank/matcher-combinators fan, and there is also https://github.com/lambdaisland/kaocha, its pretty diffs are my favourite kaocha feature.

pesterhazy14:06:12

Thanks for all the suggestions – they're all relevant. At the same time, I'm wondering if there are disadvantages to complecting together a solution to my problem (investigating a complex diff) with other concerns (IDEs, emacs, test runners etc)

👍 2
pesterhazy14:06:13

In other words, I'm wondering if there's value in treating diff investigation as a separate thing

pesterhazy14:06:30

I like @UK0810AQ2’s idea of looking at the data behind test failures and feeding that into a custom built "failure explainer"

pesterhazy14:06:19

(I'm thinking of/being inspired by the work of the Clojure team about process stacktraces)

seancorfield18:06:32

I'll +1 HTO. My clojure.test-compatible version of Expectations will activate HTO automatically if it is on the classpath and it makes a big difference (pun intended) to some types of failure reporting!

😄 2
timrichardt16:06:04

I'm a bit late to the party, but have you heard of semantic diffs?

timrichardt16:06:23

difft has no API, but uses tree sitter, whose emacs bindings are built-in since 29.

timrichardt16:06:23

I'd love to have semantic diffs in Magit. Unfortunately it does not integrate well with external diff tools, and I am having that idea of porting difftastic to ELisp for quite a while.

emacs 4
pesterhazy19:06:56

@U0ARH7B6H I hadn't seen this – that's pretty cool. The output seems quite similar to deep-diff2

pesterhazy19:06:06

Here's elucidate, a tiny tool that explores the approach of separating "failure explainer" from test runner https://github.com/pesterhazy/elucidate

clojure-spin 1
👀 1
borkdude21:06:21

@U06F82LES Also works with bb :)

$ echo '(not (= {:foo 1, :bar 2} {:foo -1, :bar 2}))' | bb -Sdeps '{:deps {io.github.pesterhazy/elucidate {:git/sha "924d710f48dc82addbe332261291bf757c5ecc69"}}}' -m elucidate.main
{:bar 2, :foo -1 +-1}

🤯 1
pesterhazy22:06:58

And bb is 8x faster than clojure for this!

roklenarcic13:06:18

When reading transit-clj documentation I’ve seen that it talks about a value cache. In which cases is this value cache applied?

p-himik13:06:10

https://github.com/cognitect/transit-format#caching I don't see it being mentioned, but I think that caching isn't applied in the verbose mode.

p-himik13:06:32

Ah yeah, it's mentioned in the types table and other places, just not in the Caching section.

roklenarcic13:06:36

I’ve used JSON, not verbose

roklenarcic13:06:53

(t/write
  (t/writer System/out :json)
  [{:a "A"} {:a "A"} {:a "A"}])
[["^ ","~:a","A"],["^ ","~:a","A"],["^ ","~:a","A"]]=> nil

roklenarcic13:06:41

hm I guess if I make larger keys: (t/write (t/writer System/out :json) [{:aaaaaaa “A”} {:aaaaaaa “A”} {:aaaaaaa “A”}]) [[“^ “,”~:aaaaaaa”,“A”],[“^ “,”^0",“A”],[“^ “,”^0",“A”]]=> nil

roklenarcic13:06:14

Hm, but values are not cached…

roklenarcic13:06:01

(t/write
  (t/writer System/out :json)
  [{:aaaaaaa "AAAAAAAAAAAAA"} {:aaaaaaa "AAAAAAAAAAAAA"} {:aaaaaaa "AAAAAAAAAAAAA"}])
[["^ ","~:aaaaaaa","AAAAAAAAAAAAA"],["^ ","^0","AAAAAAAAAAAAA"],["^ ","^0","AAAAAAAAAAAAA"]]=> nil

roklenarcic13:06:06

Looks like it works on keywords

Alex Miller (Clojure team)13:06:45

Yes, it’s just in keys of maps iirc

roklenarcic13:06:20

Nah it works for vectors too, but only if values are keywords:

(t/write
  (t/writer System/out :json)
  [:aaaaaaa "AAAAAAAAAAAAA" :aaaaaaa "AAAAAAAAAAAAA" :aaaaaaa "AAAAAAAAAAAAA"])
["~:aaaaaaa","AAAAAAAAAAAAA","^0","AAAAAAAAAAAAA","^0","AAAAAAAAAAAAA"]

roklenarcic13:06:00

Shame, I really hoped I could save on space and mem when having same strings repeating over and over

p-himik13:06:07

Correction to the above - according to the linked docs, it's not only for keywords but also for symbols and tagged entities. And strings do get cached, but only when they're keys in maps.

roklenarcic13:06:11

thanks… unfortunately not what I needed.

ghadi15:06:47

gzipping will get you the rest of the way

p-himik16:06:27

Always wondered why the Transit compression exists when there's compression built into many things already.

Alex Miller (Clojure team)16:06:06

because it's semantic and not every format is compressed

apt17:06:46

Hi folks. Does anyone know any libs or techniques that implement a react-like selective recomputation based on data changes? (for backend, not front-end) 🧵

apt17:06:00

Context: My company uses its own "intent-based" infra as code tool. It works this way: there is a repository that has EDN that files represent pieces of infrastructure (a Clojure service, say). Then, those files as somehow rendered/expanded and exposed via a REST API. The rendering phase merges default values, 'explodes' the # of resources based on our sharding strategy, applies overrides, etc. So far, so good. The issue is that, today, whenever a file is changed in the repository that has the EDN declarations, all rendered files have to be recomputed (to be precise, they are computed on demand via the REST API, but this is also problematic in terms of performance). So I'd love to have be able to, given some change (eg. some key changed in an EDN file), recompute only what could be possibly affected by it. This sounds a lot like react, but I'm not dealing with front-end here. Maybe it would be possible to use re-frame in the backend, but I'm not sure if this is the best solution. So I'm open to suggestions.

hiredman17:06:27

that isn't really react, that is an incremental build, something like what make would give you

hiredman17:06:40

the linked paper describes abstractly how a lot of different build systems work with regards to dependency tracking and includes things like excel (tracking changed cells to figure what other cells need updating)

isak17:06:08

I think that link is not what I wanted, I meant this shared recently in a post: https://github.com/yapsterapp/a-frame

respatialized17:06:08

https://clojurians.slack.com/archives/CQT1NFF4L/p1657482280025899?thread_ts=1657482280.025899&amp;cid=CQT1NFF4L There was a thread about this a while back in #find-my-lib that you may be interested in

lilactown18:06:00

re-frame doesn't really do any change detection and propagation. that's all reagent

apt18:06:09

Thanks, folks!

Ben Lieberman19:06:21

I'm definitely missing something obvious here but this throws ClassCastException: class clojure.lang.LazySeq cannot be cast to class clojure.lang.IFn

(into [] (comp distinct (mapcat (fn [m] (reduce-kv
                                         (fn [acc k v] (if-not (or (string? v)
                                                                   (nil? v))
                                                         (conj acc k)
                                                         acc)) [] (seq m))))) test-data)
It works without into, namely by calling distinct on the result of calling mapcat on test-data.

p-himik19:06:20

Without into, it's lazy. It should fail if you wrap it into doall. You didn't provide any stacktrace so I'm just guessing, but I think that (seq m) should actually be m.

p-himik19:06:28

Ah, nah - that's not it.

p-himik19:06:11

Ah, of course - distinct is a function, you have to call it without arguments to get a transducer. So, use (distinct).

p-himik19:06:33

(But that (seq m) should still be m).

Ben Lieberman19:06:08

I tried that but it does not give me the expected result. There are still non-distinct values.

Ben Lieberman19:06:22

The stack trace is as follows

clojure.core.protocols/iter-reduce (protocols.clj:49)
clojure.core.protocols/fn (protocols.clj:75)
clojure.core.protocols/fn (protocols.clj:13)
clojure.core/transduce (core.clj:6947)
clojure.core/into (core.clj:6962)
clojure.core/into (core.clj:6950)
user/eval16362 (NO_SOURCE_FILE:35)
clojure.lang.Compiler/eval (Compiler.java:7194)
clojure.core/eval (core.clj:3215)
clojure.core/eval (core.clj:3211)
nrepl.middleware.interruptible-eval/evaluate (interruptible_eval.clj:87)
clojure.core/apply (core.clj:667)
clojure.core/with-bindings* (core.clj:1990)
nrepl.middleware.interruptible-eval/evaluate (interruptible_eval.clj:87)
clojure.main/repl (main.clj:437)
clojure.main/repl (main.clj:458)
clojure.main/repl (main.clj:368)
nrepl.middleware.interruptible-eval/evaluate (interruptible_eval.clj:84)
nrepl.middleware.interruptible-eval/evaluate (interruptible_eval.clj:56)
nrepl.middleware.interruptible-eval/interruptible-eval (interruptible_eval.clj:152)
nrepl.middleware.session/session-exec (session.clj:218)
nrepl.middleware.session/session-exec (session.clj:217)
java.lang.Thread/run (Thread.java:833)

p-himik19:06:12

> There are still non-distinct values. That a completely different issue though. What is the input, the expected result, and the actual result?

p-himik19:06:00

Note that transducers are applied in the reverse order of what you usually see with comp. In your code above, assuming the described fix, distinct is applied before mapcat.

Ben Lieberman19:06:52

oh, that was it! I was working off the expected behavior of comp. TIL about the application order with transducers. Thank you!

👍 2
p-himik19:06:32

In the context of transducers, comp is like ->. :)

2
🙏 2
wei21:06:38

was there ever a spiritual successor to REBL that helped inspect forms from nrepl? i remember reading about it somewhere but I've forgotten what it was called

Alex Miller (Clojure team)21:06:16

you are thinking of Morse (also see #C055EHYAY1Z)

wei21:06:58

ah, that's it! thanks guys!

👍 2
wei21:06:30

giving it a whirl now

jpalharini21:06:11

Nice! Just keep in mind that, different from REBL, it doesn’t inspect all evaluated forms. You need to call (morse/inspect)

seancorfield21:06:38

(or tap> them?)

seancorfield21:06:41

REBL also had inspect: "Forms can be sent to REBL using cognitect.rebl/inspect. They will be evaluated and added to the eval-history"

jpalharini21:06:42

Yeah, tap> works as well. I just mentioned inspect as that’ll be the closest match to the behavior of REBL

seancorfield21:06:57

Are you confusing it with Reveal perhaps?

seancorfield21:06:15

I mean "Morse is REBL" pretty much 🙂

jpalharini21:06:44

Maybe the “inspect all evaluations” was a config I did in REBL at some point? I just remember it did inspect everything I eval’d without a call to (inspect)

seancorfield21:06:27

I created an issue to see if @U06HHF230 plans to update it to support Morse https://github.com/RickMoynihan/nrebl.middleware/issues/27

seancorfield22:06:12

(it wouldn't be hard to fork it an update it for Morse, I suspect, but I figure we might want to let Rick have first say over that 🙂 )

pesterhazy19:06:06

Here's elucidate, a tiny tool that explores the approach of separating "failure explainer" from test runner https://github.com/pesterhazy/elucidate

clojure-spin 1
👀 1