Fork me on GitHub
#expound
<
2019-05-21
>
Alex Whitt16:05:33

I use lein-test-refresh, and I want to have my functions instrumented during these tests, with expound turned on. My approach has been using profiles.clj:

{:user {:plugins      [[venantius/ultra "0.6.0"]
                       [com.jakemccrary/lein-test-refresh "0.24.1"]]
        :dependencies [[expound "0.7.2"]]
        :injections
        [(require '[expound.alpha :as expound]
                  '[clojure.tools.namespace.repl :refer :all]
                  'clojure.spec.alpha
                  'clojure.spec.test.alpha
                  'spell-spec.expound)
         (alter-var-root #'clojure.spec.alpha/*explain-out*
                         (constantly #(expound/custom-printer
                                       {:show-valid-values? true
                                        :theme              :figwheel-theme})))
         (refresh)
         (clojure.spec.test.alpha/instrument)]}}
This succeeds at instrumenting the functions, but I only get the raw :clojure.spec.alpha/problems dumped out (no expound). Am I doing something wrong, or is what I want not possible?

bbrinck16:05:08

@alex.joseph.whitt Unfortunately, I’m away from my personal computer right now, so I can’t test it. Can you create a test that prints out the value of ‘explain-out’? I wonder if you need to use ‘set!’ instead of alter-var-root?

bbrinck16:05:00

I’m not sure exactly when injections run, but if they run before Clojure sets up the default bindings, including explain-out, I think they would be incompatible with how spec expects you to configure the printer :(

bbrinck16:05:04

If they run after the default bindings, then using ‘set!’ would alter the binding whereas ‘alter-vat-root’ won’t work as you want.

Alex Whitt16:05:50

Hmm, *explain-out* is seemingly correct

Alex Whitt16:05:30

Testing clj-canopen.cob-test
"*explain-out*:" #object[expound.alpha$custom_printer$fn__2814 0x4af9217f "expound.alpha$
custom_printer$fn__2814@4af9217f"]                                                      

ERROR in (header-codec-tests) (alpha.clj:132)
Uncaught exception, not in assertion.
expected: nil
  actual: clojure.lang.ExceptionInfo: Call to #'vertiv-common.binary.codec.complex-bytes/
encode did not conform to spec.
{:clojure.spec.alpha/problems....                                          

bbrinck16:05:06

Interesting. So if you just, say, call ‘s/explain’ in a test (when it should fail), presumably you’ll see expound output?

bbrinck16:05:03

If you just call ‘instrument’ as part of your test (or in ‘:each’ fixture), does it work?

Alex Whitt16:05:14

Yep to your first question:

(deftest header-codec-tests
  (prn "*explain-out*:" s/*explain-out*)
  (s/explain int? :hi)
...)
Testing clj-canopen.cob-test
"*explain-out*:" #object[expound.alpha$custom_printer$fn__2814 0x4af9217f "expound.alpha$
custom_printer$fn__2814@4af9217f"]                                                      
-- Spec failed --------------------

  :hi

should satisfy

  int?

Alex Whitt16:05:44

No to your second question

bbrinck16:05:30

Ah, so I wonder if the issue is that “lein” is just not processing errors using explain-out

bbrinck16:05:52

My memory is that maybe newer versions of lein do this? But I’m not 100% sure

bbrinck16:05:42

Basically old versions of Clojure used to put the entire error string in the exception. Newer versions don’t: they just include the raw data, and expect the environment to format it correctly.

bbrinck16:05:48

For instance, if you did the same experiment of calling “instrument” in your tests and ran them with recent version of “CLJ”, I wonder if you’d see a different output

Alex Whitt16:05:59

Well, I'm on Leiningen 2.9.1 (newest) and Clojure 1.10.0. I suppose I could try clojure CLI, although I've never used it before

bbrinck16:05:34

Hm, maybe lein didn’t do that? I thought they did but I could be mistaken.

bbrinck16:05:01

Oh I think also newer Clojure 1.10.1-beta3 may have changed error handling in non-REPL environments

bbrinck16:05:11

You could try that as an experiment.

Alex Whitt16:05:41

I tried 1.10.1-beta3... didn't hurt anything but it didn't change the behavior 😕

Alex Whitt16:05:28

Out of curiosity, what's your workflow like? I'm not married to lein-test-refresh, but I've tried several different ways of auto-testing my code and it seems to be the most performant

bbrinck16:05:30

I use lein-test-refresh as well, but IIRC, I just set up expound in a fixture. Seems to work for me, but I can’t pinpoint what’s different off the top of my head. My test setup is in the expound source. Another point of complexity is that orchestra does something different here than vanilla instrument

Alex Whitt16:05:10

For now I'm just using the regular instrument

bbrinck16:05:15

I use orchestra (which has different bugs as you found above!) but I may be relying on that behavior.

Alex Whitt16:05:27

Oh I see what you mean

bbrinck16:05:55

There’s a lot of complex interaction right now unfortunately

bbrinck16:05:18

Sadly expound can only work within the confines of tools like lein and Clojure CLI

Alex Whitt16:05:36

Yeah. What I wouldn't give for all the niceties we want baked into Clojure / spec / clojure.test natively. We have excellent libraries like expound and such but getting to a nice workflow is not a great user story.

bbrinck16:05:38

Since ‘explain-out’ is set correctly, my guess is that lein is not formatting exceptions using ‘explain-out’. Off the top of my head, this seems like something that would need to be addressed in lein

bbrinck16:05:33

Agree! It’s a big pain and not obvious how to get it set up (And in some cases, not quite possible to do what we want!)

Alex Whitt16:05:39

Ah, I think "orchestra" was the magic word

bbrinck16:05:41

Short of a lein fix, you may be able to wrap your tests in a fixture that catches exceptions and calls expound w the data, but I realize that’s not optimal

Alex Whitt16:05:30

Only problem seems to be that it prints out the clojure.spec.alpha/problems in addition to the expound-formatted report:

........
-------------------------
Detected 2 errors

{:clojure.spec.alpha/problems ......

Alex Whitt16:05:14

I think I asked you about that last week or something but I can't see the history.

bbrinck16:05:59

IIRC, that’s still related to how lein handles exceptions. Clojure CLI may have different behavior.

bbrinck16:05:34

Lein is getting the exception and printing out the message and the data AIUI

bbrinck16:05:52

Not sure if this is configurable in lein

Alex Whitt16:05:10

Part of me wants to migrate to Clojure CLI, because I like things that smell "standard," but I'm afraid of losing everything lein buys me. I'm not sure if the simple/easy tradeoff is worth it yet.

bbrinck17:05:19

I hear ya. I haven’t moved away from lein for that reason, but it is more painful right now since it delayed in copying behavior changes in ‘clj’

bbrinck17:05:57

Not sure if CLJ CLI has a robust test refresh solution yet

Alex Whitt17:05:10

I've been looking and not turning up anything

Alex Whitt17:05:58

But anyway, I think our environments are fairly similar at this point. Do you also get the problems printed out after the expound report when using lein-test-refresh?

bbrinck01:05:15

Yep, I have the same issue in my tests

Alex Whitt17:05:43

It's more scrolling but things are livable now at least

bbrinck17:05:28

I’d have to check when I’m back on my non-work computer, but my guess is yes, I do

Alex Whitt17:05:21

Mmk. Well, I can deal. This is already 10,000% better. Thank you again for your help! It's authors like you that keep my faith in Clojure and OSS itself alive.

bbrinck17:05:19

That’s nice of you to say! Happy to help, and thanks for using expound!

🤝 4