Fork me on GitHub
#clojure
<
2023-12-15
>
flowthing06:12:43

It seems to me that making a custom version of defn such that it accepts all of the same arguments as clojure.core/defn is pretty hard. I figured I could write a macro that does alter-var-root after defn, but that won't work if direct linking is enabled, I think? Are there any other approaches I ought to look into, apart from just writing the code that covers all arg permutations of defn by hand?

Alex Miller (Clojure team)06:12:45

What are you actually trying to do?

flowthing06:12:01

I am trying to make a version of defn that wraps the function body such that that code will be instrumented. Basically wrapping it in an Elastic APM span. So for example, this:

(defnspan my-sum
  [x y]
  (+ x y))
Would expand to something like this:
(defn my-sum
  [x y]
  (with-apm-span [...]
    (+ x y)))

flowthing06:12:32

Additionally, I'd like to pick up things from the var metadata to use inside the [...].

p-himik06:12:26

If you're up for dirty hacks, you can call defn as a function with #': (#'defn nil nil 'f '[x]). The result could then be transformed and returned from a macro.

p-himik06:12:23

Also, you can just copy the definition of defn, it's rather small.

☝️ 1
flowthing06:12:34

Hmm. Interesting, thanks!

Ivar Refsdal08:12:18

I did something like this https://github.com/ivarref/defnlogged/blob/main/src/com/github/ivarref/defnlogged.clj#L160 a while ago.. (I'm not sure that matches everything though.) That code is basically taken from https://github.com/taoensso/tufte/blob/master/src/taoensso/tufte.cljc#L531 macro..

👍 1
Ben Sless09:12:18

I think alter var root will also work with direct linking, it's a question of when the var is dereferenced, at compile time or run time. As long as no one has dereferenced it, why not bang away at it?

flowthing09:12:51

Hmm, good point. The alter-var-root impl seems simplest to me. It'd look something like this:

(defmacro defnspan
  [fname & body]
  `(let [v# (defn ~fname ~@body)
         m# (meta v#)
         span-opts# (into {}
                      (comp
                        (filter (fn [[k# _v#]] (= (namespace k#) "apm")))
                        (map (fn [[k# v#]] [(-> k# name keyword) v#])))
                      m#)]
     (alter-var-root v#
       (fn [f#]
         (fn ~fname [& args#]
           (let [opts# (merge {:name (str *ns* "/" '~fname)} span-opts#)]
             (with-apm-span [span# opts#]
               (apply f# args#))))))
     v#))
That will have some performance overhead, I think, but I'm not sure how much.

Ben Sless11:12:30

You can get the argslist from the meta and emit instrumentation for every arity to avoid apply

flowthing11:12:01

True, good point! Thanks! 👍

Ben Sless11:12:13

Note that if the lower arities call the higher arities you'll instrument multiple times, so you may want to only instrument the max arity

👍 1
Noah Bogart13:12:53

we have something very similar for OpenTracing spans, @U4ZDX466T. It works great.

Noah Bogart13:12:09

@UK0810AQ2 good call about the lower arities, I hadn't thought of that

Ben Sless13:12:10

What can go wrong will go wrong, etc

Ben Sless13:12:39

What about cases where the argslist was set via metadata and not parsed from the function form?

Ben Sless13:12:13

What about cases where different arities have different implementations

Ben Sless13:12:57

the correct solution is to default to the maximum arity but to first check metadata of :instrument-arities which will be a set of numbers, akin to :inline-arities

Noah Bogart13:12:45

we're not messing with the :argslist in ours, just setting spans based on the name of the function, so we've avoided that issue, but it's worthwhile to think about

flowthing14:12:26

@UEENNMX0T something similar to that alter-var-root impl, you mean?

Noah Bogart14:12:14

yeah, we do (skipping details cuz it's work code):

(alter-var-root! v#
  (fn [f#]
    (fn ~fname [& args#]
      (let [span# (start-span! ...)]
        (set-data-on-span! span# ...)
        (try (apply f# args#)
             (catch ...)
             (finally (end-span! span#)))))))

flowthing14:12:59

Great, thanks! Good to have a precedent. 🙂

👍 1
Noah Bogart14:12:02

i surveyed our code and we do use this with multiple arity functions, but start-span! and set-data-on-span! and end-span! are all idempotent

flowthing14:12:15

I think I might just do what partial does and write out the arities by hand until four args and if there's more, do (apply f args#).

👍 1
flowthing14:12:39

@UEENNMX0T just to clarify: do you use direct linking in prod?

emccue15:12:51

so you would do

emccue15:12:40

^{::m/aspects [apm-span]}
(m/defn my-sum
  [x y]
  (+ x y))

flowthing15:12:04

Thanks! That looks very nice and clean. I'm not sure I can take a dep here, but I'll look into it.

escherize16:12:55

You can use malli to parse the data structures. e.g. This accepts a superset of the usual defn syntax: https://github.com/metabase/metabase/blob/master/src/metabase/util/malli/fn.clj#L58-L84

flowthing16:12:16

Malli is an even bigger dep, but thanks. 🙂

flowthing16:12:43

Eh, looks like things break with direct linking even before getting to alter-var-root!... given:

(defmacro my-defn
  [fname & body]
  `(defn ~fname ~@body))
Then:
(binding [*compiler-options* {:direct-linking true}] (compile 'my.ns))
And:
clj -Sdeps '{:deps {co.elastic.apm/apm-agent-api {:mvn/version "1.38.0"}} :aliases {:x {:extra-paths ["classes"]}}}' -A:x
This happens:
user=> (my-defn f [x] (inc x))
Unexpected error (ClassNotFoundException) macroexpanding my-defn at (REPL:1:1).
clojure.core$seq__5479

😅 1
flowthing20:12:15

Never mind, it works after all. I was compiling on Clojure 1.12.0-alpha5 and running on Clojure 1.11.0. facepalm

clojure-spin 3
Samuel Ludwig21:12:38

are there any tap> idioms that you use at your companies/projects? i.e., is there some sort of extra meta-data you always pass along in taps? some common wrapper you like to use?

p-himik21:12:52

Two things: • Wrap everything in a vector with extra data that helps you see which tap that was, e.g. (tap> [:doing-stuff data]) instead of just (tap> data) • Whatever metadata Portal supports

2
R.A. Porter21:12:04

Similar to ☝️, but I usually use wrapper maps instead of vectors, so I can include arbitrary context. Could, of course, use meta for that instead, but the maps are quick and dirty.

☝️ 1
Samuel Ludwig21:12:50

Both of those make a lot of sense! I'm currently going with the map approach but it's very ad hoc, usually in the format

{:event ::event-name
 :data x}
i should write some kind of wrapper though, maybe with timestamps too or something

R.A. Porter21:12:34

You certainly could. You could write a macro that included line number and other data if you wanted, but I tend to find that ad hoc works best for my workflow.

isak21:12:28

Usually I use a vector as above, but otherwise if it is just quick one: here is an idiom I learned here from Sean:

(-> (range 20)
    (doto tap>)
    (->> (map inc)))
This is a quick way to inspect something if it in a thread (`->`).

seancorfield22:12:53

I have a Joyride script that will either wrap a form with (doto .. tap>) or add (doto tap>) into a -> pipeline: https://github.com/seancorfield/vscode-calva-setup/blob/develop/joyride/scripts/tap.cljs

😎 1
seancorfield22:12:05

(and I have that bound to ctrl+alt+d ctrl+alt+t -- Doto Tap 🙂 )

Samuel Ludwig22:12:48

the doto is a neat trick 😃

🎯 1
Samuel Ludwig22:12:02

browsing through these snippets now : )

djblue22:12:44

If you are a portal user, there is https://github.com/djblue/portal/blob/master/src/portal/console.cljc which will tap a value with its context and return it. It will also tap and re-throw exceptions and works with portals goto facilities 👌 also works for clj/cljs/cljr.

markaddleman22:12:33

wrapping stuff in (portal.console/debug …) is a pretty regular thing for me. It’s great.

awesome 1
markaddleman22:12:56

One nit: Have you thought about adding reader macros?

markaddleman22:12:52

I currently use playback but I don’t like how it displays values in the portal console

1
djblue22:12:57

A little, just haven't taken the time to put it together 😬

markaddleman22:12:44

Great 🙂 Portal has been a game changer for me. Thank you!

🙏 1
lukasz23:12:30

I have a bunch of helper functions to tap> values in threading macros and also a namespace to help me decipher errors coming from Postgres (most likely caused by me messing up HoneySQL syntax ;-)) https://github.com/lukaszkorecki/rumble/blob/1194ba15f1867384e1d67befa58fb30664154441/src/r/sql.clj#L68-L73

seancorfield23:12:53

@U02EMBDU2JU Maybe the with-logging stuff in next.jdbc would help you there?

lukasz23:12:53

wrong person ;-) but.. I was not aware of with-logging :man-facepalming: that said, I'm pulling this pure-java SQL syntax formatter and that looks great in Portal :-)

hifumi12300:12:03

When debugging helix components, I like to do stuff like

(defnc component-to-debug [props ref]
  (tap> {:id `component-to-debug ...})
  ...)
then inspect values with the Shadow CLJS inspector on localhost:9630

hifumi12300:12:01

In general, I will tap> maps, and since I always use structured logging in my projects, it feels just like logging, just without messages.