Fork me on GitHub
#clojuredesign-podcast
<
2024-04-09
>
Marcel Krcah15:04:19

The growing-bag-of-data and IO-on-the-edges techniques are often discussed in the podcast. With a threading macro, I'm wondering about how to best propagate errors in the orchestrating function, that is, to stop further processing on any error. For the first attempt, I keep the possible error in the context under :error key and have each step check for presence. See the example below. I'm wondering, is there a better/more-idiomatic way of achieving the same?

; helpers for error handling
(defn ok? [ctx]
  (-> ctx :error nil?))

(defn when-ok [ctx f]
  (if (ok? ctx)
    (f ctx)
    ctx))

; example usage
(defn fastmail-parse-emails [ctx]
  (when-ok 
    ctx
    (let [jmap-response ... ]
      (assoc ctx ... ))

; orchestration
(defn fetch-fastmail-emails! [ctx]
  (-> ctx
      fastmail-prep-auth-request
      (fastmail-call-api!)
      fastmail-parse-auth-response
      fastmail-prep-email-request
      (fastmail-call-api!)
      fastmail-parse-emails))

Nick17:04:36

Convert your threading macro into a let block like this: (defn fetch-fastmail-emails! [ctx] (let [ 1 (fastmail-prep-auth-request ctx) 2 (fastmail-call-api 1) 3 (fastmail-parse-auth-response 2) 4 (fastmail-prep-email-request 3) 5 (fastmail-call-api! 4) 6 (fastmail-parse-emails 5)] ;; Debug return a datastructure with what you want to see, I do this in the repl as you are building it up or debugging [1 2 3 4 5 6] ;; Otherwise return what would have been the final form in the threading macro ;;6 ))

Nick17:04:34

At least that is what I do for visibility. You could add a "7" to the let block with [1 2 3 4 5 6] as the bound value. Then have another function to error test it. either with Malli/spec or manually via another function. Then add the :error key to the ctx that you return based on whatever that function does

Nick17:04:03

if you wanted to break the flow along the chain from 1-6 and stop processing due to an error, I wouldn't mix side effects and checks in a threading macro or function with let blocks. You'd need separate functions, and the orchestration uses something like multi-methods or core.match to dispatch to the right function based on the state of the data

slipset18:04:25

Don’t know if I’d recommend it, but you could do something like that with reduce as well:

(def todo [fastmail-prep-auth-request
      fastmail-call-api!
      fastmail-parse-auth-response
      fastmail-prep-email-request
      fastmail-call-api!
      fastmail-parse-emails])

(defn execute-step [ctx step]
   (let [r (step ctx)]
     (cond-> r (not (ok? r)) reduced)))

(reduce execute-step ctx todo)

slipset18:04:14

Basically leveraging reduced to short circuit reducing the list of steps

slipset18:04:57

And as a bonus, you can use reductions to debug 🙂

👍 1
Nick18:04:06

I like it the reduce approach quite a bit. @U04V5VAUN why do you say "I don't know if I'd recommend it". What are the downsides that you have from this approach that I may be missing?

slipset18:04:54

Not sure, it seems to me like a nifty little trick which might lead to harder to read code?

slipset18:04:32

I kind’a like the straight forwardness of the original code, and the reduce thing might feel a bit like golfing?

slipset18:04:06

There is also https://github.com/fmnoise/flow But it feels in a way like you’re running into monadic territory.

❤️ 1
slipset18:04:36

Another approach would be to some-> and rely on nil returns from the fns you’re threading.

slipset18:04:06

Which then makes you wonder wether nil-punning and monads are related, and there goes your evening 🙂

slipset18:04:02

Or you could just have the fns throw exceptions, and you’d be good to go…

Nick18:04:31

I love how readable threading macros are. When you're threading a single "ctx" map through them they are straight forward to understand. What I didn't like, is 1. Visibility - I couldn't easily see each step without a debugger or putting in printlns/debug statements. 2 - Error catching also is hard. With this reduce approach you handle 1 & 2, and the data structure you pass to it, it is just as easy to read as the threading macro. But, it has the added benefit that you can construct it a lot easier than the internal implementation of a threading macro.

Marcel Krcah14:04:08

Thanks for taking the time to answer this. Oh, I didn't know about reduced, that is neat. I'll experiment with that reducing approach; it looks like a simplified version of the think-do-assimilate.