clojuredesign-podcast

Marcel Krcah 2024-04-09T15:54:19.534529Z

The growing-bag-of-data and IO-on-the-edges techniques are often discussed in the podcast. With a threading macro, I'm wondering about how to best propagate errors in the orchestrating function, that is, to stop further processing on any error. For the first attempt, I keep the possible error in the context under :error key and have each step check for presence. See the example below. I'm wondering, is there a better/more-idiomatic way of achieving the same?

; helpers for error handling
(defn ok? [ctx]
  (-> ctx :error nil?))

(defn when-ok [ctx f]
  (if (ok? ctx)
    (f ctx)
    ctx))

; example usage
(defn fastmail-parse-emails [ctx]
  (when-ok 
    ctx
    (let [jmap-response ... ]
      (assoc ctx ... ))

; orchestration
(defn fetch-fastmail-emails! [ctx]
  (-> ctx
      fastmail-prep-auth-request
      (fastmail-call-api!)
      fastmail-parse-auth-response
      fastmail-prep-email-request
      (fastmail-call-api!)
      fastmail-parse-emails))

Nick 2024-04-09T17:09:36.174779Z

Convert your threading macro into a let block like this: (defn fetch-fastmail-emails! [ctx] (let [ 1 (fastmail-prep-auth-request ctx) 2 (fastmail-call-api 1) 3 (fastmail-parse-auth-response 2) 4 (fastmail-prep-email-request 3) 5 (fastmail-call-api! 4) 6 (fastmail-parse-emails 5)] ;; Debug return a datastructure with what you want to see, I do this in the repl as you are building it up or debugging [1 2 3 4 5 6] ;; Otherwise return what would have been the final form in the threading macro ;;6 ))

Nick 2024-04-09T17:12:34.551509Z

At least that is what I do for visibility. You could add a "7" to the let block with [1 2 3 4 5 6] as the bound value. Then have another function to error test it. either with Malli/spec or manually via another function. Then add the :error key to the ctx that you return based on whatever that function does

Nick 2024-04-09T17:16:03.738269Z

if you wanted to break the flow along the chain from 1-6 and stop processing due to an error, I wouldn't mix side effects and checks in a threading macro or function with let blocks. You'd need separate functions, and the orchestration uses something like multi-methods or core.match to dispatch to the right function based on the state of the data

slipset 2024-04-09T18:35:25.508419Z

Don’t know if I’d recommend it, but you could do something like that with reduce as well:

(def todo [fastmail-prep-auth-request
      fastmail-call-api!
      fastmail-parse-auth-response
      fastmail-prep-email-request
      fastmail-call-api!
      fastmail-parse-emails])

(defn execute-step [ctx step]
   (let [r (step ctx)]
     (cond-> r (not (ok? r)) reduced)))

(reduce execute-step ctx todo)

slipset 2024-04-09T18:36:14.547699Z

Basically leveraging reduced to short circuit reducing the list of steps

slipset 2024-04-09T18:40:57.920559Z

And as a bonus, you can use reductions to debug 🙂

👍 1
Nick 2024-04-09T18:45:06.944319Z

I like it the reduce approach quite a bit. @slipset why do you say "I don't know if I'd recommend it". What are the downsides that you have from this approach that I may be missing?

slipset 2024-04-09T18:45:54.931319Z

Not sure, it seems to me like a nifty little trick which might lead to harder to read code?

slipset 2024-04-09T18:46:32.701249Z

I kind’a like the straight forwardness of the original code, and the reduce thing might feel a bit like golfing?

slipset 2024-04-09T18:48:06.176249Z

There is also https://github.com/fmnoise/flow But it feels in a way like you’re running into monadic territory.

❤️ 1
slipset 2024-04-09T18:48:36.197289Z

Another approach would be to some-> and rely on nil returns from the fns you’re threading.

slipset 2024-04-09T18:49:06.227579Z

Which then makes you wonder wether nil-punning and monads are related, and there goes your evening 🙂

slipset 2024-04-09T18:50:02.909569Z

Or you could just have the fns throw exceptions, and you’d be good to go…

Nick 2024-04-09T18:54:31.983289Z

I love how readable threading macros are. When you're threading a single "ctx" map through them they are straight forward to understand. What I didn't like, is 1. Visibility - I couldn't easily see each step without a debugger or putting in printlns/debug statements. 2 - Error catching also is hard. With this reduce approach you handle 1 & 2, and the data structure you pass to it, it is just as easy to read as the threading macro. But, it has the added benefit that you can construct it a lot easier than the internal implementation of a threading macro.

Marcel Krcah 2024-04-10T14:26:08.314339Z

Thanks for taking the time to answer this. Oh, I didn't know about reduced, that is neat. I'll experiment with that reducing approach; it looks like a simplified version of the think-do-assimilate.