Clojurians Log v2

> I'm not smart enough to devise a grand plan without hands-on experience. @leif.eric.fredheim I'm thoroughly convinced that no one is "smart enough". Those that think they are often build a beautifully incorrect solution. The best way to learn about a new domain is to mess around with it in the connected editor. Like you said, you'll get lots of small wins and stay motivated. You will also be working with the real dependencies and learning how they behave and fail. After enough time, you'll have learned enough to come up with a design that will work for the problem.

Beautifully incorrect - I sometimes feel that way when I look back at my old code. 🙂 I liked the idea of using a map to hold intermediate values. I hadn't thought to do that. It's a useful idea. Also, I had to come here to say that I was associated with a turtle-themed team. As you surmised, no highlight reels were created. 🐢

Are you a fan of bottom-up development? Tell us why.

Yes! Because I'm not smart enough to devise a grand plan without hands-on experience. I need to try things and see what happens, learn, try again, etc. I prefer a bottom-up and exploratory approach because it helps me gradually work towards my goal and learn incrementally at the same time. There are many small wins along the way, so it keeps things interesting and prevents me from feeling lost or demotivated. You also get a little dopamine rush every time you evaluate an expression in the REPL, especially if it works on the first try.

I'd like to see the highlights for the Dancing Hippos vs the Rampaging Sloths, New Year's Day

@leif.eric.fredheim Funny you mention it, because I was just chatting with @nate about why I think a REPL-based workflow feels so satisfying. It's definitely a dopamine rush! Like you're saying, it's a steady stream of little wins! And yes, on the practical side, bottom-up keeps you grounded in the way things are.

@leif.eric.fredheim I understand where you are coming from. Using let bindings for single usages is something I normally shy away from. In this case, one reason to prefer let bindings is the second reason you gave (give something a meaningful name). This helps with understanding the data as development progresses. It also makes it easy to tap out one or more of the bindings for inspection. To make this a bit more concrete (using concepts from the series), here's some example code:

(defn query-database
  [_team-names _game-date]
  [,,,])

(defn fetch-clips-from-mam
  [_clip-tags]
  [,,,])

(defn fetch-videos-from-s3
  [_clips]
  [,,,])

(defn concatenate-clips
  [_files]
  ,,,)

(defn assemble-highlight
  [team-names game-date]
  (-> (query-database team-names game-date)
      fetch-clips-from-mam
      fetch-videos-from-s3
      concatenate-clips))

(defn assemble-highlight
  [team-names game-date]
  (let [clip-tags (query-database team-names game-date)
        clip-infos (fetch-clips-from-mam clip-tags)
        clip-files (fetch-videos-from-s3 clip-infos)
        final-highlight-file (concatenate-clips clip-files)]
    final-highlight-file))

Looking at your two functions, I have the same reaction that Jason did, the fetch-all function looks good and I would introduce the let in fetch for understandibility. I was tripped up with this:

(cond-> cont-key
        (assoc-in [:query-params (get-in request [:page :key-fn])] cont-key))

Kudos for using iteration. I've used it a few times and I think it's a great addition to the language.

Aha, yes, I see what you mean about cond-> I felt like that was an elegant way of adding the continuation token to the request if it exists. But I get how that can be confusing now.

And I'm embarrassed how long it took to understand how iteration works, haha 😂 That was a doozie of a higher order function.

hahaha, me too, it took lots of tapping and iterating to get it right

but once I did, it's a thing of magic

Referring to the cond->, another idea is to format it slightly different:

(cond->
        cont-key (assoc-in [:query-params (get-in request [:page :key-fn])] cont-key))

Yeah! That happens a lot: I look at the docs and think, "what in the everloving hell is going on here?!" and then I get lost in the valley of dispair for a few hours or days… And then suddenly it clicks. And after that I think, "hey, that's actually quite easy!"

too true

The key for me to get iteration was to remember that keywords are also functions. I was writing these complicated anonymous functions for iteration, when I simply needed to pass it a keyword.

yeah, that design decision (making keywords work as functions) has been so useful in Clojure

I suspect I'm also committing a sin here, by threading into an anonymous function to get the parameter in the right place 😅 https://gist.github.com/leifericf/242fb222966c8cfc8beb04539aaf8fab#file-pulumi-clj-L99

I think that one is fine. You are constructing a mapper function, not something at the top level of the pipeline

@leif.eric.fredheim @jason.bullers @nate Thanks for the amazing fetch and fetch-all. I think this is some of the most beautiful code I’ve seen, because of how easy iteration becomes — I rewrote my Google Photos queries, and am frankly a little shocked at how easily iteration fell out of it. My API calls will never look the same! PS: I had some problems with getting a NPE deep within hato http library. The problem was I was using uri as a map entry, which I copied from @leif.eric.fredheim’s gist. I had to change it to url in order for it to work with hato and clj-http libraries. (Otherwise, (name scheme) threw exception. 🤷‍♂️ PPS: I couldn’t get a stack trace, so I had no idea what was going on. I’m finding that I’m adding -XX:-OmitStackTraceInFastThrow` to all my JVM opts now.

@leif.eric.fredheim Your style of fetch/fetch-all is finding itself multiplying in my code. I’m using it everywhere. TY!

You're welcome, and I'm glad it was helpful! It's incredible how much one can get done with so little code and maintain readability 😄

@genekim, thank you for the kind words! I've been a fan of your writing for a long time. Long before I started learning Clojure. It means a lot to me that you found it useful. It feels like getting complimented by Yngwie Malmsteen on my comparatively mediocre guitar skills.

Just wanted to share a something. iterations is a variation of unfold, which is the opposite of a fold, which we in Clojure call reduce, which again is a generalization of recursion. From this we have that an unfold is really co-recursion. As it were, that’s also the name of the podcast (Corecursive) which taught me most of this. https://corecursive.com/046-don-and-adam-folds/ is the episode where it all unfolds.

I loved that episode!!

I loved that episode!!

Me too! It helped me understand how iteration worked, along with help from my good friends in #clojure-norway, who also helped me arrive at the code in my gist. We have several long threads about it somewhere in that channel (but they're in Norwegian).

https://clojuredesign.club/episode/086-let-tricks/

I'm not sure if it's in that episode, but I recall some discussion around keeping the form that returns the function result as small and readable as possible. This means using let to build up all the pieces, including intermediate calculation results. Personally, I like that idea a bit better than just a single threading macro because it gives a place for naming and destructuring. Granted, I haven't built anything significant with Clojure, but in some coding puzzles I've done, I've found that nice little threading macro pipelines were fun to write, but difficult to understand a few weeks later.

This might be relevant https://stuartsierra.com/2018/07/06/threading-with-style

Thanks for the input, guys! I'll check out those links. I'm curious to hear what you think about https://gist.github.com/leifericf/242fb222966c8cfc8beb04539aaf8fab#file-pulumi-clj-L7-L24 I recently wrote. They make use of ->, and fetch even has a "nested" cond->. I find this style quite readable, but perhaps it would be even more readable with let blocks? Or is this an example where using threads makes sense?

I think fetch-all is pretty clear. fetch I think would be clearer by breaking up the request and response pieces, and the dynamic part of the nested path. Maybe something like this:

(defn fetch
  "Perform an HTTP request, injecting a continuation key if supplied."
  [request & [cont-key]]
  (let [key-fn (get-in request [:page :key-fn])
        request (cond-> (:request request)
                        cont-key (assoc-in [:query-params key-fn] cont-key))
        response (http/request request)]
     (json/read-str (:body response) keyword))

the other thing I was going to suggest was destructuring the input params to its keys, largely because IMO it feels like a gap in naming to call (:request request); which I read as "get the request from the request"