😂 Which episode is this?

Stealing @U5FV4MJHG's thunder a bit here: new episode dropped early this morning in my podcast app, so I listened on my drive

It was just too funny a quote to not share

Ha! Something to look forward to for me!

I’m glad you like it! I was relistening to the episode to write up notes and I had forgotten about that. I started laughing pretty hard myself! It’s funny how much your past self can surprise your future self. 😆

I was headed to bed last night when all of a sudden I realized that I had never hit publish on the episode! Oops!

I went and published it right away, and I'll be posting announcements shortly.

Hey 👋 Just listened to the episode. Thanks a lot for that breakdown of common kinds of pure functions, it's the first time I've heard of it. It seems to me that just the knowledge of such kinds directly helps with naming and decomplecting, and also nicely complements the categorization that https://www.youtube.com/watch?v=ROor6_NGIWU. Thanks for the podcast, it was a great listen.

@U0609QE83SS Thank you for your feedback! I'm happy to hear you enjoyed it and you're finding it helpful. As for Rich's talk, are you referring to the following categories ("Avoiding Objects" slide)? • Transform • Move • Route • Record/remember • Keep separate • flow vs places

Yes, yes, that is the one; I should have probably included that in my first response

@U02PB3ZMAHH Thanks for sharing! I'm happy to hear it clicked! In my own bizarre imagination, I sometimes picture a class as an egomaniac that just can't let data be the focus of attention. When someone walks up to Data and tries to talk to it, Class has to jump in front of Data to answer all the questions. "I'll take it from here." In that world, Class only has significance because of Data, so Class can't let Data go! If Data breaks free, Class is just one of many others opining on Data, not Data's one and only. See? Bizarre imagination! 😂

There's a dilemma about decomplecting that I keep pondering about after listening to the composition serie, and I'm quite curious how others approach this. (sorry if this is a question with an obvious answer, I'm only getting into the clojure way of coding) Say that one needs to call a REST API, say with a clj-http. Such a call might have two concerns: 1. constructing the request (url, method, body, etc) - functionality is pure, without side-effects 2. calling the API, with a side-effect How to capture this in code? Two approaches come to mind: • Approach A is to mix the two concerns in one function • Approach B is to separate two concerns into one pure function and one side-effect function. Here's a real-life example from an app I'm building right now:

; Approach A: Mixing concerns
(defn fetch-from-public-api [currency-pair period]
  (let [url (str "" (:start period) ".." (:end period))]
    (client/get url
      {:accept :json
       :as :json
       :query-params {:to (:quote currency-pair)
                      :from (:base currency-pair)}
       })))

; Approach B: Separating concerns
(defn prepare-request [period currency-pair]
  {:method :get
   :url (str "" (:start period) ".." (:end period))
   :accept :json
   :as :json
   :query-params {:to (:quote currency-pair)
                  :from (:base currency-pair)}})

(-> (prepare-request period currency-pair)
    (client/request))

@U0609QE83SS Approach B is exactly what we're talking about! Yes, it feels strange at first, but it allows you to build up the entire request so 100% of it is specified as pure data and all that's left is to just execute it. I'll give you another real world example. I've been working with the https://developers.cloudflare.com/api/operations/stream-videos-retrieve-video-details. It requires authentication, so I want to factor that out. First, I'll define some endpoints I want to work with. I create pure functions for just the part unique to each endpoint.

(defn check-status-req [id]
  {:method :get
   :path id})

(defn delete-req [id]
  {:method :delete
   :path id})

(defn full-req
  [cloudflare api-req]
  (let [{:keys [api-key account-id]} cloudflare
        {:keys [method path], throw? :throw} api-req]
    {:async true
     :method method
     :uri (format "" account-id path)
     :headers {"Authorization" (str "Bearer " api-key)
               "Accept" "application/json"}
     :throw throw?}))

(def cloudflare
  {:api-key "super-secret"
   :account-id "42424242"})

(full-req cloudflare (check-status-req "abcdefg123456789"))

{:async true
 :method :get
 :uri ""
 :headers {"Authorization" "Bearer super-secret"
           "Accept" "application/json"}
 :throw nil}

@(http/request (full-req cloudflare (check-status-req "abcdefg123456789")))
@(http/request (full-req cloudflare (delete-req "abcdefg123456789")))

(defn request!
  [cloudflare req]
  (-> @(http/request (full-req cloudflare req))
      (update :body #(json/parse-string % true))))

(defn check-status!
  [cloudflare id]
  (request! cloudflare (check-status-req id)))

(defn delete-stream!
  [cloudflare id]
  (request! cloudflare (delete-req id)))

(check-status! cloudflare "abcdefg123456789")
(delete-stream! cloudflare "abcdefg123456789")

Hey @U5FV4MJHG, thanks a lot for the reply. I've been thinking about this approach a lot. Inspired by the 099 - Repl Your World, I have started exploring much more stuff with REPL. This includes several json-based REST API servers we have to deal with at work. For that, I've put together a thin wrapper for my REPL fiddling: https://gist.github.com/mkrcah/50554447517520f4c4954f9430d0f1fa One of the things I'm wondering is how to conceptually handle the side-effecting function; see the two approaches below. I think there's a guiding principle that I yet need to discover, so I'm now listening to the Effectively Isolated episodes, trying to get better grasp on composing side-effects.

; v1 has a side-effect at the start, and requires a full request; but I can eval the full-req only if needed
(defn req-v1! [full-req]
  (-> full-req
      (http/request)
      (friendly-res))),

; v2 has a side-effect married from both sides to prepping the request and prepping the response; more ergonomic perhaps; but I cannot see the full request
(defn req-v2! [req server-conf]
  (-> req
      (full-req server-conf)
      (http/request)
      (friendly-res)))

(comment
  (req-v1! (full-req get-status server-conf))
  (req-v2! get-status server-conf))

Perhaps more on the debugging side of things. Let's say that sth goes wrong with the check-status!

(defn request!
  [cloudflare req]
  (-> @(http/request (full-req cloudflare req))
      (update :body #(json/parse-string % true))))

(defn check-status!
  [cloudflare id]
  (request! cloudflare (check-status-req id)))

(check-status! cloudflare "abcdefg123456789")

I’m on vacation, so I have a short reply for the moment… I usually have very tightly defined “entities” that are more like a record. (They can even be defrecord for performance.) I also have pretty big maps for collecting lots of data together. I view the http response as one of those. There are all kinds of top level keys in there. The structure of the values can be tightly controlled, or not, depending. Eg “:headers”

For the record-like things, I have all the functions for it in a namespace. I tend to think of that as a strong entity. I may even have schemas for it that I check on function call boundaries.

But for the open maps, it’s all about the specific keys you care about. Fetch out what you care about and know the structure of.

Those open maps are more like scratchpads of data than a data model. It’s a convenient way of grouping a bunch of data. For me, it tends to be at the top-level of the call stack and whatever sequence my code is going through.

As for visualizing deep things, you can “select-keys” to just see the parts you want, or use portal to browse the data interactively.

@U0609QE83SS @jason.bullers Some short replies from my phone. I’ll be back at my computer next week.

@U0510902N may be able to elaborate on his Portal setup.

I have some small bits where I've extended Portal to provide a dashboard into a long-running process, but for the most part my Portal set up is just Portal itself. It's an amazing tool for inspecting runtime values quickly.

Hi @U5FV4MJHG, @U0510902N thanks a lot for taking the time to reply. so given we want to do some logic on a large http response, how would one typically go about that? thread macro, start with http client giving the response, maybe in a map with a single :raw or :response key, then enrich that map with a tight "entity", maybe with a schema , and then do the logic? so the final map would be sth this?

{
 :raw { ... } ; the full big bag of response data
 :person/name "adam" ; tight entity extracted from the big response map, added here by the enricher
 :person/age 34
}
; logic is then done mostly on the :person entity, while the :raw is there for easy introspection and easy enriching in future

@U0609QE83SS I like the idea of talking about this for a podcast. I made a note of that. Thanks for the suggestion In my own code, I have a couple of cases where I use open maps: 1) an open bag entities, and 2) an open entity. Consider this:

{:request { ... } ; the full HTTP request
 :response { ... } ; the full HTTP response
 :account
 {:person/name "adam"
  :person/age 34
  :user/role :moderator
  :user/:groups [:alpha, :gamma, :omicron]}}

@U5FV4MJHG, That is great news, I'd love listening to that episode. Thank you for taking the time to reply and continue this discussion. I need time to think about and process what you wrote, I will get back till the end of the week.

Going through your answer, I see the pragmatism and advantages of the two uses-cases you mentioned, thanks for enumerating those. As I think about what you wrote, two questions emerged: • Using raw data: Does your business logic ever reach to the raw request/response, or do you always go through the parsed known entity and keep the raw data only for diagnostics/forensics? ◦ always going through parsed data gives us control, less surprises, consistency in key naming; but it creates a layer between the raw data and domain logic that might consume mental bandwidth and require code to write. what are your thoughts about this? • Rich keys vs nested? ◦ In your open bag of entities example, the IO is a rest api, and you used a map of :request, :response and the known entity :account , where the known entity :account is flat with rich keys, as per https://clojuredesign.club/episode/045-why-have-derived-fields-in-data-when-i-can-just-calculate-derived-data-as-needed-with-a-function/ and https://clojuredesign.club/episode/051-maps-maps-maps/. ◦ However, in the log serie (https://clojuredesign.club/episode/029-problem-unknown-log-lines/), the IO was a file, and you put together in one flat map with rich keys both the raw data (`:raw/line`) and the known entity (`:log/date, :log/time`, etc) ◦ When to use an open-bag of entities and when a flat map with rich keys? ◦ Thinking on this, maybe: if raw data is complex (maybe 5+ fields, has headers/metadata/etc), use nested open entity, otherwise use flat map with rich keys.

@U0609QE83SS, I have found raw data to be useful in a couple of scenarios. For data collection, I always save the raw data because it has everything in it. I create a loader (an "ETL") that just pulls out the useful parts into a data structured used by the application. That approach is useful because I can always create another loader or extend my current one and then go back and re-ingest. For transient information, such as an API call, I will save the raw data for debugging purposes. I've used rolling logs for that kind of information. I can search it or load it when trying to figure out what went wrong. If I hit an especially strange edge case, I will have data that can reproduce it!

As for rich keys. I tend to use them for entities, but I do not generally use them for bags of entities. For me, it has more to do with something I want to pull apart or use together. So, for the "bag of entities", I can pull them apart easily using destructuring:

(let [{:keys [account request response]} data]
 ...)

(let [{name :person/name, :user/keys [role groups]} account]
 ...)

(Thank you very much for continuing this. I'm on a short family vacation till the end of this week, I will look into your replies next week. )

Hi @U5FV4MJHG Thanks again for replying here. I've listened to the latest episodes (Testify and Extractify) and it starts to come together. For data collection, I like the idea that you mentioned of having working data that gives us sth to reason with. I've refactored my code accordingly and see the benefits you were mentioning. For rolling logs and transient data, I don't fully get what you mean, but I believe the topic of transient data for debugging will be discussed this week on the podcast, which I'm all much looking forward to 🙌 Thanks also for that metaphor of tearing off part of the tree 💡