Fork me on GitHub
#beginners
<
2023-02-11
>
Matthew Twomey06:02:59

I frequently use a pattern of shadowing my own let bindings. For example (simplified example taken from the web):

(let [s "Eric Normand"
      s (str/upper-case s)
      s (str/trim s)
      s (str/replace s #" +" "-")]
  (println s))
Obviously in this case, it’s easy to thread this instead. However, I have many more complex cases where shadowing my let variables to build up a map is very handy. My question / call for opinion is: Am I developing a bad habit here, or is this generally an “ok” use of the capability?

lispyclouds07:02:16

Do you have an example of when this felt more handy than threading? In general shadowing works but I’ve seen some slippery slopes like shadowing a clojure.core var or the function argument and having a massive headache later trying to debug the alien error 😅 specially if the let is quite big

lispyclouds07:02:27

If you’re talking about something like conditionally building up a map, using cond-> there is better. That’s one place i used to use some shadowing before.

daveliepmann07:02:40

Opinions vary but I try to avoid this technique. Once in a blue moon it's perfect, but most of the time I prefer to structure my code so it's not necessary.

4
Matthew Twomey08:02:21

So as for the additional example, this may not make too much sense, out of context, but this is what I just wrote that prompted my question:

Matthew Twomey08:02:40

(defn gravitee-get-config [config]
  (let [gravitee-config (config :gravitee)
        gravitee-config (assoc gravitee-config :token (gravitee-get-management-token :config gravitee-config))
        gravitee-config (assoc-in gravitee-config [:endpoints :organizations] (str (-> gravitee-config :endpoints :management) "/organizations"))
        gravitee-config (assoc-in gravitee-config [:endpoints :organization] (str (-> gravitee-config :endpoints :organizations) "/" (gravitee-config :organization)))
        gravitee-config (assoc-in gravitee-config [:endpoints :environments] (str (-> gravitee-config :endpoints :organization) "/environments"))
        gravitee-config (assoc-in gravitee-config [:endpoints :environment] (str (-> gravitee-config :endpoints :environments) "/" (gravitee-config :environment)))
        gravitee-config (assoc-in gravitee-config [:endpoints :domains] (str (-> gravitee-config :endpoints :environment) "/domains"))
        gravitee-config (assoc-in gravitee-config [:endpoints :domain] (str (-> gravitee-config :endpoints :domains) "/" (gravitee-get-domain-id :config gravitee-config)))
        gravitee-config (assoc-in gravitee-config [:endpoints :applications] (str (-> gravitee-config :endpoints :domain) "/applications"))
        gravitee-config (assoc-in gravitee-config [:endpoints :identity-providers] (str (-> gravitee-config :endpoints :domain) "/identities"))]
    gravitee-config))
This is extending an existing config object (by creating a new one which augments it). Now I may be missing some obvious basic clojure functionality here - I’m still pretty new.

Matthew Twomey08:02:59

I mean I could easily use uniquely named variables (thus avoiding shadowing), but I see no point in that - I don’t care about anything except for the final outcome and each builds on the previous one.

lispyclouds08:02:42

well -> should work here too right? gravitee-config is always the first arg

lispyclouds08:02:31

(-> (config :gravitee) (assoc ...) (assoc-in ...) ...)

Matthew Twomey08:02:32

lol - yeah I guess you’re right :man-facepalming:

lispyclouds08:02:41

in general whenever you see a need for shadowing, probably there's a nicer threader somewhere in the stdlib 😄

Matthew Twomey08:02:08

It actually didn’t start out this “clean”. It was more of a mess and at that point threading wouldn’t have worked. However I cleaned it all up and didn’t notice after it was “clean” that it was now threadable lol.

Matthew Twomey08:02:15

Thanks. Now I’m curious if I run into an example where I still want to do this shadowing - or if each time it’s just because I’m missing something. Time will tell.

lispyclouds08:02:15

also another pattern i could see is since this fn is a single arg, you can do (defn gravitee-get-config [{:keys [config]}] ...) and then use config in body and drop the gravitee-config (config :gravitee)

Matthew Twomey08:02:44

Yeah good idea. Thanks for the tips - I’ve had a lot of fun so far learning clojure. I often like to take fairly simple things like this and just keep refining them, helps me learn.

lispyclouds08:02:11

staring at the code to find these nice elegant nuggets its the best part for me in clojure 😄

Matthew Twomey08:02:39

Yeah, I’ve spent way waaaaaaay longer that I should on this little script of mine lol. Especially since it did what I needed hours ago. But I’ve learned a lot.

lispyclouds08:02:00

well, all the subsequent coding sessions would now be faster and elegant! its all about the long term gains.

Matthew Twomey08:02:58

Actually I can’t thread this

Matthew Twomey08:02:26

because I’m already threading inside these and they’re also referring to newly established values right above them.

Matthew Twomey08:02:16

I mean I could thread it, but I think it would be even uglier than how it is now

Matthew Twomey08:02:52

Meaning: gravitee-config (assoc-in gravitee-config [:endpoints :environment] (str (-> gravitee-config :endpoints :environments) "/" (gravitee-config :environment))) is building upon: gravitee-config (assoc-in gravitee-config [:endpoints :environments] (str (-> gravitee-config :endpoints :organization) "/environments")) right above it.

Matthew Twomey08:02:57

(referring to the environments key)

lispyclouds08:02:26

got it, yeah shouldve seen that, thinking of a way to do this

lispyclouds08:02:23

yep, youre right, with the internal dependencies this is a fair way to do it and others would convolute it. still fun problem to think of, will spend a bit more staring time soon 😄

lispyclouds09:02:13

i guess the other way is to restructure the code before hand if possible like Dave said to not have to do gnarly edits like this

Matthew Twomey09:02:02

Yeah - I considered that also. I’m not unhappy with how it is, I think it’s fairly clear. But I like thinking about this stuff. I also am very often just missing some basic clojure function I didn’t know about, so I like to ask here.

lispyclouds09:02:32

hopefully someone else sees some better way 😄

Matthew Twomey09:02:47

yep yep! Thanks for the conversation.

phill10:02:54

You could thread with as->

phill10:02:35

In the assignment entries where you need to refer to the thing that was passed in, you could write (as-> x (assoc x :q (inc (:r x)))) etc.

lispyclouds10:02:04

@U0HG4EHMH since there is a chained update of the thing at multiple levels, should all the nested updates go inside the (as-> ...)?

lispyclouds10:02:40

the shadowed name is getting changed at multiple levels and shadowed again, i guess the as-> would introduce more nesting?

Ed11:02:47

I think the code looks really dense and I find it hard to pick appart the data flow through this code. There's a lot of repitition that I think is obsuring the elements of the config that are acutally changed by this function. I've had a go a restructuring it a bit, and I probably went too hard on destructring, but I think that it's a bit easier for me to follow. What I prefer is that only the :endpoints and :token keys are changed, so I would rather see only 2 assoc/`update`/etc calls. Also, I think that it's easier to track which elements in :endpoints depend on which other elements because they're all individually named. I find it easier to see that doms depends on env than to visually scan [:endpoints :domains] and (-> gravitee-config :endpoints :environment) to see what the dependency is between those two properties. But I guess that's just me ;)

(defn gravitee-get-config [{{{:keys [management]} :endpoints :keys [organization environment] :as gravitee-config} :gravitee}]
    (let [orgs (str management "/organizations")
          org  (str orgs "/" organization)
          envs (str org "/environments")
          env  (str envs "/" environment)
          doms (str env "/domains")
          dom  (str doms "/" (gravitee-get-domain-id :config gravitee-config))
          apps (str dom "/applications")
          idp  (str dom "/identities")]
      (-> gravitee-config
          (assoc :token (gravitee-get-management-token :config gravitee-config))
          (update :endpoints merge {:organizations      orgs
                                    :organization       org
                                    :environments       envs
                                    :environment        env
                                    :domains            doms
                                    :domain             dom
                                    :applications       apps
                                    :identity-providers idp}))))

🙌 2
Matthew Twomey05:02:17

Thanks @U0P0TMEFJ another nice approach to consider.

Joe12:02:04

I have a side-effecty function (swaping an atom in another ns) I need to map over, but I'm having trouble guaranteeing the side effects happen. Usually I'd fiddle with doall or doseq until it worked, but I'm coming up short this time. The only thing that seems to guarantee the side effects will happen is printing the results. And I can't think why that might be. Is there something I'm missing here?

(defn spin-up-sandbox!
  [config-filename seed-events]
  (let [config ,,,stuff,,,]}]
     (ref-master/reset-context)
     (setup-db! db)
     (mapv #(pipeline/process-event! config %) seed-events)         ;; doesn't work
     (doseq [ev seed-events] (pipeline/process-event! config ev))   ;; doesn't work
     (run! #(pipeline/process-event! config %) seed-events)         ;; doesn't work
     (let [x (map #(pipeline/process-event! config %) seed-events)]
       (println (map :transactor-results x)))                       ;; DOES work
     (reset! system-config config)))

(deftest seeding-test ;; fails
  (let [seed (edn/read-string (slurp "resources/seed_1.edn")) ;; seed events contain organization and account
        config (spin-up-sandbox! seed)]
    (is (every? (set (keys (deref ref-master/ref-data)))
                #{:organization :account}))))

jumar15:02:50

How do you know that it doesn't work? Or what does it mean really? If you compare these two, what effects do you observe?

(mapv #(pipeline/process-event! config %) seed-events)         ;; doesn't work


(let [x (map #(pipeline/process-event! config %) seed-events)]
   (println (map :transactor-results x)))                       ;; DOES work
mapv indeed is eager and should block until all the elements are processed. However it's not clear what pipeline/process-event! is/does, so hard to tell. You are also doing (map :transactor-results x) before the println so it may be doing something differently

Ed18:02:54

Usually if printing forces the sequence but doall doesn't, it would hint to me that it's a lazy seq of lazy seqs, because printing is a recursive operation whereas doall will only force the top level lazy seq. I think it's best to avoid missing lazy operations with side effects.

Ed18:02:19

You can do that by returning an "intent' to describe the side effect you would like to happen and then something that iterates over that and applies the side effect. This makes your logic much easier to test. Also, when dealing with sequences of sequences, mapcat is your friend ;)

Joe20:02:03

> Usually if printing forces the sequence but doall doesn't, it would hint to me that it's a lazy seq of lazy seqs Ahhh, that's a great tip. That was it - the :transactor-results were a lazy seq, which weren't being evaluated until the print A 1 letter change (`map` to mapv ) fixed it Thanks both!

Mattias13:02:17

Hey, any (up to date) tips on local workflow for Clojure backends with Docker? Working on Mac OS primarily. 🙂

jumar14:02:51

Forget Docker if you can and just run it via REPL on the host

2
facepalm 2
practicalli-johnny16:02:32

Most of the time a backend system is simple enough to simply run via a repl. I occasionally use docker compose with a build stage when I have other services I want to run, especially when doing system integration tests Being able to compose systems together (especially non-clojure services) and start from a known state is very useful and helps move the systems through different environments

Zed16:02:45

I use docker-compose to rally together my dev database, redis, clojure backend and clojure-script front-end.

Mattias17:02:37

Thanks all! 👍 😀

Oliver Marks14:02:29

Got an issue with ring / jetty where I am generating an image when I request the image I get the exact same image if the requests all happen very close together, I guess some kind of race condition putting in a few prints and as soon as I add (flush) it stops happening, hoping some one can give me some idea as to the cause ?

Oliver Marks14:02:24

A bit more information I am generating svg documents which I then generate an image from, I know its the svg that's coming out are the same as i spit the document to a file to check, some of the data comes from crux so I wonder if it could be related to that.

jumar14:02:09

It could help if you posted the handler code

Oliver Marks14:02:10

quite a bit going on in the handler, but this is the higher level code

Oliver Marks14:02:47

will try and post later slack is ignoring line breaks so the code looks aweful

jumar14:02:00

Slack doesn’t ignore line breaks, usually. Use triple backquote

Oliver Marks14:02:59

yeah I am normally it works fine, but what ever I copy is being concatenated on paste only in slack never had it happen before 😞

Oliver Marks14:02:18

lol come to try and solve one problem and hit a totally different one.

Oliver Marks14:02:24

son has woken up so I will have to get back to this, thought it might have been a common thing and some one might have some direction on what to look for.

Jakub Šťastný21:02:16

Hey guys. I'm processing a CSV going line by line and adding an entry. Quite simple, the tricky bit is the calculation that calculates the value of the entry added is based on: • All the previous lines (but not the whole of data). • The previous line in particular. What's the best way to do that? I obviously started with (map (fn [..] ...) data), but then found out about this requirement and I'm no longer sure how to do it. Thank you!

seancorfield21:02:02

If you can fit the entire thing in memory, you could reduce and build a vector of previous lines so you can access that accumulated data from the reducing function...

Jakub Šťastný21:02:26

Thanks @U04V70XH6, in memory is fine. How would it work though? No need for the whole code, but I'm not sure even how to start, I know how reduce works, but then the vectors part I'm less clear on and how would that then feed back into the reduce. Sorry it's been a long time without coding!

seancorfield21:02:13

(reduce (fn [acc row] (conj acc (compute-new-row acc row))) [] csv-data)

🙏 2
seancorfield21:02:53

acc is the vector of rows processed so far. So you could peek to get the "previous line" or compute over all lines seen.

Jakub Šťastný21:02:39

Thanks! Very elegant. I hope to learn to write this sleek soon 🙂

pppaul00:02:38

it may help to describe how you may approach solving this problem in imperative pseudo code, and then people can give you ideas of how things work in clojure.

pppaul00:02:18

reduce operates on each item of the list, and the accumulated data. however you want to operate on one extra item, which is something that can be solved in a few different ways

pppaul00:02:44

you could process your list so that these 2 items are wrapped together. you would have a preprocessing step before a fairly normal reduce. you could do this via (map vector mylist (rest mylist))

pppaul00:02:39

when I started learning clojure I used clojuredocs website. they have lots of examples of using the standard library functions, like map reduce filter, etc...

pppaul00:02:14

also, it's helpful to give us an example of your data, and what you want your output to look like. it's pretty common that people will offer many ways to solve the problem, which may give you insight into interesting tools that clojure provides

☝️ 2