Clojurians Log v2

There are questions I feel difficulty with 😅

@krishnansh710 If you watch that whole London Clojurians talk/video, I'll be interested to hear whether it helps answer those Qs for you, and what specifics you still want more detail on afterward?

@krishnansh710 Jack Rusher recently gave a talk walking through his use of the REPL for a boring task: https://www.youtube.com/watch?v=i_dUvhEIGBQ

Thanks @seancorfield I will try to find time on weekends and do it!

@krishnansh710 Are you asking that as a question, or just suggesting those as topics for Noel's video?

(In my London Clojurians talk/video I highlighted the differences between a Clojure REPL and what other languages call a "REPL", and tried to cover some of those other points too to some degree -- but it's a long video and I think a lot of video consumers don't have the patience for an hour of video these days! 🙂 )

not what you asked but related: there's a splash about find-doc, apropos, dir, doc, source , javadoc etc. but it's easy to just ignore boilerplate - programs give us useless walls of text all the time. it's worth calling attention to those functions and showing how useful they are

TIL! I wasn't aware of some of them; thank you for mentioning those.

it might be the splash has changed since I last looked at it closely

maybe it was a lein thing, clj and clojure have no splash

probably a good idea to cover most of the functions in repl-requires since they are there specifically when you get an interactive repl

user=> clojure.main/repl-requires
[[clojure.repl :refer (source apropos dir pst doc find-doc)] [clojure.java.javadoc :refer (javadoc)] [clojure.pprint :refer (pp pprint)] [clojure.repl.deps :refer (add-libs add-lib sync-deps)]]

lein repl does show a splash message that mentions some of those functions. clj repl does not.

also, a nice trick is (apply require clojure.main/repl-requires) to make any namespace useful as a working namespace in the repl

clojure is one of the best explorable / self documenting languages for figuring things out without leaving a repl

I'm still discovering that part, but, it is! It's one of the main motivations that brought me to clojure.

For this video, though, I would focus on more basic (or fundamental) things that are often taken from granted in tutorials and posts, but whose understanding is empowering: just what the REPL is, what happens when, say, Calva connects to a running nREPL, what happens when you "jack-in", etc.

Showing the potential is important, though, so I'll think about weaving some of the exploration tools in the demo 🤔

IMHO the tools clojure provides for introspection are simpler than the editor tooling stuff (classic simple vs. convenient dichotomy I guess)

I guess the hard part is a new simple thing you have to learn is more work mentally, than a thing you already know that is extremely complicated

are you comparing between REPL via CLI vs REPL via Calva or the like, or comparing between clojure's introspection vs, say, TypeScript IDE tooling?

Yeah, Simple made Easy 🙂

I'm comparing what I experience when using cider / calva / whatever vs. what I experience when I use the tools built into the repl. this includes the fact that the editor tools in my experience break frequently, and broken complex things are a lot worse than broken simple things

but I don't need to backseat drive your tutorial, I'm just very opinionated about this haha

haha, nah, I appreciate the insights; I'm learning new things here —which is the main drive behind even attempting the tutorial; I'm a beginner myself 🤷‍♂️

I came to this approach of preferring the built in repl tools because I spent a lot of time at work helping coworkers understand clojure, and it was easier to know the repl built ins and show them things on their machine, rather than force them to use my tool stack or make them watch me do it on my machine

also the repl stuff works on a remote machine (to diagnose things that only happen on deploy and never locally), and the tools often don't

oh - I just remembered another repl thing that's super useful

(require 'some.ns 'my.other.ns :reload)

Not REPL, but CLI-related:

# show functions available in deps tool:
clojure -X:deps help/dir
# find out more about a specific function in deps:
clojure -X:deps help/doc :fn find-versions
# now try that function:
clojure -X:deps find-versions :lib ring/ring-defaults

is find-versions a placeholder here?

And if you work on a project with a :build alias and build.clj:

clojure -A:deps -T:build help/dir
# or help/doc

I'm going to make an updated short version of my REPL-Driven Development video with my latest Clojure/editor setup, and that's one of the commands I will show in the video.

> clojure -X:deps find-versions :lib ring/ring-defaults
{:mvn/version "0.3.0"}
{:mvn/version "0.3.1"}
{:mvn/version "0.3.2"}
{:mvn/version "0.3.3"}
{:mvn/version "0.3.4"}
{:mvn/version "0.4.0"}
{:mvn/version "0.5.0"}
{:mvn/version "0.6.0"}

In my next.jdbc project:

> clojure -A:deps -T:build help/dir
ci
deploy
test

@seancorfield your https://www.youtube.com/watch?v=gIoadGfm5T8&t=2621s has been a great source as I try to wrap my head around all of this 🙂

A new, simpler one is coming "soon". Just VS Code + Calva + Calva Power Tools, and the CLI.

And now I'll add those commands above to the list of things to digest

Looking forward to that one!

ctrl+shift+space d s is the default hot key in the Power Tools to sync deps, and accepts an optional alias (so you can sync your :test deps, for example).

(assuming you're using Clojure 1.12 of course)

by sync deps you mean loading them from deps.edn instead of using add-libs?

Well, sync'ing what is in deps.edn with what's loaded into your REPL -- so the workflow is: edit deps.edn, sync deps, now you can require those deps. If you're just experimenting, add-lib / add-libs is fine, but I think it's good to keep records of your experiments, so adding a new dep under an alias in deps.edn and then sync'ing it means you keep the dep listed for next time.

yeah, I think the main use for add-lib in a repl is to pull in debug specific deps

Yup. IIRC, in Sean's video, he uses add-lib and mentions the need to sync that to deps.edn manually. I don't know if the tooling has changed and now allows loading from deps but didn't before, or if it was just part of the demonstration flow in that case.

That was alpha tooling that was only on a branch. Prior to Clojure 1.12.

(and it was add-lib only as I recall back then... maybe...)

I rewatched the video the other night and all that #_ shenanigans... ugh! Glad we don't have to do that any more.

Ah, that's the reason you mentioned 1.12 earlier; gotcha

It's why I'm making a new video.

yeah, that seemed like a lot of friction

@seancorfield are you planning on making a similar session for the new video?

just want to chime in and say 1000% agree with @noisesmith. using the language built in tools is amazing. makes it easy to find your way around a dev setup or connected to an uberjar. I have C-c h bound to require the repl utils in a namespace and use them constantly. Find-doc is a super power, if a bit verbose. Apropos and dir should be everyone’s first line of learning. Or at least not overlooked 🙂

@hola I'll do it live on Zoom and record it, but it won't be part of a user group meeting or anything organized. Not sure when. Probably next week, in an afternoon or maybe the evening.

What is a REPL, benefits of repl driven development, how to use repl driven development in Clojure with a text+editor. Most used functionalities

bit late to this thread but I'll chime in that @seancorfield's hour-long videos are the gold standard if you're already invested in Clojure and looking for a comprehensive breakdown of REPL workflows, but my go-to reference for quickly showing off what RDD 'feels like' viscerally has been this demo by @tonsky (for even shorter attention spans, forward to 1:30 where the interesting bits start) https://www.youtube.com/watch?v=XEMI5-MBgaM

Also the bonus fact that it's 9+ years old at this point but indistinguishable from modern Clojure is a nice nod to language stability :)

If you're fine with process-api-chunk being a "normal", blocking function, and you only want to use core.async so it can do its own job in parallel internally, you could write it as:

(defn process-api-chunk
  [callouts]
  (let [c (chan)]
    (go (doseq [callout callouts]
          (>! c (send-soap-request "getAccountDetailList" callout))))
    (<!! (go-loop [responses []]
           (if-let [res (<! c)]
             (recur (conj responses (:body res)))
             responses)))))

looking at this again after sleep / brain reset. you should avoid doing io inside go blocks, they use a limited thread pool that can get clogged up easily and should be used for coordination, not work. you can use thread which runs in an expandable thread pool and returns a channel

(>! c (<! (thread (send-soap-request ...))))

Here's the tricky bit, I want the responses back. I did take the loop out of the go block and used blocking <!! but the return value doesn't happen and the repl hangs. I got my "Clojure for the Brave and True" book handy and I need to read the parallel/async chapters. In terms of just getting the job done I may just use a normal loop and return the mapped data

if the go-loop isn't exiting, then you are into debugging territory. because async + println is a mess, and so is async + step debugging, it can be useful to use an atom as a sort of log: (def event-log (atom [])) then inside other code: (swap! event-log conj {:event "add-to-queue" :item {:a 0} ...})

then you can use standard clojure functions to investigate what did or didn't happen, or how many times - frequencies group-by etc.

Not even a go-loop, it's just a loop

(defn process-api-chunk [callouts]
  (let [c (chan)]
    (thread (doseq [callout callouts]
              (println "Loading a callout")
              (>!! c (send-soap-request "getAccountDetailList" callout))))
    (loop [responses []
           callout 0]
      (if-let [res (<!! c)]
        (do
          (println "Received response from callout " callout)
          (recur (conj responses (:body res))
                 (inc callout)))
        responses))))

I then see this with (def responses (process-api-chunk (take 3 prepared-chunks)))

Loading a callout
Received response from callout  0
Loading a callout
Received response from callout  1
Loading a callout
Received response from callout  2
(we hang here)

this is because you don't close c, so res is never nil

Heheheh

I thought I might be gapping with close!

if you add that after the doseq, inside the same thread, the code should work as written

OMG that was the missing piece. I should never try to work while exhausted. Thanks @noisesmith

(defn process-api-chunk [callouts]
  (let [c (chan)]
    (thread (doseq [callout callouts]
              (println "Loading a callout")
              (>!! c (send-soap-request "getAccountDetailList" callout)))
            (close! c))
    (loop [responses []
           callout 0]
      (if-let [res (<!! c)]
        (do
          (println "Received response from callout " callout)
          (recur (conj responses (:body res))
                 (inc callout)))
        responses))))

right now it's not using much of the core.async machinery, I bet you can come up with a simplified version of this code with less blocking operations now that you have a version that works

for example thread returns a channel, so ( can be used to wrap the io operations inside go or go-loop instead of wrapping the whole doseq