Fork me on GitHub
Bobbi Towers07:11:45

I just spent like 45 minutes writing a function to walk a parse tree and produce a sequence before realizing all I needed was tree-seq 😆

🔥 1

As a beginner, it was really valuable to study

Davide Fucci08:11:06

What is the idiomatic way of mapping a set of functions to a list of maps and triggering side effects? I have got

(dorun (map #(do (f1 (:key1 m)) (f2 (:key2 m))) mymap))

Ben Sless08:11:28

First, to create a function which "projects" a bunch of functions: (apply juxt fs) will return a new function Then you want to map it to a bunch of maps. If you care about the results use mapv, if not, run! or doseq


I'd like to call 3 different external APIs, wait for the results (or swap in a default result on timeout), Then call another API with the combined data (or default values) from the first 3. Http-kit client calls return a promise, so would that be enough to hold the 4th call untill the first 3 reply? Or would I need to wrap the first three calls in a promise or something else that delays the 4th call until ready. Background: this is done inside a handler function of a Clojure service using reitit & http-kit server, in response to an external post request.

Ben Sless08:11:52

yeah, sounds like: (all p1 p2 p3) (then call4)

Ben Sless08:11:36

It can be easier with completable futures

Ben Sless08:11:58

I'd create a completable future then deliver it in the callback http-kit provides


i'd do something like (run! deref list-of-httpkit-futures) and then @(fourth-call) too

Ben Sless08:11:28

^ If you're okay with blocking


if youre passing data to the fourth call maybe: (->> list-of-httpkit-futures (map deref) (dorun) (fourth-call))


or return then whole thing in a promise like Ben said, this would be bad for long running things.


I would put a timeout on all API calls, http-kit client has a timeout key, for a couple of seconds. Then if it times out I'd use a default value instead of what ever a timeout call returns.


yeah the thing is would you be okay holding onto the client connection while this chain runs?


and also the thread servicing this particular request


I have a very generous 5 seconds to respond from what I understand, so hanging on to the initial request that triggers all these calls shouldn’t be an issue (although that will be tested 🙂 )


Somewhat related to the above (and apologies in advance for the question length): I have a question I have more or less answered for myself in the java world: "when is it worth it to do reactive programming". Reactive in this context refers to the pattern of programming where you use async/await, completablefuture, reactor/javarx in java, js promises etc. My conclusion in the java world is more or less in line to the one this presentation (which is great by the way) by Tomasz Nurkiewicz reaches: . I.e. if you are not netflix-scale, the drawbacks usually outweigh the benefits by a pretty wide margin. Much more complex code base, way higher cognitive load which means you need a sharper level of resource to work with it, debugging context is pretty much totally lost (your code is probably not even in the stack trace and where do you go from there?), etc etc. Would you say that this still holds true in clojure land or are there features or patterns that somehow alleviate some of the pain points?


tl;dr: It depends on the specific reactive solution. And not the HLL. "reactive" is a seriously overloaded (and sometimes abused term; I am looking at you FaceBook). More relevantly to your specific question, the programming language involved is largely unrelated. I say "largely" because most things turn out better when done in a Lisp. Anyway, Reactive at best is simple and transparent, but rarely do reactive libraries make things transparent. And your stated example, the pattern of combined Java features, does not sound very transparent or automatic, however excellent it may be at delivering reactivity. The reactive paradigm can feel like magic, but not if we have to hand-wire it. As for the scale issue, I would cite Tilton's Law: If a library should be used only at scale, then it should not be used. Getting back to HLL agnosticism, I have my own Clojure reactive library ( and one of the better write-ups is based on the Javascript implementation: btw, other nice reactive frameworks are the original MobX, binding.scala, and Clojure's own Hoplon/Javelin.


ps. I lied. 🙂 HLL-specific things like: • Clojure/Common Lisp macros; • first class functions; and • dynamic/special variables... ...can do wonders when it comes to hiding wiring/boilerplate. But JS does not have macros or special variables and Matrix/JS turned out OK, so maybe I only fibbed. Java, OTOH, with strong static-typing and -- well, how is it doing on first class functions? I hear it has finally gone there. Watching that video now. I am. concerned already. :)


Thank you @U0PUGPSFR Very informative. I will dig into the links and meditate over this.


@U0PUGPSFR reading through the matrix docs. I’m especially looking forwards to reading the origin story of cl cells : ) Given that matrix is somewhat geared towards solving the ui problem, how good a match would you say it is for the backend-server-at-scale case? (incidentally the case the yt video linked above deals with) I.e where you use reactive to be able to handle more requests per second etc…


And specifically for writing reactive server side code in clojure

Peter Boling12:11:29

In Javascript-world I believe that SvelteJS has cracked the nut on making reactive programming abstract and simple as pie. The perceived complexity is due to the tools, not the paradigm, IMO.


Should be noted that in the server-side java-world, the complexity and cognitive load of the code is only one part of the problem. When shit hits the fan and you get an exception, it is quite normal to lose most if not all of the context and for example end up with a stack trace where your code is nowhere in the stack trace...i.e. all you see is library code and there is no way to see what part of your code actually blew up or caused the issue.


UI is simply the poster child for the actual Matrix use case: "any application involving an interesting amount of long-lived state and a stream of unpredictable inputs." I did apply Matrix usefully to a RoboCup simulation league client, which simply gets a complete game view every 100ms. The unpredictability there came from the other players also acting on the simulated game state. @U4VDXB2TU Meanwhile, nice catch on the origin story! I almost mentioned that, very early on in the evolution of Matrix (aka Cells, ne Semaphor), you will see we make an effort to hide the wiring. We also leave the programmer freedom to handle the same property different ways, without resorting to subclassing. The point being that a powerful developer D/X comes from deliberate design. At the time I had had already a lot of experience with UI frameworks that were absolutely brutal to make dance, so I was on guard at every turn to keep things simple for the developer.


also on a totally separate track - project loom and virtual threads might be one solution to the server side scalability issue


@U0495TEG9C1 Ah, thanks, I was not aware of Svelte3. But why did they write a compiler to make count += 1 reactive? Matrix/JS uses defineProperty. Ah, here it is:

Importantly, we can do all this without the overhead and complexity of using proxies or accessors. 
...but in the preceding paragraph they said:
Since we're a compiler, we can do that by instrumenting assignments behind the scenes:
Sounds like overhead -- we can run but not hide! Meanwhile we have a preprocessing step. Ewww! 🙂 Anyway, kudos for their attention to D/X!

Peter Boling14:11:59

Yeah, beautifully simple comes at a cost 🥳


Well, I would invoke Occam. In Lisp we say never to use a macro where a function would suffice. I think the same goes for preprocessing. Ha. Come to think of, they are the same thing! 🙂


@U4VDXB2TU I watched most of the video. I agree with the last slide: think before using reactive. I have done a lot of ETL lately and never considered reactive: it fails the "interesting amount of long-lived state" criterion. As for managing a heavy request load, it is not clear to me how reactive was helping, so not sure what to say. I made good use of core.async for massive ETL, fwiw. I agree somewhat about the stacktrace problem, wherein we cannot see the whole process that was active when an exception is thrown, but that seems to be true of any system that has communicating subprocesses passing things around in queues. This is why we log, right? The exception shows the immediate local failure, the log lets us see the larger context? Mostly I go back to my first point: what reactive mechanism are we discoussing? Mr. Nurkiewicz twice says he is describing all reactive mechanisms. Hmmm....


@U0PUGPSFR on phone so limited bandwidth - in javaland the default behavior (at least until project loom and virtual threads) is for each incoming request to be served by a java thread which in turn has a one-to-one relationship with os threads. Thus if you are Netflix and have 10 000 concurrent connections to your server process, you will need 10 000 Java threads and 10 000 os threads and 10 000 sockets - which in turn is a problem for the os to handle (see c10k problem) So then you need to either buy a second machine to run a second server process or you use reactive to make fewer java/os threads do the same work by parking the thread when it is waiting for io or otherwise blocking and allowing the java thread to serve another request while that happens. Again, from a phone, not perfect but captures the gist.


Thx, @U4VDXB2TU. What I am hearing is that Java has issues because of that 1-1 relationship with a scarce OS resource, and that there is sth about some reactive tool that lets NetFlix work around that. Not sure, but I think this just drives home the ambiguity of "reactive". I guess Matrix could be used to the same end, with one thread serving many customers because each customer was behind its own Matrix cell...OK, now I see why HLL comes up. Would Erlang sneeze at the C10k problem? I am no expert, but methinks Clojure core.async has the same lightweight process "win" as Erlang.


Haha, great minds google alike! Saw that, too.

Peter Boling13:11:48

I been trying to figure out if there is a centralized package server / service, like Ruby has, and Rust has ... what is the equivalent for Clojure, if any?


Adding to the previous answers, with the Clojure CLI/deps.edn there is a trend (at least to some extent) to switch from maven to git as the delivery mechanism, so, if you use that, you might also consider your favorite code hosting service as a package server.

👏 1
Peter Boling13:11:12

On that note, I saw that moved from GitHub to, which I applaud heartily. Is there also a trend to move away from GitHub in Clojure generally, or was that a one off? (I am also leaving GitHub as fast as my feet can carry me)


no, but Clojure use GitHub only as a public repository. issues and ci delegated to over tools like Jira

🚀 1

also Clojure deps.edn doesn't care about actual hosting of the sources. it just must be available via git protocol

👍 1

Some deps.edn features though favor Github over the others, e.g. the reverse-notation syntax for defining coordinates, and I don't think there is a request to support more hosts/make it more generic (let alone a plan).

dpsutton14:11:33 lists shorthand for GitHub, gitlab, bitbucket, beanstalk, and one more I don’t recognize. I believe Alex adds in these as the community desires. Don’t think it’s particularly favored towards GitHub although that is surely the most popular


Alex is extremely responsive to community requests/questions! I just meant that nobody asked for the less popular ones, which makes sense by definition. 🙂 > beanstalk, and one more Those must be recent additions because last I checked (months ago) I only saw a couple (github, gitlab and bitbucket I think). It's great that support is improving. By the way the one you don't recognize is!

👍 1
Peter Boling03:11:45

I am a fan of (and $ supporter of) sourcehut! Very happy to see it included!

👍 1

I'm looking for a persistent async task scheduler (like sidekiq or delayed_job for Ruby). Is there anything like that for Clojure?

Ben Sless13:11:01

Maybe #C03RCGK0P8S

👍 1

thanks i'll check that out!

Jared Langson21:11:14

Is it bad to use for for impure functions? I want to read a series of files and transform the data into a map and return it. A sketch

(defn files->map
  (for [file-path file-paths
        ls (line-seq (io/reader file-paths))]
    ;;does stuff to generate a map from the contents of ls
What's the idiomatic way for doing this?

Ben Sless21:11:00

In short, yes, but you can get away with it wrapping it in doall or (into [] ,,,)

Mario Trost21:11:05

because for (and map) return lazy sequences, you won't be able to do this

Mario Trost21:11:16

That's a good article about that

Ben Sless21:11:20

Sometimes the solution is easier to read with a for comprehension

Ben Sless21:11:38

Just make sure to strictify the result before you leave 🙃

Mario Trost21:11:51

And that is the author of that article 😄 Take his advice, not mine

Ben Sless21:11:06

Ensure it is evaluates strictly, using something to realize the entire lazy computation like doall or into

Jared Langson21:11:24

As an alternative to for, is using map multiple times better or an abomination? I can change it to mapv to make it eager

(defn files->map!
    (let [line-seqs (->> file-paths (map io/reader) (map line-seq))
          delimeters (->> line-seqs (map second) (map detect-delimeter))
          headers (->> line-seqs (map first) (map transform-header))
          rows (->> line-seqs (map rest) (map split-row delimeters))]
      (map transform-header-rows->map headers rows)))


I think map is particularly good at hiding side effects from someone reading your code, because of the implication of its meaning. But mapv would at least bring me pause, and in fact make me suspicious of side effects, because "Why else would you not just use map?" 👍


I would consider this a good spot for You can separate out the "what we're doing" by combining your calls to map into a single transducer, which gives you the benefit of only doing a single pass over your data instead of multiple. You can then use the transducer with either into for eager evaluation or sequence for lazy.


Alternatively for this simpler case, instead of using transducers, you can combine your multiple calls with a comp. E.g., instead of (->> file-paths (map io/reader) (map line-seq)) you can do (map (comp line-seq io/reader) file-paths). This gets you the single pass over your data.