Fork me on GitHub
Anik Chowhdury08:08:29

Hello, i am newbie to clojure. Can anyone guide me in core async about how to check if a channel is empty or not...


You'll know when you take from a channel and receive nil


AFAIK there's no way to know without taking a value from the channel. That's by design I believe.


you dont want to check that a channel is empty. That's "non blocking". Channels have blocking semantics. That's the whole point.

💯 4

You can provide your own implementation of clojure.core.async.impl.protocols/Buffer to chan function


what are you trying to do?


(a closed channel will always immediately return nil when pulled from... but that's different to "empty")

👍 8

maybe what you want is a process-of-time construct. like an atom?

Adrian Smith12:08:25

How do I get an infinite set of odd numbers, where if I take say 5 numbers, when I ask again for 5 numbers they'll be the "next" numbers in the set?


you would make a new var and bind to the rest of the sequence. for example:


(let [all-odds (filter odd? (range))
      first-five (take 5 all-odds)
      remaining (drop 5 all-odds)]
  ;; do things with vars here


there is a clojure functions split-at that does both the take and drop together, so once you wrap your head around that, you could express it like:


(let [all-odds (filter odd? (range))
      [first-five remaining] (split-at 5 all-odds)]
  ;; do things with vars here


if you were doing it over and over again, consider this piece of code:


(loop [remaining (filter odd? (range))]
  (let [[head tail] (split-at 5 remaining)]
    ;; do something with the 5 in head here
    (when (continue?)
      (recur tail))))


where continue? is some function that returns false when you want to stop (or could be some code in here)

Ahmed Hassan17:08:06

Great, so dropping is necessary.


I used this re-frame template for shadow-cljs (, but when I do shadow-cljs release app weird things happen to the paths/routes, I need to manually remove the trailing / from index.html and also when I click on any of my links they redirect to C://link for some reason, any ideas about what's going on?


@shin those examples assume that you are using a webserver. when loading the files directly from disk you need to adjust the paths


so everything should be fine if I serve the public folder?


greats, thanks! 🙂


@sfyire To get an infinite sequence of odd numbers, you could either write a lazy-seq, or just do (filter odd? (range))

wagner leonardi16:08:19

Good morning everyone! I'm still trying to achieve "multiple blocking IO requests asynchronously" with clojure.async:

(def channel (a/chan 1))

(defn get-articles []
  (doseq [article articles]
    (let [url (get-url article)]
      (a/go (a/>! channel (request url))))))

(defn print-articles []
  (dotimes [_ (count articles)]
    (println (a/<!! (a/go (a/<! channel))))))

(defn -main [& args]
It's working, but I want feedback because my biggest concern is to be accepted in a Clojure job.. 😅 , and it feels too "loose".. I'm not secure that I'm doing the "right way".


(a/go (a/>! channel (request url))) is just put!


theres no point in a go block with only one <!,>! op


can you explain at a high level whats it trying to do?


get all the URLs in parallel?


get-articles does blocking IO (network request) in a Go block which is a big no-no, if you do that you can end up exhausting the core.async thread pool which means that all go blocks will grind to a halt

wagner leonardi16:08:00

@UKH2HDSQH yes, getting URLs in parallel.. without blocking the "main thread"


I would do something like have a single channel like you have. Issue the requests one after the other non blocking with a callback put! ing the response on a a channel


and then just pull n times from that channel


let me try


Im more used to cljs using the js fetch API, but Ill try with http-kit

wagner leonardi16:08:24

@UKH2HDSQH my goal is to achieve that with using blocking APIs


if you want to use a blocking API in a go block, you have to call that api with something like a Java Executor, blocking APIs just don’t work in go blocks

wagner leonardi16:08:07

@U3L6TFEJF so, last week I brought a solution using (future but I received many feedbacks that I'm still blocking threads.. so I thought (go would be a solution for that. So basically, threads is better due performance but will still blocks due IO (and I just need to take care with how many threads do I spawn), and go blocks really solve that blocking IO but has worse performance? did I get it right?


not the way I'd want to do it, but here's an untested first cut:


(defn get-urls [urls options]
  (let [c (chan)]
    (doseq [url urls]
      (http/get url options #(put! c %)))

(defn get-all-articles [urls]
  (let [num (count urls)
        result-chan (get-urls urls)]
    (loop [results []]
      (if (< (count results) num)
        (recur (conj results (<!! result-chan)))


dont like the (< (count results) num)


you need to close the channel somewhere, if you do that you can replace that conditional with checking for nil from (<!! result-chan)


but how do you know when to close the channel?


I would probably just spawn getters and have them swap! into an atom with conj


you could put the close in the callback of the last url


no that wont work

wagner leonardi16:08:20

@U3L6TFEJF let me say I'm creating an webserver that I'll always listen to requests. Do I really "need to close the channel somewhere" as well?


no, you’re right, spoke to soon


you don't know the order they will come back in. the last one could be a small file. and the first one huge


right, I think the way you’re doing it seems good enough 🙂


the thing about channels is they are about "inverting control". That is, turning callback style async, into blocking code


but, if you don't want to block... then they're kind of moot


so, if you wanted to get one url at a time


but all you had is callbacks


bingo. channels are ace


if you don't care about blocking one thing, before doing another... clojure has awesome proceeds of time constructs


let me write using atoms...

wagner leonardi16:08:41

the idea is not blocking at all, as I'm used in Node.js


just as an excercise?


yeah exactly


so core.async takes things that are not blocking (think callbacks) and makes them appear to be blocking


so you can do A... and then do B... and then make a decision and do C or D...


when A, B, C and D are not blocking code


have you seen rob pikes go example in core.async?


so that example is an example of a good use case


these things fire off in parallel


but I only want the fastest 3


with a global timeout


if I just want them all, I could just use an atom


because atoms are not blocking, thread safe, time constructs

wagner leonardi16:08:25

@UKH2HDSQH does it is similar as Promise.all from js? (the behavior not the concept)


have you used atoms yet? they are like one of the core things


I use them all the time

wagner leonardi16:08:20

@UKH2HDSQH yes, If I get it right, does it is similar as actors, right?


a part of an actor


let me do it with an atom for you to compare


the only problem with the atom is being non blocking, you don't know when its "finished"

wagner leonardi16:08:01

@UKH2HDSQH could you tell me why you "use them all the time" ? I can only think on them to manipulate state where are happening concurrent stuff so I avoid all the time 😂 that's why I'm curious


but I'll just hack that bit to show the case

wagner leonardi16:08:26

@UKH2HDSQH or do you use atoms just like variables to mutate data?


(def results (atom []))

(def get-options {})

(defn get-urls [urls]
  (doseq [url urls]
    (http/get url get-options #(swap! results conj %))))

(defn get-all-articles [urls]
  (get-urls urls)

  ;; now we just watch the atom till its "full"
  (let [num (count urls)]
    (while (< (count @results) num)
      (Thread/sleep 1000)))

  ;; atom is now full
  (println @result))


the waiting till full is hacky... but two points here


1. in practice you normally don't need to "wait" until its full. programs tend to be more dynamic and reactive than the naive example


and 2. there are better ways. For example you can register a 'watcher' on the atom


so derefing the atom @results is non-blocking


at any time, in any thread, you can look at @result and get the present state


without locks or blocking anything else (persistent data structures)


I'll add a watcher to detect when it's done (but be careful your program might exit too soon)

wagner leonardi16:08:39

@UKH2HDSQH thank you so much! I only wouldn't be much comfortable with this "listening part" for a job part, but would you accept it if you was the approver?

wagner leonardi16:08:04

@UKH2HDSQH I did (.join (Thread/currentThread)) to program stay awake hehe


(def results (atom []))

(def get-options {})

(defn get-urls [urls]
  (doseq [url urls]
    (http/get url get-options #(swap! results conj %))))

(defn get-all-articles [urls]
  (let [num (count urls)
        finished (chan)]
    (get-urls urls)
    (add-watch results :finished?
               (fn [_ _ _ _ val]
                 (when (= num (count val))
                   (close! finished))))

    (<!! finished))
  ;; atom is now full
  (println @result))


using a channel to signal the finish...


but maybe even better, lets push the result down the channel when ended


wait have to fix that 1 sec. didnt call get-urls

wagner leonardi16:08:15

@UKH2HDSQH I got the Idea, thank you so much!


now bring the atom into the let scope and push result down channel

wagner leonardi16:08:55

yes, I'm gonna try that ! I own you several beers if I get a Clojure job 😂


(defn get-urls [result urls]
  (doseq [url urls]
    (http/get url {} #(swap! results conj %))))

(defn get-all-articles [urls]
  (let [result (atom [])
        num (count urls)
        finished (chan)]
    (get-urls result urls)
    (add-watch results :finished?
               (fn [_ _ _ _ val]
                 (when (= num (count val))
                   (put! finished val))))

    (println (<!! finished))))


now its looking tighter


all of this Im just hacking from my head. havent run any of it


just remember channels are to turn a non-blocking callback thingie... into a blocking peice of code


so in that case, we want to block until its finished... a-lah the channel


but the watcher is a callback... so we just put! on the channel


Ill get the epoch model of time diagram for you to visualise atoms

wagner leonardi17:08:36

@UKH2HDSQH thank you! That's something I'm missing, I can only think in single channels always (I think due my Node.js background) that was a good definition.. "everytime I want to "wait" for non-blocking calls" I can create a new channel?


yeah. if you don't need to wait, dont wait!


you can create many channels, or just use one and stuff many things on em


also different ends of channels can be different forms (go vs threads)


and can have multiple readers or writers


its not a 1-1


watch RIch's are we there yet talk to get an overarching view of how to think about time in a clojure way with immutable data


your atom is the baseball match


stop trying to stop the baseball match


just take a photo of it


and then another


and compare the photos


this is core clojure stuff. core.async is an addon library


you want a clojure job


get this core stuff 100%

wagner leonardi17:08:07

@UKH2HDSQH I was afraid to look for solutions that I get used in the past (ex. JS promises, C# async) rather than "understand a clojure way" .. that's why I'm trying to learn clojure.async as "default" way before trying to look for other solutions And thank you so much for your effort, I really appreciate, I'm going to experiment your suggestions.. and watch that talk. The previous one you recommended was very helpful! Thank you so much 😄


I love riches talks. opened my mind to new ways of thinking about the problems


we get stuck in old paradigms and patterns


like "locks". Hey stop the world. I want to do something.


but as you will see in this talk he advocates for not stopping the world


to get highly concurrent, highly parallel, we have to stop trying to "stop the world"


and just let the river flow


after that talk try "the value of values"


another great talk


hello, I successfully completed the Armstrong numbers exercise on exercism and am seeking advice: 1) I feel my algorithm to break up a number into a collection of it's digits (the base portion of my let bindings) is clunky. Any advice there? I usually just generate such expressions in the repl, adding more to the front as I see my given output. Probably not the best way to learn the fundamentals? Is it bad form to use threading macros inside a let binding? 2) Is the use of the numeric-tower library idiomatic? I feel I shouldn't need another dependency but using Math/pow didn't work on the largest test case so wondering if I should just find a way to convert to a big int (?) somewhere instead? Any other advice?

(ns armstrong-numbers                                                              
  (:require [clojure.math.numeric-tower :as math]))                                
(defn armstrong? [num]                                                             
  (let [base (map #(Integer/parseInt %) (map str (seq (str num))))                 
        exp (count (str num))]                                                     
    (= num (apply + (map #(math/expt % exp) base)))))


some quick pointers (can dive into more detail if you are interested): - you can use threading macros (`->>`) or transducers to clean up the base expression. it’s absolutely not bad form to use threading in a let, quite the contrary! - prefer reduce or transduce to apply +... - converting to a BigInteger sounds like a good idea to avoid the extra dependency!


or you can use +' to for arbitrary precision arithmetic:'


awesome. thanks again for your help!


I'm curious why no one wants to use apply when it's technically what I want it to look like (+ ...)?


in this case it’s about performance: reduce will be faster, transduce faster still


apply is mostly used to “spread” arrays as arguments to varargs functions:


good stuff!


found a cleaner way to do my base: (map #(Character/digit % 10) (str num)) but not sure if that aids readability for others or not.

Val Baca18:08:38

Morning all. I've got a fn that takes an 4 args and returns the first 3 modified based on the 4th.

(defn my-fn
  "takes a b c and instruction z. Returns a' b' c'"
  [a b c z] ...)
Where a b c are values, z tells the function what should happen. So the outputs are the new values: a', b', and c' I want to loop over a seq of zs In non-lisp pseudocode I want to do:
a, b, c = ...
for z in zs:
  a, b, c = my-fn(a, b, c, z)
return a, b, c
Is my best bet to turn abc into a vector so I can use reduce? or is there a better way?


Yup. My suggestion is to do something like this: (reduce f {:a a :b b :c c} zs) where f is a function that takes two args: a map with keys :a, :b and :c, and a z, and returns a new map with same keys, a, b, and c are initial values of a, b and c and zs is a sequence of z's you want to apply.

Alex Miller (Clojure team)18:08:13

you don't have to turn abc into a vector (you can use an anonymous function that applies things appropriately), but I think conceiving of it as a reducing function certainly is a good conceptual match

Val Baca21:08:04

First, thanks again for the help y'all have given me so far. I'm trying to learning clojure by working through Advent of Code problems. I don't know if this is done here, but I was wondering if I could get some feedback (like a code review) on my first clojure code here: For context, Here's the problem:


BTW. There's also #adventofcode


@UM1CNA7FA Just had a quick look, and it looks pretty good. The first part, where you’re just getting the absolute distance, is pretty much the way I would have written it.


The second part, where you find the first location visited twice, I think I would do differently. You’re using reduce and iterate and so on, which is good, functional programming style, but I think in this case you might be better off using a loop/`recur`. It may seem like cheating--“I’m trying to learn functional programming and this is just a plain old loop!“--but I think it’s actually a better choice here because you want to be able to stop as soon as you find the first previously-visited location. The iterate/`reduce` approach works, but once you find the visited location, you have to no-op through all the rest of the instructions. With a loop, you can drop out as soon as visited is no longer nil.


Also check out the reduced function, it might come in handy if you’re using reduce to go through the list of instructions.

Val Baca03:08:56

Thank you for the feedback @U06CM8C3V! I find the hardest thing coming to Clojure/Lisp from a Java/JS background is just finding things to replace the good ol' imperative for-loop. I'll look into reduced. Thanks!


I had the same experience coming from PHP, but once you get used to it, it’s really natural to start thinking in terms of map and filter and reduce and so on. Good old loop/`recur` still has its uses though. 🙂

Brandon Olivier22:08:19

I’m having a weird problem with hugsql, where I can’t reload the queries sql file without getting an error if (and only if) the queries file contains clojure expression comments

Brandon Olivier22:08:53

I’m using cider to reload this


@brandon149 Given that it is SQL, I would not expect CIDER to be able to (re-)load it. It is not Clojure code, after all.

Brandon Olivier22:08:04

I should clarify, I initialized this project with the luminus template and there is a user ns that has a function to reload the queries


And you're trying to re-load user.clj?

Brandon Olivier22:08:06

I was calling this from cider originally, but I tried it from the repl, and the error persisted. I’m not sure why, but it’s erroring on the final sexp

Brandon Olivier22:08:15

I’m trying to call this restart-db function


And in the sql/queries.sql file you have...?


It's SQL so "clojure expression comments" won't work, only SQL comments.

Brandon Olivier22:08:38

It’s a mostly normal sql file, but I’m using hugsql to add some special clojure expressions that get interpreted.

Brandon Olivier22:08:02

Everything with it seems to be working, including normal comments that get extracted as fn names


What do you mean by "normal comments" in this context?

Brandon Olivier22:08:45

hugsql uses sql comments to extract function names for the queries you define

Brandon Olivier22:08:00

something like

-- :name get-person
select * form persons;


OK, SQL comments.

Brandon Olivier22:08:49

I’m wondering if I need to add some other function call to that bind-connection call to get those comments set up properly

Brandon Olivier22:08:13

the clojure expression comments. I don’t recall if the template came with hugsql or if I added it right afterwards


Can you explain about the Clojure code you're embedding?


You are using --~ for single line Clojure code in the SQL comments?

Brandon Olivier22:08:12

But sometimes those comments leave an empty WHERE clause, for instance.


Share the code you're having problems with...

Brandon Olivier22:08:53

I have this snippet in my queries.sql file. It works normally, but if I try to reload the file, it produces an error


Why do you have ; in those strings?

Brandon Olivier22:08:04

That’s an oversight on my part. I was tinkering with this. They were meant to terminate the sql expression


The examples in the HugSQL docs don't seem to need ; to terminate statements...? (and you already have a ; there to terminate that one, so you'd end up with ;; in the substituted code I think...

Brandon Olivier22:08:50

That sounds correct. I don’t recall which sql dialect the docs are using. I’m using postgres, which I believe does require the semicolon. It does in psql, at least.


Yes, in the console tool, I'd expect it to require ; to terminate. But not in HugSQL files.


HugSQL uses blank lines between blocks to indicate the end of a SQL statement, based on the docs.


Anyways, if you can repro reliably, share the exact code that causes a problem when you attempt that bind-connection call. That way we'll have something concrete to try to debug to help you...

Brandon Olivier22:08:10

What’s the best way to share something like that?


Pastebin or a Gist or some similar service.


If it's a small snippet, you can share it directly in Slack.


It sounds like this would be a single line containing the --~ form that causes the problem?

Brandon Olivier23:08:04

I started a new project with luminus to narrow everything down. It’s the one line with the comment

Brandon Olivier23:08:45

With this code, calls to restart-db error out saying that clojure.lang.Symbol cannot be cast to clojure.lang.Namespace

Brandon Olivier23:08:57

That’s the entire contents of that file


And that's with calling restart-db in the REPL?

Brandon Olivier23:08:32

With these contents, the function works normally


And what command did you use to create the new project? (so I can replicate it here)

Brandon Olivier23:08:17

lein new luminus tester +postgres

Brandon Olivier23:08:55

The template has one migration, but I replaced it with this.


And exactly what steps did you take to get to the point of calling (restart-db)? (if I just run lein repl in the new project and then (restart-db) I get mount.core.DerefableState cannot be cast to clojure.lang.IFn so I assume some other steps are needed first?)


Just noticed in your code above you have (if (nil? (:username) params) which is illegal -- I think you meant (if (nil? (:username params)) instead


(Idiomatically, it's better to write (if (:username params) instead and swap the then/else expressions BTW)


The code you share should give this error Wrong number of args (2) passed to: clojure.core/nil? so I'm not sure that's the actual cause...


(as an aside, this all illustrates why I think beginners should not start out with Luminus -- but should instead start out with a simple Ring app and gradually add Compojure, Mount (or Component), (or next.jdbc), then perhaps HugSQL, one at a time, to learn how all the pieces work)


Luminus is great when it works... but when something goes wrong and you're just starting out, there's so many moving parts in the Luminus template that it can be really hard for a beginner to figure out what is going wrong.

Brandon Olivier23:08:28

In an ideal world, I’d agree, but I also want to move more quickly to get something finished. Which I guess in this case has bit me in the rear end.

Brandon Olivier23:08:43

I was thinking that because the error mentions not being able to cast symbols to namespaces, there might be something necessary in the call to bind-connection rather than in the queries file itself.

Brandon Olivier23:08:04

I managed to get a fix working but because I don’t totally understand some of the specifics of namespaces, I’m not certain of all the consequences

Brandon Olivier23:08:07

I moved the binding of the queries into a function inside the db.core ns and directly called that ns function instead of using (binding [*ns* 'tester.db.core] ...)

Brandon Olivier23:08:32

@seancorfield is that something that makes sense to you as a fix?


Not really, no. I mean a freshly generated template works when used properly so the right "fix" is to figure out what's going wrong in the changes you've made.


This is exactly what I mean about beginners and Luminus -- if something "doesn't work", there's so much machinery in the template that debugging it is hard if you're a beginner. There's just no short cut here.


As you can see "moving quickly" within such a large pile of code that you don't yet understand is likely to continue to bite you, unless you take a step back and learn the basics I think.


I generated a fresh project from the template. I started the REPL. I figured out I needed to run (start) first and then (restart-db) -- and that all worked so far.


When you're making changes, you need to work with a very small, fast feedback loop. So, make a single small change and test it works. If it doesn't, you know the breakage was caused by that single change so it's easier to debug.


As you get more confident and more comfortable, you can make bigger changes in each cycle. But at first you need to be very systematic about testing each change.


do any of you use monads in your projects? Like the Maybe monad or a Log Writer monad?


I think there are some problems for which the natural solution happens to be structured like a particular monad but monads are nearly always tied to type systems so "using monads" in Clojure looks somewhat different to, say, Haskell, and if you really try to "use monads" (such, as, for example, the algo.monad Contrib library), you end up with rather non-idiomatic Clojure code a lot of the time.


I work with @seancorfield and at one point we had this pipeline of code doing a data migration from an old system to a new one, and was basically built around what would be the Either monad, each step in the pipeline either apply some function to some value, which might return an error, or it would pass through the error from an earlier step. We didn't say "oh look a monad", we didn't write a monad library for it, it was just kind of there with a few helper functions to make it nicer, and it went away when the data migration served its purpose.


I'd say to some degree monads in clojure are more about just recognizing patterns when they arise vs. using a library of monads, and are often more ephemeral, because of the lack of emphasis on types


yea, I thought they wouldn't be such a big deal as a library in clojure -- however, I think the Either/`Maybe` monad has a lot of use in clojure and I'm surprised I don't see it more. Perhaps because it's so easy to just (try ...


At one point we also used this library that I wrote and it's essentially monadic -- but code written with it turns out to be harder to read and debug than "regular" idiomatic Clojure, in my opinion.

👍 4