This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-08-05
Channels
- # announcements (3)
- # beginners (225)
- # calva (3)
- # cider (110)
- # circleci (18)
- # clj-kondo (15)
- # cljdoc (1)
- # cljsrn (12)
- # clojure (77)
- # clojure-dev (39)
- # clojure-europe (3)
- # clojure-houston (2)
- # clojure-italy (9)
- # clojure-nl (16)
- # clojure-romania (1)
- # clojure-spec (5)
- # clojure-uk (20)
- # clojuredesign-podcast (28)
- # clojurescript (89)
- # core-async (4)
- # cursive (10)
- # datomic (3)
- # defnpodcast (5)
- # emacs (17)
- # events (1)
- # figwheel (4)
- # graalvm (6)
- # juxt (1)
- # pathom (4)
- # pedestal (5)
- # re-frame (4)
- # remote-jobs (3)
- # rewrite-clj (4)
- # shadow-cljs (90)
- # spacemacs (2)
- # sql (7)
- # tools-deps (4)
- # vim (52)
- # xtdb (7)
Hello, i am newbie to clojure. Can anyone guide me in core async about how to check if a channel is empty or not...
You'll know when you take from a channel and receive nil
AFAIK there's no way to know without taking a value from the channel. That's by design I believe.
you dont want to check that a channel is empty. That's "non blocking". Channels have blocking semantics. That's the whole point.
You can provide your own implementation of clojure.core.async.impl.protocols/Buffer to chan
function
http://tgk.github.io/2013/10/inspect-core-async-channels.html
(a closed channel will always immediately return nil when pulled from... but that's different to "empty")
How do I get an infinite set of odd numbers, where if I take say 5 numbers, when I ask again for 5 numbers they'll be the "next" numbers in the set?
(let [all-odds (filter odd? (range))
first-five (take 5 all-odds)
remaining (drop 5 all-odds)]
;; do things with vars here
)
there is a clojure functions split-at
that does both the take and drop together, so once you wrap your head around that, you could express it like:
(let [all-odds (filter odd? (range))
[first-five remaining] (split-at 5 all-odds)]
;; do things with vars here
)
(loop [remaining (filter odd? (range))]
(let [[head tail] (split-at 5 remaining)]
;; do something with the 5 in head here
(when (continue?)
(recur tail))))
where continue?
is some function that returns false when you want to stop (or could be some code in here)
Great, so dropping is necessary.
I used this re-frame template for shadow-cljs (https://github.com/shadow-cljs/examples/tree/master/re-frame), but when I do shadow-cljs release app weird things happen to the paths/routes, I need to manually remove the trailing / from index.html and also when I click on any of my links they redirect to C://link for some reason, any ideas about what's going on?
@shin those examples assume that you are using a webserver. when loading the files directly from disk you need to adjust the paths
@sfyire To get an infinite sequence of odd numbers, you could either write a lazy-seq, or just do (filter odd? (range))
Good morning everyone! I'm still trying to achieve "multiple blocking IO requests asynchronously" with clojure.async:
(def channel (a/chan 1))
(defn get-articles []
(doseq [article articles]
(let [url (get-url article)]
(a/go (a/>! channel (request url))))))
(defn print-articles []
(dotimes [_ (count articles)]
(println (a/<!! (a/go (a/<! channel))))))
(defn -main [& args]
(get-articles)
(print-articles))
It's working, but I want feedback because my biggest concern is to be accepted in a Clojure job.. 😅 , and it feels too "loose".. I'm not secure that I'm doing the "right way".get-articles
does blocking IO (network request) in a Go block which is a big no-no, if you do that you can end up exhausting the core.async thread pool which means that all go blocks will grind to a halt
this blog post should be of use: https://martintrojer.github.io/clojure/2013/07/07/coreasync-and-blocking-io
@UKH2HDSQH yes, getting URLs in parallel.. without blocking the "main thread"
I would do something like have a single channel like you have. Issue the requests one after the other non blocking with a callback put! ing the response on a a channel
@UKH2HDSQH my goal is to achieve that with using blocking APIs
if you want to use a blocking API in a go block, you have to call that api with something like a Java Executor, blocking APIs just don’t work in go blocks
@U3L6TFEJF so, last week I brought a solution using (future
but I received many feedbacks that I'm still blocking threads.. so I thought (go
would be a solution for that.
So basically, threads is better due performance but will still blocks due IO (and I just need to take care with how many threads do I spawn), and go
blocks really solve that blocking IO but has worse performance? did I get it right?
(defn get-urls [urls options]
(let [c (chan)]
(doseq [url urls]
(http/get url options #(put! c %)))
c))
(defn get-all-articles [urls]
(let [num (count urls)
result-chan (get-urls urls)]
(loop [results []]
(if (< (count results) num)
(recur (conj results (<!! result-chan)))
results))))
you need to close the channel somewhere, if you do that you can replace that conditional with checking for nil from (<!! result-chan)
@U3L6TFEJF let me say I'm creating an webserver that I'll always listen to requests. Do I really "need to close the channel somewhere" as well?
you don't know the order they will come back in. the last one could be a small file. and the first one huge
the thing about channels is they are about "inverting control". That is, turning callback style async, into blocking code
if you don't care about blocking one thing, before doing another... clojure has awesome proceeds of time constructs
@UKH2HDSQH I actually care 😞
the idea is not blocking at all, as I'm used in Node.js
so core.async takes things that are not blocking (think callbacks) and makes them appear to be blocking
@UKH2HDSQH yes I did
@UKH2HDSQH does it is similar as Promise.all
from js? (the behavior not the concept)
@UKH2HDSQH yes, If I get it right, does it is similar as actors, right?
the only problem with the atom is being non blocking, you don't know when its "finished"
@UKH2HDSQH could you tell me why you "use them all the time" ? I can only think on them to manipulate state where are happening concurrent stuff so I avoid all the time 😂 that's why I'm curious
@UKH2HDSQH or do you use atoms just like variables to mutate data?
(def results (atom []))
(def get-options {})
(defn get-urls [urls]
(doseq [url urls]
(http/get url get-options #(swap! results conj %))))
(defn get-all-articles [urls]
(get-urls urls)
;; now we just watch the atom till its "full"
(let [num (count urls)]
(while (< (count @results) num)
(Thread/sleep 1000)))
;; atom is now full
(println @result))
1. in practice you normally don't need to "wait" until its full. programs tend to be more dynamic and reactive than the naive example
I'll add a watcher to detect when it's done (but be careful your program might exit too soon)
@UKH2HDSQH thank you so much! I only wouldn't be much comfortable with this "listening part" for a job part, but would you accept it if you was the approver?
@UKH2HDSQH I did (.join (Thread/currentThread))
to program stay awake hehe
(def results (atom []))
(def get-options {})
(defn get-urls [urls]
(doseq [url urls]
(http/get url get-options #(swap! results conj %))))
(defn get-all-articles [urls]
(let [num (count urls)
finished (chan)]
(get-urls urls)
(add-watch results :finished?
(fn [_ _ _ _ val]
(when (= num (count val))
(close! finished))))
(<!! finished))
;; atom is now full
(println @result))
@UKH2HDSQH I got the Idea, thank you so much!
yes, I'm gonna try that ! I own you several beers if I get a Clojure job 😂
(defn get-urls [result urls]
(doseq [url urls]
(http/get url {} #(swap! results conj %))))
(defn get-all-articles [urls]
(let [result (atom [])
num (count urls)
finished (chan)]
(get-urls result urls)
(add-watch results :finished?
(fn [_ _ _ _ val]
(when (= num (count val))
(put! finished val))))
(println (<!! finished))))
just remember channels are to turn a non-blocking callback thingie... into a blocking peice of code
@UKH2HDSQH thank you! That's something I'm missing, I can only think in single channels always (I think due my Node.js background) that was a good definition.. "everytime I want to "wait" for non-blocking calls" I can create a new channel?
watch RIch's are we there yet talk to get an overarching view of how to think about time in a clojure way with immutable data
@UKH2HDSQH I was afraid to look for solutions that I get used in the past (ex. JS promises, C# async) rather than "understand a clojure way" .. that's why I'm trying to learn clojure.async as "default" way before trying to look for other solutions And thank you so much for your effort, I really appreciate, I'm going to experiment your suggestions.. and watch that talk. The previous one you recommended was very helpful! Thank you so much 😄
to get highly concurrent, highly parallel, we have to stop trying to "stop the world"
hello, I successfully completed the Armstrong numbers exercise on exercism and am seeking advice: 1) I feel my algorithm to break up a number into a collection of it's digits (the base
portion of my let
bindings) is clunky. Any advice there? I usually just generate such expressions in the repl, adding more to the front as I see my given output. Probably not the best way to learn the fundamentals? Is it bad form to use threading macros inside a let
binding? 2) Is the use of the numeric-tower
library idiomatic? I feel I shouldn't need another dependency but using Math/pow
didn't work on the largest test case so wondering if I should just find a way to convert to a big int (?) somewhere instead? Any other advice?
(ns armstrong-numbers
(:require [clojure.math.numeric-tower :as math]))
(defn armstrong? [num]
(let [base (map #(Integer/parseInt %) (map str (seq (str num))))
exp (count (str num))]
(= num (apply + (map #(math/expt % exp) base)))))
some quick pointers (can dive into more detail if you are interested):
- you can use threading macros (`->>`) or transducers to clean up the base
expression. it’s absolutely not bad form to use threading in a let, quite the contrary!
- prefer reduce
or transduce
to apply +...
- converting to a BigInteger sounds like a good idea to avoid the extra dependency!
or you can use +'
to for arbitrary precision arithmetic: https://clojuredocs.org/clojure.core/+'
I'm curious why no one wants to use apply
when it's technically what I want it to look like (+ ...)
?
apply is mostly used to “spread” arrays as arguments to varargs functions: https://clojuredocs.org/clojure.core/apply
found a cleaner way to do my base: (map #(Character/digit % 10) (str num))
but not sure if that aids readability for others or not.
Morning all. I've got a fn that takes an 4 args and returns the first 3 modified based on the 4th.
(defn my-fn
"takes a b c and instruction z. Returns a' b' c'"
[a b c z] ...)
Where a b c are values, z tells the function what should happen.
So the outputs are the new values: a', b', and c'
I want to loop over a seq of zs
In non-lisp pseudocode I want to do:
a, b, c = ...
for z in zs:
a, b, c = my-fn(a, b, c, z)
return a, b, c
Is my best bet to turn abc into a vector so I can use reduce? or is there a better way?Yup. My suggestion is to do something like this: (reduce f {:a a :b b :c c} zs)
where f
is a function that takes two args: a map with keys :a
, :b
and :c
, and a z, and returns a new map with same keys, a
, b
, and c
are initial values of a, b and c and zs
is a sequence of z's you want to apply.
you don't have to turn abc into a vector (you can use an anonymous function that applies things appropriately), but I think conceiving of it as a reducing function certainly is a good conceptual match
First, thanks again for the help y'all have given me so far. I'm trying to learning clojure by working through Advent of Code problems. I don't know if this is done here, but I was wondering if I could get some feedback (like a code review) on my first clojure code here: https://github.com/valbaca/AdventOfCode2016-Clojure/blob/master/src/advent_2016/day01.clj For context, Here's the problem: https://adventofcode.com/2016/day/1
@UM1CNA7FA Just had a quick look, and it looks pretty good. The first part, where you’re just getting the absolute distance, is pretty much the way I would have written it.
The second part, where you find the first location visited twice, I think I would do differently. You’re using reduce
and iterate
and so on, which is good, functional programming style, but I think in this case you might be better off using a loop
/`recur`. It may seem like cheating--“I’m trying to learn functional programming and this is just a plain old loop!“--but I think it’s actually a better choice here because you want to be able to stop as soon as you find the first previously-visited location. The iterate
/`reduce` approach works, but once you find the visited location, you have to no-op through all the rest of the instructions. With a loop, you can drop out as soon as visited
is no longer nil.
Also check out the reduced
function, it might come in handy if you’re using reduce
to go through the list of instructions.
Thank you for the feedback @U06CM8C3V! I find the hardest thing coming to Clojure/Lisp from a Java/JS background is just finding things to replace the good ol' imperative for-loop. I'll look into reduced. Thanks!
I had the same experience coming from PHP, but once you get used to it, it’s really natural to start thinking in terms of map
and filter
and reduce
and so on. Good old loop
/`recur` still has its uses though. 🙂
I’m having a weird problem with hugsql, where I can’t reload the queries sql file without getting an error if (and only if) the queries file contains clojure expression comments
I’m using cider to reload this
@brandon149 Given that it is SQL, I would not expect CIDER to be able to (re-)load it. It is not Clojure code, after all.
I should clarify, I initialized this project with the luminus template and there is a user
ns that has a function to reload the queries
And you're trying to re-load user.clj
?
I was calling this from cider originally, but I tried it from the repl, and the error persisted. I’m not sure why, but it’s erroring on the final sexp
I’m trying to call this restart-db
function
And in the sql/queries.sql
file you have...?
It's SQL so "clojure expression comments" won't work, only SQL comments.
It’s a mostly normal sql file, but I’m using hugsql to add some special clojure expressions that get interpreted.
Everything with it seems to be working, including normal comments that get extracted as fn names
What do you mean by "normal comments" in this context?
hugsql uses sql comments to extract function names for the queries you define
something like
-- :name get-person
select * form persons;
OK, SQL comments.
correct
I’m wondering if I need to add some other function call to that bind-connection call to get those comments set up properly
the clojure expression comments. I don’t recall if the template came with hugsql or if I added it right afterwards
Can you explain about the Clojure code you're embedding?
You are using --~
for single line Clojure code in the SQL comments?
Correct.
But sometimes those comments leave an empty WHERE
clause, for instance.
Share the code you're having problems with...
I have this snippet in my queries.sql file. It works normally, but if I try to reload the file, it produces an error
Why do you have ;
in those strings?
That’s an oversight on my part. I was tinkering with this. They were meant to terminate the sql expression
The examples in the HugSQL docs don't seem to need ;
to terminate statements...? (and you already have a ;
there to terminate that one, so you'd end up with ;;
in the substituted code I think...
That sounds correct. I don’t recall which sql dialect the docs are using. I’m using postgres, which I believe does require the semicolon. It does in psql, at least.
Yes, in the console tool, I'd expect it to require ;
to terminate. But not in HugSQL files.
HugSQL uses blank lines between blocks to indicate the end of a SQL statement, based on the docs.
Anyways, if you can repro reliably, share the exact code that causes a problem when you attempt that bind-connection
call. That way we'll have something concrete to try to debug to help you...
What’s the best way to share something like that?
Pastebin or a Gist or some similar service.
If it's a small snippet, you can share it directly in Slack.
It sounds like this would be a single line containing the --~
form that causes the problem?
I started a new project with luminus to narrow everything down. It’s the one line with the comment
With this code, calls to restart-db
error out saying that clojure.lang.Symbol cannot be cast to clojure.lang.Namespace
That’s the entire contents of that file
And that's with calling restart-db
in the REPL?
With these contents, the function works normally
correct
And what command did you use to create the new project? (so I can replicate it here)
lein new luminus tester +postgres
The template has one migration, but I replaced it with this.
And exactly what steps did you take to get to the point of calling (restart-db)
? (if I just run lein repl
in the new project and then (restart-db)
I get mount.core.DerefableState cannot be cast to clojure.lang.IFn
so I assume some other steps are needed first?)
Just noticed in your code above you have (if (nil? (:username) params)
which is illegal -- I think you meant (if (nil? (:username params))
instead
(Idiomatically, it's better to write (if (:username params)
instead and swap the then/else expressions BTW)
The code you share should give this error Wrong number of args (2) passed to: clojure.core/nil?
so I'm not sure that's the actual cause...
(as an aside, this all illustrates why I think beginners should not start out with Luminus -- but should instead start out with a simple Ring app and gradually add Compojure, Mount (or Component), clojure.java.jdbc
(or next.jdbc
), then perhaps HugSQL, one at a time, to learn how all the pieces work)
Luminus is great when it works... but when something goes wrong and you're just starting out, there's so many moving parts in the Luminus template that it can be really hard for a beginner to figure out what is going wrong.
In an ideal world, I’d agree, but I also want to move more quickly to get something finished. Which I guess in this case has bit me in the rear end.
I was thinking that because the error mentions not being able to cast symbols to namespaces, there might be something necessary in the call to bind-connection
rather than in the queries file itself.
I managed to get a fix working but because I don’t totally understand some of the specifics of namespaces, I’m not certain of all the consequences
I moved the binding of the queries into a function inside the db.core
ns and directly called that ns function instead of using (binding [*ns* 'tester.db.core] ...)
@seancorfield is that something that makes sense to you as a fix?
Not really, no. I mean a freshly generated template works when used properly so the right "fix" is to figure out what's going wrong in the changes you've made.
This is exactly what I mean about beginners and Luminus -- if something "doesn't work", there's so much machinery in the template that debugging it is hard if you're a beginner. There's just no short cut here.
As you can see "moving quickly" within such a large pile of code that you don't yet understand is likely to continue to bite you, unless you take a step back and learn the basics I think.
I generated a fresh project from the template. I started the REPL. I figured out I needed to run (start)
first and then (restart-db)
-- and that all worked so far.
When you're making changes, you need to work with a very small, fast feedback loop. So, make a single small change and test it works. If it doesn't, you know the breakage was caused by that single change so it's easier to debug.
As you get more confident and more comfortable, you can make bigger changes in each cycle. But at first you need to be very systematic about testing each change.
do any of you use monads in your projects? Like the Maybe
monad or a Log Writer monad?
I think there are some problems for which the natural solution happens to be structured like a particular monad but monads are nearly always tied to type systems so "using monads" in Clojure looks somewhat different to, say, Haskell, and if you really try to "use monads" (such, as, for example, the algo.monad Contrib library), you end up with rather non-idiomatic Clojure code a lot of the time.
I work with @seancorfield and at one point we had this pipeline of code doing a data migration from an old system to a new one, and was basically built around what would be the Either monad, each step in the pipeline either apply some function to some value, which might return an error, or it would pass through the error from an earlier step. We didn't say "oh look a monad", we didn't write a monad library for it, it was just kind of there with a few helper functions to make it nicer, and it went away when the data migration served its purpose.
I'd say to some degree monads in clojure are more about just recognizing patterns when they arise vs. using a library of monads, and are often more ephemeral, because of the lack of emphasis on types
yea, I thought they wouldn't be such a big deal as a library in clojure -- however, I think the Either
/`Maybe` monad has a lot of use in clojure and I'm surprised I don't see it more. Perhaps because it's so easy to just (try ...
At one point we also used this library that I wrote https://github.com/seancorfield/engine and it's essentially monadic -- but code written with it turns out to be harder to read and debug than "regular" idiomatic Clojure, in my opinion.