This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-12-28
Channels
- # announcements (5)
- # asami (34)
- # babashka (15)
- # beginners (222)
- # calva (5)
- # cider (2)
- # circleci (4)
- # cljfx (11)
- # clojure (34)
- # clojure-europe (10)
- # clojure-nl (2)
- # clojure-taiwan (2)
- # clojure-uk (5)
- # clojurescript (27)
- # conjure (8)
- # cryogen (11)
- # cursive (11)
- # datomic (7)
- # depstar (6)
- # fulcro (29)
- # graalvm (2)
- # joker (3)
- # kaocha (9)
- # nrepl (4)
- # off-topic (9)
- # pathom (1)
- # shadow-cljs (11)
- # spacemacs (4)
- # sql (1)
- # tools-deps (12)
Has anyone here used Langhor for RabbitMQ? If so, how can I have multiple threads consuming in parallel? I have tried the :executor
option with a fixed thread pool but that results in connection refused for all but one consumer.
Supposedly you can just create multiple consumers and they are executed in a threadpool by default, but my messages seem to be processed in a single thread. 😕
(defn init
[f]
(if (false? @started)
(let [conn (rabbit/connect {:uri (System/getenv "RABBITMQ_URL")})
chnl (channel/open conn)
queue-name "goodstats.queue"]
(timbre/info "Starting AMQP subscriber at: " (System/getenv "RABBITMQ_URL"))
(queue/declare chnl queue-name {:exclusive false :auto-delete false})
(consumers/subscribe chnl queue-name f {:auto-ack false})
(consumers/subscribe chnl queue-name f {:auto-ack false})
(consumers/subscribe chnl queue-name f {:auto-ack false})
(consumers/subscribe chnl queue-name f {:auto-ack false})
(reset! started true))))
They can share a thread pool. Really what I want is for each consumer to be able to process one message at a time, but N consumer to process N threads at once
When using Java I've used one channel per thread, I think I read it in the java api docs.
I saw that you could provide something like a threadfactory to langhor which might do something, but I wouldn't count on it handling concurrency (well) by itself
Apologies for a shameless plug - but it's coming from a good place 😉 My team's RabbitMQ client comes with multi-threading (and adjustable prefetch) support out of the box - and it's based on the Java client directly, rather than use Langhor (which we started with years ago). One caveat is that it has a hard-ish dependency on Component, but you can use it without it with relatively small effort: https://github.com/nomnom-insights/nomnom.bunnicula
My org quite recently moved towards using rabbitmq to "just do pub-sub" but it seems amqp + rabbit has quite a bit of complexity to handle, so having libraries with opinions is great.
Yeah, the amount of configuration options is quite overwhelming on the client and server side. We have basically set everything to be as safe as possible, while trading off throughput a bit (although we push 10s (or 100s now) of thousands of messages every hour and it's been fine. One thing to watch out for - there's v3 coming which will not be backwards compatible
Would you prefer Clojure or ClojureScript for FaaS? Personally I am concerned about the start up times for JVM.
Depends on use-case. If you’re expecting fast response to infrequent HTTP-requests then I’d probably pick CLJS. However I think the startup time is one of the smallest problems to solve when entering FaaS territory. 🙂
If you’re worried, you can write most of your code in cljc
and easily swap between runtimes. 🙂
this new feature might change things a little https://aws.amazon.com/blogs/aws/new-provisioned-concurrency-for-lambda-functions/
@U6N4HSMFW could you elaborate on the other problems when using clojure with FaaS, I would be curious to know.
@U01HN2UCH9Q oh, I didn’t mean that Clojure would be a problem with FaaS. Actually I think the language itself is a great fit for FaaS. What I wanted to say is that in my experience people are too worried about startup penalty, which I perceive as a minor (technical) thing that can be worked around. I think there are more fundamental problems to be worried about, such as splitting the problem domain into distinct deployable artefacts and managing dependencies between them.
About cognitect.anomalies
: (not sure if it's the right channel)
There is a field :cognitect.anomalies/message
When I'm using anomalies
in a ex-info
, should I use the same value for ex-info msg
and for :cognitect.anomalies/message
?
Something like (let [msg ...] (throw (ex-info msg {:cognitect.anomalies/message msg ....})))
?
Or they have different meanings?
EDIT: I just see that :cognitect.anomalies/message
is :opt
, so may I skip it in ex-info?
You're supposed to return them instead. so you would not throw an anomaly, just return one
right. so caller may write it own msg
to ex-info with a possibly generic error, and :cognitect.anomalies/message
will came from "the cause"
ATM "just fun" https://gist.github.com/ericnormand/3a39e9093692b1ee3dbbb612d739e468#gistcomment-3575422
@U050ECB92 "The caller needs to check for success" sounds a lot like C return codes. I haven't really dealt with such an approach in Clojure. Does it not get overwhelming, to have to check the result in every single function that can get an "anomaly" from one of its callees?
I know, i catch some exceptions from there and I can dig into it. But as a non-oss, is harder to see
Does anybody have a recommendation or pointers for a tutorial/into on cljfx? I've got the basic example at https://github.com/cljfx/cljfx running, but I'm not sure how to move past this. I'm relatively new with only moderate Clojure + Leiningen experience.
you can try the #cljfx channel. they're pretty helpful there.