This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-15
Channels
- # announcements (71)
- # architecture (2)
- # aws (41)
- # babashka (23)
- # beginners (80)
- # calva (26)
- # chlorine-clover (11)
- # cljfx (4)
- # cljs-dev (12)
- # clojure (78)
- # clojure-berlin (5)
- # clojure-czech (3)
- # clojure-dev (9)
- # clojure-europe (45)
- # clojure-france (16)
- # clojure-gamedev (2)
- # clojure-italy (3)
- # clojure-nl (4)
- # clojure-spec (8)
- # clojure-sweden (1)
- # clojure-uk (37)
- # clojurescript (18)
- # community-development (15)
- # conjure (30)
- # cursive (51)
- # datomic (16)
- # duct (19)
- # figwheel-main (3)
- # fulcro (23)
- # java (7)
- # jobs (2)
- # joker (10)
- # off-topic (7)
- # parinfer (1)
- # pathom (6)
- # reagent (5)
- # reitit (1)
- # remote-jobs (1)
- # sci (1)
- # shadow-cljs (55)
- # slack-help (3)
- # specter (4)
- # sql (21)
- # tools-deps (11)
- # vim (5)
- # xtdb (14)
Hi guys, I have this project https://istd50043.github.io/project#implementation-requirements I'm trying to consider the possibility of doing this solo (so that I can do Clojure full stack) I'm thinking the "Production System" can be done with Clojurescript. "Analytics System" can be done with Clojure. But I also need to do automation scripts for AWS - not sure how possible is it to do this with babashka :o I know might be pretty intense to tackle alone. But am wondering how intense/possible is it :x (Also not sure if this is the best channel for this)
For simpler shell scripts you might consider using #joker and wrapping the AWS CLI tool rather than importing the AWS SDK into your clojure of choice.
example
@zackteo If you're not already a member, I suspect the #aws and maybe #aws-lambda channels will be helpful to you. I don't know if there are channels specific to automating ops stuff on AWS.
It's a big enough project that different parts of it will likely be best served by asking questions in several different channels but I think some of the core questions about AWS will need answering first and those are almost certainly "non-Clojure" questions. I think it's an interesting project and should be a good challenge.
I think it will be very good for me to attempt to tackle it myself. Especially with groups you end up missing out on learning the different parts. But the wonder is will it be too overwhelming a task that it would be better to take a step back
There are some Clojure-based projects for managing AWS ops stuff (Pallet comes to mind, there's a #dda-pallet but I don't know how active it is) but I'm not sure how active any of those ops projects are. I think people went back to more mainstream tech for that sort of thing?
@zackteo aws is a recurring topic in #babashka - welcome to join and thinking about it
@seancorfield am gonna do the project in Clojure after all! :D I miraculously found 2 peers to join me to form a 3 man team :) so it isnt as ridiculously daunting anymore
What would be the best way to organize such a program in clojure ? I want to concurrently build a set of unique things existing across a cluster of "locations" (DBs in reality). Let's say for simplification that I have an expensive func (get-ids location)
which might take a while to return. I have a list of locations
, say a few thousands of them, and I want to share the load of calling get-ids
for each of them, across, say, 16 workers, and finally get a unique set of values out of this. I have to limit the number of concurrent workers because in effect, each worker opens a connection.
just out of curiosity, how would you go about organizing such a thing with the core libs, as in, for an ad-hoc solution for a one time problem?
(require '[com.climate.claypoole :as cp])
(let [adresses [address1 address2 address3]]
(cp/upmap 10 (fn [address] (connect-address address)) addresses))
Tried my way around with pipeline
but ended up having bloated parts to rebuild the final list of results out of channels that I'm not sure they're done fetching.
I would write a pipeline that fetches the location in a thread pool or asynchronously and plug the distinct
transducer on the output channel.
I think I went more or less that way, wasn't sure how to ensure my output channel gave me all expected results though
If you close your input channel after posting your locations, the pipeline-blocking
function will close your destination channel.
So when your code detects that the destination channel closes, you can assume that you have the complete set of ids and can dump them in the file.
And if you build your input-channel with to-chan
on the collection of locations. you don't even have to bother closing this input channel, it will be done automatically.
nil is not an allowed value in a chan
so if you take and get a nil, you know the chan is closed and drained
what's a good way to express "at least 2" elements in a sequence? I believe count
will traverse an entire seq?
you can use bounded-count
bounded-count
came from one recent release, if you have older clojure, you can use simply (->> x (take 2) count)
oh, cool thanks :) tbh I wish there were a many?
in clojure.core that did this; I find myself having to sprinkle this around in a few places for UI-related stuff since we humans like our grammatical numbers
Might also be worth looking at https://clojuredocs.org/clojure.pprint/cl-format
actually, (count (range ..))
appears to be constant time in cljs, so it seems to always be ~4x faster to use count
instead of bounded-count
there -- not that it's meaningfully slow either way, at about 25 nanos for bounded-count
according to simple-benchmark.
not necessarily
only finite long ranges
infinite ranges obvs can't be ICounted
if "works" = wrong :)
yes. in cljs it seems they are never infinite. ([] (range 0 (.-MAX_VALUE js/Number) 1))
I mean, technically even in Clojure counting is int-bounded due to Java collection apis right now
ah, the clojure version of (range)
is (iterate inc 0)
which obviously won't implement Counted
whereas the "infinite" version of (range)
in cljs does
also, the non-long version of range in Clojure does not implement ICounted
as there are many weird edge cases
user=> (counted? (range 0.0 10.0))
false
user=> (counted? (range 0 10))
true
the first is a persistent list, which is realized (and thus countable, like all persistent colls)
the second is a sequence
sequences are not (necessarily) realized so not necessarily countable
why it's a sequence is pretty obscure and probably should be considered an implementation detail
very interesting. think that's mentioned here https://clojure.org/reference/reader#syntax-quote
Hi, I'm trying to test a highly asynchronous test, and found out some bizarre behavior of clojure.test
: sometimes, my assertions simply are not registered if they are on another thread.
I was able to shring to the following code:
(t/deftest bar
(t/testing "Why this does not work?"
(.submit (java.util.concurrent.ForkJoinPool/commonPool)
^Runnable (fn []
(Thread/sleep 200)
(t/do-report {:type :fail, :expected 10, :actual 12})))
(Thread/sleep 1000)))
The task submitted to the ForkJoinPool
, when it calls do-report
, prints the failure on my terminal (but not on nREPL / Socket REPL) but does not register on clojure.test that a failure occurs
It doesn't register that there was an assertion, nor register the failure.
So run-tests
thinks that it's an empty test and returns a success.
@mauricio.szabo I think @ghadi is right -- clojure.test
is full of dynamic bindings so that's likely to be your problem.
*testing-vars*
and *testing-contexts*
are the two used by do-report
that jump out at me immediately.
Oh well... So I'll need to bring these asserts into the main thread, right?
If you don't put the assertions on the main thread you also have to make sure your assertions on other threads have run to completion before the main thread exits, which can also be tricky
@hiredman yes, I was in fact working on a library to ease async tests in ClojureScript, and wanted feature-parity with Clojure. Seems that it'll not really be possible, unfortunately