This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-02-11
Channels
- # announcements (16)
- # aws (4)
- # babashka (30)
- # beginners (58)
- # bristol-clojurians (4)
- # cider (9)
- # clj-kondo (2)
- # clojure (229)
- # clojure-europe (25)
- # clojure-gamedev (1)
- # clojure-italy (4)
- # clojure-nl (13)
- # clojure-sanfrancisco (1)
- # clojure-uk (97)
- # clojured (7)
- # clojurescript (27)
- # code-reviews (2)
- # cursive (30)
- # data-science (39)
- # datomic (14)
- # emacs (12)
- # events (2)
- # fulcro (6)
- # graalvm (8)
- # graphql (14)
- # jackdaw (10)
- # jobs (2)
- # lambdaisland (5)
- # malli (4)
- # off-topic (28)
- # protorepl (13)
- # quil (7)
- # re-frame (2)
- # reagent (1)
- # reitit (3)
- # remote-jobs (5)
- # ring-swagger (1)
- # shadow-cljs (72)
- # sql (4)
- # tools-deps (182)
- # uncomplicate (4)
- # vim (9)
- # xtdb (19)
Hey! I just finished the intro to cloure
course on
any advices on how to proceed from there would be appreciated. I got a grasp of the data-driven approach but it’s all super basic.
Oh, perfect, thank you!
hi, anyone familiar with redis? I'm trying to get all results back from scan using carmine, but I'm having some trouble with the loop part. I need to call the scan command until the first value is a 0, and want collect all results in a vector and let it be the return value: https://paste.ofcode.org/92qxSsER6tY7FXwZAL9Znk
Hi 🙂 Is there a way with clojure spec to express the idea that the value for :foo
at one spot in a nested map should be equal to the value of :bar
on another branch of the same map? …. like in {:foo 3 :baz {:bar 3 :qux 6}}
… I’ve been able to express that :foo
and :bar
are required and should be integers … but not sure how to express that they should be equal
maybe I could write a predicate function and then combine it with what I already have using s/and
?
If I have an object that is not thread-safe. Could I wrap it around some Clojure concurrency constructs, say Agents, and do parallel processing with it?
Do you mean wrap a Clojure concurrency construct around the non-thread-safe object?
It might be possible to do that safely, at least in some cases, I think.
For example, if the code you use to update the non-thread-safe mutable object is itself single-threaded (i.e. it does not internally split off part of the updating work to multiple threads, but keeps it in the calling thread), then I think that doing all such calls inside of a Clojure agent updating function should have the necessary Java Memory Model synchronization primitives around it, that even though different agent update functions might be called from different JVM threads, they would be synchronized properly.
A Clojure atom would not be safe, since most or all of the function that you pass to swap!
can be run in parallel in different threads.
Similarly for ref
transactions - unsafe
The only reason it might work when wrapped inside of an agent update function is because for a single agent, those update functions are guaranteed to execute at most one at a time, so no parallelism there.
pmap
is explicitly about parallelism, so if different functions are called on different elements of the sequence are not thread safe, you are asking for trouble there.
Yes, agents can be used for this
The only Clojure concurrency thing I can imagine right now that might be safe is doing such work inside of an agent update function, which as mentioned above, makes the update operations inherently one-at-a-time.
If you have different mutable objects, say o1 and o2, and it is thread safe to in parallel call methods on o1 and o2, then you could have one agent for o1, and another for o2, and get 2-way parallelism, or N-way for N objects. But that only works if the underlying Java methods implement thread-safety when called on different objects.
I am reading large excel files using apache poi. In order to speed things up I am using a pmap to read/set some cell/sheet info. It works most of the time but then I get exceptions (at random) thrown trying to read a cell. Looking at the docs they mention it is not thread safe to access the same wb/sheet in parallel. So trying to find a way to accomplish this
if it is not safe to access in parallel, nothing you do will make it safe to do that
But would the serialization still happen in parallel? I mean the bulk of the work barely does anything with the object. I dont see why I can't do that work in parallel
and what exactly is safe and unsafe to do in parallel from the operations you are doing
There is no magic to change a non-thread-safe implementation into a thread-safe one. You can't build one from the other automatically.
for example, if the unsafe this is you do things with the same cell in two threads, you can create a lock per cell and only serialize operations on per cell (there are issues with that approach but conceptually you could do it)
You can look for other implementations for accessing excel sheets that do have thread-safe implementations, or ask people who intimately know the apache poi implementation whether there are subsets of operations that are thread-safe, and see whether those operations are things you would like to parallelize.
if the thing that is unsafe is concurrent operations on a given sheet, you can have something that serializes per sheet
if you are are really unlucky you'll decide on a given level of serialization, and then realize you have operations that cross that level of serialization
It sounds like Mario wants to parallelize operations on a single wb/sheet, which is exactly what the library doesn't guarantee safety for.
Wrapping Clojure agents around things won't increase parallelism in a thread-safe way. It can give you the same possible parallelism in a thread-safe way, but not more parallelism.
wrapping Clojure agents won't increase parallelism.
agreed.
do it as an etl job, dump the sheets to something threadsafe in a single threaded way, manipulate that, dump that back in to the sheets
But yes, the app is going cell-by-cell to build this sheet-structure. I figured it doesn't make sense to do this one by one, since once doesn't depend on the other.
depending what you're doing, reducers can be a good tool for fine-grained embarrassingly parallel work
I have a project created with: lein new re-frame mojo +garden +10x +cider +test +handler +routes It works fine giving me some simple http pages to my broweser. So I wanted to play around with some time intervals. So I found [clj-time "0.15.2"] So I added to my dependencies. (defproject mojo "0.1.0-SNAPSHOT" :dependencies [[org.clojure/clojure "1.10.1"] [org.clojure/clojurescript "1.10.597" :exclusions [com.google.javascript/closure-compiler-unshaded org.clojure/google-closure-library org.clojure/google-closure-library-third-party]] [thheller/shadow-cljs "2.8.83"] [reagent "0.9.1"] [re-frame "0.11.0"] [secretary "1.2.3"] [garden "1.3.9"] [ns-tracker "0.4.0"] [compojure "1.6.1"] [yogthos/config "1.1.7"] [ring "1.7.1"] [clj-time "0.15.2"]]i And then when into my /mojo/src/views.cljs file and added: (require '[clj-time.core :as t]) [:app] Build failure: The required namespace "clj-time.core" is not available, it was required by "mojo/views.cljs". "clj_time/core.clj" was found on the classpath. Should this be a .cljs file? Am I note able to use clj-time in a ClojureScript .cljs file? Or?? Total ClojureScripte Noob :-). But guess you figure that :-) Best regards happyb3at
also, in views.cljs, you should add requirements inside of the ns
declaration, not as statements using the require
function. that's for when you're working at a repl
example of doing requirements inside of ns
declaration: https://github.com/clj-time/clj-time/blob/master/src/clj_time/coerce.clj#L11
documentation of the ns macro https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/ns
most libraries will be explicitly either clojure or clojurescript btw, if they support both they'll call it out
oki so on clojars if stated nothing = clojure and if explicitly stated ClojureScript is ClojureScript only?
Oki so you would just go to the projects github page and get the info I gues. Thx for the help was awesome for the NOOB helpout 🙂
Some more googling gave: Glad you asked @Webdev_Tory ;-), because there is now a fully cross-platform Clojure(Script) date/time library, ‘tick’: https://github.com/juxt/tick 62. The api sits atop java.time on the jvm and a js implementation of java.time (jsr310). .... JUST IF ANYONE WANTED to know 🙂 FYI 🙂
Even if you are entirely on the JVM using Clojure, you shouldn't use clj-time
these days (the project readme explains why and offers some alternatives).