Fork me on GitHub
#beginners
<
2020-02-11
>
Piotr Brzeziński08:02:35

Hey! I just finished the intro to cloure course on any advices on how to proceed from there would be appreciated. I got a grasp of the data-driven approach but it’s all super basic.

Piotr Brzeziński12:02:26

Oh, perfect, thank you!

jakuzure16:02:27

hi, anyone familiar with redis? I'm trying to get all results back from scan using carmine, but I'm having some trouble with the loop part. I need to call the scan command until the first value is a 0, and want collect all results in a vector and let it be the return value: https://paste.ofcode.org/92qxSsER6tY7FXwZAL9Znk

fappy17:02:27

Hi 🙂 Is there a way with clojure spec to express the idea that the value for :foo at one spot in a nested map should be equal to the value of :bar on another branch of the same map? …. like in {:foo 3 :baz {:bar 3 :qux 6}} … I’ve been able to express that :foo and :bar are required and should be integers … but not sure how to express that they should be equal

fappy18:02:11

maybe I could write a predicate function and then combine it with what I already have using s/and ?

✔️ 4
fappy18:02:01

Thanks — this is such a great place, and it has bacon

Mario C.18:02:33

If I have an object that is not thread-safe. Could I wrap it around some Clojure concurrency constructs, say Agents, and do parallel processing with it?

andy.fingerhut19:02:13

Do you mean wrap a Clojure concurrency construct around the non-thread-safe object?

Mario C.19:02:44

yes, exactly

andy.fingerhut19:02:42

It might be possible to do that safely, at least in some cases, I think.

andy.fingerhut19:02:23

For example, if the code you use to update the non-thread-safe mutable object is itself single-threaded (i.e. it does not internally split off part of the updating work to multiple threads, but keeps it in the calling thread), then I think that doing all such calls inside of a Clojure agent updating function should have the necessary Java Memory Model synchronization primitives around it, that even though different agent update functions might be called from different JVM threads, they would be synchronized properly.

andy.fingerhut19:02:04

A Clojure atom would not be safe, since most or all of the function that you pass to swap! can be run in parallel in different threads.

andy.fingerhut19:02:30

Similarly for ref transactions - unsafe

Mario C.19:02:48

I would be doing this within a pmap's mapping fn

Mario C.19:02:04

So its sounds like I might be able to get away with it, if I understand correctly

andy.fingerhut19:02:31

The only reason it might work when wrapped inside of an agent update function is because for a single agent, those update functions are guaranteed to execute at most one at a time, so no parallelism there.

andy.fingerhut19:02:07

pmap is explicitly about parallelism, so if different functions are called on different elements of the sequence are not thread safe, you are asking for trouble there.

Alex Miller (Clojure team)19:02:03

Yes, agents can be used for this

andy.fingerhut19:02:12

The only Clojure concurrency thing I can imagine right now that might be safe is doing such work inside of an agent update function, which as mentioned above, makes the update operations inherently one-at-a-time.

andy.fingerhut19:02:31

If you have different mutable objects, say o1 and o2, and it is thread safe to in parallel call methods on o1 and o2, then you could have one agent for o1, and another for o2, and get 2-way parallelism, or N-way for N objects. But that only works if the underlying Java methods implement thread-safety when called on different objects.

Mario C.19:02:00

I am reading large excel files using apache poi. In order to speed things up I am using a pmap to read/set some cell/sheet info. It works most of the time but then I get exceptions (at random) thrown trying to read a cell. Looking at the docs they mention it is not thread safe to access the same wb/sheet in parallel. So trying to find a way to accomplish this

hiredman19:02:52

if it is not safe to access in parallel, nothing you do will make it safe to do that

hiredman19:02:10

all you can do is use different constructs to serialize access

Mario C.19:02:31

But would the serialization still happen in parallel? I mean the bulk of the work barely does anything with the object. I dont see why I can't do that work in parallel

hiredman19:02:53

it will depend on how fine grained you are

hiredman19:02:56

and what exactly is safe and unsafe to do in parallel from the operations you are doing

andy.fingerhut19:02:23

There is no magic to change a non-thread-safe implementation into a thread-safe one. You can't build one from the other automatically.

hiredman19:02:32

for example, if the unsafe this is you do things with the same cell in two threads, you can create a lock per cell and only serialize operations on per cell (there are issues with that approach but conceptually you could do it)

andy.fingerhut19:02:40

You can look for other implementations for accessing excel sheets that do have thread-safe implementations, or ask people who intimately know the apache poi implementation whether there are subsets of operations that are thread-safe, and see whether those operations are things you would like to parallelize.

hiredman19:02:59

if the thing that is unsafe is concurrent operations on a given sheet, you can have something that serializes per sheet

hiredman19:02:59

if you are are really unlucky you'll decide on a given level of serialization, and then realize you have operations that cross that level of serialization

andy.fingerhut19:02:04

It sounds like Mario wants to parallelize operations on a single wb/sheet, which is exactly what the library doesn't guarantee safety for.

andy.fingerhut19:02:18

Wrapping Clojure agents around things won't increase parallelism in a thread-safe way. It can give you the same possible parallelism in a thread-safe way, but not more parallelism.

Alex Miller (Clojure team)19:02:35

wrapping Clojure agents won't increase parallelism.

hiredman19:02:27

do it as an etl job, dump the sheets to something threadsafe in a single threaded way, manipulate that, dump that back in to the sheets

Mario C.19:02:18

Thats a good idea actually

Mario C.19:02:41

But yes, the app is going cell-by-cell to build this sheet-structure. I figured it doesn't make sense to do this one by one, since once doesn't depend on the other.

Alex Miller (Clojure team)19:02:50

depending what you're doing, reducers can be a good tool for fine-grained embarrassingly parallel work

happyb3at20:02:41

Hi Everyone 🙂

happyb3at20:02:56

I have a project created with: lein new re-frame mojo +garden +10x +cider +test +handler +routes It works fine giving me some simple http pages to my broweser. So I wanted to play around with some time intervals. So I found [clj-time "0.15.2"] So I added to my dependencies. (defproject mojo "0.1.0-SNAPSHOT" :dependencies [[org.clojure/clojure "1.10.1"] [org.clojure/clojurescript "1.10.597" :exclusions [com.google.javascript/closure-compiler-unshaded org.clojure/google-closure-library org.clojure/google-closure-library-third-party]] [thheller/shadow-cljs "2.8.83"] [reagent "0.9.1"] [re-frame "0.11.0"] [secretary "1.2.3"] [garden "1.3.9"] [ns-tracker "0.4.0"] [compojure "1.6.1"] [yogthos/config "1.1.7"] [ring "1.7.1"] [clj-time "0.15.2"]]i And then when into my /mojo/src/views.cljs file and added: (require '[clj-time.core :as t]) [:app] Build failure: The required namespace "clj-time.core" is not available, it was required by "mojo/views.cljs". "clj_time/core.clj" was found on the classpath. Should this be a .cljs file? Am I note able to use clj-time in a ClojureScript .cljs file? Or?? Total ClojureScripte Noob :-). But guess you figure that :-) Best regards happyb3at

bfabry20:02:13

yes clj-time is clojure only, not clojurescript

bfabry20:02:10

also, in views.cljs, you should add requirements inside of the ns declaration, not as statements using the require function. that's for when you're working at a repl

bfabry20:02:36

most libraries will be explicitly either clojure or clojurescript btw, if they support both they'll call it out

happyb3at20:02:33

oki so on clojars if stated nothing = clojure and if explicitly stated ClojureScript is ClojureScript only?

bfabry20:02:26

I'm not actually sure if clojars documents that

happyb3at20:02:32

Oki so you would just go to the projects github page and get the info I gues. Thx for the help was awesome for the NOOB helpout 🙂

bfabry20:02:41

you're welcome

happyb3at20:02:32

Some more googling gave: Glad you asked @Webdev_Tory ;-), because there is now a fully cross-platform Clojure(Script) date/time library, ‘tick’: https://github.com/juxt/tick 62. The api sits atop java.time on the jvm and a js implementation of java.time (jsr310). .... JUST IF ANYONE WANTED to know 🙂 FYI 🙂

happyb3at20:02:44

Yup just what this happy camper wanted 🙂

seancorfield20:02:36

Even if you are entirely on the JVM using Clojure, you shouldn't use clj-time these days (the project readme explains why and offers some alternatives).

👍 4