This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-10-15
Channels
- # announcements (8)
- # beginners (65)
- # calva (25)
- # cider (11)
- # clj-kondo (9)
- # cljsrn (14)
- # clojure (103)
- # clojure-europe (15)
- # clojure-greece (1)
- # clojure-italy (28)
- # clojure-nl (39)
- # clojure-spec (9)
- # clojure-uk (28)
- # clojuredesign-podcast (37)
- # clojurescript (56)
- # cursive (41)
- # data-science (10)
- # datomic (25)
- # duct (1)
- # emacs (1)
- # events (3)
- # figwheel-main (7)
- # fulcro (9)
- # graalvm (7)
- # graphql (10)
- # jobs (2)
- # nrepl (17)
- # off-topic (40)
- # quil (12)
- # reitit (11)
- # remote-jobs (5)
- # rum (2)
- # shadow-cljs (387)
- # sql (22)
- # tools-deps (8)
- # vim (26)
- # xtdb (47)
- # yada (9)
Can you give an example of what you mean? Perhaps you mean that because pmap is lazy, execution of the later forms can start before pmap finishes applying the function f
you give it to all sequence elements?
If you mean that, then any lazy expression can be forced to evaluate completely, pmap
or otherwise, by enclosing it inside of a call to doall
@andy.fingerhut humm, I think it´s that I want, only return when all pmap elements finish
Clojure has many other functions that return lazy sequences besides pmap
, e.g. map
, filter
, and a lot of functions that return sequences.
I´m asking this because each pmap call will execute a network call and I don´t want it to be a flood
I am doing something similar like this futures (mapv #(future (api-call %)) list-of-args)
then (map deref futures)
it depends a lot on the context, but I would either use Executors directly or core.async's pipeline-blocking
using executors directly you tell it how large of a threadpool to use, and pipeline-blocking takes an 'n' which more or less limits the amount of concurrency (it limits it to n+2 if I recall)
I can´t figure out how to convert it to core.async. It´s "obscure" yet for my knowleadge
@fabrao, have you checked pipeline-async
? It seems appropriate for the scenario you’re describing(network calls).
https://clojuredocs.org/clojure.core.async/pipeline-async
I very strongly recommend against using pipeline-async if you are already having trouble managing concurrency
Could you please elaborate a bit more on that? 🙂 What would be your concerns?
https://clojure.atlassian.net/plugins/servlet/mobile?originPath=%2Fbrowse%2FASYNC-163#issue/ASYNC-163 the comments here are a little more detailed about pipeline-async and how it is a very different animal from the other two pipelines
Thanks… 🙂
pipeline-blocking is probably better for this
look what I did
(defn prmap
"Like map, except f is applied in parallel. Semi-lazy in that the
parallel computation stays ahead of the consumption, but doesn't
realize the entire result unless required. Only useful for
computationally intensive functions where the time of f dominates
the coordination overhead."
{:added "1.0"
:static true}
([f coll]
(let [n 3;; (+ 0 (.. Runtime getRuntime availableProcessors))
rets (map #(future (f %)) coll)
step (fn step [[x & xs :as vs] fs]
(lazy-seq
(if-let [s (seq fs)]
(cons (deref x) (step xs (rest s)))
(map deref vs))))]
(step rets (drop n rets))))
([f coll & colls]
(let [step (fn step [cs]
(lazy-seq
(let [ss (map seq cs)]
(when (every? identity ss)
(cons (map first ss) (step (map rest ss)))))))]
(prmap #(apply f %) (step (cons coll colls))))))
Got a question about namespaces and their use. I’m used to declaring a namespace and requiring others (renaming as necessary), but what are the boundaries for clojure’s standard library?
For instance, I’ve tried including the clojure.data.generators
namespace to get at the uuid
function, but when I use it in one source (or even enter the namespace in a repl), it tells me it’s unable to resolve the symbol
user> (in-ns 'clojure.data.generators)
#namespace[clojure.data.generators]
clojure.data.generators> (uuid)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: uuid in this context, compiling:(*cider-repl Development/geometer:localhost:52209(clj)*:22936:26)
clojure.data.generators>
that ns is noto auto-loaded
in-ns
doesn't look for or load anything, if the namespace doesn't exist yet it creates a new broken namespace
you can mostly fix that namespace by runnign (clojure.core/refer-clojure)
but that still doesn't load the source for that ns- it just makes a bare bones non-broken ns
the right way to do that is requiring first
here's a helpful idiom: (doto 'clojure.data.generators (require) (in-ns))
- this lands you in that ns, unless the load fails
clojure.data.generators> (doto 'clojure.data.generators (require) (in-ns))
FileNotFoundException Could not locate clojure/data/generators__init.class or clojure/data/generators.clj on classpath. clojure.lang.RT.load (RT.java:463)
in-ns
is "switch context to this namespace" -- it is almost certainly an error if the namespace hasn't yet been loaded
right - but it doesn't error, it just creates a broken ns
that in-ns
fails though, unless you use (clojure.core/in-ns)
or do the refer-clojure first (maybe I'm misunderstanding)
Okay, that also gives me the most recent error. I guess I have things Not Setup Properly™.
you probably haven't restarted your repl since adding that lib (or never added the lib correctly)
it doesn't come with clojure.core
user=> (in-ns 'nonexistent)
#object[clojure.lang.Namespace 0x6e5bfdfc "nonexistent"]
nonexistent=> (require 'foo) ;; ah crap clojure.core isn't available in here
Syntax error compiling at (REPL:1:1).
Unable to resolve symbol: require in this context
nonexistent=> (in-ns 'user)
#object[clojure.lang.Namespace 0x17ae98d7 "user"]
user=> (map symbol [:back :to :reality])
(back to reality)
@zbir This page might help https://clojure.org/community/contrib_libs -- those are all "external" libraries that need to be added as dependencies.
It also lets you know the development status (`clojure.data.generators` is considered Inactive).
while inactive, it's still perfectly fine to use - clojure itself uses it extensively in its own test suite
Yeah, just setting expectations in case @zbir finds things he might want changed 🙂
Is the idea that for things that act as barest proxies to underlying java implementations, just use the underlying implementation?
that's what I'd usually suggest yeah - there's no real abstraction layer barrier between clojure and the vm (except the compiler / language semantics) - using interop throughout your code is fine if the raw classes and their methods do the thing you need
for some constructs a clojure idiom can be a lot cleaner than the java factory and dep-injection centered approach, but that should end up being a pretty thin thing in most cases
I used to be a big fan of wrapper libraries to hide raw interop but I'm getting less and less so these days. The wrapper is only worthwhile if it substantially improves readability and/or reduces boilerplate (such as around JDBC stuff 🙂 )