This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-05-26
Channels
- # ai (1)
- # announcements (7)
- # babashka (2)
- # beginners (23)
- # biff (3)
- # calva (1)
- # chlorine-clover (3)
- # cider (2)
- # clj-kondo (12)
- # clojure (25)
- # clojure-brasil (4)
- # clojure-europe (33)
- # clojure-nl (1)
- # clojure-norway (7)
- # clojure-uk (1)
- # clojurescript (30)
- # clr (6)
- # consulting (1)
- # cursive (14)
- # data-science (1)
- # datalevin (4)
- # datomic (3)
- # events (4)
- # graphql (1)
- # gratitude (2)
- # hoplon (10)
- # inf-clojure (3)
- # interceptors (1)
- # introduce-yourself (1)
- # off-topic (13)
- # pathom (2)
- # pedestal (12)
- # rdf (14)
- # releases (6)
- # sci (17)
- # shadow-cljs (12)
Hi All, Below code calculates the status correctly but runs into NPE, Exception in thread "async-thread-macro-2" Exception in thread "async-thread-macro-4" java.lang.NullPointerException at clojure.core.async.impl.channels$chan$fn__5906.invoke(channels.clj:300) Can you please help me understand if the syntax is correct and pipeline has been called correctly:
(defn get-all-records-with-status [context filtered-port-data target-study-data
actual-record-map]
(let [split-list (reduce #(let [c-s-id (or (-> (get actual-record-map
[(:dbid %2) (:sid %2)])
:c_s_id)
"NA")]
(if (contains? target-study-data [(:dbid %2) (:sid %2)])
(assoc %1 :with-target
(conj (:with-target %1)
(let [chan (async/chan 1)
parallelism 5
data (assoc %2 :csid c-s-id)
data-processor
(fn []
(let [status (try (get-status-message-match context (:uuid data)
(:dbid data) (:sid data))
(catch Exception e
(log/error e (.getMessage e)
(format "for record: db-id %d, study-id %d" (:dbid data) (:sid data)))
(log/error (Throwable->map e))
"ERROR"))]
(assoc data :actualvstarget status)))]
(async/pipeline parallelism chan (data-processor) (async/to-chan! data)))))
(assoc %1 :without-target (conj (:without-target %1) (assoc %2 :actualvstarget "NA" :csid c-s-id)))))
{:without-target [] :with-target []} filtered-port-data)]
(concat (:without-target split-list)
(->> (:with-target split-list)
(async/merge)
(async/reduce conj [])
(async/<!!)))))
TIA!!That's a big blob of code and I am on my phone, but the usage of pipeline-async is nonsense, what you are doing with data-processor there is entirely made up, I would check the docstring for pipeline-async to see what argument is pass there
You also seem to be expecting the return value of pipeline-async to be some transformation of the data your poured into it, which it is not
pipeline is supposed to make the call to data-processor n number of times in parallel, during the run of data-processor the channel buffer somehow becomes null, I am not closing the channel explicitly, thanks for your response.
You are passing the result of calling the data-processor function to pipeline-async (looks like a map?) instead of a function
The return value of pipeline-async is a channel that closes when the go loop running the pipeline exits
The to chan, where pipeline writes its output you are just ignoring and doing nothing with
I think you need to stop, be clear about what you are implementing first, maybe write a simple single threaded version, then revisit core.async
pipeline-async is almost certainly not what you want here anyway, pipeline-async is typically for when you want to do some core.async stuff in parallel but you have something like a specific executor you want to shift the execution to
But you might want to use pipeline-async here anyway so you can execute blocking io in a good context for it, so you may need to introduce an executor for that, but this is already such a mess adding more complicated things (an executor) is just going to make it more so
Initial code had executor :
(.execute @thread-pool-executor
(fn []
(try
(async/put! chan (data-processor))
(finally
(async/close! chan)))))
I was trying to replace it with pipeline, to see if the performance improvesI suspect it's related to the fact that none of my autocompletions are working... defn
can't be unresolved lol.
I am an absolute beginner.
Working on a kit-clj projeect w/cljs frontend
I hate it when this happens. 🙂 And it always waits until I have forgotten the fix. I think it is in the Cursive FAQ, even.
Have you tried File/Invalidate caches...
, taking all the defaults? And are you getting any warnings from Intelli?
You can also try the #C0744GXCJ channel.
Is this considered a reasonable way of implementing functions with optional parameters that have default values? :thinking_face:
(defn get-projects
([] (get-projects " "))
([org] (shell (str "az devops project list --org " org))))
it is, but you might also want to consider an options map or kwargs - if you have one option now, you might have more later...
Is there anything like a python PEP8 for clojure outlining a thing such as "best practices" for documenting functions? If there are any community resources with such recs I'd be happy to see them
can't say I'm super-familiar with 'PEP8', but there's a clojure style guide at <https://guide.clojure.style/>
https://github.com/clj-kondo/clj-kondo might be interesting to you also