This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-07-08
Channels
- # announcements (22)
- # aws (4)
- # babashka (25)
- # beginners (78)
- # calva (9)
- # cider (22)
- # cljdoc (2)
- # cljsrn (2)
- # clojure (27)
- # clojure-australia (7)
- # clojure-europe (22)
- # clojure-nl (15)
- # clojure-uk (26)
- # clojurescript (20)
- # datahike (3)
- # datomic (15)
- # events (1)
- # helix (5)
- # honeysql (4)
- # kaocha (1)
- # malli (1)
- # meander (1)
- # off-topic (84)
- # pathom (14)
- # re-frame (3)
- # reagent (28)
- # reitit (6)
- # sci (1)
- # shadow-cljs (78)
- # tools-deps (30)
Hey everyone, how do I access values nested under a multi-word keyword? My use case is reading a CSV and turning it into a map. I keywordise the column headers Some of the column headers have multiple words.
For instance my key is multi keyword
(:multi keyword mymap)
returns nil. Running (keys mymap)
will show :multi keyword
as a valid key however.
Real solution: Don’t parse CSV headers into keywords. Then you can use (get mymap "multi keyword")
Bad solution: ((keyword "multi keyword") mymap)
In general, parsing external input as keywords isn’t the best idea. Clojure-readable keywords have restrictions, but the keyword
function doesn’t do any validation.
When you say restrictions, what are they actually? Are multi-word keywords not a supported use case, and the fallback option is to just use string keys?
There are many ways to approach this. When the CSV structure is known in advance, I simply write an adapter:
(def csv-header->keyword
{"some long header" :some-long-key
"aw3i~rd header" :a-weird-header
...})
;; for round-tripping back to the OG CSV headers
(def keyword->csv-header
(clojure.set/map-invert csv-header->keyword))
When the CSV structure is under my control, I may also choose to guarantee my CSV processor a clean regular format.
When the CSV is not in my control at all ... what others said above ☝️just out of curiosity — who holds the reference for this object after the var o
got a new value? it was not cleared by the GC
(def o (Object.))
=> #'learn-clojure.mapf/o
(def wo (WeakReference. o))
=> #'learn-clojure.mapf/wo
(.get wo)
=> #object[java.lang.Object 0x604a6096 "java.lang.Object@604a6096"]
(def o nil)
=> #'learn-clojure.mapf/o
(System/gc)
=> nil
(.get wo)
=> #object[java.lang.Object 0x604a6096 "java.lang.Object@604a6096"]
this is what i mean in java:
public class WeakR {
public static void main(String[] args) {
Object o = new Object();
Reference<Object> wo = new WeakReference<>(o);
out.println("first: " + wo.get());
o = null;
System.gc();
out.println("second: " + wo.get());
}
}
yields
first: java.lang.Object@6f195bc3
second: null
Hey, can I make this function from a coding exercise "point-free" and remove all the s
's with something like cond->
?
(defn response-for [s]
(cond
(and (question? s) (upper-case? s)) "Calm down, I know what I'm doing!"
(question? s) "Sure."
(upper-case? s) "Whoa, chill out!"
(str/blank? s) "Fine. Be that way!"
:else "Whatever."))
Yes; but i wouldn't recommend it:
(condp #(%1 %2) "abc"
str/blank? "Fine. Be that way !"
"Whatever")
if i made any change it might be to have all the conditions at the same level of abstraction. All of this stuff is overkill for toy problems, I just wanted to play around with some concepts. I did this awhile back:
(defn response-for
"Simulate a lackadaisical teenager"
[s]
(letfn [(question? [s] (= \? (last s)))
(silence? [s] (blank? s))
(shouting? [s] (and (= s (upper-case s))
(re-find #"\p{L}" s)))]
(cond
(shouting? s) "Whoa, chill out!"
(question? s) "Sure."
(silence? s) "Fine. Be that way!"
:else "Whatever.")))
Oh, interesting @UHJH8MG6S, why not do it like that?
Nice @U1Z392WMQ! I like that.
Mostly because it's uncommon. condp
is generally used for values with a certain pred; so
(let [x 5]
(condp > x
10 "Big!"
4 "Small!"))
=> "Big!"
Consider my snippet more a code golf 😛 The suggestions by tws and FVR are more clojuresqe (but it always depends on your usecase)Got it, thanks. I wasn't familiar with condp
on these toy problems my goal is still practicing readability and maintainability. I think condp is cleveritis and less readable in this case.
When a function has many test expressions, you can place them in a map like above. This has the benefit of making tests into data, easy to modify, extend and reuse.
also, you get twice as many lines to do the same job and completely lose the ordering of your conditions
Nice, I like the idea of using a map! When would you use that approach versus cond
in practice @frwdrik?
When you benefit from separating rules from functions using those rules. Examples: you want to A) easily extend the rules, B) use several functions on the same rules, C) use different rules in different circumstances, D) store rules in edn. For a single coding exercise it's maybe overkill, I agree.
if order matters, a vector of pairs is often better than using array-map
unless you want to call it as a function ... also the run-rules
fn looks a lot like some
and the or
condition looks like a rule to me 😉 ... maybe this sort of structure would work?
(let [question? #(= \? (last %))
upper-case? (complement #(.matches % ".*[a-z].*"))
rules [[(every-pred question? upper-case?) "Calm down, I know what I'm doing!"]
[question? "Sure"]
[upper-case? "Whoa, chill out!"]
[str/blank? "Fine. Be that way!"]
[(constantly true) "Whatever"]]
rule-matches (fn matches? [v [tst ret]] (when (tst v) ret))
s "TEST?"]
(some (partial rule-matches s) rules))
Yes, there are many ways to implement this, once you have "coordinate-free" rules. The or
could be another rule, or you can keep it to let different functions have different standard responses.
i suppose there is a significance that the name of some interfaces in core starts with I
, others dont — ISeq
, IPersistent*
etc vs Sorted
, Sequential
etc. whats that?
I don't think there is a huge significance in the name, but Sequential is a marker interface (no methods)
I think in general, the Java interfaces use I when it's an abstraction and not I when it's more of a "trait"
thanks, but why not all interfaces then? (`Counted`, Sorted
are not markers) — eariler where i saw it either all java interfaces were using it or dont. anyway, probably doesnt matter
(see https://insideclojure.org/2016/03/16/collections/ for some more background re Clojure collections)
Sequential, Counted, and Sorted are all traits of different kinds of collections
I am confused as of the usage of promesa
. As stated the p/let
macro should wait for promises to be resolved before evaluating the body - but I'm not seeing that happen.
Can somebody guide me to where I'm going wrong? This is the method (With a println for debugging)
(defmethod -fetch-resource "http" [uri params]
(p/let [response (http/get (str uri) params)]
(println response)
(:body response)))
and in the print I get: #org.httpkit.client/deadlock-guard/reify--6182[{:status :pending, :val nil} 0x1db2fe5b]
- this looks to me like I still get the promise in pending stateNot according to the docs. But I think I have tracked down the issue... not quite there yet but it seems that http-kit does not return a proper promise
Now I'm even more confused...
http-kit returns a clojure promise and promesa doesn't seem to accept a clojure promise as a promise it can handle.
And if I try to use (p/promise (promise))
it just wraps the promise as a value :thinking_face:
For now I get around by using a wrapper:
(defn wrap-clojure-promise [clj-promise]
(p/create (fn [resolve reject]
(try
(resolve @clj-promise)
(catch Exception e
(reject e))))))
but I would still like to know if there is a more idiomatic approach to dealing with clojure promises through promesaI think (p/do! @clj-promise)
might do it
it's annoying that they decided to use a two names that is already used in clojure.core, with completely different meanings (promise, delay)
Thanks for the response! I tested this and on the way noticed that both don't do what I expected - they both block at the time of the creation of the promise until it is resolved instead of instantly returning a pending promise. I will take another look - or wrap this up and move to core.async
The only solution I find until now is:
(defn wrap-clojure-promise [clj-promise]
(p/create (fn [resolve reject]
(try
(future (resolve @clj-promise))
(catch Exception e
(reject e))))))
Wrapping the deref into a future. I'm not so sure that this is a good solution as there might be quite an overhead for dealing with the extra threadI think the library was started in the context of ClojureScript, where promise generally refers to js/Promise which has future-like semantics
I have tried it in the context of ClojureScript before and there I didn't see any weirdness. Probably because promesa works directly on vanilla js promises
I'm guessing you're trying to get the code working in both Clojure and ClojureScript? Cause if not, on Clojure you don't need p/let, you can just deref the Clojure promise returned by http-kit
I'm not yet sure if I shouldn't move this over to core.async but I'm not sure of the tradeoffs in simple scenarios where all I want is issuing multiple tasks and wait for them all. I'm using it already in other places.
I'm not working with ClojureScript here, only Clojure. I wanted some "simple" handling for awaiting multiple parallel jobs
In Clojure, if http-get returns a Clojure promise, you don't need any library, just do:
(let [res1 (http/get ...)
res2 (http/get ...)]
(println @res1)
(println @res2))
Oh I got a bit light-headed there I guess, thanks for the pointer 😄 I guess I can just map
over the promises returned by http-get and then realize the sequence.
If you really want to wait for them all, but that seems like it just make things less optimal
You can do:
(run! deref [res1 res2])
Which will just block until res1 and res2 are available.But why wouldn't you want to say print as soon as res1 is available and then block for res2? Why wait for them all?
You are right that I probably don't have to wait for them all, it just seemed like a simpler solution as that can be done in a normal loop. The context is that I try to pull in resources and those resources have dependencies. Once I have the first batch of resources I resolve their dependencies and so on. The goal is to resolve as many resources as possible in parallel. Maybe a queue approach would be a better fit here :thinking_face:
I think there's an Executor just for this type of thing. Hum... I always forget it's name
Here's how you use it:
(import 'java.util.concurrent.ExecutorCompletionService)
(import 'java.util.concurrent.Executors)
(defn do-concurrently
"Executes each task in tasks with concurrency c, assuming side-effects,
and run handler on their results as they complete. Handler is called
synchronously from the calling thread."
[tasks c handler]
(let [executor (Executors/newFixedThreadPool c)
cs (ExecutorCompletionService. executor)
initial (take c tasks)
remaining (drop c tasks)]
;; Submit initial batch of tasks to run concurrently.
(doseq [task initial]
(-> cs (.submit task)))
(doseq [task remaining]
;; Block until any task completes.
(let [result (-> cs .take .get)]
;; When there remains tasks, submit another one to
;; replace the one that just completed.
(-> cs (.submit task))
;; Handle the result of the task that just completed.
(handler result)))
;; Since we submitted an initial batch, but only handled a remaining
;; number of tasks, some tasks are left un-handled, and we need to handle
;; them.
(doseq [_ initial]
(handler (-> cs .take .get)))
;; shutdown executor once all tasks have been processed
(-> executor .shutdown)))
;;; Run io 10000 times at 10 ms per io call with up to 100 concurrent calls
;;; and sum up all results.
;;; Then print the time it took and the resulting sum.
(let [sum (atom 0)]
(time
(do-concurrently (repeat 10000 (partial io 10)) 100 #(swap! sum + %)))
(println @sum))
The trick is that you first submit c
number of tasks to be executed concurrently. In this case, I've chosen to make 100 concurrent calls at a time. The call to submit
is non blocking and will return immediately. After you've initiated your first batch, you block on cs
, which will wait till any of them complete, and when one does, it will unblock and return the result of the task that just completed. When that happens, we will submit another task, so that we maintain our concurrency level, and we will call our handler with the result. In effect, we're saying, perform n number of calls up to c at a time. We are handling the results on the thread which submits the remaining tasks as they complete. This means that if our handler is very slow, it will delay our re-queuing of remaining tasks, so that's something to keep in mind. Finally, we have to handle the remaining batch of un-handled tasks, and shutdown the executor
to release the resources associated with it.This might be a bit advanced. But what it does is run tasks concurrently and as soon as any of them completes it lets you handle what to do next, where you could also submit even more tasks.
Thanks! I just noticed that its not so streight-forward as I thought as the handler theoretically has to add more tasks to the executor but I might be able to adjust it
(try
(future (resolve @clj-promise))
(catch Exception e
regarding this snippet, the catch here does nothing, as future doesn't blow up until you deref and that would be outside the try/catchI managed to get it to work with the ExecutorCompletionService
now by rewriting the do-concurrently
function from @U0K064KQV such that it also handles additional tasks returned from the handler