Fork me on GitHub
#beginners
<
2021-07-08
>
Krishan V04:07:06

Hey everyone, how do I access values nested under a multi-word keyword? My use case is reading a CSV and turning it into a map. I keywordise the column headers Some of the column headers have multiple words. For instance my key is multi keyword (:multi keyword mymap) returns nil. Running (keys mymap) will show :multi keyword as a valid key however.

potetm04:07:23

Real solution: Don’t parse CSV headers into keywords. Then you can use (get mymap "multi keyword") Bad solution: ((keyword "multi keyword") mymap)

potetm04:07:35

In general, parsing external input as keywords isn’t the best idea. Clojure-readable keywords have restrictions, but the keyword function doesn’t do any validation.

Krishan V04:07:56

Thanks @U07S8JGF7. Using string keys would work better in this case!

✔️ 3
Krishan V04:07:56

When you say restrictions, what are they actually? Are multi-word keywords not a supported use case, and the fallback option is to just use string keys?

hiredman04:07:56

That is for the reader

hiredman04:07:51

You can create whatever you want calling the symbol or keyword functions

hiredman04:07:37

Oh, pardon me, just missed it on my phone

adi04:07:38

There are many ways to approach this. When the CSV structure is known in advance, I simply write an adapter:

(def csv-header->keyword
     {"some long header" :some-long-key
      "aw3i~rd header" :a-weird-header
      ...})

;; for round-tripping back to the OG CSV headers
(def keyword->csv-header
  (clojure.set/map-invert csv-header->keyword))
When the CSV structure is under my control, I may also choose to guarantee my CSV processor a clean regular format. When the CSV is not in my control at all ... what others said above ☝️

bnstvn10:07:23

just out of curiosity — who holds the reference for this object after the var o got a new value? it was not cleared by the GC

(def o (Object.))
=> #'learn-clojure.mapf/o
(def wo (WeakReference. o))
=> #'learn-clojure.mapf/wo
(.get wo)
=> #object[java.lang.Object 0x604a6096 "java.lang.Object@604a6096"]
(def o nil)
=> #'learn-clojure.mapf/o
(System/gc)
=> nil
(.get wo)
=> #object[java.lang.Object 0x604a6096 "java.lang.Object@604a6096"]

bnstvn10:07:32

this is what i mean in java:

public class WeakR {
    public static void main(String[] args) {
        Object o = new Object();
        Reference<Object> wo = new WeakReference<>(o);
        out.println("first: " + wo.get());
        o = null;
        System.gc();
        out.println("second: " + wo.get());
    }
}
yields
first: java.lang.Object@6f195bc3
second: null

bnstvn10:07:15

ah got it — done in the repl, and i had refs via *1 *2 etc!

Yosevu Kilonzo12:07:30

Hey, can I make this function from a coding exercise "point-free" and remove all the s 's with something like cond-> ?

(defn response-for [s]
  (cond
    (and (question? s) (upper-case? s)) "Calm down, I know what I'm doing!"
    (question? s) "Sure."
    (upper-case? s) "Whoa, chill out!"
    (str/blank? s) "Fine. Be that way!"
    :else "Whatever."))

thumbnail12:07:03

Yes; but i wouldn't recommend it:

(condp #(%1 %2) "abc"
  str/blank? "Fine. Be that way !"
  "Whatever")

tschady13:07:38

if i made any change it might be to have all the conditions at the same level of abstraction. All of this stuff is overkill for toy problems, I just wanted to play around with some concepts. I did this awhile back:

(defn response-for
  "Simulate a lackadaisical teenager"
  [s]
  (letfn [(question? [s] (= \? (last s)))
          (silence?  [s] (blank? s))
          (shouting? [s] (and (= s (upper-case s))
                              (re-find #"\p{L}" s)))]
    (cond
      (shouting? s) "Whoa, chill out!"
      (question? s) "Sure."
      (silence?  s) "Fine. Be that way!"
      :else         "Whatever.")))

Yosevu Kilonzo13:07:25

Oh, interesting @UHJH8MG6S, why not do it like that?

thumbnail13:07:25

Mostly because it's uncommon. condp is generally used for values with a certain pred; so

(let [x 5] 
  (condp > x
    10 "Big!"
    4 "Small!"))
       
=> "Big!"
Consider my snippet more a code golf 😛 The suggestions by tws and FVR are more clojuresqe (but it always depends on your usecase)

Yosevu Kilonzo13:07:53

Got it, thanks. I wasn't familiar with condp

tschady14:07:42

on these toy problems my goal is still practicing readability and maintainability. I think condp is cleveritis and less readable in this case.

💯 6
Fredrik12:07:23

When a function has many test expressions, you can place them in a map like above. This has the benefit of making tests into data, easy to modify, extend and reuse.

jsn13:07:03

also, you get twice as many lines to do the same job and completely lose the ordering of your conditions

Fredrik13:07:14

I changed the map to array-map , that should respect ordering

jsn13:07:53

(condp apply [expr] ...) could be used, but even that seems excessive to me

😬 3
Yosevu Kilonzo13:07:17

Nice, I like the idea of using a map! When would you use that approach versus cond in practice @frwdrik?

Fredrik13:07:17

When you benefit from separating rules from functions using those rules. Examples: you want to A) easily extend the rules, B) use several functions on the same rules, C) use different rules in different circumstances, D) store rules in edn. For a single coding exercise it's maybe overkill, I agree.

💯 3
Ed14:07:41

if order matters, a vector of pairs is often better than using array-map unless you want to call it as a function ... also the run-rules fn looks a lot like some and the or condition looks like a rule to me 😉 ... maybe this sort of structure would work?

(let [question?    #(= \? (last %))
        upper-case?  (complement #(.matches % ".*[a-z].*"))
        rules        [[(every-pred question?  upper-case?) "Calm down, I know what I'm doing!"]
                      [question?                           "Sure"]
                      [upper-case?                         "Whoa, chill out!"]
                      [str/blank?                          "Fine. Be that way!"]
                      [(constantly true)                   "Whatever"]]
        rule-matches (fn matches? [v [tst ret]] (when (tst v) ret))
        s            "TEST?"]
    (some (partial rule-matches s) rules))

Fredrik14:07:17

Yes, there are many ways to implement this, once you have "coordinate-free" rules. The or could be another rule, or you can keep it to let different functions have different standard responses.

bnstvn14:07:42

i suppose there is a significance that the name of some interfaces in core starts with I , others dont — ISeq, IPersistent* etc vs Sorted , Sequential etc. whats that?

nate sire14:07:01

is I for immutable?

nate sire14:07:13

let me research that

ghadi14:07:30

I is for interface

👍 3
ghadi14:07:10

I don't think there is a huge significance in the name, but Sequential is a marker interface (no methods)

dpsutton14:07:19

The naming convention is called Hungarian Notation

nate sire14:07:49

ahhhh... so just one char on the front of the name... to signal what kind of type

Alex Miller (Clojure team)14:07:55

I think in general, the Java interfaces use I when it's an abstraction and not I when it's more of a "trait"

bnstvn14:07:02

thanks, but why not all interfaces then? (`Counted`, Sorted are not markers) — eariler where i saw it either all java interfaces were using it or dont. anyway, probably doesnt matter

Alex Miller (Clojure team)14:07:16

Sequential, Counted, and Sorted are all traits of different kinds of collections

dpsutton14:07:40

oh that's an interesting distinction i hadn't realized

bnstvn14:07:54

thanks, that makes sense!

alpox18:07:52

I am confused as of the usage of promesa. As stated the p/let macro should wait for promises to be resolved before evaluating the body - but I'm not seeing that happen. Can somebody guide me to where I'm going wrong? This is the method (With a println for debugging)

(defmethod -fetch-resource "http" [uri params]
  (p/let [response (http/get (str uri) params)]
    (println response)
    (:body response)))
and in the print I get: #org.httpkit.client/deadlock-guard/reify--6182[{:status :pending, :val nil} 0x1db2fe5b] - this looks to me like I still get the promise in pending state

telekid18:07:02

shot in the dark (haven’t used it,) but do you need to deref response?

alpox18:07:21

Not according to the docs. But I think I have tracked down the issue... not quite there yet but it seems that http-kit does not return a proper promise

👍 3
alpox18:07:52

Now I'm even more confused... http-kit returns a clojure promise and promesa doesn't seem to accept a clojure promise as a promise it can handle. And if I try to use (p/promise (promise)) it just wraps the promise as a value :thinking_face:

alpox18:07:53

For now I get around by using a wrapper:

(defn wrap-clojure-promise [clj-promise]
  (p/create (fn [resolve reject]
              (try
                (resolve @clj-promise)
                (catch Exception e
                  (reject e))))))
but I would still like to know if there is a more idiomatic approach to dealing with clojure promises through promesa

noisesmith20:07:00

I think (p/do! @clj-promise) might do it

noisesmith20:07:28

it's annoying that they decided to use a two names that is already used in clojure.core, with completely different meanings (promise, delay)

alpox20:07:12

Thanks for the response! I tested this and on the way noticed that both don't do what I expected - they both block at the time of the creation of the promise until it is resolved instead of instantly returning a pending promise. I will take another look - or wrap this up and move to core.async

alpox20:07:05

The only solution I find until now is:

(defn wrap-clojure-promise [clj-promise]
  (p/create (fn [resolve reject]
              (try
                (future (resolve @clj-promise))
                (catch Exception e
                  (reject e))))))
Wrapping the deref into a future. I'm not so sure that this is a good solution as there might be quite an overhead for dealing with the extra thread

didibus20:07:44

I think the library was started in the context of ClojureScript, where promise generally refers to js/Promise which has future-like semantics

alpox20:07:28

I have tried it in the context of ClojureScript before and there I didn't see any weirdness. Probably because promesa works directly on vanilla js promises

didibus20:07:41

Ya it does

didibus20:07:29

I'm guessing you're trying to get the code working in both Clojure and ClojureScript? Cause if not, on Clojure you don't need p/let, you can just deref the Clojure promise returned by http-kit

alpox20:07:32

I'm not yet sure if I shouldn't move this over to core.async but I'm not sure of the tradeoffs in simple scenarios where all I want is issuing multiple tasks and wait for them all. I'm using it already in other places.

alpox20:07:32

I'm not working with ClojureScript here, only Clojure. I wanted some "simple" handling for awaiting multiple parallel jobs

didibus20:07:30

In Clojure, if http-get returns a Clojure promise, you don't need any library, just do:

(let [res1 (http/get ...)
      res2 (http/get ...)]
  (println @res1)
  (println @res2))

alpox20:07:53

Oh I got a bit light-headed there I guess, thanks for the pointer 😄 I guess I can just map over the promises returned by http-get and then realize the sequence.

didibus20:07:24

If you really want to wait for them all, but that seems like it just make things less optimal

didibus20:07:51

You can do:

(run! deref [res1 res2])
Which will just block until res1 and res2 are available.

didibus20:07:13

But why wouldn't you want to say print as soon as res1 is available and then block for res2? Why wait for them all?

alpox20:07:59

You are right that I probably don't have to wait for them all, it just seemed like a simpler solution as that can be done in a normal loop. The context is that I try to pull in resources and those resources have dependencies. Once I have the first batch of resources I resolve their dependencies and so on. The goal is to resolve as many resources as possible in parallel. Maybe a queue approach would be a better fit here :thinking_face:

didibus20:07:03

I think there's an Executor just for this type of thing. Hum... I always forget it's name

alpox20:07:21

That would be nice! Otherwise I may go the core.async route.

didibus20:07:27

ExecutorCompletionService

didibus20:07:31

That's the one

didibus20:07:00

I have a old draft blog post about it haha

didibus20:07:08

Here's how you use it:

(import 'java.util.concurrent.ExecutorCompletionService)
(import 'java.util.concurrent.Executors)

(defn do-concurrently
  "Executes each task in tasks with concurrency c, assuming side-effects,
   and run handler on their results as they complete. Handler is called
   synchronously from the calling thread."
  [tasks c handler]
  (let [executor (Executors/newFixedThreadPool c)
        cs (ExecutorCompletionService. executor)
        initial (take c tasks)
        remaining (drop c tasks)]
    ;; Submit initial batch of tasks to run concurrently.
    (doseq [task initial]
      (-> cs (.submit task)))
    (doseq [task remaining]
      ;; Block until any task completes.
      (let [result (-> cs .take .get)]
        ;; When there remains tasks, submit another one to
        ;; replace the one that just completed.
        (-> cs (.submit task))
        ;; Handle the result of the task that just completed.
        (handler result)))
    ;; Since we submitted an initial batch, but only handled a remaining
    ;; number of tasks, some tasks are left un-handled, and we need to handle
    ;; them.
    (doseq [_ initial]
      (handler (-> cs .take .get)))
    ;; shutdown executor once all tasks have been processed
    (-> executor .shutdown)))

;;; Run io 10000 times at 10 ms per io call with up to 100 concurrent calls
;;; and sum up all results.
;;; Then print the time it took and the resulting sum.
(let [sum (atom 0)]
  (time
   (do-concurrently (repeat 10000 (partial io 10)) 100 #(swap! sum + %)))
  (println @sum))
The trick is that you first submit c number of tasks to be executed concurrently. In this case, I've chosen to make 100 concurrent calls at a time. The call to submit is non blocking and will return immediately. After you've initiated your first batch, you block on cs, which will wait till any of them complete, and when one does, it will unblock and return the result of the task that just completed. When that happens, we will submit another task, so that we maintain our concurrency level, and we will call our handler with the result. In effect, we're saying, perform n number of calls up to c at a time. We are handling the results on the thread which submits the remaining tasks as they complete. This means that if our handler is very slow, it will delay our re-queuing of remaining tasks, so that's something to keep in mind. Finally, we have to handle the remaining batch of un-handled tasks, and shutdown the executor to release the resources associated with it.

❤️ 3
didibus21:07:42

This might be a bit advanced. But what it does is run tasks concurrently and as soon as any of them completes it lets you handle what to do next, where you could also submit even more tasks.

alpox21:07:59

Looks perfectly understandable to me. Many thanks! Mind if I take that snippet? 😉

alpox21:07:28

This is exactly what I was looking for, great!

didibus21:07:14

Go for it!

alpox21:07:17

Thanks! I just noticed that its not so streight-forward as I thought as the handler theoretically has to add more tasks to the executor but I might be able to adjust it

noisesmith22:07:19

(try
                (future (resolve @clj-promise))
                (catch Exception e
regarding this snippet, the catch here does nothing, as future doesn't blow up until you deref and that would be outside the try/catch

alpox23:07:41

Thanks for the heads-up - that passed by me in my testing.

alpox23:07:54

I managed to get it to work with the ExecutorCompletionServicenow by rewriting the do-concurrently function from @U0K064KQV such that it also handles additional tasks returned from the handler

👍 4