Fork me on GitHub
#beginners
<
2021-10-20
>
Benjamin08:10:03

app.event('app_home_opened', ( this java api (nvm) has the signature this java.lang.Class... what do I pass there in the place of 'single_quoted_stuff' ? (fixed)

Benjamin08:10:38

nvm the snippet was javascript. The java api wanted a class and I achieved that by putting the class symbol name

noom12:10:39

Hey everyone, I'm having some trouble understanding how I should use core.async after having read some doc on it. I am coding a program that is long running and I'm trying to make it repl friendly so that I can have access to the marvels of interactive coding I have a start and a stop function. The start function initialize a jetty handler to listen to http requests and also run an infinite loop running the rest of the logic of the program. I run this infinite loop with (thread (run-loop)) so that it doesn't hang the repl. My question is, how should I communicate with this new thread to be able to tell it to stop for example?

noom12:10:32

I currently have a shared atom (def loop-running (atom false)) that I set to true when I run start and to false when I run stop interactively, the loop check that value in order to decide whether to recur or do nothing and exit.

noom12:10:21

This doesn't sound very optimal though and can lead to errors in some cases so I'm wondering if there is a better way? I am aware of chan s but it seems like there is no way to check if a signal arrived on the channel without blocking the loop

Apple13:10:22

either the run-loop actively checks for a signal to change like in your case an atom, or run-loop has to be passively signaled to stop. i dont think there can be much magic here...

noom15:10:43

I guess the atom solution isn't so bad then. What do you mean "has to be passively signaled to stop" though?

Apple03:10:31

never mind. now that i think about it again, eventually it's some code checking some variable and decides to staying running or quit, be the code on the caller side of callee side it's the same essentially.

noom12:10:50

Am I missing something with channels? Or is there another more idiomatic way to make the loop not hang the repl without another thread?

Yab Mas13:10:17

Hi, I'm building my first clojure(script) app and I'm having trouble debugging my backend. I'm trying to place printlines in a handler to see what's going on. Whenever I evaluate this hanlder-function from my editor it does print the values into the repl as expected. When I call my backends rest-api from the frontend it doesn't print any values. A response is returned and there's only one possible handler that could be called in this scenario, so I'm really sure the handler must have been called. Does anyone have an idea what's going on here? Please let me know if I should supply more information.

Apple13:10:00

regarding more info: perhaps some essential code to start with

Yab Mas14:10:51

Sure. I didn't include any as I suspect I'm missing a high-level/conceptual point, not so much wrong implementation. But here is an example of a handler function I would like to print some value of: (defn update-wall-by-id   [{{{:keys [wall-id holdmarkings]} :body} :parameters :as request}]   (prn request)   (create-or-update-holdmarkings {:wall-id wall-id :holdmarkings holdmarkings})   {:status 200    `:body (db-api/get-wall-by-id wall-id)})`

Apple14:10:11

did you reload your app? or do you have wrap-reload?

Apple14:10:44

"Whenever I evaluate this hanlder-function from my editor it does print the values into the repl as expected" how did you do this part?

Yab Mas14:10:34

I call the function from a comment block within the same file/namespace where the handler-function is defined. This is the code: (update-wall-by-id {:parameters {:body {:wall-id 1                                           :name "Modified Wall"                                           :image-url "test.jpg"                                           :holdmarkings [{:hold-id 1                                                           :points [{:x 1 :y 1}]}                                                          `{:hold-id 2`                                                           :points [{:x 2 :y 2}]}]}}})

Yab Mas14:10:16

I did evalute the namespace + core namespace + restart server several times.

Apple14:10:29

how about route matching?

Apple14:10:56

testing from editor is ok so the handler is ok.

Apple14:10:16

testing from frontend fails suggests that the handler was not called.

Yab Mas14:10:28

Im getting a response and there's only one possible handler for this route, so Im pretty sure this part works as well. But here's the code. Relevant part of routes: ["/:wall-id"     {:parameters {:path {:wall-id s/Int}}      `:get get-wall-by-id`      `:put {:parameters {:body {:wall-id s/Int`                                `:name s/Str`                                `:image-url s/Str`                                `:holdmarkings [{:hold-id s/Int`                                                `:points [{:x s/Num`                                                          `:y s/Num}]}]}}`            `:handler update-wall-by-id}}]`

Yab Mas14:10:40

I started to suspect the app uses a different output for the printline then my running repl when called from "outside", but I've no idea if that makes sense, Im new to working with repls so Im not sure what behaviour to expect.

Apple14:10:54

the prn output has to showup somewhere, either in your editor where repl is or the cli where repl is started.

Apple14:10:29

seems like this is routit make sure your frontend is testing the right url and routit is properly routing to that handler

Yab Mas14:10:53

I start my repl from my editor (VSC/Calva) so I guess that means the printline should show up in the same repl where I see the results of my in-editor evaluations.

Yab Mas14:10:39

Yes, I use reitit. The app is actually completely working, so Im pretty sure routing is fine.

Benjamin15:10:02

If I talk between 2 clojure applications is there any reason to use transit instead of edn? (performance is not critical in my use case)

sova-soars-the-sora16:10:04

Hi, want make functions spit out what sort of data they are expecting and what sort of data they output, maybe by adding logging lines via with-redefs (?) but I also want the function to do what it does normally. does it make sense to wrap it in a macro that tells me things and then just invokes the function?

Ed16:10:21

you can use alter-var-root to wrap functions with what the lisp peeps unsed to call advice ... which sounds a bit like what you want to do??

sova-soars-the-sora16:10:28

Oh hey that looks like a perfect fit. Gracias mucho

Ed16:10:33

(defn my-func [arg]
    arg)
  
  (alter-var-root #'my-func (fn [orig-fn] (fn [& args] (prn '> args) (apply orig-fn args)))) 

Ed16:10:36

actually ... you could add the orig fn as metadata just in case you want to undo it later

(alter-var-root #'my-func (fn [orig-fn] (with-meta (fn [& args] (prn '> args) (apply orig-fn args)) {:orig-fn orig-fn})))
??

sova-soars-the-sora17:10:38

Hmmm interesting idea

Benjamin17:10:56

what is an algorithm that will "raise" arbitrarly nested :bar in my map? {:foo1 {:foo2 {:bar 'baz}}} => {:foo1 {:foo2 'baz}} , while preserving the rest of the map structure

Benjamin17:10:44

Isn't recuring on maps wierd because of the non-linear structure?

lassemaatta17:10:29

perhaps (update m :foo :bar) might do the trick (except the arbitrarily nested part, where update-in might help..)

R.A. Porter17:10:39

Something like this might do it (using clojure.walk...

(defn pull-up-bar [m]
  (walk/postwalk
    (fn [{:keys [bar] :as x}]
      (if bar
        bar
        x))
    m))

R.A. Porter17:10:05

Hat tip to @U8QBZBHGD for the insight.

💜 1
R.A. Porter17:10:43

I'm not clear why one would want to do that...short of it really lending itself to a fn named "pull-up-bar". 😂

😂 2
Ben Sless17:10:29

You can also leverage libraries like malli and do it with transformers.

Ben Sless17:10:03

It's useful when you want to make data you get via some api look less silly

Benjamin17:10:48

Thanks for the suggestions, will check it. The use case is actually that I have a config map and I'd like to do something simple for defining things :prod and :dev next to each other or something not sure yet. The idea was that I could then do (get-in config {endpoint}) but yea not sure anymore if that's such a great idea -> what is a simple way to have 2 configs just 2 files?

Alex Appetiti17:10:18

I think that works - if I understood you correctly, it’s similar to the approach taken by https://github.com/juxt/aero#profile

1
bhenry17:10:21

i like to have a config slurped into a map, then the prod config can be (merge config prd-config) so that prd-config only has to override things that are specific to prod.

👀 1
👍 1
Scott Starkey20:10:00

Hi there. I have a bit of weirdness in my REPL, and I'm having a problem making it go away. I'm needing Java 1.8 now for a particular application.

scotto@Scotts-MBP ~ % lein version
Leiningen 2.9.7 on Java 1.8.0_292 OpenJDK 64-Bit Server VM
scotto@Scotts-MBP ~ % lein repl
nREPL server started on port 53638 on host 127.0.0.1 - 
REPL-y 0.5.1, nREPL 0.8.3
Clojure 1.10.3
OpenJDK 64-Bit Server VM 1.8.0_292-b10
    Docs: (doc function-name-here)
          (find-doc "part-of-name-here")
  Source: (source function-name-here)
 Javadoc: (javadoc java-object-or-class-here)
    Exit: Control+D or (exit) or (quit)
 Results: Stored in vars *1, *2, *3, an exception in *e

user=> (System/getProperty "Java.version")
nil
user=> quit
Bye for now!
scotto@Scotts-MBP ~ % java -version
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (AdoptOpenJDK)(build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (AdoptOpenJDK)(build 25.292-b10, mixed mode)
Would anyone know why (System/getProperty "Java.version") is evaluating to nil? How can I get it to recognize Java 1.8?

dpsutton20:10:47

(System/getProperty "java.version") use a lowercase j

❤️ 1
dpsutton20:10:40

(keys (System/getProperties)) if you're ever unsure

Scott Starkey20:10:49

I've been driving myself crazy. All for a capital letter! 🤪

sova-soars-the-sora04:10:29

been there just about every day this week xD

rmxm22:10:18

Concurrency question, what is a more common path? Threads or async? Async seems like a big buy-in, as you have to use nonblocking io etc. Is async default nowadays in clojure or do you typically start project with threads?

didibus04:10:51

Threads is more common in Clojure, but async is the way to go in ClojureScript.

hiredman22:10:32

deeply project dependent and mixable

💯 5
dpsutton22:10:48

i consider core.async to be a coordination and communication library and not really an alternative to threads.

Zach Mitchell, PhD22:10:13

Could you elaborate on this?

dpsutton22:10:37

channels are communication channels. and you can lock up the threadpool used for go blocks if you hog the execution time. There are pipeline functions but those just use standard threads and have mechanism to use channels to communicate. core.async just doesn't really offer these kinds of primitives and is just communication (imo)

didibus04:10:20

It is very much an alternative to threads, it is just that there are not good alternative to threads for heavy compute or blocking operations. But you can use its fibers effectively for async programming.

rmxm22:10:08

ok small folllow up, what is default threadpool size for futures?

rmxm22:10:03

> Creates a thread pool that creates new threads as needed, Is there any path towards discovering this number 🙂?

dpsutton22:10:21

its not a number. It is a cached thread pool

seancorfield22:10:33

As I said: unbounded.

hiredman22:10:50

if a new task is created and there is no thread already existing to run it a new threaded is created

Alex Miller (Clojure team)22:10:14

More importantly, why are you asking?

dpsutton22:10:30

has a description of thread creation policy

seancorfield22:10:35

If you run (future (Thread/sleep 10000)) in a loop, you'll likely get an OoM error based on not being able to create any new threads.

seancorfield22:10:57

@U11BV7MTK I linked to that above and @US9EF3BGU was quoting from that text.

seancorfield22:10:16

(so I guess the javadoc page is not clear enough about it being unbounded?)

rmxm22:10:17

asking to know if there is a limit in futures, or they can be invoked freely until they reach threadpool size (given they take time)

dpsutton22:10:32

wow. yes you did. sorry about that 🙂

seancorfield22:10:01

@US9EF3BGU See my comment about creating threads in a loop. It is unbounded and you will simply run out of threads and hit an OoM error.

hiredman22:10:08

there is no limit on the threadpool size, but you will run into other limits, os limits on threads or running out of memory for all the stacks of the threads, etc

rmxm22:10:37

yes, asking due to worry around backpressure from system, so can the threadpool from future be limited?

dpsutton22:10:52

threadpools can be limited. that particular one is not

seancorfield22:10:01

user=> (dotimes [n 10000] (future (Thread/sleep 10000)))
[35.261s][warning][os,thread] Failed to start thread - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
Execution error (OutOfMemoryError) at java.lang.Thread/start0 (Thread.java:-2).
unable to create native thread: possibly out of memory or process/resource limits reached

hiredman22:10:11

don't use that pool if you want limits

dpsutton22:10:36

https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ThreadPoolExecutor.html has some really great documentation about the way to think about them

seancorfield22:10:55

i.e., don't use a bare future call. Use a specific executor and submit tasks to it.

hiredman22:10:55

clojure has another built in pool that is limitted, but future doesn't use that one

hiredman22:10:17

core.async also has two pools, limitted and unlimitted

hiredman22:10:38

and the jvm has some concept of a default executor these days so it also may have two such

hiredman22:10:22

it is sort of a mess, but the pattern is clear enough

hiredman22:10:53

(the jvm's is the fork join common pool, and there is just the one, which would the equiv of the bounded executor case)

rmxm22:10:59

ok, so the question now, how would i run future(or something looking like a future) over a pool rather than unbound

hiredman22:10:14

you do it yourself

seancorfield22:10:29

As I said "Use a specific executor and submit tasks to it."

hiredman22:10:37

future is just a macro that wraps up the code and runs it on another executor

hiredman22:10:11

the core executor interface just takes Runnables, which all clojure functions are

hiredman22:10:00

I would strongly suggest sitting down with something like Java Concurrency in Practice, which is old, but very good

Alex Miller (Clojure team)23:10:45

also you can swap out the executor pool under futures with your own pool, but I would recommend making your own pool as they say https://clojure.github.io/clojure/clojure.core-api.html#clojure.core/set-agent-send-off-executor!

Alex Miller (Clojure team)23:10:17

the reason being that you don't know what other lib may also be using that pool and limiting it could violate those assumptions

seancorfield00:10:56

Perhaps the docstring of set-agent-send-off-executor! should mention that it also affects future? And perhaps future's docstring could mention that it uses the agent send-off executor? Or is that considered an implementation detail that shouldn't be relied on @U064X3EF3?

seancorfield00:10:35

(I was this week old when I discovered this connection -- courtesy of @U0NCTKEV8 at work this week 🙂 )

Alex Miller (Clojure team)00:10:50

On future, I'd say that's an implementation detail, and on the setter, if you're messing with this, you should be intimately familiar with the impl details

thom00:10:52

https://github.com/TheClimateCorporation/claypoole is a library that gives you more control over these things (including pmap and more).

didibus05:10:31

@US9EF3BGU This article: https://purelyfunctional.tv/guide/clojure-concurrency/ is really good to introduce you to all the options you have for concurrency.

didibus05:10:33

That said, since you mentioned being worried about backpressure. That's a bit of a different concern in my opinion. Core.async handles backpressure really well. ExecutorService can handle backpressure, but it requires you to create the ExecutorService carefully. They are often backed by a queue where tasks sit until there is a thread available. This queue can also be unbounded. For backpressure, you'll want to limit the size of that queue, but then you need a strategy for what should happen if someone tries to submit a task and the queue is full? Should the task be dropped? Should the caller thread block until there is room in the queue again? Should an exception be thrown?

rmxm08:10:09

So much good answers. Thanks all.