Fork me on GitHub
#clojure
<
2021-03-09
>
kah0ona08:03:37

(future
    (try
      (println "inside future body")
      (assert false) ;;
      (catch Exception e
        (println "error" (.getMessage e))))) ;; no error printed

  (try ;; not in a future, but just normal
    (println "inside future body")
    (assert false) ;;
    (catch Exception e
      (println "error" (.getMessage e)))) ;; error printed

  (future
    (try
      (println "inside future body")
      (throw (ex-info "throwing" {}))
      (catch Exception e
        (println "error" (.getMessage e))))) ;; error printed

flowthing08:03:12

user=> (assert false)
Execution error (AssertionError) at user/eval17874 (REPL:1).
Assert failed: false
user=> (instance? Exception *e)
false
user=> (instance? Throwable *e)
true

flowthing08:03:01

That is, assert throws AssertionError if it fails. AssertionError is not an Exception, but it is a Throwable.

👆 3
kah0ona08:03:12

the top one doesn’t go to the catch clause

kah0ona08:03:18

but the others do

dangercoder10:03:01

How do you handle interopt with Java's Functional Interfaces? Right now it's Reify for me and some nice to have functions & macros. I've looked into libraries such as https://github.com/athos/power-dot

vemv10:03:36

I use reify - it's simple, transparent and flawless. The only problem IMO is discovery (what interfaces/methods should one implement). Extending https://github.com/alexander-yakushev/compliment/ might help if you use cider-nrepl

vemv11:03:15

clj-refactor.el's cljr-add-stubs also helps. I discovered a minor bug in it though https://github.com/clojure-emacs/clj-refactor.el/issues/479 (discovered while doing Functional stuff)

dangercoder11:03:01

cool, thanks for the insight @U45T93RA6 🙂

Alex Miller (Clojure team)13:03:44

I’m very curious what your use case is leading you to ask - the core team has been looking at this

Alex Miller (Clojure team)16:03:12

I'm familiar with all these, what I'm looking for is: what problems are you having in your code base where you reach for these?

Alex Miller (Clojure team)16:03:36

"I wanted to do X but I couldn't or it was cumbersome... "

schmee17:03:40

you might be familiar with all these, but @UBN9SNVB4 might not be, and he asked the original question 🙂

Alex Miller (Clojure team)17:03:49

oh for sure, thx for dropping them

dangercoder18:03:38

@U064X3EF3 I asked because the functional interface interopt becomes cubersome due to the proliferation of interfaces that one has to explicitly implement. example in the kafka streams DSL Java

streams
.filter((k, v) -> value.equals("Purple haze, all in my brain"))
.mapValues(value-> value.toUpperCase());
Clojure
(-> streams
    (.filter (reify Predicate
               (test [_ k v]
                 (= "clojure-rocks!" v))))
    (.mapValues (reify ValueMapper
                  (accept [_ v] (clojure.string/upper-case v)))))

Alex Miller (Clojure team)19:03:00

but why are you using streams at all? why not just filter / map in clojure

Alex Miller (Clojure team)19:03:55

where are you getting the streams from?

Alex Miller (Clojure team)20:03:22

do you have a link to where you're getting that example?

dangercoder20:03:55

you're correct @U064X3EF3, it's an example from the official java client kafka streams lib: Simple Clojure example (not mine, but identical to the code that I gave as an example) https://github.com/perkss/clojure-kafka-examples/blob/master/kafka-streams-example/src/kafka_streams_example/core.clj Java example mapValues in the official docs: https://kafka.apache.org/20/documentation/streams/developer-guide/dsl-api.html

dangercoder20:03:27

If you prefer a gist with 100% identical java/clojure I can fix that

Alex Miller (Clojure team)20:03:23

that's helpful, thx

✌️ 3
schmee21:03:56

@UBN9SNVB4 you’ve probably seen this, but there is also https://github.com/fundingcircle/jackdaw

dangercoder21:03:59

I have seen it, it seems nice but I prefer embracing java-interop

👍 6
aratare11:03:52

Hi there. Is there a way to restart the agent thread pool after shutdown-agents has been called?

emccue15:03:40

@rextruong technically you can set the agent executors to something custom that could support restart

pinkfrog15:03:40

Can we set the thread pool size used by future?

Alex Miller (Clojure team)15:03:44

those methods @emccue linked can be used to customize the executors

emccue15:03:50

That uses clojure.lang.Agent/soloExecutor

emccue15:03:10

which is what set-agent-send-off-executor! changes

Alex Miller (Clojure team)15:03:30

that said, there is no thread pool size for that pool - because it can do IO, you can lock up the runtime if you constrain it

pinkfrog15:03:08

Yup. That’s why I want to make the threadpool a little bit larger.

Alex Miller (Clojure team)15:03:16

there is no thread pool size

Alex Miller (Clojure team)15:03:24

it adds threads as needed

emccue15:03:25

volatile public static ExecutorService pooledExecutor =
	Executors.newFixedThreadPool(2 + Runtime.getRuntime().availableProcessors(), 
		createThreadFactory("clojure-agent-send-pool-%d", sendThreadPoolCounter));

volatile public static ExecutorService soloExecutor = Executors.newCachedThreadPool(
	createThreadFactory("clojure-agent-send-off-pool-%d", sendOffThreadPoolCounter));

pinkfrog15:03:22

Thanks. Was checking that too.

emccue15:03:58

and none of this is there in the docs of future

emccue15:03:00

Takes a body of expressions and yields a future object that will
invoke the body in another thread, and will cache the result and
return it on all subsequent calls to deref/@. If the computation has
not yet finished, calls to deref/@ will block, unless the variant of
deref with timeout is used. See also - realized?.

emccue15:03:35

but it is alluded to in a comment in one of the community examples

emccue15:03:59

;; Note: If you leave out the call to (shutdown-agents), the program
;; will on most (all?) OS/JVM combinations "hang" for 1 minute before
;; the process exits.  It is waiting for a thread created by the
;; future call to be shut down.  shutdown-agents will shut them down
;; immediately, or (System/exit <exit-status>) will exit immediately
;; without waiting for them to shut down.

Alex Miller (Clojure team)15:03:17

if you want to file a docstring request, please log a question at https://ask.clojure.org

James Carr15:03:11

What is a really good fullstack demo clojure application? Looking for something I can just clone and run docker-compose up, play around with some metric tracking that interests me on the devops side of the house

borkdude15:03:48

Maybe juxt/edge?

James Carr15:03:03

Looks close... was hoping to find something working with a database like postgresql that I could just run docker-compose up on

p-himik15:03:33

Not sure about the "really good" part - I've never seen it before. Just something I was able to find quickly.

afry15:03:25

Hey all, do you have a favorite process manager for your apps? I've got multiple Clojure/ClojureScript apps running on a single VM, and I'm trying to find a good way to wrangle them. Not only for things like automatically restarting apps on server reboot, but also for development (streaming logs for debugging, stopping the app, restarting the app, etc.) I've enjoyed using Supervisor a lot (http://supervisord.org/), but I'm finding that there are logs coming out of my apps which don't end up in either the stdout or stderr logfiles. It's got such a nice CLI, but if I'm missing errors or failed ShadowCLJS compilations, then it's hurting me more than it's helping me. Anyway, any recommendations?

ghadi15:03:54

systemd if you're not going to go the container route

afry16:03:21

I do like systemd for automatically starting processes and stuff like that, but it would only solve part of my problem.

ghadi15:03:34

Is this is for a dev setup then maybe something different

afry16:03:45

What I'm trying to achieve is a unified prod/dev kind of thing. I want to develop in the same environment that my production app will run in. I've got everything in a Vagrant VM, so I've got the environment part of it figured out, but what I'd like is something that will make developing on the VM as "easy" as developing directly on my local. If I was developing on my local for example, I'd probably open up a few different terminal windows and invoke clj -M:alias or whatever the startup command is for each app. My dream VM version of that is to have a little CLI that can invoke all of the startup commands, can foreground/background the apps easily, and can still give me that same direct access to streaming logs. Supervisor can almost do all that, but logging everything from one app to one streaming output doesn't seem to be possible, at least as far as I can see.

lukasz16:03:47

To be honest, I'd go with Docker and Docker compose for single-machine, multi-app deployment. Your dev environment and production will always be different

afry17:03:55

What would be the benefits of Docker over Vagrant? I'm fairly comfortable in a *nix environment, but pretty new to developing on a VM. After a little research, Vagrant seemed like a good option for my goals. And I do think I've achieved that 1:1 parity between dev and prod to be honest (complete with SSL and everything). I do like the idea of hitting all of my "production problems" early on so I don't have to encounter and solve all of them in the 11th hour before I deploy, but it does seem to add a lot of complication too. I'd be curious to get your perspective though.

lukasz17:03:01

I've used Vagrant at multiple shops as means of standarizing dev environments, and it never looked like production (even before Docker), on top of that developer experience is not so great, especially when it comes to 'edit locally see changes in the VM', that part is always flaky. Of course you can work in the VM directly, and that (IMHO) works great if you accept some trade-offs (performance being the biggest one). Now for production, even if we're talking about a single machine - Docker simplifies this in many ways: you can test and build software locally with the same compose setup, but you get the additional benefit of not tying your application's runtime to the host, and forces you early on to figure things like secrets management, repeatable builds and so on. Sorry for the random thoughts, I'm just speaking from experience of trying to achieve dev-prod parity and never seeing it realized fully (or even in 50%)

p-himik19:03:32

As an alternative to Docker, it might be worth looking into podman. It's almost a drop-in replacement that doesn't require root privileges.

lilactown16:03:03

I'm once again running into the wall that is my ignorance of JVM multithreaded constructs. looking for advice: I working on a bit of CLJC code that schedules a task to be run in some way that doesn't hog the current thread. It sounds like in the JVM nomenclature this could be easily handled by an ExecutorService?

Jakub Holý (HolyJak)16:03:48

Yes. Or just use (future..) if you don't need to set up a dedicated thread pool and are ok with using the default one...

lilactown16:03:34

ah, I guess I should have specified that multiple tasks could be queued up at once and I want to process them in some order

lilactown16:03:08

hence the desire for something more than (future ,,,)

lilactown16:03:46

The eventual goal is to "chunk" the task up so that it can be paused & resumed later based on the "priority" of the task, e.g. if a high-priority task while a low-priority task is being processed, it could pause the current task, process the high-priority task, and then resume. If I already have a way of "chunking" a task up like this, is it a good idea to wrap this in an ExecutorService? Or do I need something fancier?

Jakub Holý (HolyJak)18:03:52

Do tasks support pausing? I guess look at the E.S.api,does it do what you want?

lilactown19:03:48

I don't think so out of the box

lilactown19:03:16

I'm tempted rn to build my own task running/scheduling machine using core.async but I also really don't want to depend on core.async

Ben Sless21:03:55

core.async already has a running/scheduling machine for tasks - pipelines. They guarantee order maintenance and predetermined parallelism. They're pretty good at their job

Ben Sless21:03:40

But not suitable for pause-resume

Jakub Holý (HolyJak)16:03:08

Any good ideas for extracting all calls to (log/warn ...) from .cljc files? I thought about simply using clojure.edn/read and then prewalk the data but it cannot be read because of #?(..) Is there some cljc aware reader? Thanks!

lilactown16:03:10

this might be more than you're looking for, but https://github.com/borkdude/edamame I think fits the bill

lilactown16:03:34

you can configure it to process reader conditionals

flowthing16:03:58

clojure.core/read-string also takes :read-cond :allow, but note that it can execute code if *read-eval* is true.

flowthing16:03:55

I think you can also use https://github.com/jpmonettas/clindex or clj-kondo to find var usages.

vemv19:03:31

there's tools.reader also with .cljc support

👍 3
lilactown16:03:46

The eventual goal is to "chunk" the task up so that it can be paused & resumed later based on the "priority" of the task, e.g. if a high-priority task while a low-priority task is being processed, it could pause the current task, process the high-priority task, and then resume. If I already have a way of "chunking" a task up like this, is it a good idea to wrap this in an ExecutorService? Or do I need something fancier?

jmckitrick19:03:28

Are there any current libraries for SFTP? Or does it make sense to just wrap the apache commons functions for SFTP?

borkdude19:03:34

@jmckitrick I've been using this one for a long time and it hasn't disappointed me https://github.com/miner/clj-ftp

jmckitrick19:03:17

I saw that, but didn't see sftp support on the home page. Guess I should dig a little deeper?

borkdude19:03:13

hmm, I guess you're right. sorry

jmckitrick19:03:31

Rats. I was hoping you were right.

vemv19:03:10

The repo is active though so you may have luck requesting that feature. Or maybe simply hacking that .clj file

viesti06:03:50

> SFTP is also supported. > My colleagues used probably this library to monitor file drops on a file server. Long time ago so don't remember if this was the same lib, but looks similar enough :)