A thread dump might provide some details. But calling (shutdown-agents) at the end of your test suite will probably fix the issue.

I suspect an agent thread pool is the culprit here, but need to gather more context on what the program is doing before I can be sure; getting more familiar with thread dumps will be useful beyond just this one problem, I think!

you can take a thread dump with ctrl-\ if you are in proc, or via jstack or kill -3 if external

in the case of pending agents, the tell-tale sign is that it waits for 1 minute, then exits, but also you would see no RUNNING threads in the thread dump

It waits indefinitely, longer than one minute 😕

"dirigiste-executor-controller-0" #14 daemon prio=5 os_prio=0 cpu=515.81ms elapsed=193.28s tid=0x00007f669360d5d0 nid=0x6b87e waiting on condition  [0x00007f6661cf8000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep([email protected]/Native Method)
	at io.aleph.dirigiste.Executor.startControlLoop(Executor.java:511)
	at io.aleph.dirigiste.Executor.access$1000(Executor.java:9)
	at io.aleph.dirigiste.Executor$1.run(Executor.java:188)
	at java.lang.Thread.run([email protected]/Thread.java:833)

"manifold-pool-1-1" #15 daemon prio=5 os_prio=0 cpu=15.72ms elapsed=193.28s tid=0x00007f6692f7aa80 nid=0x6b87f waiting on condition  [0x00007f6661bf8000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x000000070b0eba90> (a java.util.concurrent.SynchronousQueue$TransferStack)
	at java.util.concurrent.locks.LockSupport.parkNanos([email protected]/LockSupport.java:252)
	at java.util.concurrent.SynchronousQueue$TransferStack.transfer([email protected]/SynchronousQueue.java:401)
	at java.util.concurrent.SynchronousQueue.poll([email protected]/SynchronousQueue.java:903)
	at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:52)
	at manifold.executor$thread_factory$reify__29401$f__29402.invoke(executor.clj:47)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.lang.Thread.run([email protected]/Thread.java:833)

"dirigiste-pool-controller-0" #16 daemon prio=5 os_prio=0 cpu=959.23ms elapsed=193.27s tid=0x00007f66936169d0 nid=0x6b880 sleeping [0x00007f6661af8000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep([email protected]/Native Method)
	at io.aleph.dirigiste.Pool.startControlLoop(Pool.java:367)
	at io.aleph.dirigiste.Pool.access$500(Pool.java:8)
	at io.aleph.dirigiste.Pool$1.run(Pool.java:387)
	at java.lang.Thread.run([email protected]/Thread.java:833)

those threads are all marked as "daemon" so will not prevent the process from exiting. Look for ones that are either RUNNING or non-daemons - those are the ones preventing exit

"pool-1-thread-1" #29 prio=5 os_prio=0 cpu=6.27ms elapsed=739.69s tid=0x00007f65b10af660 nid=0x6b8b6 waiting on condition  [0x00007f65b42fe000]
   java.lang.Thread.State: WAITING (parking)
	at jdk.internal.misc.Unsafe.park([email protected]/Native Method)
	- parking to wait for  <0x000000070b159e78> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
	at java.util.concurrent.locks.LockSupport.park([email protected]/LockSupport.java:341)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionNode.block([email protected]/AbstractQueuedSynchronizer.java:506)
	at java.util.concurrent.ForkJoinPool.unmanagedBlock([email protected]/ForkJoinPool.java:3463)
	at java.util.concurrent.ForkJoinPool.managedBlock([email protected]/ForkJoinPool.java:3434)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await([email protected]/AbstractQueuedSynchronizer.java:1623)
	at java.util.concurrent.LinkedBlockingQueue.take([email protected]/LinkedBlockingQueue.java:435)
	at java.util.concurrent.ThreadPoolExecutor.getTask([email protected]/ThreadPoolExecutor.java:1062)
	at java.util.concurrent.ThreadPoolExecutor.runWorker([email protected]/ThreadPoolExecutor.java:1122)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run([email protected]/ThreadPoolExecutor.java:635)
	at java.lang.Thread.run([email protected]/Thread.java:833)

"DestroyJavaVM" #40 prio=5 os_prio=0 cpu=66468.32ms elapsed=683.18s tid=0x00007f669001efb0 nid=0x6b823 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=173.21ms elapsed=776.13s tid=0x00007f669016e400 nid=0x6b82b runnable

"GC Thread#0" os_prio=0 cpu=1815.82ms elapsed=776.15s tid=0x00007f66900890c0 nid=0x6b826 runnable

"GC Thread#1" os_prio=0 cpu=1789.21ms elapsed=775.93s tid=0x00007f6658005180 nid=0x6b838 runnable

"GC Thread#2" os_prio=0 cpu=1783.05ms elapsed=775.93s tid=0x00007f6658005bd0 nid=0x6b839 runnable

"GC Thread#3" os_prio=0 cpu=1763.74ms elapsed=775.93s tid=0x00007f6658006620 nid=0x6b83a runnable

"G1 Main Marker" os_prio=0 cpu=6.88ms elapsed=776.15s tid=0x00007f6690099f00 nid=0x6b827 runnable

"G1 Conc#0" os_prio=0 cpu=916.13ms elapsed=776.15s tid=0x00007f669009ae80 nid=0x6b828 runnable

"G1 Refine#0" os_prio=0 cpu=26.22ms elapsed=776.15s tid=0x00007f669013e830 nid=0x6b829 runnable

"G1 Service" os_prio=0 cpu=569.62ms elapsed=776.15s tid=0x00007f669013f740 nid=0x6b82a runnable

"VM Periodic Task Thread" os_prio=0 cpu=692.39ms elapsed=776.11s tid=0x00007f66901a4750 nid=0x6b835 waiting on condition

that top one is the important one, for sure

it is a non-daemon thread, blocked and waiting for something (which presumably will never arrive)

whose thread that is, I don't know - it's not Clojure or core.async afaik

How might I go about finding that out?

I guess by the naming, that's an executor thread pool, so could really be created by anyone

you or one of your libs is creating a ThreadPoolExecutor with non-daemon threads so could probably catch the construction point in a debugger would be my best tip

if you look at cognitect's aws-api it does this with the json descriptions of aws services

@U0NCTKEV8 neat, I'll take a look! Thanks

generating the client from an api description is one of my favorite tricks (I've even multiple times generated clients from html documentation, html is structured data too)

the big change in how I do it once I saw aws-api is I shifted from generating functions for client operations to instead having a single invoke operation that takes a data description of the operation

Oh yeah. Seems so unnecessary to redefine same thing more than one time. On that note, have you used anything for gRPC?

Yes, that sounds like much easier to grasp

I have used various google apis, which I think are technically defined as grpc services, but they all have ways to interact with them as a rest+json api

and they publish api specs that are mostly in the openapi format, so no dealing with protobufs

if you are dealing with a pure grpc+protobuf service that is going to be much more annoying I think, the whole ecosystem is built on ahead of time code generation

Ah. That's a miss for my usecase then, as I'm building a test client for our internal services, gRPC only 🙂

I've found a good library to ping back and forth between clojure DS and protobuf definitions, but no good one for server/client implementations so far, so I digress to Java interop for now.

there is no reason you couldn't have a clojure protobuf library that could decode protobuf messages for a given schema without having to do code generation with protoc, but to my knowledge no one has written such a thing (I've fiddled with writing it but usually give up when it comes to parsing .proto files)

Yeah, no, code generation is alright. Although without it, would be even better, but still. 🙂

The lib I've fiddled with is com.appsflyer/pronto, if anything. Seems fine enough, though can't say I've used it extensively. It depends on generated Java code though.

While I can't recommend gRPC, I worked with pronto's author for a couple of years, and I can vouch for the quality of his work. One of the best software developers I've met.

@U0NCTKEV8 why not just use instaparse with the official bnf?

the official bnf is an illusion

so much so that some startup just launched their own version of the spec with what they claim is the complete real bnf

https://protobuf.com/docs/language-spec > Google's documentation can be found https://developers.google.com/protocol-buffers/ (with separate grammars for https://developers.google.com/protocol-buffers/docs/reference/proto2-spec and https://developers.google.com/protocol-buffers/docs/reference/proto3-spec syntax). But these grammars are incomplete. In the face of these documentation shortcomings, the actual implementation in the protoc compiler prevails as the de facto spec. Without complete and accurate documentation, it is very hard for the community to contribute quality tools around the language.

I have actually tried at various points feeding google's bnf (heavily modified) to instaparse, then using the resulting parser to parse all the .proto files in the protoc repo as an initial test, but ran into issues with the bnf and if I recall I wasn't to happy with the performance of parsing either

If you're ever interested in tackling it again let me know, I would be interested in lending a hand

you can pass options to the read functions to conditionally preserve reader conditionals

(read-string {:read-cond :preserve} "#?(:cljs (:require [goog.string :as gstr]\n [goog.string.format])\n :clj (:refer-clojure :exclude [format]))")

but maybe I'm completely not answering your question

oh ok, and then edit the result of that and pr-str again

I'm looking for a way of dealing with that situation using rewrite-clj, but :read-cond :preserve is hacky but maybe helpful there

or maybe it is easier to don't even bother reading the string and just do str/replace

the problem I'm trying to solve is that I have a namespace with functions that can instrument code (for FlowStorm debugger) and I want it to be able to instrument itself, then I need a copy of the namespace, maybe someone has a better idea for that

see also #rewrite-clj

Hi @U0739PUFQ, I am a maintainer on rewrite-clj, I can help you out. But… I’m not sure why you want to rename namespaces, if it is to inline them so they do not conflict with other libs, then you might be interested in https://github.com/benedekfazekas/mranderson. Like @U45T93RA6 suggests, drop by #rewrite-clj and we’ll try to help you out.

thanks @UE21H2HHD, will repeat the question there

I hadn't heard that, and I am not sure what it means

As far as I remember the sentiment of that person was that if you have longer chains of functions, it might be more readable to split them up into several bindings in the same let expression and have named intermediate results than chaining them through a threading macro.

I disagree with that, no reason to force names for things that don't need to be named

the idea that (f (a b)) is ok, but if you write (-> b a f) suddenly you need to be writing (let [x (a b)] (f x)) doesn't hold water

Thanks, that makes sense to me. I guess names for intermediate results can be helpful for longer call chains, but it’s too dependent on context, generalizing rules don’t help here

I remember sharing https://gist.github.com/hiredman/184831 in the clojure irc years ago, and rhickey being somewhat disgusted by it, and shortly afterwards clojure.core/->> got added

Ha, then good things came from it

I'm with hiredman on this: I don't see any value in creating a bunch of single-use names just to avoid threading pipelines.

https://stuartsierra.com/2018/07/06/threading-with-style is a nice and nuanced stance.

I can definitely see using let to extract an important, slightly complex, or less-than-totally-straightforward anonymous function from a threaded pipeline... with the plan of shoving that binding back in where the function was.

(let [foo (fn [a] 
            (do-some-semi-elaborate-computation-with a)]
  (-> bar
      baz
      foe
      foo
      fee))

Not exactly what was asked for but this made me think of https://mobile.twitter.com/borkdude/status/1250442846089310210