2023-11-02 cljdoc | Clojure Slack Archive

cljdoc

liebs 2023-11-02T23:20:13.079769Z

Hi there, I'm having some issues with the analysis job timing out. The CircleCI logs appear to always hang at this stage:

2023-11-02 23:12:42,868 INFO  cljdoc-analyzer.runner - launching metagetta for: org.clojars.slothrop/clj-baseball languages: [clj]

and then it runs to the 10min cap. I'm not really sure how to debug this.

liebs 2023-11-03T14:03:01.078359Z

hey @lee! Thanks for this really thoroughgoing reply. I'll make this change today and see if it helps. Guess I gotta be more careful about these things.

lread 2023-11-03T14:32:01.400109Z

You are welcome @bhlieberman93! We all learn from each other here, glad I could help.

lread 2023-11-03T00:47:40.789189Z

Hi @bhlieberman93! I'll take a peek.

lread 2023-11-03T01:33:08.118919Z

Hmm.... analysis actually finishes but never exits... Does loading any of your namespaces start up a daemon thread?

lread 2023-11-03T02:06:21.754629Z

So, I think this is the culprit: https://github.com/bhlieberman/clj-baseball/blob/7bb29be285c90ffeec960bb0ea176525ee70ae3b/src/com/slothrop/clj_baseball/player/lookup.clj#L24-L28 This will get run every time the namespace loads. It is causing some threads to start, which is keeping whatever process loaded this namespace (including cljdoc-analyzer) from closing naturally.

lread 2023-11-03T02:14:16.114379Z

To diagnose, I poked around your code. Tried loading various namespaces while looking at threads with visualvm. And then verified by starting up a REPL, like so:

clj -Sdeps '{:deps {org.clojars.slothrop/clj-baseball {:mvn/version "0.4.2"}}}'

and looked at the REPL process threads with visualvm. After loading this I noticed the new threads:

user=> (require '[com.slothrop.clj-baseball.player.lookup])

So I closed my REPL and started another one to see if I could isolate. This also starts up those threads:

user=> (require '[clj-http.client :as client])
nil
(def register (client/get ""
                          {:async? true
                           :as :stream}
                          (fn [resp] resp)
                          (fn [err] (throw (ex-info "Request for player lookup table failed" {:cause err})))))
#'user/register

lread 2023-11-03T02:18:54.205399Z

Another way to test, from an empty dir, create a test.clj with:

(require '[clj-http.client :as client])

(def register (client/get ""
                          {:async? true
                           :as :stream}
                          (fn [resp] resp)
                          (fn [err] (throw (ex-info "Request for player lookup table failed" {:cause err})))))

(println "all done")

And run:

clj -Sdeps '{:deps {org.clojars.slothrop/clj-baseball {:mvn/version "0.4.2"}}}' -M test.clj

Notice that it just sits there and does not exit.

lread 2023-11-03T02:22:45.568889Z

Here's a screenshot of threads from visualvm (new threads are ones starting at "I/O dispatcher"):

lread 2023-11-03T02:25:33.983449Z

So... probably best not to create threads from a def var.

Clojurians Log v2

cljdoc