graalvm

2024-11-01T12:09:01.808899Z

Another problem arised. I have a library, let's call it my-lib. This library builds to native image well and pass tests when I run testing image with tests running entry point. I set :aot :all and made lein install . In another application I put [my-lib "0.1.3"] in dependencies ant trying to build testing native image with same entry point as in library, with same initialization configs and native-image properties (lib config passes via META-INF, I checked reports in both cases). But it fails on initialization stage with Detected a started Thread in the image heap. The application have extremely small code base ans there is nothing that can interfere... but anyway entry point is inside a library. Why it can be like that? I'd appreciate for any lead, because I already don't know what to try...

borkdude 2024-11-01T12:12:25.724039Z

Try to replace :aot :all with :aot [only.your.entry-point.namespace]

borkdude 2024-11-01T12:12:43.187469Z

aot all compiles the same namespace dependency multiple times which can lead to odd behavior

borkdude 2024-11-01T12:13:17.762119Z

now on the warning about the thread: this can happen when a thread, e.g. with future , is started on the top level of a namespace

borkdude 2024-11-01T12:13:53.303729Z

It would be actually nice if you could report back if your previous problems were fixed ;)

2024-11-01T12:36:13.990059Z

> It would be actually nice if you could report back if your previous problems were fixed 😉 Sure I will report all of them when solve complex issue. They all are connected to each other

👍 1
whilo 2024-11-25T17:25:16.516969Z

@octo221 look at this suggestions by @borkdude

🙏 1
2024-11-02T20:11:02.464699Z

Okay. I found out that root cause of all troubles was in usual dependency conflict. Unfortunately I wasn't so experienced to recognize it in native-image error messages

whilo 2024-11-14T08:30:17.318789Z

Found this, will try to see whether I can make progress https://github.com/clj-easy/graal-docs?tab=readme-ov-file

borkdude 2024-11-14T10:00:46.602279Z

@whilo native-image + clojure doesn't work without initialize-at-build-time

whilo 2024-11-14T16:06:23.298889Z

Right, I figured that out. Our older GraalVM version build pipeline through warnings at me about it, that is why I had removed it.

borkdude 2024-11-14T16:07:29.224719Z

We built this to deal with the warning: https://github.com/clj-easy/graal-build-time

whilo 2024-11-14T16:12:33.424639Z

Yes, I added it, but now my binary is even bigger. I fear there is a resolve call somewhere, but couldn't find it in the tracing reports yet.

borkdude 2024-11-14T16:13:47.185849Z

I've added this to babashka once to detect requires at runtime:

(def enable-require-scan
  "(do
    (def old-require require)
    (def old-resolve resolve)

    (def our-requiring-resolve (fn [sym]
                                 (let [ns (symbol (namespace sym))]
                                   (old-require ns)
                                   (old-resolve sym))))

    (defn static-requiring-resolve [form _ _]
      (prn :req-resolve form :args (rest form))
      `(let [res# (our-requiring-resolve ~@(rest form))]
         res#))

    (alter-var-root #'requiring-resolve (constantly @#'static-requiring-resolve))
    (doto #'requiring-resolve (.setMacro))

    (defn static-require [& [&form _bindings & syms]]
      (when (meta &form)
        (prn :require &form (meta &form) *file*))
      `(old-require ~@syms))
    (alter-var-root #'require (constantly @#'static-require))
    (doto #'require (.setMacro))

    (alter-var-root #'clojure.core/serialized-require (constantly (fn [& args]
                                                                    (prn :serialized-req args)))))



    (defn static-resolve [& [&form _bindings & syms]]
      (when (meta &form)
        (prn :require &form (meta &form) *file*))
      `(old-resolve ~@syms))
    (alter-var-root #'resolve (constantly @#'static-resolve))
    (doto #'resolve (.setMacro))
")


(when (System/getenv "BABASHKA_REQUIRE_SCAN")
  (load-string enable-require-scan))

❤️ 1
whilo 2024-11-14T16:16:35.933309Z

Oh, cool. Thanks!

whilo 2024-11-14T04:16:11.410999Z

@borkdude I am trying to help @sasha_bogdanov_dev by trying to shrink the datahike native image size (including pod) finally as we discussed in the past. I get " Could not locate clojure/core__init.class" at runtime when I remove "--initialize-at-build-time" from the compilation flags. In babashka you did not seem to using tracing like this https://www.graalvm.org/latest/reference-manual/native-image/guides/configure-with-tracing-agent/. How did you make sure you don't have these runtime errors?

2024-11-14T07:27:15.519019Z

Thanks a lot, @whilo, but in particular this moment I am more trying to understand why snap start is not working for my case…