graalvm

sb 2023-09-05T12:48:23.952379Z

Hi, is that possible use libpython-clj in graalvm? I tried but I got after this error:

(ns simple.search
    (:require [libpython-clj2.python :refer [py. call-attr] :as py])
  (:gen-class))

(println "Initializing...")
(require '[libpython-clj2.require :refer [require-python]])

sb 2023-09-05T12:48:38.858359Z

Error message (not full)

INFO: Reference thread starting
Initializing libraries
Initializing test
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to jdk.internal.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of tech.v3.datatype.UnsafeUtil. Detailed failure reason(s): Could not determine the field where the value produced by the call to jdk.internal.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store. 
[2/7] Performing analysis...  [**********]                                                             (100,5s @ 3,61GB)
  16 016 (92,00%) of 17 408 classes reachable
  18 997 (30,45%) of 62 382 fields reachable
  61 021 (61,65%) of 98 985 methods reachable
     306 classes,   559 fields, and 1 089 methods registered for reflection
      68 classes,    88 fields, and    56 methods registered for JNI access
       6 native libraries: -framework CoreServices, -framework Foundation, dl, pthread, stdc++, z

------------------------------------------------------------------------------------------------------------------------
                        6,9s (5,8% of total time) in 27 GCs | Peak RSS: 5,72GB | CPU load: 4,23
========================================================================================================================
Failed generating './target/cli4clj-sample-project' after Error: Unsupported features in 6 methods
Detailed message:
Error: Detected a started Thread in the image heap. Threads running in the image generator are no longer running at image runtime.  To see how this object got instantiated use --trace-object-instantiation=java.lang.Thread. The object was probably created by a class initializer and is reachable from a static field. You can request class initialization at image runtime by using the option --initialize-at-run-time=<class-name>. Or you can write your own initialization methods and call them explicitly from your main entry point.
Trace: Object was reached by
  reading field java.util.concurrent.ThreadPoolExecutor$Worker.thread of constant 
    java.util.concurrent.ThreadPoolExecutor$Worker@4af7ac25: java.util.concurrent.ThreadPoolExecutor$Worker@4af7ac25[State = 0, empty queue]

borkdude 2023-09-05T12:53:32.028969Z

I'd ask @chris441 directly

πŸ‘ 1
sb 2023-09-05T13:06:07.313959Z

If this doesn’t work, then bbin will be the solution for me, with a little bit of pods. πŸ™‚

souenzzo 2023-09-05T17:30:07.873459Z

btw, why use libpython-clj in graalvm? you can access python via polyglot API

πŸ‘ 1
sb 2023-09-05T17:38:35.378549Z

Yes, this is third option, just I need inject into Python a Clojure function. I didnt do similar with Polygot

sb 2023-09-05T17:43:11.998779Z

I filter in one Python indexed db with Clojure logic (rules). I will share with you on the ~ next week. Vector search engine like ChromaDB. ((I can share because the investor is out.. core of this, written in Clojure))

souenzzo 2023-09-05T17:44:04.416209Z

seems a good case to use polyglot API

❀️ 1
souenzzo 2023-09-05T17:44:25.551659Z

or simply Clojure+libpython on a regular JVM

sb 2023-09-05T17:48:49.148189Z

I do the last. Works fine, just I thought I can write a native cli for this. I test in first way

sb 2023-09-05T17:49:07.740429Z

Thank you!! πŸ‘

chrisn 2023-09-05T21:37:21.750409Z

Have you tried to load C extensions via the polyglot API?

sb 2023-09-06T03:47:33.621049Z

Not, but I check today. I tried import numpy and hnswlib.. via polygot, I played with that in the night.. ginstall have numpy package, but not hnswlib. (I check with venv)