Fork me on GitHub
#clojure
<
2021-06-15
>
marciol17:06:15

Hi all, I'm reading about the FFI and Java, and I wondering what the JEP 393 [1] will mean in the world of native integration regarding Clojure. Seems that the public state of the art is condensed in this article [2] from @chris441, but I'd like to know what you all think about it. [1]: https://openjdk.java.net/jeps/393 [2]: https://techascent.com/blog/next-gen-native.html

marciol17:06:30

Let's discuss on this thread 😄

phronmophobic17:06:35

I've recently been trying dtype-next and it's pretty great.

phronmophobic17:06:20

I've used ffi to access a few c libraries from clojure and it's been interesting to see that the c apis are often more clojure-friendly then many of the java APIs. Corollary: using ffi to directly talk to a native library is almost always better than trying to use a Java library that wraps a native library.

phronmophobic17:06:41

There are a couple libraries I have my eye on: • libclang: I want to parse c headers and have it produce the c<->clojure api for other libraries • graphviz: shelling out to dot is ok, but directly talking to the c api could be better. • ffmpeg: there's already avclj, but more of ffmpeg's capabilities can still be wrapped more idiomatically.

chrisn12:06:23

The foreign pathway definitely works and is quite fast, faster than JNA direct mapping or JNR by a bit but for instance when I used it with Python the python runtime could not find the numpy module which was very very odd so there exist some low level differences there to be worked out.

chrisn12:06:28

It worked fine with avclj, that is for sure. Speaking of which to take avclj (or something like it) further there is a cool python wrapper done in the same way (linking to the ffmpeg shared librarries) that has a good sort of filter-based interface. I can't remember the name offhand.

marciol23:06:18

Nice to hear about your experiences. I think that OpenJDK is condensing all improvements for java 17 on this umbrella https://openjdk.java.net/jeps/412 This is a subject that caught my attention, so I intend to understand how this stuff interacts with the dtype-next.

chrisn13:06:28

The memory model stuff (memory segments, writing to native memory) I don't use much aside to get an integer pointer as I already use sun.misc.unsafe for writing/reading to native memory. The ffi stuff, CLinker and friends I use when I generate jdk-16 FFI bindings and I do this using a library named insn https://github.com/cnuernber/dtype-next/blob/master/src/tech/v3/datatype/ffi/mmodel.clj. There could conceivably be a JDK-16 specific version of dtype-next that doesn't use unsafe at all and thus doesn't required restricted access but some parts of the memory model specification - regions - I have absolutely no use for as dtype-next uses the https://github.com/techascent/tech.resource to bind objects that need a dispose-or-free-type operation to the GC or stack frame as needed.

chrisn13:06:57

One thing to note as that graalvm has two completely different and unrelated FFI pathways of which I support one of.

antonmos18:06:58

Does anyone use datadog's javaagent in their clojure app? I gave it at try, but saw that the app took 5x time to start up.

👋 3
2
kenny18:06:30

We use it and have not experienced that. If it matters, we AOT compile prod artifacts.

jmv19:06:57

i have only noticed a very slow start when instrumentation debugging is on

lilactown19:06:11

what are the cases when Throwable->map would return multiple maps under the :via key?

Faiz Halde19:06:35

chained exceptions? if an exception has a cause set using initCause

Faiz Halde19:06:08

that's all i can see from the source code of that function

Alex Miller (Clojure team)19:06:49

yes, chained exceptions

Faiz Halde19:06:43

also, initCause isn't the only way to chain them. Throwable constructor accepts the cause as well

stephenmhopper21:06:56

Can I use the fancy new protocol :extend-via-metadata true functionality with Java interfaces somehow? Here’s a contrived example that doesn’t work. What’s the quickest way to make this work?

(def thing
  (with-meta {}
    {'clojure.lang.IDeref/deref
     (fn [this]
       "some value")}))

(deref thing)

seancorfield21:06:41

@stephenmhopper No, only protocols. You probably want to look at reify for interfaces.

dpsutton21:06:52

example

(let [x (reify clojure.lang.IDeref
          (deref [_] :foo))]
  @x)
:foo

2
☝️ 2
Joshua Suskalo22:06:34

Quick question about print-dup, is it intended to preserve metadata?

Joshua Suskalo22:06:03

I'd imagine not

Joshua Suskalo22:06:40

Although it looks like it does based on the implementation for persistent map

dpsutton22:06:13

can you explain your question a bit more?

hiredman22:06:11

it is, but print-dup is super rarely used

Joshua Suskalo22:06:15

print-dup is a multimethod for basic serialization, most of the time it constructs read-time evaluation forms. I'm just curious if it's in general intended to produce readable metadata forms as well as the actual data structure itself.

hiredman22:06:13

to some degree the use of print -dup is an anti-pattern because it is too specific

Joshua Suskalo23:06:57

That's fair, but I'm not really one to judge what the usage of the library would be. I'm just writing a general ropes library since the existing one only works on byte arrays.

Joshua Suskalo23:06:23

And since the core data structures support print-dup, so will I

hiredman23:06:53

user=> (doc *print-dup*)
-------------------------
clojure.core/*print-dup*
  When set to logical true, objects will be printed in a way that preserves
  their type when read in later.

  Defaults to false.
nil
user=> (alter-var-root #'*print-dup* (constantly true))
true
"user=> "{}
#=(clojure.lang.PersistentArrayMap/create {})
"user=> "
the one place I've seen print-dup used outside of the compiler (and I am pretty sure that ended up being a mistake) prn was used and *print-dup* was bound to true, print-dup wasn't called directly

Joshua Suskalo23:06:06

Yeah, it's intended mostly to be used with prn or pr-str to my understanding

didibus04:06:26

Ya, I got the impression print-dup is very much an internal detail, I don't know for what exactly. But for the user, you would want to extend print-method instead, see here: https://groups.google.com/g/clojure/c/R-9Pwk3HcFk

didibus04:06:44

This is true for prn and pr-str as well

hiredman23:06:08

in the compiler print-dup gets used for serializing arbitrary objects into byte code

hiredman23:06:49

so for example if a macro expansion contained one of your rope data structures (not the code to create one)

Joshua Suskalo23:06:20

Ah, makes sense.