Fork me on GitHub
#graalvm
<
2020-08-25
>
plexus05:08:28

anyone know of a way to check at runtime if you're running inside a native image vs on a regular JVM?

plexus06:08:26

perfect! thanks!

plexus08:08:12

took a lot of trial and error but I managed to get funnel to daemonize/background itself

🎉 3
plexus08:08:06

some gnarly JNI shenanigans... still need to clean this up but might do a write-up later

👀 3
plexus08:08:36

actually the thing that took the most time was trying to somehow statically include my native code in the final executable, because it's just one tiny function, it seemed so silly to have to distribute a separate .so for that... tried all kinds of linker options and read tons of stuff online until finally decided that it can't be done 🙂 then figured out a great hack around it. Include the .so as a resource, copy it to the filesystem when starting up and then load it from there

sogaiu08:08:04

sounds pretty nifty! so for linux you have .so, macos .dylib, and windows .dll and choose among them at runtime?

borkdude08:08:19

@plexus If you are going to do a writeup, might want to also put it in clj-graal-docs

sogaiu08:08:49

oh yes, that would be nice

borkdude08:08:29

@plexus The resource hack has already been done before as part of spire, https://github.com/borkdude/clojure-rust-graalvm and some others

borkdude08:08:04

Spire was the first one that did it I think

sogaiu08:08:33

mmm, lacking .dll 😛

borkdude08:08:55

I leave .dll as an exercise to the reader ;)

sogaiu08:08:07

he he 🙂

borkdude08:08:09

This is only a POC

sogaiu08:08:27

a very nice one at that 🎩

borkdude08:08:06

The background / daemonizing I would be interested to see

borkdude08:08:29

Here are already some docs about interacting with native libraries: https://github.com/lread/clj-graal-docs#interfacing-with-native-libraries

plexus08:08:51

yeah haven't gotten to the macos part yet, since I don't have access to a mac.

borkdude08:08:21

@plexus CircleCI and Github Actions have so that should not be too hard to get around

plexus08:08:30

I know, but for debugging that's not very helpful

borkdude08:08:48

CircleCI offers ssh access

borkdude08:08:50

Not too long ago I didn't have access to Windows so I debugged everything on appveyor. A bit tedious but it works. Having SSH access makes all the difference for getting fast feedback though.

borkdude08:08:23

You might want to poke @marc-omorain for macOS access if it's not enabled for your open source project yet, if you go with CircleCI

plexus08:08:28

this is the one project where we're using GH actions actually, although generally my intention is to migrate towards GH actions for the other LI projects as well

borkdude10:08:32

Impressive, so you're using C to daemonize the Graal process

borkdude10:08:22

what's this for?

if ((chdir("/")) < 0) {
      exit(EXIT_FAILURE);
    }

plexus10:08:34

some code I cargo culted from this example http://jnicookbook.owsiak.org/recipe-no-029/ as far as I understand it changes the working directory of the process to "/", which I'm guessing is so that it becomes independent of the location where it was started, for instance if the directory it was started from gets deleted

plexus10:08:10

yeah it's really just using JNI so I can do fork + setsid

borkdude10:08:19

what if the process doesn't have permission to chdir to /?

plexus10:08:40

then I suppose it fails. When would that be the case though?

borkdude10:08:51

don't know :)

borkdude10:08:05

does daemonize/chdir/etc also work via JNI in the JVM?

borkdude10:08:27

I've been considering some function in babashka to do this, but the JVM itself doesn't support this

plexus10:08:42

ls -ld /
drwxr-xr-x 21 root root 4096 Jun  1 12:43 /

plexus10:08:56

read+execute permissions globally, I'm guessing that's pretty standard

borkdude10:08:37

does chdir("/") also work on Windows with the forward slash?

plexus10:08:53

you could in theory do the same thing on the JVM, but I believe it's not considered safe. It may also not be safe in this case, time will tell

plexus10:08:11

re. windows, probably not. I'm not doing windows builds at this point.

plexus10:08:19

https://blog-old.headius.com/2009/05/fork-and-exec-on-jvm-jruby-to-rescue.html > Update: The biggest problem with using fork+exec in this way is that you can't guarantee nothing happens between the fork call and the exec call. If, for example, the JVM decides to GC or move memory around, you can have a fatal crash at the JVM process level. Because of that, I don't recommend using fork + exec via FFI in JRuby, even though it's pretty cool.

borkdude10:08:50

I guess you can also just run funnel &?

plexus10:08:30

yes, but then it's not detached from the parent shell process. If you close the shell it will kill the child

plexus10:08:40

I think you could do nohup funnel &

borkdude10:08:41

I guess you can install funnel as a service, systemd kind of thing?

plexus10:08:03

yes, totally

plexus10:08:17

the direction I'm going now is that tools like kaocha-cljs2 that need it will try starting it in the background. If there's already one running then it's a no-op.

plexus10:08:42

although at this point I'd like users to install and run it themselves, so they know it exists, since it's a good place to look when troubleshooting

borkdude10:08:34

@plexus I guess you can also use java.lang.ProcessBuilder to run one in the background, without daemonizing

borkdude10:08:59

each test run would then have to run its own instance

borkdude11:08:12

or rely on some pid file or something to check if there's already one

plexus11:08:21

but then it would only live as long as the test runner process, that defeats the purpose

jumar11:08:00

If you start a subprocess from a Java process the child isn’t terminated when parent exits - that’s a typical behavior in Linux

plexus11:08:50

> Funnel provides persistence and discoverability. It keeps connections to long lived processes (like a browser), so that short-lived processes (like a test runner) can discover and interact with them . This is why we recommend running Funnel as a separate long-lived process, rather than for instance embedding it into the tool that uses it.