Fork me on GitHub
#babashka
<
2022-04-13
>
Jeffrey Bay17:04:29

is there a way in babashka to do an actual unix exec? i.e., not the java "exec" which is actually fork/exec and retain parent/child relationship, but rather, "exec child and exit self"?

borkdude17:04:59

@jbay Theoretically that could be supported, but isn't right now.

Jeffrey Bay17:04:10

okay, thanks for the quick answer!

borkdude17:04:38

The closest is (System/exit (:exit @(babashka.process/process ["child_process"] :inherit true)))

Jeffrey Bay17:04:05

the babashka process still hangs around for the life of child, though, right?

Jeffrey Bay17:04:48

this is why lein has this whole shell wrapper and trampoline file do-si-do which seems like madness. it baffles me that java still doesn't support a native unix-exec semantic

borkdude17:04:32

For Java this makes more sense than bb: bb doesn't use a lot of system resources. So why is this important?

Jeffrey Bay17:04:18

bb process itself is using 15M of memory, so there's that, and you have to correctly handle the child shell in order to exit cleanly when really you just want to exec and be done, so the code could be simpler if the parent process didn't need to deal with the child after it was ready to exec. those are the two main concerns, though they are both reasonably minor i guess.

Jeffrey Bay17:04:40

thanks for the issue reference - the "how to do the closest thing" is very helpful. It seems like it could be on option to do something like this in babashka.tasks/shell?

borkdude17:04:33

don't know: shell already throws automatically on non-zero exit codes, so there isn't much to add there

lread17:04:42

I don’t know much about the subject, so… …but a while ago I was dabbling a project that used https://github.com/JetBrains/pty4j which does seem to do some native work to support exec.

borkdude17:04:59

To repeat, exec can be supported in bb, the question is more should we.

Jeffrey Bay17:04:06

there was an ancient (like 15 years ago?) library i thought was called spork that did something similar with JNI but I can't find it now. borkdude's suggestion is a decent alternative for my purpose though.

borkdude17:04:06

It's technically not difficult, since there is a Graal API for it

Jeffrey Bay18:04:28

i think my answer would be yes but i'm happy to move on with my life without it. The java "exec" fork/exec model means that you can't use it to write a truly clean "process starter" - you briefly double the memory footprint of the child process even if the child is going to be super tiny, and you wire the parent process to the child in a way that is unnecessary when that's what you are trying to build, which is, to my mind, a core use case of babashka - it's super fast to start up, so making a reasonably complex process starter is a natural fit for babashka, but this is a minor problem with actually doing so. a) doubles memory footprint before new process is fully realized b) requires handling in babashka code unnecessarily c) leaves a pid/ps entry in the process table which shows up in greps and other tools rather than just dropping out after the kick off is complete

Jeffrey Bay18:04:38

i get why you might not want to do it, but those are the immediate reasons that come to mind.

lread19:04:20

yeah, I only chimed in because I remembered something that might be useful to the discussion (or not! simple_smile)

Gunnar20:04:33

So... a "process starter" would be used to spin off a long-running process, after doing some preparation? Or do you have an example use case to share? I'm envisioning a trivial example: Start firefox with a URL argument, after the babashka script has looked up a bookmark abbreviation in some database/file.

$ ffbm myblog
# which execs firefox <full URL>
If I understood that correctly I agree having an extra parent process stick around for no good reason is minor but still feels a bit icky. It is often annoying IMHO to have too many processes that are just waiting around, when searching for a particular one in output from "ps".

borkdude21:04:52

I reopened the issue: https://github.com/babashka/babashka/issues/858 Please leave your feedback there, if enough people would find this useful, perhaps we could add exec to babashka.process , with the limitation that exec would only work inside native images

borkdude12:05:23

OK, y'all, I implemented babashka.process/exec . If you want to test, please mention your OS, and I provide the link. Example call:

./bb -e '(babashka.process/exec ["ls" "-la" ])'

borkdude12:05:26

Actually, just merged to master

Jeffrey Bay18:05:45

sorry, missed the thread notification - i'm on osx

Jeffrey Bay18:05:53

did it land in 0.8.2? if so i can try it there

Jeffrey Bay18:05:27

nice, upgrading now

Jeffrey Bay12:05:27

hmm - @U04V15CAJ i'm not finding exec symbol in babashka.process on 0.82. not sure why - I'm on osx if that matters...

Jeffrey Bay12:05:10

----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Could not resolve symbol: babashka.process/exec
Phase:    analysis
(ns utils
  (:require babashka.fs
            babashka.process
            
            clojure.string)
  (:import (java.nio.file Path)))

borkdude13:05:10

Hmm, crap, forgot to add the mapping for that var

borkdude13:05:00

I pushed a commit to master now. Which OS are you using? Then I'll give you the link to the new binary in a few minutes

Jeffrey Bay15:05:54

not sure if it is working and i'm not, or if there's a problem. I see (babashka.process/exec ["ls" "-l"]) works as expected, but a more complex command:

(babashka.process/exec ["ssh" "-t" "<user@host>" "..."]) 

Jeffrey Bay15:05:01

just silently hangs

Jeffrey Bay15:05:13

until i ctrl-c

Jeffrey Bay16:05:23

(does exec support the same options that process does, btw? I thought to try {inherit: true} but probably not helpful, and possibly not even supported 🙂

Jeffrey Bay16:05:49

the :dir option will likely be used, though

Jeffrey Bay16:05:58

ah the map is the env then, got it. no options.

borkdude16:05:37

the map supports :env and :extra-env

borkdude16:05:43

those are the only options supported in exec

Jeffrey Bay16:05:24

it seems to work as advertised. there are definitely some gotchas around how the exec works that will have to fiddle around with (and the fact that it's in the process of an exec makes it a bit harder 😆 but this is what I asked for)