The example on the wiki doesn't make sense. The exit code matters. You can submit an improvement to the wiki if you want.

The example will be more verbose in bb for sure

yeah this might be good:

(or (try (shell "foo") (catch Exception _ nil)) (shell "bar"))

already changed in wiki, thanks for reporting

Oh nice, didn't think of using try/catch like that. Thank you!

Neat trick. Did you have to modify bb somehow in advance to support this? How is it possible that the bb binary correctly finds out the mybb.jar, given that it's placed after the actual bb binary content? It'd be awesome if you could just do $ ./mybb.jar 1 2 3, but that wouldn't be easy to do, right? bb binary would probably have to somehow self-inspect itself, realize that there's "an uberjar glued to itself" and behave accordingly (?)

(btw this sorcery probably doesn't work in bb.exe, does it?)

> It'd be awesome if you could just do $ ./mybb.jar 1 2 3, but that wouldn't be easy to do, right? bb binary would probably have to somehow self-inspect itself Indeed, that would be more awesome. I don't know if it works on Windows, my Windows machine currently doesn't power up

This works by virtue of zip files allowing to have a prepended blob which is just ignored

and by virtue of executables allowing to have an appended blob as well, apparently

in such case it might work even in windows :thinking_face:

if Windows executables allow this, yes

If the self-inspection of the binary can be done very cheaply, I'd be willing to look into it

if it does not affect startup time very negatively

is there and end marker of a zip file, for example? then inspecting the very last bytes may be the check, don't know yet

https://games.greggman.com/game/zip-rant/ (interesting article 😄)

4.3.1 A ZIP file MUST contain an "end of central directory record". A ZIP file containing only an "end of central directory record" is considered an empty ZIP file. Files MAY be added or replaced within a ZIP file, or deleted. A ZIP file MUST have only one "end of central directory record". Other records defined in this specification MAY be used as needed to support storage requirements for individual ZIP files.

0x50 0x4b 0x03 0x04   // a local file record
0x50 0x4b 0x01 0x02   // a central directory file record
0x50 0x4b 0x06 0x06   // an end of central directory record

btw just tried bb.exe + foo.jar and it seems to work as well

c:\foo>type src\foo.clj
(ns foo)

(defn -main [& args]
  (prn args (+ 1 2 3)))

c:\foo>bb -cp src uberjar foo.jar -m foo

c:\foo>bb --version
babashka v1.3.185

c:\foo>copy /b bb.exe + foo.jar mybb.jar
bb.exe
foo.jar
        1 file(s) copied.

c:\foo>mybb.jar mybb.jar 1 2 3
("1" "2" "3") 6

Very promising

so the only thing left is some cheap heuristics to see if there is a jar/zip file appended. maybe the first heuristic could be: the executable name is not bb

$ bb -e '(-> (java.lang.ProcessHandle/current) .info .command .get)'
"/Users/borkdude/dev/babashka/bb"

so the only thing left..Yes, this and.. • re-evaluate if this functionality is something you really want to support "forever". 😉 Other than that... I like the first heurestics idea 😄 (ie bb or bb.exe) Btw there's one issue I had on windows.. The scenario described ^^^ didn't work for me at all first. Then I realized I had bb.exe installed via scoop. Ie. it was sitting on my %PATH% somewhere, pointing to C:\Users\brdloush\scoop\shims\bb.exe When I do: copy /b C:\Users\brdloush\scoop\shims\bb.exe + foo.jar mybb.jar then the execution of mybb.jar fails with:

C:\foo>mybb.jar mybb.jar 1 2 3
Cannot open shim file for read.
Could not read shim file.

Btw on windows: ✅

C:\foo>bb -e "(-> (java.lang.ProcessHandle/current) .info .command .get)"
"C:\\foo\\bb.exe"

the bb.exe in scoop is perhaps a symlink or so?

I got this code from Stackoverflow to detect if the file is a zip file:

(ns buber)

(import [ File RandomAccessFile IOException])

(defn is-archive? [f]
  (let [file-signature (atom 0)]
    (try
      (with-open [raf (RandomAccessFile. f "r")]
        (reset! file-signature (.readInt raf)))
      (catch IOException _
        ;; Handle the exception if you like
        (prn :dude)))
    (prn @file-signature)
    (or (= @file-signature 1347093252)
        (= @file-signature 1347093766)
        (= @file-signature 1347094280))))

(-> (File. "./buber") #_(java.lang.ProcessHandle/current) #_#_#_#_.info .command .get (File.)
    is-archive? prn)

that probably only works on zip content only, not bb+zip

ah

ok, this might work then:

(let [is-zip (try (.entries (java.util.zip.ZipFile. (first args)))
                    true
                    (catch Exception _ nil))]
    (prn "Is zip:" is-zip))

no idea what additional stuff the scoop version of bb.exe has inside it, but it's whopping 126MB.

C:\foo>C:\Users\brdloush\scoop\shims\bb.exe --version
babashka v1.3.182

C:\foo>dir C:\foo>C:\Users\brdloush\scoop\shims\bb.exe --version

C:\foo>dir C:\Users\brdloush\scoop\shims\bb.exe
 Volume in drive C has no label.
 Volume Serial Number is 009D-E031

 Directory of C:\Users\brdloush\scoop\shims

05.08.2023  10:50           126 464 bb.exe

28.09.2023  14:14        64 466 944 bb.exe

haha I don't know, perhaps ask in #C01JDDEBRU1

Yup. It's a bit surprising. I mean.. twice the size, that's a bit too much.

as long as (-> (java.lang.ProcessHandle/current) .info .command .get) works reliably, can just use that

> as long as (-> (java.lang.ProcessHandle/current) .info .command .get) works reliably, can just use that Ok, we have mac (you're on mac, right?), windows build from GH releases.. Here's Ubuntu with bb installed via brew

➜  ~ uname --hardware-platform --kernel-name --kernel-version --operating-system && bb -e '(-> (java.lang.ProcessHandle/current) .info .command .get)'          
Linux #33~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Thu Sep  7 10:33:52 UTC 2 x86_64 GNU/Linux
"/home/linuxbrew/.linuxbrew/Cellar/babashka/1.3.184/bin/bb"

so far so good

All together now, this could be the handling of the self-invoking jar:

(ns buber
  (:require [clojure.string :as str]
            [babashka.fs :as fs]))

(defn -main [& args]
  (let [cmd (-> (java.lang.ProcessHandle/current) .info .command .get)]
    (when-not (str/ends-with cmd (if (fs/windows?)
                                   "bb.exe"
                                   "bb"))
      (let [is-zip? (try (with-open [zip-file (java.util.zip.ZipFile. cmd)]
                           (.entries zip-file))
                         true
                         (catch Exception _ nil))]
        (when is-zip?
          ;; invoke uberjar main function of cmd
          ))))
  )

worth a try 🤞

Ah, just (java.util.zip.ZipFile. cmd) is already sufficient for the check

@U01LFP3LA6P Pushed the functionality to master. When current CI build finishes, these binaries should work as self-invoking jars: https://github.com/babashka/babashka/releases/tag/untagged-2718428094ee3428b0e6

The nice thing about this is that you can build self-invoking jar-binaries all on linux, no need to build on OS-specific CI. Just download bb for each OS and append the jar, that's it :)

and the speed of "building" is amazing, right? 🙂

Indeed, sub-second :)

just gave it a try and it works fine on linux

➜  ./bb --version                   
babashka v1.3.186-SNAPSHOT
➜  bb -cp src uberjar foo.jar -m foo
➜  cat bb foo.jar > foo             
➜  chmod +x foo                     
➜  ./foo 1 2 3
("1" "2" "3") 6

I'll try it with windows .exe in the evening. Update: works fine.

Lovely to see this coming together in real time

Tomáši 🙂 in scoop\shims directory, it's only a stub executable. it has size of 126KB ! not MB! true binary is in different directory scoop\apps\babashka\current

if you want to do the bundling with bb.exe you have to avoid the stub, since it only captures execution environment and runs the big binary.

Reason for this is practical, one of the goals of scoop is to avoid poluting of your environment. They went with these shims which are on the single location where all the applications can be put on the single path.

something like: PATH=%PATH%:SCOOP_SHIMS_DIR%

Thanks for bumping this thread @U042D1SQ56U 🙏. I realized that there was one more important information in this thread - that I wasn't able to use the bb.exe installed via scoop succesfully. So I asked @UBLU3FQRZ (who's maintaining https://github.com/littleli/scoop-clojure) whether he has some idea why it's so. Thanks for explaining, @UBLU3FQRZ! Yeah, it's 126KB, not MB 😄, Pitty I haven't noticed that. If I did, I'd probably get the idea it's sort of link/loader of the true bb exe sitting somewhere else. Now it makes sense. @U04V15CAJ should it be mentioned in the documentation part of this new feature that with SCOOP (+ possibly other packagers if they use similar approach as scoop), people should be extra-sure that they're using the "true bb binary" and not just some shim, when gluing the binary + jar?

Btw I didn't know about the concept of shims. Here seems to be some nice summary describing the benefts https://docs.chocolatey.org/en-us/features/shim#benefits It's from chocolatey, not scoop, but I guess the shim idea is the same.

> • The exe can be called from powershell, bash, cmd.exe, or other shells just like you would call the target. This is one of the bigger benefits of deps.clj for Windows as well instead the powershell module

@U01LFP3LA6P The documentation suggests downloading a version of bb from Github releases, I think this is the safest approach since you are not coupled to the bb you have installed, nor the bb for your own operating system

@U01LFP3LA6P no worries man. I'm glad I could help a bit

I bet someone could make a library to make self-contained uberjar binaries for each os. It involves downloading the versions from Github releases and then appending the uberjar, making the file executable, done! Possibly: upload to Github releases (https://github.com/borkdude/gh-release-artifact)

There was a concept of overlays inside executables in old good 90s when I was developing using Borland Pascal. Dunno what happened since then.

Basically payloads appended after the code part of the program.

@UBLU3FQRZ This is exactly what all of the above is about and how self-contained bb executables work

And it feels surprisingly useful for things like babashka (or anything that supports interpreted execution with something like SCI). Using this payload appending approach, you can have CI pipeline step that generates all the executables for combinations of OS/CPU_platform.. and it can be executed anywhere + executes in couple of seconds.

Been waiting for this. @U0479UCF48H told me it covers the relationship between bb and sci.

it does

Just finished a 2X speed. Very good talk. 🙂

Really helpful to see how SCI relates to CLJ/CLJS.

without a repro, can't say

let me try to reproduce into simple scenario because my project is huge.

the different thing is that the source code came from :deps {fernando/abrao {:local/root "lib/testing"}}, is that affect something?

classpaths are cached

is there any way to force not cache?

or remove the cache, where is located?

it's still not clear what your problem exactly is/was though. classpaths are cached so you don't have to shell out to java to calculate the classpath.

it does NOT mean that bb does not see code changes on your classpath, or course

so if you change your local lib, bb will see that

that's the problem what is happening

I make a change into local lib, and it's not seeing the change

it could also be that you don't use unique coordinates for one and the same library and another version of your library is also on the classpath

can you paste the output of bb -e '(babaska.classpath/get-classpath)' ?

bb -e '(babaska.classpath/get-classpath)'
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Could not resolve symbol: babaska.classpath/get-classpath
Data:     {:type :sci/error, :line 1, :column 1, :file "<expr>", :phase "analysis"}
Location: <expr>:1:1
Phase:    analysis

fix the obvious typo

babaska -> babashka

:melting_face: sorry

the code is here /home/fabrao/radiant/core-modeling-api/libs/modeling-common/src

ok, and can you make a top level change in one of the clojure files that gets required in that directory, like (ns foo) (prn :hello)

I did, but it's getting the version before I changed

OH, something is strange , the source code of classpath is came from other source

I suspected as much

/home/fabrao/radiant/reviews/core-modeling-api -> current

it should be lib/testing?

can you do bb clojure -Sdeps-file bb.edn -Stree?

it should be /home/fabrao/radiant/reviews/core-modeling-api/libs/modeling-common/src

and paste the output here

assuming that your deps are in bb.edn

and also paste your bb.edn here

and does that look good or not?

yes

i see schema-registry-v2 on the classpath twice under different coordinates. So if you are seeing stale specs, you’ve got that lib there twice and it’s nondeterministic which one will be found at run time

this is in the right place, but the classpath not

can you please paste your bb.edn as well

why this bb -e '(babaska.classpath/get-classpath)' is showing something different?

where is > the different thing is that the source code came from :deps {fernando/abrao {:local/root "lib/testing"}}, is that affect something?

@U11BV7MTK ignore this, was only a test

sorry, but I can't debug this unless you share the project on github. you can use -Sforce to recalculate the classpath

-Sforce where?

sorry, I can't share the project

usually these problems arise when people use multiple names like fernando/abrao and com.foo.bar/baz for the same library so the library appears multiple times on the classpath and you don't know which one gets loaded

bb -Sforce ...

ok, I'll try it

> why this bb -e '(babaska.classpath/get-classpath)' is showing something different? what exactly is different? and if it’s a classpath, not just “the whole classpath is different” but exactly which items on the classpath are different or not as expected

@U04V15CAJ Worked with -Sforce

thank you !!!

we still haven't found the root cause of the problem though

did you change the deps of the local root referenced lib?

why not?

(the message seems to be gone)

what version of bb are you running

babashka v1.3.182

maybe try the very newest, it uses the latest tools deps, just in case

ok, thank you

Yes: https://github.com/babashka/http-client/blob/main/API.md#babashka.http-client.websocket

To the naïve observer, the list of what bb doesn’t do would be shorter than the list of what it does.