babashka-sci-dev

borkdude 2022-03-25T14:53:21.718199Z

@cap10morgan Some feedback on the PR: I think it's worth figuring out why resources will be in the uberjar by default, other than "it seems to work". Also I think it's better to use a temporary directory to copy bb.edn into than the resources directory, as there might already be such a directory and you would be overwriting the contents.

borkdude 2022-03-25T14:53:47.794289Z

babashka has its own old version of depstar built in for making uberjars, perhaps we could tweak that to include certain files as well

borkdude 2022-03-25T14:54:50.292199Z

or we could move to the new tools.build uberjar code instead. it's fully compatible with bb except for the basis stuff, which we don't really need

borkdude 2022-03-25T14:55:00.031239Z

the uberjar code is pretty thin

borkdude 2022-03-25T14:55:17.924939Z

but let's move in the smallest step possible and just do with what we have right now

cap10morgan 2022-03-25T14:57:00.004619Z

looks like in depstar it's just b/c its on the classpath: https://github.com/seancorfield/depstar/issues/27

cap10morgan 2022-03-25T14:57:13.621289Z

so we could just put a temp dir on the classpath while we're building the uberjar

cap10morgan 2022-03-25T14:57:15.777539Z

instead

borkdude 2022-03-25T14:57:40.164169Z

@cap10morgan yes, but resources isn't always on bb's classpath, only if you add that

cap10morgan 2022-03-25T14:57:43.287949Z

I tried out the tools.build api a little but got confused by the basis stuff and how it would relate here

borkdude 2022-03-25T14:57:48.748939Z

so I don't understand how it ends up working

cap10morgan 2022-03-25T14:58:20.807319Z

probably just on the classpath where I tried it (e.g. in the :paths vector in my deps.edn files)

cap10morgan 2022-03-25T14:58:28.146299Z

so I'll just do it the way you suggest instead

borkdude 2022-03-25T14:58:30.448589Z

you only need a classpath in the end. if you have that, you don't need the basis stuff which depends on mvn etc

borkdude 2022-03-25T14:58:55.151059Z

bb should not look at deps.edn, unless you tell it to

👍 1
cap10morgan 2022-03-25T14:58:58.254279Z

seemed like the uber task in tools.build wanted a basis, but maybe it isn't required or it doesn't need much of it

borkdude 2022-03-25T14:59:21.136899Z

yes, the uber task wants a basis but once you replace that with a classpath, the rest of the code works.

cap10morgan 2022-03-25T14:59:28.946289Z

ah ok

cap10morgan 2022-03-25T14:59:29.881479Z

cool

cap10morgan 2022-03-25T14:59:37.488869Z

I can try converting it to that if you want

borkdude 2022-03-25T15:00:31.288549Z

let's save that for another time

👍 1
borkdude 2022-03-25T15:00:38.331059Z

separate issue

cap10morgan 2022-03-25T18:00:34.848549Z

@borkdude for the temp dir that we copy the bb.edn into and add to the classpath while building the uberjar, you think it matters if it's inside the project root (vs. just the system's temp dir)?

cap10morgan 2022-03-25T18:00:49.244329Z

it gets deleted once the jar is built

cap10morgan 2022-03-25T18:03:47.257079Z

pushed the system temp dir version just now, but happy to change if you want

borkdude 2022-03-25T19:05:28.356479Z

@cap10morgan I haven't tried, but won't deps from bb.edn be fetched again when you execute the uberjar, while they are already within the uberjar, when you start that uberjar? We could also write something in the uberjar's manifest about the config.

cap10morgan 2022-03-25T19:06:20.191019Z

not sure, I haven't tried that either

borkdude 2022-03-25T19:06:48.525989Z

Well I don't see any special handling not to do this, it seems the bb.edn is picked up as if you're not inside an uberjar

borkdude 2022-03-25T19:06:57.488809Z

which should not happen

cap10morgan 2022-03-25T19:07:20.244779Z

not sure I follow

cap10morgan 2022-03-25T19:07:29.327689Z

like a bb.edn in the same dir as the .jar file?

borkdude 2022-03-25T19:07:38.416079Z

yes?

cap10morgan 2022-03-25T19:07:57.450729Z

yeah I saw that too

cap10morgan 2022-03-25T19:08:21.932969Z

I can make it not do that

borkdude 2022-03-25T19:09:37.484989Z

Also tasks etc do not really apply

borkdude 2022-03-25T19:09:54.224959Z

or do they...?

borkdude 2022-03-25T19:10:03.141349Z

bb uberjar.jar run task1...

borkdude 2022-03-25T19:10:04.662499Z

hmm

borkdude 2022-03-25T19:10:16.805309Z

maybe not

borkdude 2022-03-25T19:10:54.174259Z

I think we should be conservative and only copy the pods config into some uniquely named file and write something in the manifest file that pods should be loaded from there

cap10morgan 2022-03-25T19:12:39.685299Z

I think the latest code in the PR already skips using a bb.edn in the same dir as the .jar file. Because of the cond. It will hit the jar branch and either find a file in the jar or not, but if it doesn't it should just return nil, not skip down to the :else where the ./bb.edn loading happens.

borkdude 2022-03-25T19:13:10.695509Z

yeah that's good but the uberjar invocation will still trigger dependency downloading which should not happen anymore

borkdude 2022-03-25T19:13:22.831049Z

also tasks should probably be elided from the uberjar

borkdude 2022-03-25T19:13:43.057459Z

that's why I think eliding most of the bb.edn in the uberjar would be better

borkdude 2022-03-25T19:13:55.250759Z

e.g. (select-keys bb.edn [:pods])

cap10morgan 2022-03-25T19:14:17.388029Z

seems reasonable

borkdude 2022-03-25T19:14:17.452939Z

until we have a reason to support more of it

borkdude 2022-03-25T19:15:09.000439Z

in the future we could support bundling the pods with the uberjar too but this is out of scope for now ;)

cap10morgan 2022-03-25T19:32:53.747499Z

hmm... it's not putting deps into the uberjar

cap10morgan 2022-03-25T19:35:17.268479Z

oh, perhaps b/c if its only declared in bb.edn's :deps, I need to manually add it to my uberjar classpath? b/c clojure -Spath will only give me the deps.edn ones, huh?

borkdude 2022-03-25T19:35:54.376249Z

I don't follow, why are you talking about clojure -Spath?

cap10morgan 2022-03-25T19:36:25.187689Z

from the babashka book: bb -cp $(*clojure* -A:remove-clojure -Spath) uberjar foo.jar -m foo

cap10morgan 2022-03-25T19:36:41.637549Z

that's how I'm getting my classpath for the uberjar I'm testing with

borkdude 2022-03-25T19:36:59.562979Z

that's an example from before bb.edn

cap10morgan 2022-03-25T19:37:13.722349Z

ah

borkdude 2022-03-25T19:37:29.124269Z

we should update this

cap10morgan 2022-03-25T19:38:22.227779Z

ok, yeah, eliding that worked (even better)

cap10morgan 2022-03-25T19:38:53.003399Z

:pods only is pushed

borkdude 2022-03-25T19:41:14.201389Z

a minor thing but:

->> bb-edn-pods (hash-map :pods)
can be written as {:pods bb-edn-pods}, I find that a little easier to read

cap10morgan 2022-03-25T19:41:47.298119Z

Yep, no problem

cap10morgan 2022-03-25T19:43:24.655659Z

pushed

borkdude 2022-03-25T19:45:14.021969Z

ok, now we should also have tests for this. One thing to note is that local sources are always resolved relative to the bb.edn file, unless you set --deps-root but I don't know how that plays out if you invoke the uberjar from another directory now, e.g.

/tmp/foo $ bb /tmp/bar/uberjar.jar  

borkdude 2022-03-25T19:46:21.701339Z

I suspect that will error out unless we set --deps-root (or unset it)

borkdude 2022-03-25T19:46:34.145279Z

but maybe not, I'm just not sure :)

cap10morgan 2022-03-25T19:47:17.777439Z

my simple tester jar works from a different dir

cap10morgan 2022-03-25T19:47:22.763359Z

w/ a dep

borkdude 2022-03-25T19:47:28.812379Z

ok, that's assuring

cap10morgan 2022-03-25T19:47:48.155419Z

writing up a test now

borkdude 2022-03-25T19:49:04.656599Z

Maybe this part of the code isn't hit anymore: https://github.com/babashka/babashka/blob/72ae6638422c542055dda8fc068f995cbfd9f61f/src/babashka/impl/deps.clj#L63 in the uberjar, since we cut out the deps part, so that makes sense.

cap10morgan 2022-03-25T21:13:05.457819Z

ok just pushed a test. let me know if there are other aspects you'd like tested

borkdude 2022-03-25T21:21:45.891329Z

cool, I'll take a look tomorrow again, thanks!

👍 1