Fork me on GitHub
#babashka
<
2023-11-19
>
ccfontes09:11:40

I want to prepare deps using two separate cmds: ā€¢ for ship: prepares with :deps only ā€¢ for test: prepares with :deps + :extra-deps What is the idiomatic way of achieving this separation? Thanks. In :tasks, I was thinking of reusing a built-in bb task like prepare, but I don't know how to do this. Use cases Quicker dev feedback loop: we use prepare to fetch deps only when bb.edn file changes (docker img cache busting) docker img size optimization: Separation of ship and test's .m2

borkdude10:11:38

Currently there's no direct way to do this but you could do this as follows:

bb -Sdeps "$otherdeps" prepare 
and then script the otherdeps variable using some bb scripting

šŸ‘ 1
ccfontes09:11:16

It's working for the most part, but have a last call with -Sdeps that is an https://stackoverflow.com/questions/51642221/dockerfile-how-to-set-env-variable-from-file-contents when using ENV This doesn't work:

ENV fprocess="bb -Sdeps `cat ship-deps.edn` run-function"
neither does this:
ENV fprocess="SHIP_DEPS=$(cat ship-deps.edn) bb -Sdeps $SHIP_DEPS run-function"
Can we have an alternative CLI opt to pass deps as a file?

borkdude09:11:22

I think it's better to wrap the $(...) expression in \"...\" to prevent spaces messing things up

borkdude09:11:33

same for \"$SHIP_DEPS\"

borkdude09:11:41

always quote environment variables

borkdude09:11:17

but that might not be your issue here

borkdude09:11:53

you can do it another way: bb --config ship-deps.edn prepare

ccfontes10:11:06

ship-deps.edn is not intended to replace bb.edn . it's actually non standard, only supporting :deps map Isn't -Sdeps merging, while --config replaces? I want merge behavior.

borkdude10:11:31

yes, but for prepare, does it matter?

šŸ˜† 1
ccfontes10:11:52

Sure, and for prepare it's working with -Sdeps The problem is that the task run-function here is not like "prepare". It's running code under :paths classpath:

ENV fprocess="bb -Sdeps `cat ship-deps.edn` run-function"
See task:
{:paths ["."]
 :tasks {run-function index/-main
         test function.test.run-tests/-main}}

borkdude10:11:08

ok, maybe prepare should have an option to prepare all the :extra-deps dependencies in tasks (or do so by default)

borkdude10:11:51

I don't get your last remark though, why would you want to "prepare" a local directory?

ccfontes10:11:04

I did prepare. Wait, are you saying I don't need to pass Sdeps to run the code after preparing?

borkdude10:11:24

I don't think so?

borkdude10:11:44

hmm, -Sdeps gives a different .cache hash than without it

borkdude10:11:17

Perhaps it's best if you prepared an uberjar with everything in it and then execute with bb -cp deps.jar <task>

borkdude10:11:28

this way you're never going to hit dependency stuff

borkdude10:11:52

bb uberjar lets you make this uberjar

bb uberjar deps.jar ...

ccfontes10:11:24

Unless there is any runtime speed improvement in prod, like with standalone native binary (which I couldn't get it working because maybe I'm not smart enough), creating uberjar for dev time would end up being a massive drag while iterating code and check the results. Context: This is an OpenFaaS function template to write Functions in babashka.

borkdude10:11:30

if this is for dev, why are you preparing stuff?

ccfontes10:11:07

I am preparing first. I didn't show that part of the code. That is working well. In the meantime, I'll try without Sdeps . > hmm, -Sdeps gives a different .cache hash than without it I don't understand yet the implications of this yet, but I'll try now and see.

borkdude10:11:10

you could just move all the :extra-deps stuff to :deps , run prepare and call it a day :)

borkdude10:11:00

Well, deps.edn files (or bb.edn files) are hashed and their hash is saved to .cpcache. If the hash matches, java isn't called to calculate a classpath and download dependencies

borkdude10:11:17

and if you prepare the .cpcache file is produced, so next time you won't hit the java machinery

borkdude10:11:30

the same happens if you just run the task once btw

borkdude10:11:51

ok, let's take a few steps back then. > I am preparing first. I didn't show that part of the code. That is working well. Can you explain what is the remaining problem?

ccfontes10:11:52

oh ok, sweet, so should work just fine without Sdeps

ccfontes10:11:32

There is no remaining problem, I think.

borkdude10:11:54

ok, feel free to ping me again if you run into any other issues

borkdude10:11:18

in the docker image, you might need to save the ~/.clojure/.cpcache directory as well, since that is where the hash is saved by default

borkdude10:11:46

but I guess that happens by default if you're still in the same environment

ccfontes10:11:33

Yes, it will be there by default, reusing the build files. Massive thanks. I'll ping if there's a similar blocker.

šŸ‘ 1
ccfontes10:11:32

Sorry, I missed your point here: > if this is for dev, why are you preparing stuff? Preparing first is essential for FaaS cold starts. It's for dev in the sense that trying changes to the Function's code only makes sense by rebuilding the underpinning docker image, but the resulting image will be used in prod as well.

ccfontes10:11:57

Btw, logs show it can't find ns of the ship dep: Could not find namespace: bell.core I'll do some debugging to confirm everything is in place.

borkdude10:11:37

perhaps you can just upload your code somewhere, talking about this stuff is hard when you don't have concrete stuff to look at

borkdude11:11:00

one weird thing I'm seeing here: https://github.com/ccfontes/faas-bb/tree/run-docker-cmds-together-for-smaller-img/examples/http/bb-routes-bell is that bb.edn doesn't mention :paths ["."] to add the current directory to the classpath

borkdude11:11:12

either put sources in src and use :paths ["src"] or add :paths ["."]

ccfontes11:11:26

that bb.edn is non-standard. it's just for being used by with-bb-deps.clj to merge together with test/bb.edn and the result passed to Sdeps in bb prepare . The bb.edn that is used as config file by bb commands is located in template/bb/bb.edn.

ccfontes13:11:14

> you could just move all the :extra-deps stuff to :deps , run prepare and call it a day šŸ™‚ Did this as you suggested and now it works, of course haha šŸ˜† 1. merge function/bb.edn + function/test/bb.edn + template/bb/bb.edn as ship.edn/test.edn 2. then bb --config ship.edn or bb --config test.edn Thanks for your help!

šŸ‘ 1