Fork me on GitHub
#babashka
<
2022-07-01
>
mmz07:07:00

@borkdude I really liked the Babashka news https://github.com/babashka/babashka/blob/master/doc/news.md ... still checking periodically to see if there are updates. It was the easiest way (for me at least) to keep up with the babashka news with a very high signal to noise ratio. I'm sure you stopped it for good reasons ... just wanted to let you know it was highly valued (n=1) 馃槄

borkdude07:07:47

You can follow the babashka hashtag on Twitter as well :)

mmz07:07:24

Ah, good one (a bit noisy though). Thanks. https://twitter.com/hashtag/babashka

borkdude07:07:23

If you find anything that should be highlighted in the news than maybe post it here

borkdude09:07:02

@U088NU894 Maybe it's also interesting to read my OSS updates on my blog: https://blog.michielborkent.nl/oss-updates-may-jun-2022.html

1
pieterbreed12:07:23

Does uberscript package pods? Will babashka.pods/load-pod work for an uberscript (or maybe an uberjar?)

borkdude12:07:31

@pieterbreed Uberscript does not package pods. load-pod will work in an uberscript. Also pods declared in bb.edn will work in an uberjar, but not in an uberscript

pieterbreed12:07:09

I'm trying to package for an aws lambda layer. Sounds like I must use an uberjar.

borkdude12:07:56

There is no need, as long as you package the pod in your layer in the right location

borkdude12:07:08

You can also set BABASHKA_PODS_DIR

pieterbreed12:07:32

ok, I'll try this, thank you :thumbsup:

Karol W贸jcik13:07:25

You can look on how Holy Lambda has solved this issue.

馃憤 1
Karol W贸jcik13:07:30

Both deps and pods are cached in a layer.

Karol W贸jcik13:07:29

Also the babashka sources are provided as is to AWS Lambda, which means you can always edit code in a Dashboard.

Karol W贸jcik13:07:47

You are very welcome ;)

pieterbreed16:07:58

@borkdude > You can also set BABASHKA_PODS_DIR I'm not sure if I understood you correctly. I set the env var, but it seems that when I run the following, it looks for pods in a different folder than what I specify: This is the code I'm trying to eval

(require '[babashka.pods])
  (babashka.pods/load-pod 'tzzh/mail "0.0.2")
  (require '[pod.tzzh.mail :as m])
Here are pieces of the environment config:
+ export BABASHKA_PODS_DIR=/opt/.pods-dir
+ BABASHKA_PODS_DIR=/opt/.pods-dir
+ export DEPS_CLJ_TOOLS_DIR=/opt/.deps.clj
+ DEPS_CLJ_TOOLS_DIR=/opt/.deps.clj

+ ls -alpR /opt/.pods-dir
/opt/.pods-dir:
total 0
drwxr-xr-x 3 root root 33 Jul  4 12:31 ./
drwxr-xr-x 5 root root 77 Jul  4 15:38 ../
drwxr-xr-x 3 root root 27 Jul  4 12:31 repository/
/opt/.pods-dir/repository:
total 0
drwxr-xr-x 3 root root 27 Jul  4 12:31 ./
drwxr-xr-x 3 root root 33 Jul  4 12:31 ../
drwxr-xr-x 3 root root 27 Jul  4 12:31 tzzh/
/opt/.pods-dir/repository/tzzh:
total 0
drwxr-xr-x 3 root root 27 Jul  4 12:31 ./
drwxr-xr-x 3 root root 27 Jul  4 12:31 ../
drwxr-xr-x 3 root root 28 Jul  4 12:31 mail/
/opt/.pods-dir/repository/tzzh/mail:
total 0
drwxr-xr-x 3 root root 28 Jul  4 12:31 ./
drwxr-xr-x 3 root root 27 Jul  4 12:31 ../
drwxr-xr-x 2 root root 75 Jul  4 12:31 0.0.2/
/opt/.pods-dir/repository/tzzh/mail/0.0.2:
total 4358
drwxr-xr-x 2 root root      75 Jul  4 12:31 ./
drwxr-xr-x 3 root root      28 Jul  4 12:31 ../
-rwxr-xr-x 1 root root      82 Jul  4 12:31 metadata.cache
-rwxr-xr-x 1 root root 4460544 Jul  4 12:31 pod-tzzh-mai

pieterbreed16:07:43

I get this error when I do the load-pod

Type:     java.io.FileNotFoundException
Message:  /home/sbx_user1051/.babashka/pods/repository/tzzh/mail/0.0.2/manifest.edn (No such file or directory)
Location: /var/task/cognito/auth_challenge_trigger.clj:10:3

borkdude16:07:27

So it installs in the right dir, but doesn't load from the right dir?

pieterbreed16:07:23

no. that /opt/.deps.clj comes from a zip file which I extract there

borkdude16:07:55

I meant, it installs the pod in the right dir?

pieterbreed16:07:13

I'm packaging the pods on one machine as a aws lambda layer and running in lambda

pieterbreed16:07:27

the installation happens when I'm creating the layer

borkdude16:07:37

You have to set that environment variable to the same value during installation but also during loading

borkdude16:07:25

If it's easier you can also use an absolute or relative path to load the pod:

(load-pod "./pods/tzzh-foobar")

pieterbreed16:07:13

I'm not sure how to explain: I'm trying to carry the pods-dir from a building machine, to a runtime environment. so install in one place, zip up the cache, extract to another machine, set the BABASHKA_PODS_DIR var, and invoke, but it fails. Here is the code that is running:

(require '[babashka.pods])
  (println (str "babaska-pods-dir: " (System/getenv "BABASHKA_PODS_DIR")))
  (babashka.pods/load-pod 'tzzh/mail "0.0.2")
This produces this output:
babaska-pods-dir: /opt/.pods-dir
----- Error --------------------------------------------------------------------
Type:     java.io.FileNotFoundException
Message:  /home/sbx_user1051/.babashka/pods/repository/tzzh/mail/0.0.2/manifest.edn (No such file or directory)
Location: /var/task/cognito/auth_challenge_trigger.clj:11:3
----- Context ------------------------------------------------------------------
7: (do
8: 
9:   (require '[babashka.pods])
10:   (println (str "babaska-pods-dir: " (System/getenv "BABASHKA_PODS_DIR")))
11:   (babashka.pods/load-pod 'tzzh/mail "0.0.2")
^--- /home/sbx_user1051/.babashka/pods/repository/tzzh/mail/0.0.2/manifest.edn (No such file or directory)
12:   (require '[pod.tzzh.mail :as m])
13: 
14:   )
15: 
16: (defn handle [{:keys [who]
----- Stack trace --------------------------------------------------------------
                - <built-in>

Karol W贸jcik16:07:51

@pieterbreed why you didn't want to use HL?

pieterbreed16:07:47

feeling a little dumb; what is HL?e

borkdude16:07:48

Which babashka version are you using. BABASHKA_PODS_DIR was introduced in a very recent version

borkdude16:07:24

Let me fix that right now and you can then try a version from master

pieterbreed16:07:52

@borkdude version: 0.8.157 @UJ1339K2B I'm not sure how to answer you. I'm sure HL is great, but I didn't want to use it 馃檪

borkdude16:07:52

@pieterbreed I pushed to the branch manifest-dir-with-env-var - if the build is ready you can look into artifacts and test it

Karol W贸jcik16:07:07

@pieterbreed Any particular reason why? Or just no reason :D btw if you change your mind you can always include latest HL to your bb edn and use it as a custom runtime without all of the mess of its tasks. May I ask what custom runtime are you using?

borkdude16:07:08

afk now for dinner, will check later

pieterbreed16:07:28

Thanks borkdude - I'll test, thank you for helping :thumbsup:

pieterbreed16:07:24

@UJ1339K2B I'm trying to build a solution in such a way that I can support it, because if it goes down, I'll be called. Initially I was confused with HL's support for 3 different runtimes, of which bb was only 1, and the one I'm interested in. So I had some trouble figuring out which parts of the solution was applicable to me and which was not. I'm also tied to a specific pod, and could not for the life of me figure out how to make the pods work (logically) with HL. (Not saying HL could not do it, just I could not figure out what was what from a client perspective) So I https://github.com/dainiusjocas/babashka-lambda-layer, which I could grok, and I've been tinkering at it until it's now starting to work for me. It's easier for me to integrate this solution with the terraform work I'm also doing... so just a whole bunch of teensy tiny little nudges into solving it from first principles.

pieterbreed17:07:16

@borkdude - I just tested version 0.8.158-SNAPSHOT, getting essentially the same error:

----- Error --------------------------------------------------------------------
Type:     java.io.FileNotFoundException
Message:  /home/sbx_user1051/.babashka/pods/repository/tzzh/mail/0.0.2/manifest.edn (No such file or directory)
Location: /var/task/cognito/auth_challenge_trigger.clj:11:3
----- Context ------------------------------------------------------------------
7: (do
8: 
9:   (require '[babashka.pods])
10:   (println (System/getenv "BABASHKA_PODS_DIR"))
11:   (babashka.pods/load-pod 'tzzh/mail "0.0.2")
^--- /home/sbx_user1051/.babashka/pods/repository/tzzh/mail/0.0.2/manifest.edn (No such file or directory)
12:   (require '[pod.tzzh.mail :as m])
13: 
14:   )
15: 
16: (defn handle [{:keys [who]
----- Stack trace --------------------------------------------------------------
                - <built-in>
                     - <built-in>
babashka.pods.impl.resolver/download     - <built-in>
babashka.pods.impl.resolver/pod-manifest - <built-in>
babashka.pods.impl.resolver/resolve      - <built-in>
... (run with --debug to see elided elements)
lambda.core/resolve-handler-fn           - /opt/bin/babashka:640:11
lambda.core/resolve-handler-fn           - /opt/bin/babashka:632:1
lambda.core                              - /opt/bin/babashka:650:20
clojure.core/apply                       - <built-in>
user                                     - /opt/bin/babashka:654:36

Karol W贸jcik17:07:25

Oh got it. I鈥檓 happy to make the docs easier to grok or help you get it working when you fail with the current setup. All the best!

馃憤 1
pieterbreed17:07:07

ok, I retested with the binary you pointed at and it's respecting the envvar now. I am getting a different error now. Now the load-pod is looking for a manifest.edn file which is not there:

----- Error --------------------------------------------------------------------
Type:     java.io.FileNotFoundException
Message:  /opt/.pods-dir/repository/tzzh/mail/0.0.2/manifest.edn (Read-only file system)
Location: /var/task/cognito/auth_challenge_trigger.clj:11:3
----- Context ------------------------------------------------------------------
7: (do
8: 
9:   (require '[babashka.pods])
10:   (println (System/getenv "BABASHKA_PODS_DIR"))
11:   (babashka.pods/load-pod 'tzzh/mail "0.0.2")
^--- /opt/.pods-dir/repository/tzzh/mail/0.0.2/manifest.edn (Read-only file system)
12:   (require '[pod.tzzh.mail :as m])
13: 
14:   )e

borkdude17:07:07

Can you maybe remove the entire /opt/.pods-dir directory and start with a clean slate?

borkdude17:07:22

@U054AT6KT You might also be interested in this conversation since you blogged about pods here: https://jmglov.net/blog/2022-07-04-dogfooding-blambda-1.html

borkdude17:07:59

And to you both I'd like to say: maybe also give https://github.com/grzm/awyeah-api a try instead of the pod

pieterbreed17:07:40

borkdude - I am planning on using awyeah-api... Are you saying I should use it for smtp instead of tzzh/mail ?

borkdude17:07:29

oh, no, this makes sense. but @U054AT6KT was using the aws pod in his blog, while awyeah-api might work as well

pieterbreed17:07:00

borkdude - after using your binary and cleaning .pods-dir, and rebuilding, the pod now loads.

馃憤 1
borkdude17:07:14

I'll merge to master then

jmglov10:07:35

Thanks. I saw awyeah-api, but wasn't sure if the authentication stuff would be a PITA. I'm glad I went the pod route, because that forced me to figure out how pods worked and make sure to support them in Blambda! 馃槈

borkdude10:07:24

Cool. @U054AT6KT So did you see the conversation above? There is BABASHKA_PODS_DIR but there was a bug which has been fixed on master. You can get the master version through the installer script, if you like.

bash <(curl -s ) --version 0.8.158-SNAPSHOT --dir /tmp

jmglov14:07:03

I read the whole thread, so it's all good. I'll update Blambda (which I'm going to stop writing as "Blambda!" because it makes punctuation all weird) with the latest bb version tomorrow. 馃檪

jmglov14:07:33

I need to make sure that pods actually work in my lambdas anyway. 馃槈

borkdude14:07:23

latest bb master version

1
borkdude14:07:27

not release version

dabrazhe09:07:23

@U054AT6KT Do pods work now with Blamda?

jmglov08:07:50

@U96LS78UV They should, yes. Just be warned that Blambda has not yet been used for anything real, so I wouldn鈥檛 recommend it for anything but experiments at this point.

borkdude13:07:18

@wilkerlucio It seems #meander now also works with bb, at least the meander.epsilon-test tests while I hadn't purposely tested that before releasing the new bb.

Ran 155 tests containing 316 assertions.
0 failures, 0 errors.

鉂わ笍 3
馃帀 2
wilkerlucio13:07:55

awesome! 馃槃

borkdude14:07:42

Now ran all tests, all passing:

Ran 205 tests containing 835 assertions.
0 failures, 0 errors.

borkdude16:07:08

Another cool thing I discovered today. Sean Corfields' deps-new project is using tools build, but it's only using the file system API which is already compatible with #babashka. With one very minor change (replacing java.util.Calendar for getting the current year with something else), it ran! This is the script. https://gist.github.com/borkdude/2ee8e49803b40cf0fe284c9921388f87#file-new Put it in your path and then you can call:

new app --name foo/bar --overwrite true

馃帀 6