babashka-sci-dev

cap10morgan 2022-03-23T16:04:53.313609Z

@borkdude I unfollowed the pods repo in my CircleCI account so you can follow it from yours

borkdude 2022-03-23T16:05:33.323769Z

yeah, it's activated with the bb org now

👍 1
borkdude 2022-03-23T16:05:41.786749Z

thanks a lot! I didn't realize we had no CI for that

cap10morgan 2022-03-23T16:06:00.074369Z

sure thing!

borkdude 2022-03-23T16:09:24.968729Z

I've merged the PR now, additional fixes/features can happen in future PRs

👍 1
borkdude 2022-03-23T16:09:33.828649Z

thanks a lot!

cap10morgan 2022-03-23T16:09:45.540419Z

no problem, it was fun 🙂

borkdude 2022-03-23T16:09:59.186899Z

Do you have access to an M1 machine?

cap10morgan 2022-03-23T16:10:08.137229Z

I do

borkdude 2022-03-23T16:10:48.204809Z

I think it's worth testing if intel pods work with the M1 version of babashka. I'll try it myself too, but if they work, then I think adding a fallback mechanism for macos/aarch64 to macos/amd64 might make sense.

cap10morgan 2022-03-23T16:12:18.070349Z

yep

cap10morgan 2022-03-23T16:12:38.782749Z

I'll build it and give it a shot

borkdude 2022-03-23T16:13:16.092289Z

that would save us a lot of trouble with re-compiling all those pods... while having no M1 CI available, that gets tedious

cap10morgan 2022-03-23T16:13:34.899509Z

yeah

borkdude 2022-03-23T16:26:40.956849Z

@cap10morgan This seems to work with the fallback in place:

{:pods {org.babashka/go-sqlite3 {:version "0.1.0"}}
 :tasks {:requires ([pod.babashka.go-sqlite3 :as sqlite])
         create-db {:task (do (sqlite/execute! "/tmp/foo.db"
                                               ["create table if not exists foo (the_text TEXT, the_int INTEGER, the_real REAL, the_blob BLOB)"]))}}}

borkdude 2022-03-23T16:26:55.803989Z

I'll make a PR and let you review it ;)

cap10morgan 2022-03-23T16:27:00.756049Z

great! 🙂

borkdude 2022-03-23T16:28:59.284039Z

https://github.com/babashka/pods/pull/46

👀 1
lispyclouds 2022-03-23T16:29:01.721259Z

Just chiming in for the Go pods, we can produce aarch64 mac binaries without the arm CI 😄

borkdude 2022-03-23T16:29:19.420969Z

That's great :) Golang is such a neat language to produce pods!

borkdude 2022-03-23T16:29:36.365589Z

But it already seems to work with Rosetta2 for now :)

borkdude 2022-03-23T16:32:05.272489Z

I wonder what other cool golang libraries we could expose as pods

lispyclouds 2022-03-23T16:37:22.464619Z

we have https://github.com/avelino/awesome-go to cherry pick 😄

borkdude 2022-03-23T16:41:05.204419Z

@cap10morgan While I said this in #babashka : > Also, when you declare the pods in your bb.edn, you no longer have to use load-pod in your code, the pod is automatically loaded whenever a related namespace is required. I wondered, do we check if a pod has already been loaded? E.g. if you require multiple namespaces related to the same pod, is the pod started only once?

cap10morgan 2022-03-23T16:41:51.981739Z

I believe so b/c it shouldn't even hit the load-fn at that point. Instead it should see that it already has that namespace loaded.

cap10morgan 2022-03-23T16:42:09.631549Z

But let me know if you observe behavior to the contrary! 🙂

borkdude 2022-03-23T16:43:04.578539Z

Ah, because when you load one namespace the other ones are automatically defined and then you no longer hit load-fn, right?

cap10morgan 2022-03-23T16:43:26.428069Z

yes. and I am 99% sure I tried it and saw it behave that way too 😉

borkdude 2022-03-23T16:45:00.260609Z

I think that makes sense

borkdude 2022-03-23T16:45:25.537939Z

this is very cool

😁 1
borkdude 2022-03-23T16:50:56.040179Z

It always feels good when you can close an issue if you've waited long enough for a better solution: https://github.com/babashka/babashka/issues/1065

💯 1
cap10morgan 2022-03-23T16:53:03.115409Z

Guess we can close https://github.com/babashka/babashka/issues/863 too huh?

borkdude 2022-03-23T16:54:01.518809Z

Done. Should we still go for :pod/version? But what about :path... :pod/path? Blegh

cap10morgan 2022-03-23T16:54:38.566209Z

I think they're OK as-is. Next up is probably uber(jar|script) support?

borkdude 2022-03-23T16:55:03.649879Z

Yeah

cap10morgan 2022-03-23T16:55:27.184109Z

I played around with pulling the bb.edn into the jar a little, but it needed a bit more work. I think mostly just accessing it differently (i.e. as a stream) once in the jar.

cap10morgan 2022-03-23T16:55:37.389099Z

happy to dive back into that if you'd like

borkdude 2022-03-23T16:56:00.125339Z

yeah, sounds good. I could take a look at the uberscript, if you don't beat me to it. I have some other stuff to do first

cap10morgan 2022-03-23T16:56:17.713929Z

cool. I'll start with uberjar then

borkdude 2022-03-23T16:57:12.323779Z

for the uberscript, we already hacked the load-fn. and we could "emit" a load-pod into the uberscript similar to when we load the pod in bb now

borkdude 2022-03-23T16:57:51.966289Z

perhaps that approach would also work for the uberjar btw...

🤔 1
borkdude 2022-03-23T17:00:32.561999Z

what happens if you emit some code like

pod/babashka/go_sqlite3.clj
into the uberjar with the contents:
(load-pod 'org.babashka/go-sqlite3 "0.1.0")
Perhaps when loading that namespace that namespace will be replaced by the pod namespace

cap10morgan 2022-03-23T17:01:26.586569Z

yeah interesting

borkdude 2022-03-23T17:03:36.594819Z

That works :)

borkdude 2022-03-23T17:03:39.913339Z

(ns script
  (:require [pod.babashka.go-sqlite3 :as sqlite]))

(sqlite/execute! "/tmp/foo.db"
                 ["create table if not exists foo (the_text TEXT, the_int INTEGER, the_real REAL, the_blob BLOB)"])

borkdude 2022-03-23T17:03:46.245709Z

(ns pod.babashka.go-sqlite3
  (:require [babashka.pods]))

(babashka.pods/load-pod 'org.babashka/go-sqlite3 "0.1.0")

borkdude 2022-03-23T17:04:01.769879Z

So maybe that approach would work for both the uberscript and uberjar

cap10morgan 2022-03-23T17:09:11.531099Z

I can try that out first and see if it works for both

borkdude 2022-03-23T17:23:11.402979Z

@cap10morgan Another approach could be that we would offer a programmatic interface to declare pods

borkdude 2022-03-23T17:23:30.331349Z

So we copy the bb.edn part to a call to that at the beginning of the uberscript

borkdude 2022-03-23T17:23:40.482849Z

That seems a lot simpler

borkdude 2022-03-23T17:24:31.730489Z

(babashka.pods/add-pods '{:pods {...}})

borkdude 2022-03-23T17:24:34.338199Z

or so

cap10morgan 2022-03-23T17:26:04.256529Z

extract this to a separate fn, roughly? https://github.com/babashka/babashka/blob/e6022867996540f0afa2844caa57a6e9b3c903e1/src/babashka/main.clj#L794-L796

cap10morgan 2022-03-23T17:26:13.829839Z

yeah, that could be pretty nice

borkdude 2022-03-23T17:26:27.063989Z

yeah roughly

cap10morgan 2022-03-23T17:26:39.493559Z

except take the thing from @common/bb-edn as an arg

cap10morgan 2022-03-23T17:27:50.186399Z

that sounds perfect for uberscripts, but I wonder if having the bb.edn in the uberjar would pay dividends in other areas?

borkdude 2022-03-23T17:28:39.765449Z

potentially yes

cap10morgan 2022-03-23T17:29:37.630119Z

might be simpler too (vs. generating some code that has to wrap the main fn that calls this new add-pods)

borkdude 2022-03-23T17:30:20.271209Z

yes

borkdude 2022-03-23T17:37:08.841709Z

Maybe, for standalone scripts, we should support some way to inline the entire bb.edn in a script

borkdude 2022-03-23T17:37:27.864959Z

(babashka.config/set-config! '{:pods ... :deps ...})
or so

borkdude 2022-03-23T17:37:48.318129Z

(similar idea, thinking out loud)

cap10morgan 2022-03-23T17:38:55.023849Z

yep, that would be nicely consistent with the potential uberjar approach

borkdude 2022-03-23T17:42:35.963509Z

yeah, btw, in uberscripts and uberjars the :deps component should not be processed anymore since all deps should already be there

👍 1
cap10morgan 2022-03-23T20:07:19.774369Z

got it working in uberjars (w/ the "store the bb.edn in the jar" approach)

cap10morgan 2022-03-23T20:07:23.208219Z

PR coming shortly

cap10morgan 2022-03-23T20:12:24.557619Z

might make sense to (provide an option to) download every os/arch pod variant into the uberjar too

cap10morgan 2022-03-23T20:12:28.043869Z

megajar

cap10morgan 2022-03-23T20:14:07.393979Z

another crazy idea: if you have qemu in your path, we fall back to running the linux/x86_64 pod w/ it as a last resort 😈 (hello BSD, etc. support)

cap10morgan 2022-03-23T20:20:11.281969Z

assuming you can connect stdio or a socket across that emulation layer...

cap10morgan 2022-03-23T20:21:15.484489Z

I'll stop there for now, but let me know if you'd like me to dive into uberscripts too, @borkdude

borkdude 2022-03-23T21:19:19.109389Z

Thanks! Yeah, embedding the pods into the uberjar is something I've considered as well... dunno, let's consider that later ;)

👍 1