babashka-sci-dev

cap10morgan 2022-03-14T18:55:49.622669Z

@borkdude When you have a moment, I wanted to see if this all sounds reasonable to you for the "read & cache pod namespaces on installation" concept: 1. For each pod declared in bb.edn: a. Download and install it if necessary b. Look for pre-existing namespaces-cache.edn file in cache dir and load it if found c. If no cache file found, run pod, send :describe op, load returned namespaces, write out cache file, & shut down pod d. Make the first usage of any of the pod's vars start it back up and leave it running like it does now on (load-pod ...)

cap10morgan 2022-03-14T18:56:38.162469Z

assuming that all sounds reasonable, can you give me a pointer to the load-fn where you thinking it would make sense to wire up the "we're using this pod's namespace, so start it up" logic?

borkdude 2022-03-14T18:57:18.389039Z

load-fn is managed in babashka.main

borkdude 2022-03-14T18:57:37.741019Z

we could pull that out to a different namespace, but it's there now

borkdude 2022-03-14T18:58:31.062209Z

yes

cap10morgan 2022-03-14T19:08:00.973829Z

ok so perhaps after looking for built-in namespaces, we look in common/ctx in a new :pods-namespaces key or similar for a closure that starts up the pod?

borkdude 2022-03-14T19:09:38.999549Z

yes, when the namespace doesn't exist as a normal dependency, we look into the cached namespaces of each of the pods in bb.edn

borkdude 2022-03-14T19:09:56.446149Z

until found

borkdude 2022-03-14T19:10:03.446599Z

the order should not matter much since namespace should be unique

cap10morgan 2022-03-14T19:10:19.384709Z

ok

cap10morgan 2022-03-14T19:10:33.940009Z

I think I understand the concepts now, so I'll try coding it up. thanks!

borkdude 2022-03-14T19:10:46.343719Z

thank you too!

👍 1
borkdude 2022-03-14T19:11:38.061839Z

we could extract the namespaces into a separate file so we have describe.edn and namespaces.edn , the latter can just be a flat list we look into

borkdude 2022-03-14T19:11:49.744079Z

for faster lookup than parsing the bigger file

cap10morgan 2022-03-14T19:12:15.256619Z

yep

borkdude 2022-03-14T19:12:55.070279Z

or we could even store it as some more optimal format, but this is an optimization, I think .edn will be fast enough for this right now

borkdude 2022-03-14T19:13:13.644999Z

as most projects have like maybe 1-3 pods

cap10morgan 2022-03-14T19:19:00.604119Z

yeah that seems right

borkdude 2022-03-14T18:56:45.016529Z

a,b,c = sounds good d = I would say on any of the pod's namespaces, load the pod

👍 1
borkdude 2022-03-14T18:58:21.452989Z

so we only really need to cache the namespaces, but we could just cache the entire describe response, you never know what it will be good for right?

cap10morgan 2022-03-14T18:58:41.172859Z

yeah makes sense