Fork me on GitHub
#babashka-sci-dev
<
2022-03-14
>
cap10morgan18:03:49

@borkdude When you have a moment, I wanted to see if this all sounds reasonable to you for the "read & cache pod namespaces on installation" concept: 1. For each pod declared in bb.edn: a. Download and install it if necessary b. Look for pre-existing namespaces-cache.edn file in cache dir and load it if found c. If no cache file found, run pod, send :describe op, load returned namespaces, write out cache file, & shut down pod d. Make the first usage of any of the pod's vars start it back up and leave it running like it does now on (load-pod ...)

cap10morgan18:03:38

assuming that all sounds reasonable, can you give me a pointer to the load-fn where you thinking it would make sense to wire up the "we're using this pod's namespace, so start it up" logic?

borkdude18:03:18

load-fn is managed in babashka.main

borkdude18:03:37

we could pull that out to a different namespace, but it's there now

cap10morgan19:03:00

ok so perhaps after looking for built-in namespaces, we look in common/ctx in a new :pods-namespaces key or similar for a closure that starts up the pod?

borkdude19:03:38

yes, when the namespace doesn't exist as a normal dependency, we look into the cached namespaces of each of the pods in bb.edn

borkdude19:03:56

until found

borkdude19:03:03

the order should not matter much since namespace should be unique

cap10morgan19:03:33

I think I understand the concepts now, so I'll try coding it up. thanks!

borkdude19:03:46

thank you too!

👍 1
borkdude19:03:38

we could extract the namespaces into a separate file so we have describe.edn and namespaces.edn , the latter can just be a flat list we look into

borkdude19:03:49

for faster lookup than parsing the bigger file

borkdude19:03:55

or we could even store it as some more optimal format, but this is an optimization, I think .edn will be fast enough for this right now

borkdude19:03:13

as most projects have like maybe 1-3 pods

cap10morgan19:03:00

yeah that seems right

borkdude18:03:45

a,b,c = sounds good d = I would say on any of the pod's namespaces, load the pod

👍 1
borkdude18:03:21

so we only really need to cache the namespaces, but we could just cache the entire describe response, you never know what it will be good for right?

cap10morgan18:03:41

yeah makes sense