Let's make the smallest change possible first which I guess is no 1 and let's not try to optimize too much right away. I think the overhead of starting a pod at install time should only be done if the namespace files aren't there yet. Besides that there isn't that much to win when the pods are already installed right? Maybe I don't understand exactly what the benefit of 2 is.

Yeah I’m only starting it for :describe once on install either way. And yeah, I don't think it's a big perf hit either way after that. I kind of stumbled onto no. 2 and was curious if you'd think it was better. Hard to say which is the smaller change right now, but I’ll proceed with no. 1 and see how it evolves.

It seems with 2 we are introducing different code-paths when the namespaces files are already there. Then we do not have to start any pods.

@borkdude Is it safe to assume that all byte arrays in the describe response are strings?

Or should I just serialize them as e.g. base64 (in the cache files)?

the describe response has a documented schema and you should assume that schema

OK looks like they're all strings 👍:skin-tone-2:

yeah, but could theoretically change in the future

just go by the docs

sure, definitely. the question is: do I cache the entire response or selectively pull out the currently-documented keys? and also: when I deserialize the cached data, is it important to put things that were bytes back to bytes or should I just deep-convert everything to strings and deal with both cached and uncached data the same (i.e. it's all strings)?

Maybe we could cache the entire bencode response as is

yeah, perhaps. loses easy human inspectability, but that's maybe a non-goal anyway

@cap10morgan I guess we could just not cache the describe response at all

if we need more info for some other feature, we could just load the pod once, extract the info and write it to a separate file

like the namespaces files

Trying out the raw bencode cache approach now. If it works it seems fairly elegant so far. But that's a big “if” :)

@borkdude when we run the pod at installation just for the :describe, do you think it's important to run any :shutdown op the pod has when we then shut it down again?

I'm assuming "yes"

yeah

it allows to gracefully shutdown, you never know what it's good for.

yeah, what I figured. just requires that I do a little more decoding of the metadata at that point in the code than I had been. no big deal though, already done. 🙂

Is there a standard file name extension for bencode data?

I guess maybe .torrent but that doesn't seem appropriate here 😆

I'll just use metadata.cache for now

haha

maybe .bnc

or just describe.bnc

hey it works!

more commits to the PR coming shortly

🎉 🎉 🎉

uncached run: 490ms, cached run: 355ms (both with pods already installed). but that's with only one pod being required.

it should be just as fast as with manual load pod right?

yeah in theory. guess I should kick off the shutdown in a separate thread. no need to wait on that.

ok that's pushed and a new PR opened in babashka/pods

@cap10morgan Will check tomorrow