Clojurians Log v2

you mean something like a build-time constant that you dynamically fetch at build time but are then part of the static build or should they still be fetched when viewing the build

Yes. For example a SQLite database uploaded to GitHub artifacts

I don't want it to be part of the repository but it's still related to it.

A or B? yes.

🙃

What I mean is I don't want the results which may run plenty of times and produce lots of text to be part of the repository's git history

I want them to be over there and pull them in when I want to display and analyze them

I don't understand what this has to do with build. Isn't the normal workflow in clerk that you can query whatever external thing and make a notebook from it without checking that external thing into git?

This is more a question of making it available via clerk garden. I don't know if downloading artifacts from somewhere is possible and advisable

right, I'll wait for @mkvlr’s reply

not yet supported. we currently store image results as individual files when :package is :directory . It sounds like you’d want to store all files in this way, is that right?

maybe only doing that past a certain size threshold would make sense?

and then adding support for specifying a different url to load it from (default is _data).

It can be a single sqlite file, I don't need to keep all raw results

sqlite to still support pagination you mean?

I don't think it would be necessary, it's not a large amount of data. Lots of output histograms from wrk

can you say why you’d prefer sqlite over one file per result?

would sqlite require a server or can the browser read that directly?

No particular reason, it's smaller. But I can tar gz the bunch of files. It's more that I don't want to keep the results as part of the repository

so individual files would also work as long as you can upload and load them from somewhere else?

Yes.

and the files are large enough that you don’t want to inline them?

so keeping tiny results inline and only moving larger ones to files would work in your case?

I'm not sure, I'll describe what I'm doing - running wrk for many servers and configurations, saving the results with metadata describing the run parameters, then displaying the analysis over these parameters with clerk (reading all the files and doing some aggregation) Are you suggesting I read all the files during the static build process? (Just def the read?)

And how can I tell if this is too much data to def?

can you share a sample notebook? Doesn’t have to be real data ofc but something where I can better understand this would help.

afk for a bit

I'll share it in a bit Unfortunately I experimented with vibe coding it, While the analysis graphs are great, the code is somewhat slopped, so apologies in advance.

Just checking if you've seen I uploaded the archived repository here. If you don't have time to deal with it lmk I'll just experiment with best solution I can figure out

@ben.sless yes I've seen it and looked at it briefly but mostly busy with day job stuff today

Same, I'm in no rush, just checking