Clojurians Log v2
hyperfiddle
I have a simple task at work to have a web ui that can filter and search through a sequence of ~2-5k clojure maps. I need to dynamically specify a time range filter, and to filter if a string appears anywhere in the map. I don’t know electric yet, but is this something I can integrate in our project in a day? Was hoping to try it soon and wondering if this is my chance. (For our use-case, we were previously just dumping a giant text file pretty-printed as an ascii table, and loading that in the browser to get a sticky table header, and it was fast, but now we need dynamic filtering on it.)
https://www.clojure.net/ may be what you want, it has search, the dynamic time range filter may require manual URL construction for the time being
it lets you make internal UIs like this https://datomic-browser.clojure.net/(Inject,'mbrainz-1968-1973')/attributes (note filter top right) without writing frontend code
i think the default filter will do what you want with respect to checking if string is somewhere in map, if not it can be customized
got pulled into another task, but will revisit this, thank you dustin
hmm, the data I’m trying to browse is a lazy sequence of objects from s3, and we can’t retain its head without running out of memory, so I don’t think we can wrap it in hfql
Did you try it? We think this is a bug, so long as the final sequence after filtering fits into memory to support random access virtual scroll
didn’t try it since it has to work without filtering too
if you dont want random access virtual scroll because the collection doesn't fit, then instead you want to just see the top page of results? like a google search? What UX are you going for
we want to be able to scroll through all the results of the sequence on the page
(sorry notifications getting lost)
when a user requests a table of objects, our server streams s3 objects to a zipped html file on disk. user downloads it to get a simple ascii table view with thousands of rows, which seems to be smooth since it’s just plaintext. just needs filtering now.
so random access backed by disk
maybe I can avoid this ceremony and give the s3 stream directly to hyperfiddle if I can wrap it first in a cache interface that keeps only recently visited chunks in memory
if you have it on disk, how about duckdb
the s3 objects themselves aren’t currently kept on disk, only formatted strings based on what the user wants to see in the table
DuckDB can attach to structured files in s3
oh cool
the happy path here is to be able to query out the list of 2-3000 PKs and let that fit in memory easily, and then as you scroll electric will load row by row only what is visible (and dispose records when they occlude). And we want duckdb to be responsible for the filtering