Clojurians Log v2
architecture
I've got a system design question for you all today. Specifically I'm curious to ask how you all handle reports generated from Datomic data. I'm just about to embark on a reporting system for the data in our DB. The reports are going to be generated as HTML artifacts. I'm asking myself the following questions about how to go about this: 1. Should I generate the reports as straight static HTML files and save them to disk? I think I like the simplicity of this approach, but ... 2. What if I want to generate the same report twice? Or better said, what if I want to load the UI parameter settings for the report back into the web UI so a user can "get back" to the UI state that generated the report? This would enable them to debug it, or edit the parameters of it, or whatever. Where would I save the query that generated the report? In what form? Let's say I've got this query to find the number of animals in an animal shelter:
'[:find (count ?e)
:in $ ?type
:where [?e :animal/type ?type]]
sorry, answering a week later but only just read this. I think the answer to this question is "it depends", as always :) some aspects: 1. How big are queries supposed to be? If they're potentially going to be quite large, then this is more data that each peer will have to reflect. My general gut feeling is that reporting is a fundamentally different interaction with a database, and so storing the data in the same way as the rest of the app may or may not be a good idea. If queries can be quite large, I think it's a good idea to keep them out of the app (in files, S3, whatever) 2. What sort of app is the reporting app? In general I would err towards keeping more event history than less, for all the reasons everyone in the Clojure world loves, but then it seems to me weird that you would have it for reports, and not the rest of the user interactions too? Perhaps this would be something that would be better stored on an event log 3. If you choose to log URL parameters, and the parameters are as complex as a query can be, then you have defacto chosen to store the queries anyway 4. Is the expected use case for the user to be able to see their old queries to save time? In which case, perhaps some form of local storage might suffice 5. If the expected use case is to be able to go from the report to the original query, then IMO having a stateless report system and having the query in the report itself somehow (e.g. through the URL params as suggested) is the way forward 6. I don't think it's a good idea to track API versions, because then you might be inclined to change what the API would return for the same parameters 😉