Clojurians Log v2
clojuredesign-podcast
Maybe it's coming in future episodes, but do you use Malli, schema, Truss or similar to validate the context map at the top, and after each function call, or just after critical validation points determined by domain logic?
@nhamm Yes, you can definitely use schemas to validate the context map, and @nate and I have definitely done that in production. With schemas, it's nice that you can identify specific subtrees as "any"; that's what we've done if we cache raw responses. As for coverage in the podcast, we talk a lot more about schemas back in the Twitter series. See https://clojuredesign.club/episode/007-input-overflow/. We used plumatic's schema, but new libraries like Malli can be used in the same way.
Awesome. Thanks for the response, I'll go listen to that one for sure.
Also, earlier in the podcast you walk through parsing text logs. You mentioned logs again in the recent episodes for capturing data about the process as it progresses. Have you guys ever seen/used this: https://github.com/BrunoBonacci/mulog it was suggested in the data science channel. It seems like the way to go to add data to the context map as it winds it's way through execution. Wondering what your thoughts are on it.
Oh yeah, we use mulog at work and it's amazing. Makes it much easier to aggregate logs into elasticsearch or new relic for analysis. I haven't done much with the context accumulation part of mulog as I usually shy away from hidden things in my code. I would much rather pass along context as an argument.
@nhamm I have not personally used mulog, but it looks really nifty! Thanks for the link! (@nate, have you used it?) I could imagine using such a thing in the main think-do-assimilate loop. (More about that loop in your next episode.) That loop will never be pure because of the "do" step. I would avoid doing any sort of I/O in either the "think" or "assimilate" step. If I had the space for it (especially if I was proving out reliability), I could imagine writing down every context, operation, and result. As always, reliability only matters until the cost of remaining failures is so low, they're not worth fixing. The resources worth spending depend on your situation.
Credit to @afoltzm and his comment here: https://clojurians.slack.com/archives/C0BQDEJ8M/p1699024007288729?thread_ts=1699005645.107199&cid=C0BQDEJ8M
I am new to clojure so still figuring things out. But a "single-map application state" where you can get the following for every run would be really cool (image from the bottom of the mulog docs from above)
@nhamm That does look like a neat options for an imperative decomposition kind of approach. I/O is being done at multiple levels of the call stack. That's pretty typical of my first shot at any process, and often good enough for applications I've worked on. It looks like mulog could be a nice next step for profiling and debugging that approach. Very nifty! In general, I would say that some applications are more heavily tied to external data (such as integrations, reporting, etc) and others are more about the domain model of the application itself (games, productivity apps, etc). Sometimes your application doesn't have to do much aside from schlep data around, so there isn't a whole lot to test. Less need to test means less benefit from a pure core. The more an application is just sequencing external things, the more benefit you get from a system that logs what actually happened. I love how Clojure enables such good options for the different kinds of situations.