This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-02-10
Channels
- # beginners (7)
- # calva (1)
- # cider (8)
- # clojure (21)
- # clojure-austin (5)
- # clojure-dev (26)
- # clojure-europe (7)
- # clojuredesign-podcast (4)
- # clojurescript (2)
- # code-reviews (4)
- # community-development (4)
- # datomic (1)
- # emacs (14)
- # lsp (21)
- # pedestal (8)
- # polylith (16)
- # reitit (6)
- # releases (2)
- # vim (14)
- # xtdb (5)
Great episode (110)! One question that popped into my mind: You talked about logging the clip data from the MAM so that you can view it later when debugging. Clojure has excellent support for metadata. Instead of logging the clip data from the MAM, would it be a good idea to annotate the internal data structure (created by your extractor function) with the untouched clip data from the MAM? Then, you could use (meta clip-info) and compare it to the internal data structure, and you could also use the raw clip data from the MAM in your unit tests if desired. Another option (instead of using metadata) might be to assoc the clip data from the MAM into your internal data structure. Neither of these approaches would be a good idea if the data from the MAM is enormous or you're dealing with many clips (using too much memory).
I'm interested in an answer to this question too. Metadata in Clojure seems really cool, but I don't think I've seen many examples of how to leverage it well. Like for example, say you're doing a reduce and need to carry some extra state information along with the result. I've seen that done by carrying a map through the reduce and then extracting out the piece you need at the end. Could you (should you?) also do that with metadata on a single result value? One of the few things I've seen that goes hard on metadata is Clojure zippers, and it's absolutely fascinating how it works. However, I think (and this may be misunderstanding or hallucination on my part) I've seen somewhere that metadata handling can be fiddly, and not all functions properly copy it over to their output values. That means there's a chance in any data transformation pipeline that your metadata disappears.
@U01PE7630AC Thanks! I'm glad you enjoyed it! I am indeed a fan of keeping the raw response around. I like to create a map that accumulates information in it: both the raw and ingested forms. Sometimes I'll call that a "bag of data". I've experimented with metadata, but the data feels hidden to me, and I have had issues with it getting discarded (like you pointed out). I do think "tagging" (via metadata) an extracted entity with the raw data it came from is a nifty idea. I could see it being useful in that local context when debugging. I would have concerns about it getting lost, or travelling further than I would like (eg. if I forgot to strip the metadata before passing a reference to far away code). Check out https://clojurians.slack.com/archives/CKKPVDX53/p1699637695133179 I had with @U0609QE83SS. It talks about keeping raw responses around in a "bag of data" map.
@U04RG9F8UJZ I think "fiddly" is an appropriate word. In my own experiences, I generally assume the metadata won't survive any significant transformation. I think it can be useful as local side-band data for debugging and such. I would not recommend using metadata as a "first class" part of your data model.