This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-06-08
Channels
- # announcements (2)
- # babashka (6)
- # beginners (40)
- # calva (23)
- # clojure (30)
- # clojure-europe (7)
- # clojure-gamedev (8)
- # clojure-losangeles (1)
- # clojure-sweden (2)
- # clojurescript (184)
- # datahike (1)
- # dev-tooling (2)
- # hyperfiddle (3)
- # music (1)
- # nbb (1)
- # off-topic (14)
- # pedestal (5)
- # polylith (8)
- # rdf (1)
- # tools-build (14)
- # yamlscript (1)
PSA: data normalization is good for you. Just watched a talk on some implementation details of Linear (nothing too exciting) and this moment took me by surprise: "Now we have to take a leap of faith [...] just normalize all your data structures". Why on Earth would normalizing data be a "leap of faith"? It's like recommending others to wash their fruit before eating and adding "trust me" at the end. Of course, there are situations where you want some of the data to be denormalized to some degree. But I can't fathom how that can be considered the norm by anyone.
With a time stamp (haven't watched the first half with a different speaker): https://www.youtube.com/watch?v=WxK11RsLqp4&t=2193s
With Rama, you denormalize a lot, at least in my experience. The data may be implicitly normalized in the event stream, but you materialize it to support very specific queries. A general, normalized view hasn’t turned out to be that useful.
I'm not familiar with Rama but I know all the other words. :) IME it's always simpler and easier to have endpoints that return normalized data - not for a single item or a collection of items of a particular kind, but rather all the items that are needed for a particular bit of functionality that the client wants. Then the client is free to incorporate all that data in any way it wants. In my case, I also prefer when the data storage on the client side is also normalized, then any kind of updates is as trivial as it gets, and the actual denormalization that can happen on a need-to basis happens as a reaction of that data, becoming in effect a passive view.
Yeah, we used to do this as well. However, with Electric the client/server side distinction has disappeared, which made normalization over the wire superfluous. With Rama, you set up materialized views for specific purposes, and point reactive subscriptions to them (and you are free to add new ones based on the immutable event stream at any point). So far, this seems to make normalization pre-wire superfluous as well.
I guess at this point I’m less inclined to say that normalization always is the correct shape, everywhere.
I'm not sold on Electric at all at this point. Granted, I haven't studied it all that much but just the concept itself makes it look much more like CORBA than I'm comfortable with. Not to mention any potential third-party clients that AFAICT wouldn't be able to easily use the server. > this seems to make normalization pre-wire superfluous as well If nothing is normalized, how does such a system handle a point update of an item that's used by many denormalized views? Like in the case of any m2m relationship, for example.
“NoSQL” databases became the default among an entire generation of developers in the 2010s. They view normalization as a stodgy old fashioned waste of time for unenlightened greybeards. Everything gets dumped as JSON blobs into a big blob store. When they want to look something up, they write a custom one-off query engine. When they want to change the record format, they build ad hoc logic into endpoints to understand multiple possible formats of the same record type. … Yes, all the horrible trainwreck scenarios that you as a seasoned RDBMS programmer are now imagining happen.
This is not to say that there are never scenarios where a schemaless blob storage is appropriate… There are such scenarios. But 99% of the real world cases where a NoSQL database is deployed are not in that set.
By the way, I think it's very interesting. The way I'd approach it normally is I'd make a join query that would return rows of [bookid title price] and then return a similar payload, or maybe create a nested map for user that has a :fav-book key on it with the list of favourite books. Display that, and then each command would just know the right DB update query. So like edit price command would take bookid and new-price and it'll map in the backend to a update to the Price table. Etc.
But I guess in an SPA context, you can make multiple updates to the UI, and so you need like a persistent UI in-memory data-model, so you replicate the same model as the DB, and then synchronize changes to that with the backend DB. I never did SPA, so never had to think about that problem.
That was an interesting thread and glad you kind of resolved it at the end: Denormalized data definitely is a big topic for apps, where the same data is simultaneously presented in different forms (typical example: List vs details view).
Just this week I came across this JS library promising “[a]utomatic normalization and data updates for data fetching libraries” : https://github.com/klis87/normy
There’s also a short entry in the re-frame
FAQ and suggested libs providing denormalized data: https://day8.github.io/re-frame/FAQs/DB_Normalisation/
Have you ever heard about the base 3/2 (or 1.5) number system? Or base -4 (negative 4), or base phi (golden ratio base number system)? Check this Exploding Dots™ Machine by James Tanton (https://globalmathproject.org/wp-content/uploads/2020/09/Written-Guide-EXPERIENCE-9_2020_English.pdf). I also made a small gist with a small Clojure function to start on here (https://gist.github.com/avitkauskas/4e35b2d8fb4b61dad552cac343e566f9). You can have a 16/8 base system (and that happens to be base 2 system by the value of positions but with the hexadecimal numbers allowed in any position), and year 2024 is written as 88888808 in this system. Useful? Not very much yet, perhaps. But fascinating anyway.