This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-02-28
Channels
- # announcements (11)
- # babashka (66)
- # beginners (19)
- # calva (4)
- # clj-kondo (26)
- # cljsrn (33)
- # clojure (38)
- # clojure-australia (10)
- # clojure-dev (3)
- # clojurescript (18)
- # conjure (8)
- # data-science (2)
- # datahike (1)
- # datascript (5)
- # defnpodcast (3)
- # emacs (3)
- # events (2)
- # fulcro (17)
- # funcool (66)
- # girouette (28)
- # jobs-discuss (1)
- # lsp (33)
- # malli (1)
- # membrane (11)
- # off-topic (2)
- # practicalli (1)
- # re-frame (141)
- # rewrite-clj (6)
- # sci (1)
- # shadow-cljs (68)
- # sql (5)
- # startup-in-a-month (7)
- # vim (1)
- # xtdb (21)
has anyone ever implemented this benchmark for re-frame? https://github.com/krausest/js-framework-benchmark
I'm kinda interested to see how the subscribe
stuff performs in benchmark conditions
not interested in the rendering/react aspects at all but I'm interested to see what the performance of mounting/unmounting queries looks like
@thheller Not the same, but there was this (out of date) analysis: https://www.freecodecamp.org/news/a-real-world-comparison-of-front-end-frameworks-with-benchmarks-e1cb62fd526c/
@jacek.schae updates A RealWorld Comparison of Front-End Frameworks
every year, and may have done other benchmarks, or would like to.
@mikethompson https://medium.com/dailyjs/a-realworld-comparison-of-front-end-frameworks-2020-4e50655fe4c1
mostly because its an easy setup to see some common situations in webapps in benchmark conditions
(eg. no app should have 1000 items rendered in a list but in a benchmark its useful to see)
@huxley thanks
the real world bench is useful for other stuff and probably more useful overall but I'm specifically looking for examples that stress the subscribe
mechanisms
FWIW I did this benchmark for the stuff I'm working on and have two variants. one that would be similar to re-frame
(full) and the other pure reagent
(light). both perform more or less the same overall but full
performs better in updates (eg. select row
) but light
performs better in mount/unmount (create/clear rows). I would expect re-frame
to be somewhat similar in results when compared to reagent
. https://github.com/thheller/js-framework-shadow-grove
At some point, I'll free re-frame subscriptions from using reagent reactions
I'll be moving to using one map for all the subscriptions. The keys in this map will be the subscription vectors And the values in the map will be a map representing the subscription, including a dirty? flag, a set of dependent (subscriptions, identified by their subscription vectors) s, etc
So simple
yeah. as far as I understand one "common" bottleneck for re-frame apps is when one modification results in too many subscription updates
eg. in a map of a thousand items you update one and 1000 subscriptions run to figure out if they are dirty. which is fast as long as its just a (get db thing)
and compares identical?
for 999 items
We've never found it to be much of an issue
Because the identical? test shortcuts early
And because we don't display 1000 things (elements of a vector?)
My proposed rework of subscriptions is more about clean up and doing it a much simpler way
Hey, while you are here ... I know you have been working on a framework ...
... Zach Oaks has convinced me that using a map for app-db
is ultimaitely a bad idea ... flat data is better
maybe the stuff I tried to describe here with transacted
observed
would also benefit the re-frame model (it should) https://github.com/thheller/shadow-experiments/blob/master/doc/what-the-heck-just-happened.md
Indeed
yeah that looks very interesting but seems impossible to scale this to what an webapp would need (adding/removing rules seems rather expensive)
That was the reason I didn't use DataScript originally. We had too much data. And it took too long to import. But ... The problem with using a map is that it is very "placeful"
I didn't realise quite how placeful until I started writing a tutorial on writing reusable components for re-frame
It struck me a little hard.
The difficult part of creating a reusable component in re-frame is that the component's subscriptions and event handlers have to know "where" in pp-db
they need to access
placeful
That means it is hard to take components from one app to another
There are things you can do to mitigate this
You can load data into well known places within app-db
well you can always make things reusable by separating that out and just having a render function that takes the data and renders it
Yes, correct
These are all suggestions I came up with. BUT ... I was quite struck by this placefulness thing It means the subscriptions and event handlers had to get parameterised (by place)
but I agree that flat is better. my normalized db thingy is one big map with one level of nesting.
Anyway ... just a thought
Datascript is flat
but then you have to pay everwhere for turning it into maps so you can actually work with the data again
There were some libraries which stored "entities" in app-db in a very regular way. I can't find them offhand.
They might yet be a middle ground
Anyway, I don't have an answer yet, but it is on my mind.
yeah I do think so. normalizing the data to one level to avoid duplication of entities seems to be enough. making it flatter to EAV tuples means you need to reconstruct maps all the time where you really just want to do (get db id)
(which datascript has a wrapper for to emulate but thats not the same thing)
Ah, one of them was https://github.com/den1k/subgraph
Just some potentially useful musings. I ended up rewriting subgraph to accommodate my needs and I'm still quite happy with the outcome. Although, I use it only for the domain data that comes from an RDBMS in an already normalized form. One thing I have found quite helpful is having multiple indices for every foreign key attribute and being able to query based on that. "Give me all items by a coll of IDs" is useful, but sometimes "give me all items that reference that other item" is even better.
Another feature that's very useful for my use cases is to be able to store user-caused changes separately and query the changed state and the original state separately. Sort of like a transaction. It facilitation creation of complex and reusable components where you can enter a bunch of data, see what exactly you've changed, check validation results, and only then save the data.
Hmm. I'll have to read up again.
Night
Just in case - there are also https://github.com/riverford/compound and https://github.com/threatgrid/asami
btw I'm basically just waiting for someone to ask for custom schemas in autonormal. happy to riff on what a good API for that would look like if anyone is interested
I think I'm struggling with a similar problem atm, where one thing in a collection changes and that causes recalculations in multiple subscription chains
Data is preprocessed first, then indexed in various ways and postprocessed at leafs. We have something that implements incremental indexing at the root, but further change propagation requires calculations from scratch in every subscription. My current idea is to have "streaming subscriptions" that would propagate changes only, similar to the idea behind transducers
The normalized flat db is great, but there is a problem similar to datascript
, all subscriptions are recalculated with every change in the db. It is possible to do something like in posh
, check each transaction and see if it matches the query. I have no idea how efficient this would be though, but I'm going to test and find out š
A small correction - all level 2 subscriptions are recalculated on each app-db change.
I'm currently reading the penpot
code from cover to cover, they have a slightly different approach to state management there.
Penpot is UXbox right? I think posh's change inference wasn't always correct at the time, curious if that's changed
and yes, posh sometimes returned incorrect results, especially when retracting from db
That's my experience with datascript as well
simple flat db in the form of a {?id ?map}
searched with meander
is just as fast for query
and many times faster when it comes to transactions or pull
if it's interesting to y'all, I was inspired by trying to use datascript + re-frame to author this lib: https://github.com/lilactown/autonormal
it normalizes your data into a relatively flat, simple map. and gives an API to easily pull
data out using EQL; though of course you can just get-in
to get specific data out too
it's not the power of datalog but I found that datalog wasn't what I wanted most of the time anyway when relying on datascript to store my app's entities; it wasn't performant enough
i'm hoping to eventually base a "pathom-client" lib on it but in the meantime, I find it moderately useful just for getting out of the "place-oriented" ness that y'all have been talking about
it still relies on parameterizing your subs/events to talk about specific entities tho. which is why i typically only use app-db for domain data and let UI state live in local component state
I recently wrote my thoughts on pretty much the same topic. Also, pretty much everyone here wrote something that influenced how I think about user interfaces! Thanks! (Tony Kay from fulcro and the minds behind hoplon were also big influences). > An entity can use three main techniques to refer to another entity: nesting, identifiers, and stateful references. > - Clojure Applied Chapter 1 The main difference between using datascript and subscriptions isn't flat vs nested, it's the type of reference that is used. Applications that use datascript as a data model tend to use identifiers as references whereas re-frame uses stateful references. What makes a component "reusable" isn't the type of reference that is used, it's whether or not the information needed to produce a reference is passed as an argument to the component or is hard coded in the component. When using identifiers, passing all the necessary information to build a reference is trivial (it's just the identifier). For data models that use nesting or stateful references, it's less straightforward. The tricky part with users interfaces is that there tends to be a lot of incidental state. Given the choice between implicit state handling that's less reusable and explicit state handling that's more reusable, developers tend to prefer implicit state handling. Ideally, incidental state should be handled implicitly and essential state should be handled explicitly which would make it easier to write reusable components by default. Here's the long version: https://blog.phronemophobic.com/reusable-ui-components.html
Using re-frame, always at some point I have problems with data denormalization and map fatigue in general. On the other hand, when e.g. I use datascript, or other ideas for flat normalized db, the problem is performance and synchronization with backend and/or local-storage. Some data should be excluded from storage, what is trivial in case of nested maps, but more complicated in case of flat structure.
so is it more correct to say re-frame uses both nesting and stateful references or just paths/nesting?
I mention this only because I'm not sure what "stateful references" are. I know, I know, I should read your page.
I don't actually explain "stateful references"
Then I'm off the hook :-)
I just kind of ignore them, but I give re-frame as an example of a library that uses stateful references
so I can update it if I'm wrong
See the link provided above
I actually cite that page in my post, https://day8.github.io/re-frame/reusable-components/#implications
I have actually rewritten that page at some time, but never published the rewrite. I'm stuck on the final part of how best to get around placefulness
it's a tough problem.
Tradeoffs all the way down.
I've been using macros to "track" usage of data derived from props to automatically produce references based on paths/nesting, but I'm not sure it's the right solution
I can remember once spending an entire weekend trying to make Datascript faster, by turning it into a column store database and using the GPU. (GPUs are very fast with vectors) I failed. :-)
And when I say I failed, I mean I failed at the very first hurdle: using the GPU.
Back to identity http://day8.github.io/re-frame/reusable-components/#providing-identity
the other approaches I've looked at: ā¢ Om (uses proxies). After using it, it was ok, but had too many caveats for me to want to try again ā¢ macros (my current approach) ā¢ fulcro has ui components define queries as part of the component definition. I think that approach could also work, but I'd rather the queries were automatically written for me.
Sorry, it was the section above the link I supplied
I like Clojure Applied's definition: > An entity can use three main techniques to refer to another entity: nesting, identifiers, and stateful references.
I'm not reading that and feeling wiser :-)
path's within an app-db being the same as "nesting"
It's been tough finding good resources on the subject. Re-frame's docs are one of the best resources I could
It has been interesting comparing different approaches. Re-frame mainly focuses on path/nesting references and fulcro focuses on identifers as references
An identity is something which identifies an entity. And it is very context specific.
In C we used pointers
In a database, it is a unique key
In Clojure it is typically a keyword
Or a path of keywords and integers (a path)
I think it's possible to write UI components in a way such that they're independent of which reference type is required, but it's a challenge to design it in a way that it's not an abstract mess.
The identity depends on the application's data model, not the language (although languages will usually have a lot to say about what kind of data model is idiomatic).
Yeah, that's a distinction without a difference for me. Practically speaking. But maybe I'm missing something.
I have to go but I present this challenge ... What is the identity of a certain collection of customers (the big ones)
So just to be clear, I'm not talking about individual customers
I'm talking about the collection of these customers
is it a subset or the full collection?
Because I might need to distinguish good customers from bad ones
Yeah, so how do I identify the two collections.
Now I want to show the user this collection, now I want to show the other collection
It depends on the implementation.
A single customer can be ID'ed by [:customer 7]
, for example.
A collection of customers by [:customer [1 2 3]]
, for example. Of course, this precludes having a single customer having an ID that's a collection, but I think there can be reasonable limitations.
Another way would be something like [[:customer 1] [:customer 2] [:customer 3]]
, similar limitations apply.
And there are many other ways - again, depends on the implementation. And perhaps the particular set of limitations/idiosyncrasies you're willing to deal with.
;; uses nesting and identifiers
(def data
{:customers {0 {:name "Bob"}
1 {:name "Mary"}
2 {:name "Sue"}}
:lists {0 {:name "good"
:customers [[:customers 0]
[:customers 2]]}
1 {:name "bad"
:customers [[:customers 1]]}}})
;; reference is [:lists 0] or [:lists 1]
;; alternative data model
(def data
{"b4931d86-efca-4e86-b397-e54315673d32" {:name "Bob"}
"133e9085-9481-4a64-befd-ce3b071a4c8a" {:name "Mary"}
"f3c630f5-35e2-4600-8bf6-dc6fdbe76dae" {:name "Sue"}
"b97c51b5-1078-45a9-a325-b8cb71f0959d" {:name "good"
:customers ["b4931d86-efca-4e86-b397-e54315673d32"
"f3c630f5-35e2-4600-8bf6-dc6fdbe76dae"]}
"51bf61bf-5163-4fd9-a52a-060de35ab7f7" {:name "bad"
:customers ["133e9085-9481-4a64-befd-ce3b071a4c8a"]}})
;; references are "b97c51b5-1078-45a9-a325-b8cb71f0959d" and "51bf61bf-5163-4fd9-a52a-060de35ab7f7"
;; stateful references
;; not encouraged!
(let [bob (ref {:name "bob"})
mary (ref {:name "Mary"})
sue (ref {:name "Sue"})]
(def customers [bob mary sue])
(def good-customers (ref [bob sue]))
(def bad-customers (ref [mary])))
;; references are good-customers and bad-customers
@U2FRKM4TW yeah I was imagining that there might be two collections at different points within app-db. I was attempting to draw out that path is identity in re-frame, collections have identity. So too do the customer entities with them.
I see, right. Although "path is identity" and "[sub-]collections have identity" don't fit quite well in my head (by sub-collection I mean a subset of entities within some existing set).
No matter what path-as-identity a sub-collection might have, it will require some special path handling because you won't be able to simply get-in
.
@smith.adriane Some some web apps are a thin veneer over a remote, authoritative database. In that case, database identities tend to dominate. In other cases, the SPA manages quite a complicated data model itself, without much reference to an authoritative remote database, in which case path based identities will be very useful.