Fork me on GitHub

has anyone ever implemented this benchmark for re-frame?


I'm kinda interested to see how the subscribe stuff performs in benchmark conditions


not interested in the rendering/react aspects at all but I'm interested to see what the performance of mounting/unmounting queries looks like


@jacek.schae updates A RealWorld Comparison of Front-End Frameworks every year, and may have done other benchmarks, or would like to.


yeah I saw that one but I was looking specifically for this benchmark


mostly because its an easy setup to see some common situations in webapps in benchmark conditions


same thing only newer


(eg. no app should have 1000 items rendered in a list but in a benchmark its useful to see)


the real world bench is useful for other stuff and probably more useful overall but I'm specifically looking for examples that stress the subscribe mechanisms


I can set write it myself, just wondering if someone else did before me


shouldn't the results be essentially identical to the reagent?


the rendering yes but thats not what I'm interested in


FWIW I did this benchmark for the stuff I'm working on and have two variants. one that would be similar to re-frame (full) and the other pure reagent (light). both perform more or less the same overall but full performs better in updates (eg. select row) but light performs better in mount/unmount (create/clear rows). I would expect re-frame to be somewhat similar in results when compared to reagent.


At some point, I'll free re-frame subscriptions from using reagent reactions


that would be neato. then I could just plug it into my stuff šŸ™‚


I'll be moving to using one map for all the subscriptions. The keys in this map will be the subscription vectors And the values in the map will be a map representing the subscription, including a dirty? flag, a set of dependent (subscriptions, identified by their subscription vectors) s, etc


yeah. as far as I understand one "common" bottleneck for re-frame apps is when one modification results in too many subscription updates


eg. in a map of a thousand items you update one and 1000 subscriptions run to figure out if they are dirty. which is fast as long as its just a (get db thing) and compares identical? for 999 items


but if you actually compute something that becomes a bottleneck


We've never found it to be much of an issue


Because the identical? test shortcuts early


And because we don't display 1000 things (elements of a vector?)


My proposed rework of subscriptions is more about clean up and doing it a much simpler way


Hey, while you are here ... I know you have been working on a framework ...


... Zach Oaks has convinced me that using a map for app-db is ultimaitely a bad idea ... flat data is better


maybe the stuff I tried to describe here with transacted observed would also benefit the re-frame model (it should)


I assume you are referring to his o'doyle rules thing?


yeah that looks very interesting but seems impossible to scale this to what an webapp would need (adding/removing rules seems rather expensive)


That was the reason I didn't use DataScript originally. We had too much data. And it took too long to import. But ... The problem with using a map is that it is very "placeful"


I didn't realise quite how placeful until I started writing a tutorial on writing reusable components for re-frame


It struck me a little hard. The difficult part of creating a reusable component in re-frame is that the component's subscriptions and event handlers have to know "where" in pp-db they need to access


That means it is hard to take components from one app to another


There are things you can do to mitigate this


You can load data into well known places within app-db


well you can always make things reusable by separating that out and just having a render function that takes the data and renders it


and the other one that gets it from wherever and calls it


These are all suggestions I came up with. BUT ... I was quite struck by this placefulness thing It means the subscriptions and event handlers had to get parameterised (by place)


but I agree that flat is better. my normalized db thingy is one big map with one level of nesting.


yeah but how else would you solve that?


Anyway ... just a thought


Datascript is flat


but then you have to pay everwhere for turning it into maps so you can actually work with the data again


(update thing :foo inc) becomes rather difficult too


There were some libraries which stored "entities" in app-db in a very regular way. I can't find them offhand.


They might yet be a middle ground


Anyway, I don't have an answer yet, but it is on my mind.


yeah I do think so. normalizing the data to one level to avoid duplication of entities seems to be enough. making it flatter to EAV tuples means you need to reconstruct maps all the time where you really just want to do (get db id) (which datascript has a wrapper for to emulate but thats not the same thing)


Just some potentially useful musings. I ended up rewriting subgraph to accommodate my needs and I'm still quite happy with the outcome. Although, I use it only for the domain data that comes from an RDBMS in an already normalized form. One thing I have found quite helpful is having multiple indices for every foreign key attribute and being able to query based on that. "Give me all items by a coll of IDs" is useful, but sometimes "give me all items that reference that other item" is even better.


Another feature that's very useful for my use cases is to be able to store user-caused changes separately and query the changed state and the original state separately. Sort of like a transaction. It facilitation creation of complex and reusable components where you can enter a bunch of data, see what exactly you've changed, check validation results, and only then save the data.


Hmm. I'll have to read up again.


ah that looks good


currently it looks like everyone is trying to solve the same problem in a similar way


oh thanks. didn't see that one yet.


btw I'm basically just waiting for someone to ask for custom schemas in autonormal. happy to riff on what a good API for that would look like if anyone is interested


I am currently trying to solve the same problem myself as a hobby ; )


in production we use rum + patched datascript, but it is very slow and tedious


I think I'm struggling with a similar problem atm, where one thing in a collection changes and that causes recalculations in multiple subscription chains


Data is preprocessed first, then indexed in various ways and postprocessed at leafs. We have something that implements incremental indexing at the root, but further change propagation requires calculations from scratch in every subscription. My current idea is to have "streaming subscriptions" that would propagate changes only, similar to the idea behind transducers


The normalized flat db is great, but there is a problem similar to datascript, all subscriptions are recalculated with every change in the db. It is possible to do something like in posh, check each transaction and see if it matches the query. I have no idea how efficient this would be though, but I'm going to test and find out šŸ˜‰


A small correction - all level 2 subscriptions are recalculated on each app-db change.


I'm currently reading the penpot code from cover to cover, they have a slightly different approach to state management there.


exactly based on streaming


Penpot is UXbox right? I think posh's change inference wasn't always correct at the time, curious if that's changed


yes, uxbox is now a penpot


and yes, posh sometimes returned incorrect results, especially when retracting from db


by the way, datascript is not fast at any time, in fact performance is poor


with and without posh


That's my experience with datascript as well


simple flat db in the form of a {?id ?map} searched with meander is just as fast for query and many times faster when it comes to transactions or pull


if it's interesting to y'all, I was inspired by trying to use datascript + re-frame to author this lib:


it normalizes your data into a relatively flat, simple map. and gives an API to easily pull data out using EQL; though of course you can just get-in to get specific data out too


it's not the power of datalog but I found that datalog wasn't what I wanted most of the time anyway when relying on datascript to store my app's entities; it wasn't performant enough


somewhere here I posted autonormal as an example šŸ˜‰


I see that now šŸ˜„ skimmed too fast


i'm hoping to eventually base a "pathom-client" lib on it but in the meantime, I find it moderately useful just for getting out of the "place-oriented" ness that y'all have been talking about


it still relies on parameterizing your subs/events to talk about specific entities tho. which is why i typically only use app-db for domain data and let UI state live in local component state


I recently wrote my thoughts on pretty much the same topic. Also, pretty much everyone here wrote something that influenced how I think about user interfaces! Thanks! (Tony Kay from fulcro and the minds behind hoplon were also big influences). > An entity can use three main techniques to refer to another entity: nesting, identifiers, and stateful references. > - Clojure Applied Chapter 1 The main difference between using datascript and subscriptions isn't flat vs nested, it's the type of reference that is used. Applications that use datascript as a data model tend to use identifiers as references whereas re-frame uses stateful references. What makes a component "reusable" isn't the type of reference that is used, it's whether or not the information needed to produce a reference is passed as an argument to the component or is hard coded in the component. When using identifiers, passing all the necessary information to build a reference is trivial (it's just the identifier). For data models that use nesting or stateful references, it's less straightforward. The tricky part with users interfaces is that there tends to be a lot of incidental state. Given the choice between implicit state handling that's less reusable and explicit state handling that's more reusable, developers tend to prefer implicit state handling. Ideally, incidental state should be handled implicitly and essential state should be handled explicitly which would make it easier to write reusable components by default. Here's the long version:

šŸ˜€ 3

Creating components is one thing


Using re-frame, always at some point I have problems with data denormalization and map fatigue in general. On the other hand, when e.g. I use datascript, or other ideas for flat normalized db, the problem is performance and synchronization with backend and/or local-storage. Some data should be excluded from storage, what is trivial in case of nested maps, but more complicated in case of flat structure.


every solution has its weaknesses


@smith.adriane paths are identities in re-frame (paths within app-db)

šŸ‘ 3

so is it more correct to say re-frame uses both nesting and stateful references or just paths/nesting?


I mention this only because I'm not sure what "stateful references" are. I know, I know, I should read your page.


I don't actually explain "stateful references"


Then I'm off the hook :-)


I just kind of ignore them, but I give re-frame as an example of a library that uses stateful references


so I can update it if I'm wrong


See the link provided above


I have actually rewritten that page at some time, but never published the rewrite. I'm stuck on the final part of how best to get around placefulness


it's a tough problem.


Tradeoffs all the way down.


I've been using macros to "track" usage of data derived from props to automatically produce references based on paths/nesting, but I'm not sure it's the right solution


I can remember once spending an entire weekend trying to make Datascript faster, by turning it into a column store database and using the GPU. (GPUs are very fast with vectors) I failed. :-)


And when I say I failed, I mean I failed at the very first hurdle: using the GPU.


the other approaches I've looked at: ā€¢ Om (uses proxies). After using it, it was ok, but had too many caveats for me to want to try again ā€¢ macros (my current approach) ā€¢ fulcro has ui components define queries as part of the component definition. I think that approach could also work, but I'd rather the queries were automatically written for me.


Sorry, it was the section above the link I supplied


I like Clojure Applied's definition: > An entity can use three main techniques to refer to another entity: nesting, identifiers, and stateful references.


I'm not reading that and feeling wiser :-)


path's within an app-db being the same as "nesting"


It's been tough finding good resources on the subject. Re-frame's docs are one of the best resources I could


It has been interesting comparing different approaches. Re-frame mainly focuses on path/nesting references and fulcro focuses on identifers as references


An identity is something which identifies an entity. And it is very context specific.

šŸ‘ 3

In C we used pointers


In a database, it is a unique key


In Clojure it is typically a keyword


Or a path of keywords and integers (a path)


I think it's possible to write UI components in a way such that they're independent of which reference type is required, but it's a challenge to design it in a way that it's not an abstract mess.


The identity depends on the application's data model, not the language (although languages will usually have a lot to say about what kind of data model is idiomatic).


Yeah, that's a distinction without a difference for me. Practically speaking. But maybe I'm missing something.


I have to go but I present this challenge ... What is the identity of a certain collection of customers (the big ones)


So just to be clear, I'm not talking about individual customers


I'm talking about the collection of these customers


is it a subset or the full collection?


Because I might need to distinguish good customers from bad ones


Yeah, so how do I identify the two collections.


Now I want to show the user this collection, now I want to show the other collection


It depends on the implementation. A single customer can be ID'ed by [:customer 7], for example. A collection of customers by [:customer [1 2 3]], for example. Of course, this precludes having a single customer having an ID that's a collection, but I think there can be reasonable limitations. Another way would be something like [[:customer 1] [:customer 2] [:customer 3]], similar limitations apply. And there are many other ways - again, depends on the implementation. And perhaps the particular set of limitations/idiosyncrasies you're willing to deal with.

šŸ‘ 3
āž• 3

;; uses nesting and identifiers
(def data
  {:customers {0 {:name "Bob"}
               1 {:name "Mary"}
               2 {:name "Sue"}}
   :lists {0 {:name "good"
              :customers [[:customers 0]
                          [:customers 2]]}
           1 {:name "bad"
              :customers [[:customers 1]]}}})

;; reference is [:lists 0] or [:lists 1]

;; alternative data model
(def data
  {"b4931d86-efca-4e86-b397-e54315673d32" {:name "Bob"}
   "133e9085-9481-4a64-befd-ce3b071a4c8a" {:name "Mary"}
   "f3c630f5-35e2-4600-8bf6-dc6fdbe76dae" {:name "Sue"}
   "b97c51b5-1078-45a9-a325-b8cb71f0959d" {:name "good"
                                           :customers ["b4931d86-efca-4e86-b397-e54315673d32"
   "51bf61bf-5163-4fd9-a52a-060de35ab7f7" {:name "bad"
                                           :customers ["133e9085-9481-4a64-befd-ce3b071a4c8a"]}})

;; references are "b97c51b5-1078-45a9-a325-b8cb71f0959d" and "51bf61bf-5163-4fd9-a52a-060de35ab7f7"

;; stateful references
;; not encouraged!
(let [bob (ref {:name "bob"})
      mary (ref {:name "Mary"})
      sue (ref {:name "Sue"})]
  (def customers [bob mary sue])
  (def good-customers (ref [bob sue]))
  (def bad-customers (ref [mary])))

;; references are good-customers and bad-customers


@U2FRKM4TW yeah I was imagining that there might be two collections at different points within app-db. I was attempting to draw out that path is identity in re-frame, collections have identity. So too do the customer entities with them.


I see, right. Although "path is identity" and "[sub-]collections have identity" don't fit quite well in my head (by sub-collection I mean a subset of entities within some existing set). No matter what path-as-identity a sub-collection might have, it will require some special path handling because you won't be able to simply get-in.


@smith.adriane Some some web apps are a thin veneer over a remote, authoritative database. In that case, database identities tend to dominate. In other cases, the SPA manages quite a complicated data model itself, without much reference to an authoritative remote database, in which case path based identities will be very useful.

šŸ‘ 3