Fork me on GitHub
#om
<
2016-03-09
>
jaredly04:03:15

So it looks like the root query is getting run on every mutation, in addition to the query of the node that was changed. Is that expected? the root query is pretty expensive, and I’d rather it not re-run simple_smile

cjmurphy04:03:10

@matthavener: Is there a reason your top level component does not join (using {}) to the other component? You have:

(defui App
       static om/IQuery
       (query [this]
              [:parent (om/get-query Parent)]))

cjmurphy04:03:56

, but I would expect to see: [{:parent (om/get-query Parent)}].

tawus04:03:56

How do we resolve nested idents in an ident using db->tree. Say I have [:x 10] with an app-state {:x {10 [:y 20]} :y {1 10 2 20}}

tawus04:03:24

What would be the query I should use with db->tree ?

jlongster04:03:25

@jaredly: that's exactly my main complaint about the current system. you can memoize db->tree and make your query run really fast, but even then the system needs to recurse through your whole data structure to tag it with paths which isn't fast. I've found it hard to control the queries and make sure as little work happens as possible

jlongster04:03:00

if it weren't for that last step to tag the data structure, you could memoize your parser and make query execute super fast

jlongster04:03:06

so re-executing them wouldn't matter

jlongster04:03:44

I'm optimistic we'll figure it out though

tawus04:03:55

@jlongster: I have been experimenting with @tony.kay’s approach of doing away with remote reads and using read mutation. I haven’t tested it much but I find the approach simpler. The reads are then reduced to pull statements.

jlongster04:03:00

if enough people hit that, it'll become an important point

jlongster04:03:05

@tawus: I don't think that's really related. you still have normal reads and you want to avoid excessive query execution

tony.kay04:03:28

@jlongster: and you are aware of path optimization, right?

tony.kay04:03:44

We’re building production apps and are seeing no problems with query performance

jlongster04:03:13

@tony.kay: yep but that doesn't help with the situation where I change something unrelated but it still has to run a query that returns a lot of data

tawus04:03:18

Oops! This is query execution after mutation. Yes, it is not related

jlongster04:03:25

either via set-state! or something higher up in the component tree

jlongster04:03:35

it could be that my app just isn't as suitable

tony.kay04:03:23

But are you seeing user-perceptible lag, or just the warnings in the console?

tony.kay04:03:54

cause for most apps, 60fps just isn’t necessary….so who cares?

jlongster04:03:21

definitely lag, and long execution times. it's all due to path-meta that has to traverse my big list. it's not theory, I ran into just bad performance for my app.

tony.kay04:03:37

ah…you were the one doing big items in data state?

jlongster04:03:38

was really hard to do much and make it avoid that work

jlongster04:03:04

"data state"? I had a large list of data in my app state, yeah

jaredly04:03:20

so I have ~2000 nodes, arranged in a tree. and the root query loads everything (I think that’s the only way to do it?), which takes a while

jlongster04:03:27

more clearly: I'm building a personal financing app and it's all centered around big lists of transactions

tony.kay04:03:47

So, if those bigs lists are not in the query, is it a problem?

jaredly04:03:05

and then the root query apparently re-runs on every mutation, even though the root node doesn’t change

tony.kay04:03:09

e.g. they are ok in the app state…the problem is them getting hit by a query?

tony.kay04:03:27

@jaredly path optimization can help avoid the root query

jlongster04:03:45

@tony.kay: yeah, ok in the app state. but for example, I also had a current-tab in the app state that was used in a parent component, and whenever that changed it had to re-query all the data, even though it doesn't need to. I just basically wanted it to set display: none (or block)

jlongster04:03:06

that re-querying was too easily triggered

tony.kay04:03:57

So, I don;t know if this will work for you, but I imagine that you don’t need to actually show these long lists in the UI all at once, right? Why not have a transform function that builds the current “UI View” from the larger data set…then your UI query need not deal with the big data?

tony.kay04:03:48

We do this in our apps for other purposes, but the idea works well. Post-process data after loading it into small view bits that the UI queries…then things like “next page” generate the next view.

tony.kay04:03:01

The UI query need not be what you use to talk to the server

jlongster05:03:46

@tony.kay: I've tried a bunch of stuff like that simple_smile I want to show a reasonable amount of data so you can scroll through it quickly (a couple hundred at least?). I also use the react-virtualized component so that only the actual items displayed are mounted in the DOM. But even with a couple hundred, path-meta made the query slow-ish (not terrible, but was hard to control)

jlongster05:03:11

this wasn't the only problem also, I want to dump in a new list of data quite frequently and make that fast. In the end I was tempted to write my own DB format, but I'm not sure anymore. I'm prototyping this app with react/redux now but hoping to look at this again later

tony.kay05:03:11

hm. Good to know. I’ve not run into that yet. Is it a problem if you treat the list of items like a blob (e.g. don’t have UI queries for it, and instead render using just sub-functions?)

jaredly05:03:33

ok so pathopt helps a little bit for leaf nodes (though there’s still a noticable delay), but doesn’t help for branch nodes 😕

jlongster05:03:55

@tony.kay: I tried that too, and unfortunately right now it doesn't help, but @anmonteiro is going to make path-meta so that it doesn't blindly recurse through all the returned data. what it should do is only recurse down into what the query asked for, no more. I tried implementing it (https://gist.github.com/jlongster/1c661626a0860022ef26) but it's not good enough

tony.kay05:03:18

ah, ok, I am remember this from the issues list

jlongster05:03:23

once that is solved I could just treat it like a blob and then do my own normalization or whatever else

jlongster05:03:32

but then I lose some of the nice stuff, so it's a trade-off. at some point I can show you (messy) code and the app if you want details

jlongster05:03:00

it sounds like @jaredly is running into similar things

tony.kay05:03:21

Well, one other thing that comes to mind is that the full root query should not actually be what is run. The UI refresh actually focuses the query to the paths that need re-rendered (true it has to run from root, though, but the side branches should not be in the query). So, that might be of some use….though ultimately not ideal

tony.kay05:03:59

oh wait…I might be wrong on that

jlongster05:03:20

unfortunately for my tabs situation the list is a child of the tabs component anyway

tony.kay05:03:05

I thought that was how it worked…but I’m having trouble proving that to myself reading the code….it has gotten bigger since I last looked I think

jlongster05:03:46

that sounds like it could help

jlongster05:03:47

heading to bed, talk to you all tomorrow

ethangracer18:03:11

Hey all, running into a strange situation for my server-side reads. Say I’m reading

[{[:item/by-id 1] [{:content [ … big nested join here … ]}]}]
When I parse the query down to content and return {:value nil}, the query result is an empty map. But when I return {:value {}}, the result is {[:item/by-id1] [{:content {}]}. Why does returning nil wipe out the entire query path for the return value?

ethangracer18:03:23

interesting, thanks. why does it matter if the value is nil? shouldn’t I be able to return nil from a read if no error was thrown?

ethangracer18:03:40

for example, what if I want to rewrite the :content with nil

ethangracer18:03:57

but not the entire :item/by-id

iwankaramazow20:03:59

@jlongster: how is the same app in R&R (react/redux) coming along?

anmonteiro21:03:42

@dnolen: was wondering if we should also check that children have changed in shouldComponentUpdate. this is what @r0man is suggesting in #644

dnolen21:03:45

@anmonteiro: this doesn’t sound right to me - would need to think about that

anmonteiro21:03:42

@dnolen: I don't really have an opinion, will have to see how React behaves wrt. to this

dnolen21:03:02

@anmonteiro: yes but I suspect children is not part of the calculation

dnolen21:03:10

not the default anyway

dnolen21:03:18

the fact that it worked before doesn’t really mean anything

anmonteiro21:03:37

@dnolen: It worked before because shouldComponentUpdate was broken

anmonteiro21:03:55

the wrapped vs unwrapped props thing

dnolen21:03:25

maybe it’s convenient I suppose

dnolen21:03:43

I’m just confused how children could possibly change from outside

dnolen21:03:56

(sorry haven’t had time to look at the minimal case)

anmonteiro21:03:55

@dnolen: ah, I'll sum it up. the minimal case passes a <ul> (as children) to a component. the contents of the state dictate how many <li>s there will be

anmonteiro21:03:48

I'll look at how React handles this and let you know. I think we should mimic whatever behavior React implements

dnolen21:03:57

OK, agreed

jlongster21:03:37

@iwankaramazow: good but I'm very experienced with redux so that's not surprising. I'll probably try to write it again in Om after I finish a prototype and have a better understanding of how to handle my data in an efficient way

anmonteiro21:03:35

@dnolen: so React does re-render whenever children change

dnolen21:03:50

@anmonteiro: ok, then we should follow suit

anmonteiro21:03:07

I have a JSBin in case you want to look it over

dnolen21:03:58

children will be an IEquiv so this should be OK

anmonteiro21:03:31

@dnolen: that means O(1), right?

dnolen21:03:49

O(N) I think, since children is being dynamically constructed each time

dnolen21:03:22

it’s hard for me to think this will matter much wrt. perf but of course should test that with the patch applied on something realistic

anmonteiro21:03:47

@dnolen: yes, it might be O(N), I've looked at PersistentVector's -equiv implementation

anmonteiro21:03:30

I'll submit a patch for this one, should be a simple one

anmonteiro21:03:37

wrt. performance, most of the time people won't be using children anyways, so it'll be (not= nil nil)

anmonteiro21:03:01

so O(N) only for the worst case