This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2016-03-07
Channels
- # admin-announcements (5)
- # aws (2)
- # boot (313)
- # cider (69)
- # cljsfiddle (18)
- # cljsrn (17)
- # clojars (6)
- # clojure (121)
- # clojure-austin (4)
- # clojure-bangladesh (4)
- # clojure-colombia (2)
- # clojure-dusseldorf (17)
- # clojure-japan (1)
- # clojure-russia (65)
- # clojure-sg (4)
- # clojurescript (94)
- # community-development (6)
- # core-matrix (2)
- # cursive (2)
- # data-science (6)
- # datomic (28)
- # hoplon (4)
- # jobs (1)
- # jobs-discuss (1)
- # keechma (15)
- # ldnclj (2)
- # off-topic (6)
- # om (140)
- # om-next (1)
- # onyx (47)
- # parinfer (11)
- # re-frame (13)
- # reagent (4)
- # spacemacs (7)
- # specter (7)
- # yada (18)
@anmonteiro: this is how far I've gotten with the path-meta
optimizations https://gist.github.com/jlongster/1c661626a0860022ef26
a bit messy, but it solves my use case. I'm not using union or recursive queries yet though, and those will break
I don't know what to do with union queries because I don't know which subquery to use while descending
note that this won't help much if you're still querying across a lot of data, as it will still descend into it. It allows you to "bypass" the system though by returning a lot of data for a single key (don't query into it) and optimize it yourself
Can I use om.next code inside an om app? Can I somehow mount a om.next root inside a om controlled vdom?
Don't see why not, if you're happy with them being separate apps.
Yes I have to mount them separately in a real dom. But mounting the om.next part in a om controlled vdom is not possible?
@dnolen: the work that @jlongster is doing on path-meta
will probably need to use the query-template
& co. helper stuff. Would you be open to move these to a query-util namespace or something?
@anmonteiro: if we can avoid that it’s preferable - it’s not clear to me why path-meta
needs query-template
at the moment.
@dnolen: I might be misunderstanding what has to be done in path-meta
but I think we would need that stuff to choose union branches & join entries etc.
yeah I figured there might already be code for all this logic. basically path-meta
needs to walk through the query so it knows when it can stop tagging data
I'm trying to think if there's another way we could do this, like dynamically constructing the path as props are passed down
@jlongster: I'm looking at your code and might have something for you soon
something that immediately jumps to mind is that you can't have that top-level if
or rather, you need to put x
in the else
branch
@anmonteiro: awesome, thanks! I've studied the zippers & templates before but not enough to know how to leverage them (if possible). I don't know how to move forward with union queries.
@jlongster: I'm looking into it, I'll ping you when I have something
but this would involve having the zipper / query-template stuff in a .cljc file
which I did locally
@anmonteiro: I see what you mean about the if
, I wonder how it was working for me in my app, strange
also this doesn't really help for most cases, it only allows you to not query across data (just use :items
instead of {:items [:x :y]}
but still return a big list for the :items
key, and then you can optimize that part of the app yourself. If it still descends into each item it's still a bit slow
so I don't know if there's another approach altogether that is faster for the general case
@jlongster: I don't think we can avoid descending into that stuff altogether with this approach
I'm thinking that we can, however, reduce by one the number of recursion steps
by looking ahead into a query and seeing that it doesn't have joins
and not recurse into that
Are there any problems with using the Om.next-style normalized application state in an Om.now application?
(Other than having to do the normalization manually.)
@anmonteiro: if you have a list of 1000 items and a component that renders each item, you still have to tag each item individually right?
I think it's the descending into each item that is slow for me, but maybe I just need to use smaller page sizes
yes you do have to tag each one in that case, but we can avoid recursing into the single properties, it's what I'm saying
e.g. in [{:item 1 :foo 2} {:item 2 :foo 3}]
you need to tag the maps but not their properties
which is happening currently, I think
it's a bit unfortunate imho that I can return the same data as the previous read
and even with a memoized db->tree
the query can be slow because of this step
@anmonteiro: here's why this is important to me, I don't know if I should be doing things differently: I have a big list of data, but I load it incrementally in chunks of, say, 100. Loading 100 at a time is fast enough and works great. But if I've scrolled down enough and the list had loaded 5000 items by now, if I do anything in the top-level components (`set-state!` etc) it re-reruns the query which now returns 5000 items, and path-meta
is the bottleneck there
@jlongster: I also can't spend any more time on it today
I've added a comment to your gist with my working stuff
for posterity
I'm thinking that another approach could be follow the query and add meta to whatever matches it and leave everything else alone
we currently follow the data. wouldn't it be more performant to follow the query instead?
@anmonteiro: awesome, thanks!
not sure, I'd have to see the code, I suspect if there's a path that has to descend at all it still might be a problem. but I'll think about it some more.
I was just laying my thoughts here, I think it's something worth pondering. more of a rhetoric question
@anmonteiro: last thought: following the query might make it easier for optimizations, like detecting if the the components that use this part of the query don't need path
at all, maybe we could skip it (we don't need it if it doesn't use dynamic queries right?)
we always need the path meta
full-query
will use it, for example
and that's triggered after any mutation
I made a test case for the situation I described before: imagine components A->B->C, A and C have queries and B doesn't, but B has local state. If B calls set-state!
to rerender and C calls set-query!
to change its query, it seems undefined what props B passes to C https://gist.github.com/jlongster/df77ec0d37818162557f
here is that code running: http://jlongster.com/s/om-state-bug/
strangely, I can set the query once, and continue updating the number and C continues to render bar
fine. but if I set the query again (it still stays bar
, but does another set-query!
call) and then increment the number C displays foo
@jlongster: haven't looked at it, but set-query!
also accepts a list of reads as transact!
does
would this solve the issue?
@anmonteiro: doesn't seem like it. C changes its own query and only it needs to rerender the new query anyway
the problem is that B rerenders because it changes local state, but it has no queries so it doesn't re-run any queries, but it passes props down to C. It should be passing down old props before set-query!
but I'm seeing inconsistent behavior, sometimes it does and others not
@tony.kay im working on a similar issue using c3. I was thinking of diffing the next-props to be loaded with the current props in shouldComponentUpdate, and returning false is there is no difference, otherwise true. I then put the render logic in didUpdate. Is this what you were thinking? It seems reasonable.
@tony.kay I see, I didn’t think of that. Why is that an issue? What does your render function look like?
well, that's why I was asking the channel about it. If I don't touch shouldComponentUpdate at all and put my rendering in other lifecycle methods it works fine; however, technically React could mess with it.
@kenbier: For whatever reason, my sample works when I do it that way, but it seems like doing it via componentWillReceiveProps
and always returning false from shouldComponentUpdate
is better.
I see. I didn’t touch shouldComponentUpdate
and did not notice any performance issues either . So I’ll probably leave it as well. Thanks for explaining.
@anmonteiro: (can't avoid thinking about this) it seems like we could tag the data path inside db->tree
as it generates it, and in path-meta
don't recurse into data that already has the tag
it's bothering me that we are the ones generating the data and should be able to do that on the fly instead of recursing the data structure afterwards
users also could then selectively "pre-tag" data in their parsing functions if they really wanted to micro-optimize something and path-meta wouldn't recurse into it
@kenbier: I hadn't tried the alternative...just did, and it works fine and is almost certainly the way to do it.
where render-squares
is D3 code that uses the props to render/update the UI (idempotent)
@anmonteiro: alternatively, this actually would be a very simple solution that solves most of my concerns: expose path-meta
to the user and allow them to preemptively apply it in the read
function, and make path-meta
not recurse if it's already been tagged. that way I could memoize this work, and my worries go away
because I'm fine doing it once, but when any parent component changes it re-queries the big data and re-does all that work making a simple query change slow
@jlongster: hah, I know the feeling
there's a knob to disable path-meta
in the parser
@anmonteiro: yeah, but like you said we actually need it, right? wouldn't things just break?
yes we need it. I meant if you want to do it in your read
fns
also, putting it in db->tree
might not be sufficient
because we also need it in non-normalized data
it's a cool problem to think about
but yeah, in the end I basically want to disable it for part of the app state, knowing that I won't use features that will break with it
I meant people might not be using normalized data, and so they're not using db->tree
at all
this is something that needs to go in the parser
I'll prototype a version of path-meta that doesn't follow the data, but the query instead, as I suggested earlier
whenever I have the time
(might be only on Friday though)
I can live it for now, just trying to avoid any refactoring down the road, but I'll bet that we can just optimize this somehow later
I was just going through the source: why does (reconciler) `(queue! [_ ks] (swap! state update-in [:queue] into ks))` trigger a rerender?
@iwankaramazow: because reconcile!
is run on requestAnimationFrame
@anmonteiro: thanks 😄
explains a lot why my undo/redo now rerenders
is there a way to determine the current path into the denormalized state tree in a mutate?
i see private functions for path in the source, but i’m not sure if its exposed publicly anywhere?
@matthavener: just curious, why do you need this?
iwankaramazow: i want to store some ui state for a component down in the tree
and i definitely dont want to write a mutate handler for every possible path down the tree.. its not even possible for some dynamically defined paths (like a component tied to some entity)
so 'thinking-with-links' style queries don't suffice in your case?
oh hmm
let me read that, it seems useful
that's what I'm referring to
the graph magic 🙏
hey i've been slowly walking through the om next todomvc tutorial to try and understand it all, and I'm not sure what the
:find
part of this query is doing. can someone explain?(q [:find [(pull ?eid selector) ...]
:in $ selector
:where
[?eid :todo/created]] (db conn) [:todo/title :todo/id])
i know based on the :in and :where that we're querying for the entities that have :todo/created :todo/title and :todo/id attributes
@seanirby: finding all titles and ids for all todos
@seanirby: what part don't you understand
Pull will pull all attributes that has entity id ?eid, using the selector to specify the attributes
because we wanna use the query that the client has sent
we could be only asking for the title
:find ?eid
would just return a vector of ids [[id]], whereas what is is doing will return [{:todo/title <val> :todo/id <val>}]
@seanirby: that's just "declaring" variables
you need to actually use them
e.g. in pull
thanks @anmonteiro and @taylor.sando
is it correct to say that the pull expression in the :find field determines the format of the output response?
It will return a map
Hash map
actually the ...]
says it'll return a vector
Is normalization recursive? If I had a ident to a top level unique ident that contains a vector of idents, will those idents get normalized calling tree->db?
here’s a testcase showing the problem https://gist.github.com/matthavener/2d2910222c1221b719ea
basically, the contents of the map normalized don’t get normalized themselves