how relevant is the euroclojure talk on om.next, have there been many significant design changes?

there is a newer talk recorded at the lisp nyc user group about two weeks ago

@scriptor: http://livestream.com/intentmedia/events/4386134 and http://korisnamedia.com/audio/ClojureNYC_9-29-15.mp3 for the demo audio

@abp: I was there for that one! Just looking to see what other resources I can study

@scriptor: oh wow nvm then

@scriptor: mutations weren’t sorted out until a couple of weeks ago

that was the last piece of the big puzzle, there are some other finer details still but nothing major

@dnolen: ah, makes sense, thanks!

Newbie question…I’ve been playing with om-next and am seeing apply being used (for example in the Person list tutorial) to render lists. I understand what apply does, and in the cases it is being used I see that it is splicing lists in as parameters; however, I noticed that removing the apply works just fine (as if something underneath is unrolling the collections already). So, why are we still using apply? Does something break in pathological cases? It sure seems like unnecessary noise in the examples.

OK, after hours of sorting through code, I’m guessing that React improvements are the reason. I found this commit from last Oct in React: https://github.com/facebook/react/commit/c07ea0ba34df57fc54b218d6b4edcd090ed6411c which talks about making React automatically recursively traverse nested collections of children if they support ES6 iteration. Since Clojurescript’s data structures implement ES6 iteration, I’m guessing we are still using apply to prevent browser compatibility breakage?

@tony.kay: oh nice did not know that. pretty cool, seems to work with for too.

@tony.kay: one issue that comes to mind though is to when is traverseAllChildren called? might break with regards to some lazy seq semantics and things not being in render anymore.

@dvcrn: now I get what you're trying to do, after seeing the whole thing

however I still don't think it's the intended way; what I'd do is have a map inside a specific :codemirror key and then query for that

then you don't have to fabricate keys in the read methods

@anmonteiro: can you give me an example of what you mean?

you mean I should put that stuff inside a :codemirror key inside the state atom?

yep

mhmmm

so, modify your app state such that :window is inside :codemirror

like {:codemirror {:window "foo"}}

then if another component wants :window too for example, I'll have to put that twice inside the state atom, and my mutators have to edit it twice

true

so if :window is not specific to CodemirrorComponent I wouldn't wrap it into the :codemirror key when reading it 😛

and if it is, I'd nest it in the app state map

in that way I have to make state specific keys for every component and not just have my global state and allow components to filter on that. I think this could get problematic when you start sharing state between components can you maybe explain what you think is wrong with my approach?

it seems to be cleanest to structure the app state tree just like the component tree

hmm...

didn't read the full discussion, but i'm struggling with how to structure stuff as well, and this is how i feel about it now

it makes sense, definitely. Just trying to think how to implement shared state with keeping things DRY

the structure part already ate a good amount of time from me 😛 I keep finding a solution, ask about it, get feedback and then do it over again lol

@dvcrn: what piece of state are you sharing?

for example I have the application (or window) size that multiple components need

{:component-1 {:window-size [1920 1080]} :component-2 {:window-size [1920 1080]}}

then I need to add that key to every component that needs it, plus the mutator has to be adjusted everytime a new component needs this state. Seems like a lot of copying to me :x But I can see how this is working

I'm also struggling with structuring

I didn't say it's wrong, it's just not how I've seen from dnolen's examples

I'd also say that normalization might solve the issue at hands

however, it seems to me that those are more "helper keys" than something to stick into a component's identity

for me it might even be something I'd put in local state, have you considered this?

e.g. a component that has :window in its local state and other components are nested into that one

@dvcrn: adding it to every component can be trivial, (defmethod read :component-1 [env key params] {:value (merge (actual-value env key params) (shared-component-map env))})

I think I'll give this way a shot and see how it goes

if this is the intended / optimal way, it makes things a lot more clear

I'm not there yet, but how would this integrate with a server?

what do you mean?

Om next in general?

look here, this is how I've done it: https://github.com/swannodette/om-next-demo/blob/master/todomvc/src/cljs/todomvc/core.cljs#L80 https://github.com/swannodette/om-next-demo/blob/master/todomvc/src/cljs/todomvc/util.cljs#L17-L24

might require some changes if you're integrating with e.g. Json over HTTP

that code works for edn, I believe

I took only a sneak peek into it but it is transacting the entire state, right? If my state is built like my component tree, does my server has to take care of creating a dataset that looks like my component tree?

I should probably read more about it though before asking questions 😛

@dvcrn: yes, your endpoint has to take care of that. you can also make a local function that collects data from multiple http endpoints and turns it into one tree

that seems not very optimal to me on first look. So now everytime I create a new component, that function (be it server or client) has to know the name of that component so it can return data in that format plus my mutator also needs to know this information. I thought it was more like: A component requests the data it wants (:a, 😛, :c) and the server gives that data back without knowing anything about the component. With this, just changing the name of a component would cascade through the entire application

@dvcrn: the data keys don't need to be equal to the component name

yeah but if we model the state like the component tree, multiple points still need to know about the presence of a new component. That's what I meant 😇

@dvcrn only if the new component requires data from the server that wasn't accessed by the app before

then you need to think about where in the data tree to put that new data, and then it often seems possible to use a similar hierarchy to the (new) component hierarchy

that makes sense, yeah

I think I should try an actual application to get my head around it

the code/repl comments without (render) from https://github.com/omcljs/om/wiki/Components%2C-Identity-%26-Normalization seem to help to experiment with different designs

yeah I ate my way through that already

i.e. make a state tree, look at the normalization and denormalization, iterate

@sander ok another small question on top of that before I go to sleep: Given I did it the way you guys explained it, what would be the best way to exclude local state and stuff that should not go to the server. For example if I have a :window key, or keys that define which component is currently visible. It of course doesn't make sense to handle these like the rest of the data

@dvcrn: i think remote queries are only ran for attributes that aren't already returned by your local read fn

for read yes, but how about write / mutate?

can I, say, put that stuff into a :local key and tell my fn to exclude all of that?

@dvcrn: i think mutations are only done remotely if you set :remote true

ahh ok

yeah just saw it here - https://github.com/swannodette/om-next-demo/blob/master/todomvc/src/cljs/todomvc/parser.cljs#L62

all right, more stuff to hack on tomorrow thanks for the hints!

thank you, it's fun trying to understand om.next without the tutorials being all written yet

Can anyone explain to me how the index is built in om.next (`build-index*` in next.cljs)? It appears queries of nested components need to be structured in a particular way for the indexer to recurse into subcomponents. I'm probably doing it wrong because the class-path->query mapping only contains an entry for the root component and when I call om.next/transact! on a subcomponent, this results in a "No queries exist for component path (frontend.app/App frontend.components.Subcomponent) error.

@jannis does the subcomponent implement IQuery and is its query referred to in the root component query?

Yes. But perhaps I'm doing it wrong. I'm doing something like (query [_] (concat [:app/foo :app/bar] (om/get-query Subcomponent)) to separate the information the root component is interested in (`:app/foo` and :app/bar) and the information the subcomponent needs.

So the subcomponent might have (query [_] [:app/baz]), resulting in a root query of [:app/foo :app/bar :app/baz].

@jannis try making that `[:app/foo :app/bar {:subcomponent ~(om/get-query Subcomponent)}]

I'm trying that now.

Well, build-index* seems to be looking for "joins" in the query. {:subcomponent ...} is treated as a join and it seems to then look for all components that have the same {:subcomponent ...} query and recurses by indexing those - or something like that.

But I haven't managed to get it to work this way.

when indexing, the metadata of (om/get-query Subcomponent) will tell that Subcomponent is associated with that join

Also, aren't queries to refer to application state only? :subcomponent does not appear anywhere in the application state, nor is there a read method for it.

if you're going to call it :subcomponent, you need to make read handle that indeed

Right. That seems wrong to me.

I am looking for a way to pass up queries from subcomponents to parent components without having to repeat details of the subcomponent queries in the parent. If the only way to do that is via joins that refer to specific parts of the application state then I'm wondering if that's possible at all.

@jannis if you want SubComponent to receive :app/baz, RootComponent needs to provide that value to SubComponent through props

so either RootComponent knows that SubComponent needs :app/baz and provides that from (om/props this), or RootComponent does what i just described

IMHO, the most intuitive way to include :app/baz in the parent query via (concat [:app/foo] (om/get-query Subcomponent)). I'm not sure why joins need to be maps.

in that case, how does ParentComponent know what props to provide to SubComponent?

Well, okay, I'm just passing (om/props this) down to the subcomponent. That's probably very wrong because it will cause Subcomponent updates whenever anything changes.

yes, and it doesn't seem to help building/using reusable components

I keep running against a wall trying to understand how subcomponents with queries can be nested in a useful way.

The TodoMVC example is too simple to help me understand. Yes, if I have one container component (todo list) and many children (todo items), then I can say: TodoItem wants properties x y and z, so that's its query and TodoList wants all todo items, so in TodoList, the query is "all todo items but only with the properties specified in the TodoItem query".

But as soon as there are one or two intermediate layers of components, it becomes less trivial. And I don't see a way around a massive mega-query in each parent component, becoming more complicated the closer to the root you look.

depends on how well your app breaks down into a hierarchy of components

if it's just a flat list of unrelated components, it becomes one (flat) monster query

I assume a good app would be nested quite heavily.

Also, let's assume a subcomponent is interested in two independent things, e.g. [:foo :bar]. How would it's own and a parent component's query look like, without having a query selector like {:subcomponent ...} which is totally UI-specific?

@thheller: what things are not in render? It is a pure function that acts as the React render (om-next in case you were talking om). React is doing side-effects and walking these things, so I’m not sure where a lazy seq would not be expanded. At least I cannot think of an example, but I don’t know what you mean by things that are not in render.

@jannis: this becomes easier when you have concrete examples

@tony.kay: I mean in render with regards to bindings, if the lazy seq is not realized while in that binding (ie. render) this might get lost

@thheller: oh, I see…in case react somehow async defers the expansion, and reconciler is no longer bound, etc

exactly

well not async defers

hm. right…where would async happen?

I imagine react does something like (-> data (render) (diff) (apply-dom))

if traverseAllChildren happens in diff the bindings won't be there

if it happens in render we are all good, did not check when it happens

I see. thinking about what I saw when I read the code...

yeah, I don’t think I hit that part of the process.

probably worth to investigate

from an optimization standpoint, react does throw out all subtrees if the type of the root is wrong, so I could see it wanting to defer

I’m going to lead towards leaving apply out for now unless anyone can tell me why we still need it. If a case comes up that doesn’t work during testing, it is easy enough to add it back in.

@tony.kay: funny twist ... the dev version of react realizes the lazy seq immediately. the prod version does not. so lazy seqs are out.

AH….great researching

ReactElementValidator realizes the seq to check if everything is ok, the prod version does not use this and realizes the lazy seq "later"

have not exactly figured out when yet but definitely after render

well doall still makes for nicer syntax than apply

or even mapv

right

@thheller: would you mind me running my understanding of om-next query processing by you. I’m trying to make sure I’ve got it

@tony.kay: might not actually be a problem since the binding is around ReactDOM.render and not in an indiviual render. still probably better to stay away from not forcing lazy seqs

@tony.kay: I'm not quite sure I fully get them yet but I can try

well, perhaps we can grok them together. I get that the root component composes all of the children queries (via looking them up) into one big large thing…it looks like then the parser is called to convert that query into the overall app state. That sound right so far?

Any child can compose from its children, making the whole thing recursive at start

also preserves local reasoning…I only have to compose my direct children queries into mine

well to be honest I have not quite figured out how/when the query processing happens

the whole remote thing is still a mystery to me

Yeah, I’m struggling with that bit as well…You can modify the query dynamically, so I know it can re-happen…also on query parameter changes

but it looks like the parser (router) is only ever called on the root component’s query (which makes sense, I guess)

well yeah but I ran into issues where the query is affected by the data it receives

@tony.kay: yes only the root executes the query

and if you don't pass exactly that data down the tree on render, you run into issues.

As you descend the rendering tree, it requires that you’ve followed the rules (e.g. you end up with a recursively nested map). and it seems you pass the explicit data via props

right

yeah

David did say “the parent filters the state of its children"

the big question for me right now is how "dynamic queries" work. David is writing a tutorial for those.

I’m reading over factory code….still missing something about what happens in the wrapped render

wrapped render is here

https://github.com/omcljs/om/blob/master/src/main/om/next.clj#L75-L83

oh and more bindings. so lazy seq is definitely out

I was more looking at the details in reshape-map

where the actual magic plumbing occur

https://github.com/omcljs/om/blob/master/src/main/om/next.clj#L30

yeah the render is part of that reshape-map 😉

ok, so factory gives us a function that takes explicit props, and puts them on :omcljs$value

path is tracked through metadata the the parser adds while parsing

(which is why we cannot break the structure rules…the metadata won’t match)

yes

and that is tacked in by the factory as well

basically all components that implement IQuery must receive the query result as their only props

that seems to be the case….so I guess you just explicitly pass it through any interstitial components in the UI

manually

yeah but I'm a bit skeptical. with old Om you passed cursors arround, which looked like data but weren't

om.next you pass data around, but must follow very strict rules and can not modify it in any way

basically looking like cursors again

well, the pure functional aspect is right on. passing immutable data = getting the same render result. The app state changing causes reconciler to re-trigger render.

learned that the hard way yesterday ... https://github.com/omcljs/om/issues/425

Yeah I saw you post that issue

I'm still trying to figure out if I really don't ever need props

I think it was because of the shape change, right…not the fact that you wanted to patch in data

but that none of my data can have a :ref property already bothers me

but I'm not used to om.next enough yet so you might not really ever need it

still evaluating ... trust in David right now to know what is "best"

Hm. right. The indexer is supposed to play the role of finding things

the issue is that I don't want DOM-related stuff in my data

but I really need to learn more first before making hasty judgements

I get what David is trying to do ... hope it all works out ... bit skeptical though.

I’m in the same basic spot…but as I’m working through it I’m mostly just seeing some of my assumptions were wrong, not that there is anything inherently bad. Something has to denormalize all of this state across the UI, and a graph is as good as anything. Especially when you mix in persistent/transient data mixing

The thing that mostly surprised me is that the parser itself seems to need to be recursive on it’s own. The query runs on root, and then you’re done…so if you had subcomponents that want something, the top level parser has to work it out. I was hoping for the ability to disconnect subtrees that were self-rooted and could hit the “router/parser” themselves. Perhaps I’m missing something that is actually there.

Well you can always write a second "root" component and manually add that somewhere down the tree

but I think that's what subquery is about .. which I also haven't figured out yet

I have not seen subquery

A second root…how would you add it? Seems like add-root wants an existing DOM node, and it seems like a bad idea to tell add-root! to join onto something that is already under the control of react elsewhere

well it is probably a terrible idea .. let me try and sketch what I mean

I also remember from his most recent talk that “getting initial state from the server in a manner that is synchronized” is really only doable if the root node knows what all is needed. The special query syntax for remotes triggers the (remote) reads, then stuff gets merges in, I believe, depth first on the return. But I think this is also the reason for a single top-level query: you can make a single server request, gather up all the stuff, then send it back all at once.

Oh, and since queries can be parameterized, it seems like you can put query fragments in your parameters…perhaps this is dynamic query. So some component that needs to make a decision about what to render could be told to vary it’s parameters such that the query changes and then the render method looks at the data and chooses which sub-component to render based on what is in the props. This lets you dynamically change out the subcomponent tree

Params: { :subquery {:thing1 [:item/title]} } Query: [ ?subquery ] (set-params! c { :subquery {:thing2 [:other/data]} }) That sound roughly right?

alright, David is probably gonna yell at me for how bad of an idea this is

but

https://github.com/thheller/shadow-om-next-tests/blob/master/src/cljs/om/nested_root.cljs

see OtherRoot which is nested inside RootView which has no query

seems to work fine

Ah…well, credit for making something work, but I agree it is none too pretty

the remote part I actually hate, just like Falcor/Relay

seems like a step in the wrong direction

we got all this async, multiple connections, caching and force everything through ONE rpc endpoint.

but I understand the motivations for it

Seems you can also just bang on the atom and skip the remote stuff

yeah, but om.next tries to do all this automatic normalization of data

I guess I only have a problem with all that since all my data is already pretty normalized since it is living in a SQL db

but ... still learning ... there are some quite some compelling arguments to do it this way

Yeah, I’m kinda in the best circumstance: full stack clojure/clojurescript with datomic

green field

so I can totally drink the cool-aid with few side-effects

@thheller: Hey, GF is wanting to go out. I guess since it is Sunday I oughta get off my rear. Thanks for chatting about this. I feel a bit clearer on a few more things

hf ... helped me to think through some stuff too.