hmm, how to trigger renders when remotes return and the data is merged with the app state?

@noonian: Add reads for the keys you want to re-read to the transact! call.

@jannis: but what about normal reads that use remotes so no transact!?

yeah it seems like I should put reads somewhere but I’m unclear on where they should go

Component queries?

the update happens when my send function calls the callback it is provided by om

yeah, my root component wants the key :remote/msg and the read for that returns a remote ast which causes send to get data from the server which is correctly merged into the app state but no re-render

do I need to manually transact! with a read after the remote data is merged in?

on the reconciler

Nope

What does the server return exactly?

Is it a map like {:remote/msg ...}?

yeah

although I could return something else if thats wrong

No, that's correct.

but that gets merged correctly

query [:foo :bar] -> result {:foo ... :bar ...} is the format

Does your read :remote/msg return a :value?

right, everything seems to work except no re-render is caused

I’ve tried with and without, if I supply one then my root component gets the initial value but not the new one when the state is merged from the remote. If I don’t supply one then it just never is rendered with that prop

I think you have to return a :value that gets :remote/msg from the local state.

And then it should work... if it doesn't: is the component and read code public?

ahh, maybe my simplistic tests are foiling me

heh, that fixed it

I had been trying with just a hard coded value and it was always getting that. Stupid on my part.

😄

thanks

No problem

this is the read code I’m using now which works great: https://www.refheap.com/111605

If I am building an app with several distinct pages, does it make sense to use set-query! on the root component to swap the query out with a new page’s “root query” when the user switches pages?

Alternatively I think you could put the reference to a component for that page somewhere and use get-query on that. Like current-component is DetailComponent which has subcomponents. Your root is just querying current-component. On a new page, you swap that with a different component

Putting a presentation on om next together right now. One thing I noticed about relay is that you can defer elements that are not important, like if you want to have comments in your app but it's not critical, you defer it and it loads in whenever it is ready. Can I do something similar in next?

It seems like the components will render when data is there. I'd like to give them a default value like :not-loaded-yet, which I could use to display loading indicators where necessary

(defmethod read :foo/bar
  [{:keys [state] :as env} key params]
  (if (find @state key)
    {:value (get-data state key)}
    {:remote true :value :not-loaded-yet}))

@dvcrn Looks ok to me except I'd deref the state once only. It might change between find and get-data. (I really don't know anything about concurrency in JS...) What doesn't work exactly?

@dvcrn: Partial loading is something I've been interested in as well. So far no delay-like capability. Your primary option is to filter your queries based on your current state, or use query params to change their behavior. Both involve embedding extra logic in your reads to figure out if it should be elided.

Maybe @dnolen has some thoughts on this

@dvcrn just return :value :not-loaded-yet

only return :remote true when you need the data.

I get ReactDOM is not defined if I use Sablono and take out om.dom in the om.next quick-start. Is that expected? I’m using om 1.0.0-alpha19 Here’s the “broken” version...

(ns om-tutorial.core
  (:require 
            [goog.dom :as gdom]
            [om.next :as om :refer-macros [defui]]
            ;;[om.dom :as dom]
            [sablono.core   :as html  :refer-macros [html]]))

(defui Counter
  Object
  (render [this]
          (html [:div "Howdy"])))

(def counter (om/factory Counter))

(js/ReactDOM.render
 (counter)
 (gdom/getElement "app”))

@boz you need to verify that your environment is OK

lein deps :tree and make sure your deps are correct

make sure you are building clean to eliminate variables while fixing the issue

as far as I know nobody else has this problem you are describing

@boz oh oops, reading more closely - yes that changed

om.next doesn’t rely on om.dom anymore for React Native.

so that behavior is expected

You can workaround it by by requiring cljsjs.react.dom in one of your own namespaces.

Thanks @dnolen and @noonian . Requiring cljsjs.react.dom fixed it too.

@dnolen: … I’m using boot in place of lein, just for the heck of it. Not surprisingly, it works too. All good stuff.

@jannis can i ask how you wired up the server-side bits of tempids? i tried passing om.next.server/reader to my middleware and then saw you’d already attempted that with no luck.

@joshfrench: https://github.com/Jannis/copaste/blob/master/src/server/handler.clj#L37

After that, you have to return a :tempids map from mutations that create real IDs. I'm simply reusing the id part of tempids for the real IDs at the moment: https://github.com/Jannis/copaste/blob/master/src/server/parsing/copaste.clj#L78

i’ll take a look, thanks!

@joshfrench: Any questions, just ask

new wiki page https://github.com/omcljs/om/wiki/Om-Next-FAQ

I will fill out these first two questions shortly

the idea is commonly asked questions that have 5-10 line answers

feel free to add things as long as it fits these constraints

@boz: In my build.boot I have react and react-dom "0.14.0-1" in my deps and then: [sablono "0.4.0" :exclusions [cljsjs/react] [org.omcljs/om "1.0.0-alpha20" :exclusions [cljsjs/react cljsjs/react-dom]]

hmm, is tree->db intended remove values that are not included in the given components query?

@thosmos: like this?

(set-env!
 :source-paths #{"src"}
 :resource-paths #{"resources"}
 :dependencies '[[org.clojure/clojure   "1.7.0"]
                 [org.clojure/clojurescript "1.7.170"]
                 [adzerk/boot-cljs      "1.7.48-6"   :scope "test"]
                 [adzerk/boot-cljs-repl "0.2.0"]
                 [adzerk/boot-reload    "0.4.1"]
                 [pandeiro/boot-http    "0.6.3"      :scope "test"]
                 [cljsjs/react "0.14.0-0"]
                 [cljsjs/react-dom "0.14.0-0"]
                 [org.omcljs/om "1.0.0-alpha20" :exclusions [cljsjs/react cljsjs/react-dom]]
                 [sablono "0.4.0" :exclusions [cljsjs/react]]])

I don’t think sablono has a cljsjs/react dependency. At least it’s not listed on http://clojars.org or in my deps graph (before making the above change.

I still get the same behavior. e.g., have to require cljsjs.react.dom in my cljs namespace.

Oops, I just took a closer look at your example, you're calling js/ReactDOM directly, so yes you'd need to require cljsjs.react.dom

It happens with om.next’s add-root! too, which defaults to using ReactDOM … it’s documented. I just didn’t get it.

I’m curious. What else could be used in place of ReactDOM.render for add-root! ?

@noonian: yes, we could change that but would like to hear more rationale about why you have stuff not in queries

@noonian: in any serious application with a remote component that wouldn’t even be possible

@boz: I think for Om to work with React Native, you have to use something else instead of ReactDOM.render.

AH! thanks @jannis

@boz ReactDOMServer.renderToString

Oh yes, that as well.

I'm attempting to use a normalized datascript db where entity refs use the :db/id , like so: {:db/id -6 :type :dashboard/list :list/title "A List of Various Media" :list/items [-1 -2 -3 -4 -5]}

is it possible to make this work with om/Ident without creating a separate items-by-id index?

@dnolen: I’m curious why it woudn’t work for remote. But my issue fundamentally has to do with normalization. I have a parser with recursive reads (which are reflected in the query as keys that do not show up in the data I want to normalize). That means that tree->db with the root component doesn’t really work to normalize the data. It does work to normalize with the sub query fragment that is inside the recursive key. So I’m manually normalizing my data using that approach currently but if the outer component wants any keys tree->db will strip them from the data unless I “whitelist” the keys by adding them to the sub query which is dirty. It seems to me that you will run into this issue anytime you have components that leverage recursive parsers. It also seems like any sort of routing example requires a recursive parser.

If I try to set the Ident to just a plain id I get an error: (defui Post static om/Ident (ident [this {:keys [id]}] id) ... error: Uncaught TypeError: Cannot read property 'om$isComponent' of null

@noonian: a remote should never send keys a client didn’t request anyway

you could never be in this situation

@dnolen: ah yeah, this is all client side

@noonian: I see your point though

you could for example have some data with a bunch keys

and some query isn’t part of normalization that will needs those keys later

fixing

Does this approach of manually normalizing the data when using a recursive parser make sense? It also leads to questions about how to merge in new data from remotes. Currently I think I’d have to keep Om not auto-normalizing and override merge functions to do my custom normalization because Om hardcodes calls to tree->db on the root component. But if one could instead configure the reconciler with a function to get the query to normalize with it would allow someone with a recursive parser to take advantage of auto-normalization without implementing custom merge functions etc.

@noonian: well you need to be able to turn this stuff off

so that’s well supported

however I don’t really understand why you want to do that since it automates a lot of work for you

This works. I was mixing up how datascript handles references in queries with how Om knows an entity's id. (defui Post static om/Ident (ident [this {:keys [id]}] [:db/id id])

@dnolen: I think I need to post some queries to make this explicit and also my goals as far as the structure of app-state.

@noonian: yeah I don’t know how you would have ended up here

it really doesn’t make sense to me at the moment

@noonian: tree->db no longer drops data, fixed in master

@dnolen: ok here is a minimal example of what I’m talking about that builds on the “Components, Identity & Normalization” tutorial: https://www.refheap.com/111633 Left out is the motivation for the recursive read but its basically the same reason as in https://github.com/jdubie/om-next-router-example/blob/master/src/om_router/core.cljs

@dnolen: thanks

@noonian: that’s just a lot of code with no explanation of what I should be looking at

I am currently trying the Quickstart and I get the following error with the part where I start fighweel for the first time:

Analyzing src/om_tutorial/core.cljs
Analyzing jar:file:/Users/mitkuijp/.m2/repository/org/omcljs/om/1.0.0-alpha20/om-1.0.0-alpha20.jar!/om/dom.cljs
Exception in thread "main" clojure.lang.ExceptionInfo: No such namespace: cljsjs.react, could not locate cljsjs/react.cljs, cljsjs/react.cljc, or Closure namespace "cljsjs.react" {:tag :cljs/analysis-error}, compiling:(/Users/mitkuijp/Development/om-tutorial/script/figwheel.clj:17:1)

@dnolen: hehe ok sorry. The People component is the RootView component from the tutorial. All I do is nest that under the :page/content key which in the read implementation just parses the query from the root.

so now that whole tutorial is nested under a root component

in the comments below you see the result of om/get-query and om/tree->db

I cant use the root to normalize in this scenario

and the automation you were talking about hardcodes that tree->db is called with the root component

so in short I’d love to use that automation but either I need to find a different approach or configure what query om will use for normalization

because the structure of init-data does not match the shape of the root query (due to the recursive parse key)

this still isn’t making any sense to me at all (I’m willing to figure it out though)

is the problem here that you don’t want to choose a page to start on?

that is sort of why you would want to nest something that queries keys at the root under a parent component

but its more the mismatch between the shape of the app state and the shape of the query

which I believe will happen anytime you have a UI tree that uses keys that trigger recursive parses

because even if I choose which one I want, I can

can’t normalize with the root

@noonian you’ve said a couple of things that make no sense to me

I’m sorry please ask me questions so I can clarify

1) mismatch between app state and shape of query

no such thing

if you are doing this you don’t want to use Om period

hmm

2) believing this occurs because of recursive parse

absolutely not

tons of people are writing recursive parse without your problem

ok, this is the query in question that I would like to be able to normalize data with: [{:page/content [{:list/one [:name :points]} {:list/two [:name :points]}]}]

what is stopping you from doing that

?

are you saying that I can’t maintain app state of the shape:

{:list/one
   [[:person/by-name "John"]
    [:person/by-name "Mary"]
    [:person/by-name "Bob"]],
   :list/two
   [[:person/by-name "Mary"]
    [:person/by-name "Gwen"]
    [:person/by-name "Jeff"]],
   :person/by-name
   {"John" {:name "John", :points 0},
    "Mary" {:name "Mary", :points 0},
    "Bob" {:name "Bob", :points 0},
    "Gwen" {:name "Gwen", :points 0},
    "Jeff" {:name "Jeff", :points 0}},
   :om.next/tables #{:person/by-name}}

that all works fine, the problem is if I want to use tree->db

or use auto-normalization which calls tree->db

right this won’t work

let’s rewind and reset here

what does tree->db do fundamentally ...

it takes a thing which matches the query and transforms it into a database

there’s no such thing as giving it a thing which doesn’t match the query

but nothing is stopping you from writing a normalized initial app state

and leveraging normalization later

right, I understand that, but the query I want to use for normalization isn’t the root query. I think you are saying that isn’t something you want to do with Om but I am trying to understand why not now.

why not, because I just don’t care about this problem

either supply the normalized thing yourself and be done with it

or supply the correct initial data

ok, in this case what would that look like?

you need a union

but your example doesn’t use one

so wouldn’t it be something like this?:

(def init-data
  {:page/content
   {:list/one [{:name "John" :points 0}
               {:name "Mary" :points 0}
               {:name "Bob"  :points 0}]
    :list/two [{:name "Mary" :points 0 :age 27}
               {:name "Gwen" :points 0}
               {:name "Jeff" :points 0}]}})

ok I think I finally see where the issue might be

normalization on unions works correctly for collections

that makes sense to me, but throws an error in tree->db I believe because one of the components in the tree doesn’t have an ident

but not singletons

@noonian: yeah I’m pretty sure this is the problem

@noonian: https://github.com/omcljs/om/issues/473

by singletons you mean components without idents?

ah

no I mean you have some singleton value that can be many kinds of things

:value (X|Z)

but we only spuport

:value [(X|Z)*]

ok, but even then I would expect to be able to normalize this data with the root query above:

oops

{:page/content
   [{:list/one [{:name "John" :points 0}
                 {:name "Mary" :points 0}
                 {:name "Bob"  :points 0}]
      :list/two [{:name "Mary" :points 0 :age 27}
                 {:name "Gwen" :points 0}
                 {:name "Jeff" :points 0}]}]}

is that a correct expectation? if so I think theres a bug in Om, if not then Im misunderstanding something.

and the query: [{:page/content [{:list/one [:name :points]} {:list/two [:name :points]}]}]

Hmm I got the quickstart working with the following project.clj

(defproject om-tutorial "0.1.0-SNAPSHOT"
  :description "My first Om program!"
  :dependencies [[org.clojure/clojure "1.7.0"]
                 [org.clojure/clojurescript "1.7.145"]
                 [org.omcljs/om "1.0.0-alpha20"]
                 [com.cognitect/transit-clj "0.8.281" :scope "provided"]
                 [com.cognitect/transit-cljs "0.8.225" :scope "provided"]
                 [figwheel-sidecar "0.5.0-SNAPSHOT" :scope "provided"]])

@noonian: that query is perfectly valid already for that data shape

this is what everyone is already doing

@noonian: ah no it isn't

and no that’s not going to work

query and structure of your data always has to match - this is never going to change

this is in fact the whole point

so does that not match?

Yeah, I guess I really don’t understand the algorithm behind tree->db

@noonian: oh oops

it does match but it’s impossible to see this so much context

yeah

I think I can strip this down a bit

and get it closer to the original tutorial

[{:x [{:y …} {:z …}]}] matches {:x {:y … :z …}} AND {:x [{:y … :z …}]}

@noonian: and actually that issue doesn’t make sense we can’t normalize singletons since we don’t know which one to pick

@noonian: but I think now understand the conceptual difficult you are having

i’m having trouble even if its a collection

other people are having this problem as well

and the answer is always the same - fix this in the data

your initial data should look something like this

{:pages [….] :current-page [:pages :main]}

and normalization should not break this, it’s a bug if it does

your query should be

[{:pages […]} {:current-page […]}]

where those subqueries are the same

and all data for each page should go in pages and use references to point to tables of denormalized data if it represents the same data right?

yep

@noonian: sorry for taking so long to understand what you were asking about

so pages is a vector of maps?

yep

ok, lemme see if I can get data looking like that to normalize

then I’ll be happy

yeah I’m double checking things over here to make sure there isn’t a bug for this case

pretty sure there is

since I don’t have any tests around “partially” normalized data

@noonian: fixed in master

@dnolen: thanks I’ll check it out. So there was a bug then?

@noonian: btw thanks for pushing along the local-centric point of view

been a bit focused on the remote stuff so easy for me to miss cases like this

@noonian: definitely a bug

np, its what I’ve been trying to get working since I jumped into Om next but was suprised no one else seemed to run into this

you should always be able to have your own links in the data

and normalization should not touch it

@noonian: people have like @thomasdeutsch

this pretty much the exact same kind of issue

but the point remains, think about links

try to fix things in the data not in Om Next machinery

yeah, I was definitely pushed that way but I think I hit bugs that I thought were om next semantics hehe

yep

@dnolen: I had the same problems as @mitchelkuijpers trying to run through the Quick Start last night and I can confirm that adding the transit deps and using figwheel-sidecar “0.5.0-SNAPSHOT” fixes the issue

@kschrader: ah right … will fix

Awesome quickstart btw @dnolen Finally time to play with om.next

@mitchelkuijpers: thanks!

agreed, really helps getting started

well done

@dnolen: ok I’m still having trouble normalizing with a union. I have a much smaller example now but the tables created by tree->db have nils instead of the ids from props. Either a bug still or my data/query syntax is off. Here’s the example: https://www.refheap.com/111640

ident is called twice for the page during normalization and with print debugging it seems it only gets the correct props the first time called

@noonian your query doesn’t make sense given the data

the query for PageView

@noonian: oh that’s your union query

yeah, I based it off the union tutorial and tried to make sure the shape of everything was the same

so PageView corresponds to DashboardItem and RootView to Dashboard and PeoplePage to Post, Photo, or Graphic

@noonian: oh simple bug

PeoplePage is missing :id

in the query?

yes

doesn’t seem to affect anything

you need it

otherwise you can’t compute ident whenever you want

I think the issue is in the union. In the tutorial you are unioning on the type of the dashboard item. How does Om know to compute the union from the type key?

there’s nothing wrong with the union

everything looks fine to me

but PeoplePage missing :id is a bug

add :id to the query doesn’t change the behavior

ok looking but that is a bug

are the keys in the union used at all?

no they aren’t again a thing for remotes

cool

but we need the queries

locally

right

and a way to differentiate them

yep

@noonian: your code example works here

@noonian: make sure you clean your build

try again

you need :id

ok

alpha21 released with deps fixes so tutorials work again

@jannis you might want to test migrate

@jannis also I don’t know about OM-470 anymore https://github.com/omcljs/om/issues/470

@dnolen: hmm, you got tree->db to work with that code?

I don’t see why you can’t just use om/get-query

@noonian: yep worked perfectly

weird, something must be up with my environment then

om/get-query works fine

@noonian that was for @jannis

oh sorry

@jannis I don’t know why om/get-query isn’t good enough for OM-470

@dnolen: How do you suggest to use it there?

@jannis oh hrm

the problem is for the case where some other component listens on that key

@dnolen: ah I got it working. I must have git pulled too soon and still had the buggy version. Re-pulling and installing alpha22-SNAPSHOT fixed it

thanks for your help and patience

@noonian: np

1) if the component listens to it, you want to pass in the query the component uses for that key, not just the key itself, otherwise query in read will be empty on the remote end and you'll get nothing useful back

@jannis so how about this … thinking out loud ...

if a user drops a key, find all the components that listen on that key and get their queries for that key

merging them

2) if other components listen for the same key... I don't know... would we collect and re-execute all queries for keys passed to transact! and send them all over?

Ok, listening to you now

so for example :user/playlist

maybe 3 components listen to this

each need different things

actually we should probably just do the simplest thing that works

so find each component that listens on the key (the indexer knows)

and get their query for that key

done

Yep, sounds good

It's kinda what my hack does except doing it the right way (via the indexer) and for other components as well

ok

I totally forgot about other components

@dnolen: Migrate still works. What's id-key for? (Apologies, I'm really tired )

Oh, it allows to set :id or whatever represents the ID from the real ID in the tempid mapping?

@jannis yes it just for the case where you just don’t want to read anything

you just want the new id

but need to know what the standard id key is

Yeah, so you can inject it into the entity/object in the client tree. I get it. Cool

Assuming the server doesn't do anything but create a real ID and store the rest as-is, that makes no-effort transactions possible; the client doesn't have to do anything else at all unless they fail on the server side

Magic

haha yep

Did I mention migrate still works?

Oh, I did

yes, cool thanks for the report

(Mmh, parinfer is cool. Comment the last line of a function with a lot of ))))))))) out and it just moves the parens up to the last line still in use.)

added examples on delayed loads and optimistic commits https://github.com/omcljs/om/wiki/Om-Next-FAQ

If I see something like this TaggedValue in the transaction response data, the client hasn't properly handled the transit response, right? This fragment passed to the send callback and :id-key doesn't work: :tempids {[:snippet/by-uuid #object[Object [TaggedValue: om/id, ab6596ec-e154-429b-9fba-7f6a03ed619a]]] [:snippet/by-uuid ab6596ec-e154-429b-9fba-7f6a03ed619a]}

@jannis yeah that means writing didn’t work for some reason

Writing (server) or reading (client)?

oh hrm

could be either

you would need to check

Will do. Certainly not Om's fault anyway

@dnolen: Actually... shouldn't the CLJS version of om.transit/reader also use ReadHandler?

no read handlers are just functions

only write handlers are types

in transit-cljs

I can verify that CLJS transit bits round trip

i.e I can write/read a tempid

Ok

so I’m pretty excited about the Om Next caching story

requires none of the craziness that Relay & Falcor are pursuing due to lack of HTTP Caching narrative

turns out ident migration is a great time to prune the application state of unreachable information

so there’s nothing more for us to do

and I don’t need to bother with crazy logic, we can just lean on browser caching

as it should be!

How does migrate know which idents to get rid of?

@jannis it doesn’t need to

is ident migration moving data to ident tables in db->tree? if not when does that happen?

we just drop all the generated tables

and rebuild

we have to anyway for ident migrations

because all those links in the data could be bad

Ah, of course. And migrate happens for everything returned by the server, so when e.g. {:list/one ... :list/two ...} are returned, it rebuilds and all people that aren't returned anymore disappear.

@noonian: we only look at the tables in the sense that we need to build the full tree

as we do that we replace any old links

then we invoke tree->db to get a fixed up thing

any refs that weren’t in the tree of course will get discarded

I see, and what initiates that process? the send function calling the merge callback? So build the data tree by following refs, then merge in new unnormalized data, then renormalize?

basically GC

@noonian: this only is applied for the remote case where :tempids are returned for mutations

ah I see

but nothing’s stopping you from doing this yourself if you’re doing local only

I haven’t done any remote mutations yet just have remote reads going

not local only, but was working on the local story first

routing, specifically