Hrm. There must be something very wrong here. This works fine:

app.reconciler=> (cognitect.transit/read (om.transit/reader) "[\"~#om/id\",\"~u9fff7aad-e336-4d21-a582-86cc133b0ddd\"]")
<#C06DT2YSY>/id["9fff7aad-e336-4d21-a582-86cc133b0ddd"]

app.reconciler=> (ajax.core/POST "" {:params (om.next/tempid) :reader (om.transit/reader) :writer (om.transit/writer) :handler #(println % (type %))})
#object[Object [object Object]]

Nevermind. Passing :response-format :transit to POST did the trick... so this took me over an hour.

Don't forget to sleep @jannis ;-)

@leontalbot: Aww, thanks You're right

When I use om/computed componentDidUpdate is called twice. First with om/props or prev-props having the correct props and then again with {:om.next/computed {...}}. Why is that?

Did you try without computed to verify it actually has anything to do with that?

yes when I pass a map as props it works fine

man, after figuring out that normalization stuff everything just started working as I hoped/expecting things would be

@noonian: cool!

question: I have a post component which looks like this:

(defui Post
  static om/Ident
  (ident [this {:keys [id]}]
         [:post/by-id id])
  static om/IQuery
  (query [this]
         '[:id {:user [:username]} :content])

{:app/posts [{:user {:username dvcrn}, :content Hello World!, :id 1}]}

yet after normalisation, :user is always nil. What did I do wrong?

(The parent component queries like this:

(defui TimelineComponent
  static om/IQuery
  (query [this]
         (let [subquery (om/get-query Post)]
           `[{:app/posts ~subquery}]))

@dvcrn: you should be able to test this without the server bit

Isolate the issue

hmm what do you suggest?

my thinking was that the :user key will be handled like every other key in that query during normalisation. so :post/by-id 1 should be {:id 1, :user {:username dvcrn}, :content Hello World!}

ah, am I supposed to return the data flat maybe? like: {:app/posts [{:user/username dvcrn, :post/content Hello World!, :post/id 1}]}

@denik: make a minimal example

@dvcrn: just test parsing or normalization with data like the tutorials show

hrm you are right. I tested it with the normalisation tutorial (but with "name" being embedded into "user") and the problem seems to not be present

@dvcrn: might be a normalization bug now that I look at it

@dvcrn: yep bug

using your example as a test case

I think I just found a normalization bug

normalizing nested data with a ui tree where you have nested components with idents

hard to describe

@noonian: might be the same issue @dvcrn is having

just fixed in master

if it’s not fixed, need another minimal case

make sure your environment is clean if you’re testing master yadda yadda

ok, I haven’t tested master yet

@noonian: looks the same

and also didn’t understand what @dvcrn’s case was

definitely a bad bug that’s easy to hit

cool

I’ll check master in a moment

I think my case still exists

but I want to make sure what I’m trying to do is/should be supported

In my case, the first component encountered in the tree that implements ident is correctly normalized, but the data nested inside that map is not normalized i.e. the normalization does not recurse into the identified thing and replace maps with refs. It doesn’t create tables for the components getting the nested data.

@dnolen: can you go into short detail what was causing that bug?

Checking if it's fixed for me

problem is gone. user data is now being normalised correctly.

Somehow I'm getting process.hrtime is not a function now though

What version of cljs are you using? I’ve had similar issues in the past that were fixed by updating to the latest version (1.7.170 at this time).

let me check

@noonian: yep that was it. Updated clojurescript and cljsbuild and it's gone

nice

In my datascript/om.next app i would like to specify that a component should not be rendered if only a :db/id is received. Here is a scenario in 30 loc: https://gist.github.com/ThomasDeutsch/8a88f6d63063e892daee Would this make sense?

@thomasdeutsch: Sure, why not? If the subcomponent queries demand props that you can't satisfy because they don't exist / aren't set, you shouldn't pass the incomplete data down to those subcomponents anyway - you'd be violating the contract established by their queries.

Although I'd probably make sure that what ends up in the database is stuff that is actually useful (e.g. through validation / schemas / etc.) and can be rendered.

om

@noonian: so I spoke incorrectly last night, the keys of the union matter

the keys must match the first component of the idents

@noonian: I would test with master (made some other fixes) and change this and see if it still fails for you.

if does please provide the components along with the data as well so I can easily make a test case. Thanks.

UUID hashing in transit-cljs was a nasty bug. Took me an hour or two last night to figure out why in default-migrate, (get-in pure old) returned nil, resulting in all local entity props to be erased when migrating from a tempid to a real one.

@jannis ah yeah, I’m surprised that one was only just reported a couple of days ago

but maybe people just aren’t using UUIDs that much yet

I didn't notice it at first yesterday because along with the mutation, I also re-read the corresponding data. So it would migrate to something empty, then be overwritten by the new data right away. Dropped the read and ran into the bug right away.

But it's working now

ah right because I suggested that (`:id-key`)

@jannis glad to hear it, sorry about that

No problem at all

If cljs-ajax had bumped its dependency on transit-cljs, it wouldn't have been an issue. This should solve UUIDs in any client that uses cljs-ajax: https://github.com/JulianBirch/cljs-ajax/pull/113

@jannis yeah this also is why I never bother with wrappers over goog.net.XhrIo

just too much trouble for a simple but critical part of the app

You're probably right. cljs-ajax doesn't even make it possible with less code. The extra dependency/complexity may not be worth it.

I might have completely misunderstood what it is used for, but doesn't lein override the package's dependency if you use :scope "provided"?

or is it for something totally different and I'm just saying nonsense?

@anmonteiro: people keep copying that part blindly

that’s Maven scope

it doesn’t make sense to do that in anything but libraries that other people will use

thanks for clearing it up for me

it just means this is a dependency but don’t actually add it to the artifacts lists of deps

because everyone will always provide it themselves

so it doesn’t make sense for anything other than Clojure & ClojureScript

even then, only in libraries that other people will use, right?

absolutely

doesn't really make sense on our own projects, then

if I'm understanding correctly

that’s correct

by projects I mean apps

cool, thx

it’s just a way to avoid adding to a user’s dependency hell

you have to be careful though

I do it because Clojure & ClojureScript change with enough frequency

but if your library actually needs a specific version of either of those things you need to pin.

yep, that makes sense

@jannis integrated transform-reads into transact, give it a spin at your leisure

Wow, that was quick

I'll look at how you did it first, I'm really curious

haven't been following the latest updates to Om, been caught up in schoolwork; what does transform-reads do ?

@jannis wasn’t much to do since there’s all these helpers now that are needed elsewhere

@anmonteiro: it fixes up reads so that remotes get the right thing

and locally you have the convenience of just talking about some key

selector-wise?

[(change/it! …) :some/list]

query* now

that’s fine locally, but remote we need to know the queries associated with that prop

I see

so it gets those queries from the components where they are declared

and passes them to our read fn?

@dnolen: Excellent. focused-query is what I tried to "emulate" in my hack (except only for the initiating component itself). full-query is something the hack didn't even consider. Nice. Gonna try it now

@anmonteiro: Yep

cool

@jannis cool thanks

@dnolen: (get-ident this) is always (ident this (props this)) now?

@jannis yes, I could bring that back, were you actually using that?

Everywhere Mostly to pass the ident of a component to callbacks. But I can live with ident as well, it's only marginally longer.

@jannis reverted, it’s back

I just forgot why it was there

Heh, okay

Thanks

@dnolen: Somehow performing a local mutation without specifying any reads causes the entire query of the initiating component to be sent over the wire again.

@jannis well you have the implicit read of the component

but that shouldn’t be triggering a the query to get rewritten out

Yeah, that's supposed to be local only, isn't it?

@jannis if you look at transform-reads you should see it only applies if the transaction fragment is a keyword

@jannis so not clear to me how that could be happening

Same here. tx only contains the mutation by the time it arrives at transact*. I'll debug a bit more.

Okay, it happens only when the app is reloaded after changing its code. If I refresh the page in the browser it works. I had this weird behavior before...

@jannis right because you blew away the state and it would resend right?

That's the initial read but that's another thing. I've ran into situations where the app has behaved differently after refreshing and after reloading (regardless of the state). One time it was caused by doing (for [item list-of-items] (item item)) by accident (data and component had the same name...). I must be doing something like that again.

@jannis ah k

@jannis so I’m interested in the case the where you actually supply a key, does that work for you?

With my app behaving in two different ways I can't say which one is actually the right one.

It could be that it transforms [(app/something) :foo] into [(app/something) [({:foo ...} {:param :bar})]] instead of [(app/something) ({:foo ...} {:param :bar})] but I can't be sure.

I think the app is behaving normal again. @dnolen It's not transform-reads that is the problem. It only sends the entire component query over the wire if mutate returns an :action. So somewhere else it must accidentally go remote when updating the component after the action.

@jannis ok cool, that could be a bug in Om of course

Strange. It only happens if :action is provided and :send is specified.

@jannis you must be providing :remote logic somewhere and not conditionalizing it?

if parse keep returning :remote then it’s going to keep going back

(or it’s a bug)

@jannis easy way to test - parse w/ this mutation query provide :remote as 3rd argument

it should return []

if it doesn’t, that’s the bug

It does

@jannis so something else is up

@jannis I would test gather-sends next.

(gather-sends (to-env (:config reconciler)) tx [:remote])

Will do. (om/transact! reconciler '[(app/toggle-expanded ...)]) triggers the send as well, so it's not the component being added implicitly that's the problem.

@jannis if this is getting messed up it’s getting messed up at gather-sends

that’s what transact* uses

gather-sends returns {}.

heh so then this doesn’t make any sense

Exactly

I would start setting some breakpoints in the transact logic and poking around

or in queue-sends! impl of reconciler

and walk the stack backwards to see how this is possible

Is there a way to see cljs data structures and not the generated JS ones when debugging in chrome?

I just use cljs.core.pr_str

there’s a Chrome extension now too https://github.com/binaryage/cljs-devtools

Thanks

@dnolen: Right. @tony.kay pointed me to it. It may be simple. The "problem" is that something triggers schedule-render!. Could be the state watch. That calls reconcile!, that fetches the root query and gathers and schedules the read sends I'm seeing.

@jannis, ah right I see the bug, will fix in a bit

Awesome!

What's the bug exactly?

see parsef in reconciler

the gather-sends there should be first time only

it isn't

simple to fix

Ah

@dnolen: is the second one to pick up possible new stuff that needs to happen due to the state change?

perhaps I'm misunderstanding something: the parser is run with a nil target and each remote on every re-render, correct?

or is the remote set only done after transacts and query param changes?

my reading of the code says the former...gather sends is running the parser on all remotes

during parsef

following the om.next tutorial (the datascript integration part), I get :

pull_parser.cljc:223 Uncaught #error {:message "Cannot parse pull pattern, expected: [attr-spec+]", :data {:error :parser/pull, :fragment nil}}

@tony.kay: it’s just wrong

gather-sends should only run 1st time and after transactions

set-params!?

@tony.kay: ah right after query changes

I had assumed your example in todomvc of conditional remote logic and comments on conditionalized read meant you intended that

Does that mean :remote true doesn't have to be conditionalized at all because it will only be executed during initial read and transactions that ask for the key to be re-read?

So, you need the conditions either way...but I can see why you only need to do the remote rescan in special cases

@jannis: ANY transact could re-run it

so conditional it is, unless you want to repeat it always

parse goes from root

@tony.kay is right

@dnolen: but what if the incoming data from the remote read changes the app state in a way that wants more remote reads?

@tony.kay: that just sounds broken to me

e.g. new things appear on the screen that were not there.

one way binding

if a read causes another read that’s two way binding

busted

I guess the follow-on reads in transact can handle that

transact! doesn't gather keys from the root though, so you can choose whether or not to re-read remotely by including it in the transaction. If you don't - local re-read only. If you do - local and remote re-read.

I find conditional :remote confusing. The only situation where the client knows to re-read remotely is on initial render and whenever a mutation happens. That information is not available inside read :foo.

straining my brain to think of an example...had not thought about it as a two-way binding

@jannis: what's wrong with "if my data is here in app state, use it; else remote".

"refresh" becomes a transact that clears the state

or timestamps, etc.

How do I know the data in my app state is up-to-date enough not to require remote?

The client doesn't have the tools to make that decision in read. Unless I'm wrong.

up-to-date is cache invalidation problem...one of the two hard problems

(naming, cache invalidation, and off by one errors)

I don't think David has solved that one in Om

Fair enough. You could argue that polling/pushing server data into the client can happen separately to keep the app state up to date.

And the client needs to assume what it has is fine.

Although in that case... why have remotes at all and not just wait for something else to make data available in the app state?

Also: if :remote is conditional, gathering sends for (om/transact! '[(foo/bar) :baz]) would never return any.

I think you want to be able to state: after this mutation, :foo or :bar may be out of date and I want to re-read it. But if there already is a :foo or :bar in the app state, that wouldn't end up as a query at the remote end.

Unless you add explicit invalidation flags in the app state and ... ugh.

Please correct me if I'm totally wrong here.

Why can't your (foo/bar) clear the invalid stuff locally before returning?

I haven't looked at transact! implementation enough, so not sure what else it might do

@jannis: This is what I am doing... so the query ends up being more of a subscription request than an request for data. The server will send updates, and the client assumes it always has current data.

@bplatz: web sockets?

SSE... but web sockets would work fine too.

either way, the thing that remains: we need to understand better how it is supposed to work

Datascript allows the subscription to be more broad, to reduce the server-side need to re-running queries. So it can push attribute changes that may not actually be required, but then the client will re-execute the query once the changes are received. Om helps here in that it will limit re-rendering if nothing has changed.

https://github.com/omcljs/om/blob/master/src/main/om/next.cljs#L729

the code for transact does local and remote

queues and sends

@tony.kay: You could clear the invalid stuff. But think about the tempid feature, for instance. If you'd say saving a temporary entity will remove it from the local state parallel to sending to the server for creating it there, the whole tempid migration story makes no sense.

so, the parser (your parser) must do the "right thing" to get the follow-on reads to ever happen

And if you didn't clear the temporary entity but marked it as invalid instead, you'd have to clear the invalidation hints when merging the server data or during migrate.

@jannis “the client doesn’t know to read"

this just simply isn’t true

this is how caching works everywhere

there is no magic way to invalidate cache

only the programmer knows when/how and with what strategy

tempid strategy is just a special case that’s outside of any specific application (like GC)

I guess instead of deleting an entity with a tempid before transacting its real creation, read could return :remote true if the entity has a temporary id.

So if you pass the read to the transact! call, it would go remote.

@dnolen How would I import styles from a css file per component? http://ai.github.io/postcss-way/?full#29

@stephenway: Om Next doesn’t care about that, same as it doesn’t really care about DOM in the big picture

ahh ok

@stephenway: CSS is very specific to web, Om Next is designed to work with Web, iOS, Android

so that’s a community concern, not one I intend to work on myself

oh interesting, didn’t know that

like traditional routing, templating

this is all stuff I don’t care about including

@jannis I don’t really understand this point about handling tempid’s via read

another huge thing about tempid stuff is clients who cannot reach the network

tempid is just about building up the value you want before committing, regardless of when you get to commit it

Let's ignore the tempid stuff for the moment. What I don't understand is how passing a read to remotely executed transact! would be useful if the corresponding read function had a conditional :remote that is only returned if there is no data in the local app state.

How would you express "please have the post-tx read include the remote part after this transaction"?

@jannis I don’t understand the question since I don’t know why you would do that

@jannis oh sorry

you’re talking about the case where it’s already in the app state but you need a fresh thing

Yes

@jannis right so not obvious but that was addressed a while ago

[(do/it! …) ‘:please/read]

quoting will always refetch it doesn’t matter what you say in read

Oh! How did I miss the discussions around that?

@jannis I think it was before you showed in the channel

really old thing

Crazy

@jannis heh, been working on this since February

so there are quite a few things that happened before the Slack channel was a thing

How does the quoted thing know what the remote AST(s) will be if they aren't returned by read?

@jannis you can do this via metadata

@jannis this stuff wasn’t completely set in stone though since I wasn't really sure about runtime remote computation

and I definitely am convinced of that now

@jannis: you can do [(do/it! …) ~(with-meta ‘(quote :please/read) {:remote …}))]

and I’m more than happy to add a helper for that

something like

[(do/it! …) ~(force :please/read :remote)]

Ok. I haven't modified AST remotes yet, so others would be better judges on how this is done best.

I was just curious

I'll play with quoted reads

And conditional :remote.

transform-reads seems to turn [(app/dosomething) :foo/bar :baz] into [(app/dosomething) [:foo/bar] [{:baz [:db/id :baz/name]}]] where it should be [(app/dosomething) :foo/bar {:baz [:db/id :baz/name]}]

@jannis good catch fixing

should have been into tx not conj tx

fixed

@jannis also fixed the continuous gather sends problem

@dnolen: Thanks for the fix, retesting now

@dnolen: I am confused enough now... wasn't gather-sends fine after all if :remote is conditional?

@jannis not really

the logic should be very, very clean here

only 1st time and only after transactions or query changes

Doesn't that mean you could always return :remote true and it would still only be executed "1st time and only after transactions or query changes"?

And only transactions that ask to re-read the thing that always returns remote true.

@jannis in what cases would this not be desirable?

@dnolen: This is perfectly desirable, it's what I've been arguing for all along (unless I misunderstand yet again). 😉

@dnolen: transform-reads works fine now, the gather-sends on rerender is also gone and it's all working smoothly now. Good stuff!

@jannis oh ok misunderstood

great!

ok so here’s a breaking change, apologize for that

set-query! & set-params! are separate functions

but that’s a disaster for computing sends

(you’ll do it twice if you do both)

I’m going to collapse these two things into one fn

@jannis: but the parser will run from root, so anything that says remote true will be included in the remote read, I believe

so conditionals are still part of the equation

@dnolen: Ah, the quote thing is exactly what I needed for what I'm working on...I was doing something a lot more convoluted

I like the idea of a helper function for that...quote is not very self-documenting

I have a component that queries for a nested map, and then creates a set of child components. I am passing transact callback to each of those components I create. This should be enough to trigger a re-render correct? I shouldnt need to implement Ident and IQuery on the child components right?

@tony.kay: will add a helper for sure

https://github.com/omcljs/om/issues/476

set-query! and set-params! collapsed into master

update docstring on usage

ok, so everything is set-query! now

@tony.kay: yep

@tony.kay: fixing the send issue with set-query! now

I think I've seen some conflicting information about returning a vector from a join. Is that best practice, or is a map perfectly acceptable. i.e. [{:me [:name]}] returns {:me [{:name "brian"}]}, or is returning {:me {:name "brian"}} perfectly acceptable?

@bplatz: all the code is meant to handle vectors or maps

if there’s some case that doesn’t work provide a minimal example

@tony.kay: master is fixed

Both work, I just thought I saw somewhere "always return a vector from a join". In certain cases I'd prefer to return a map from a join, I just want to make sure I'm not messing up anything internally if I do so.

@bplatz: if you can find that, feel free to fix

I realized that you need to be able to return single values

1.0.0-alpha22 released

updating docs

@bplatz: I might have made such a mistake in my (unstanctioned) doc...I didn't realize singleton case was ok until fairly recently

yeah, I did say something like that in my docs...fixing

force remote read helper added to master

@dnolen: thanks, changing my ident fixed everything.

@noonian: cool!

Do you have any plans to make the relationship between the shape of an ident and how things are normalized more configurable?

@noonian: no

cool

OH: "i swear he’s watching our private slack channel, the # of times he’s just happened to drop whatever feature we’ve been wanting is uncanny"

lol

@dnolen I assume doesnt force sending reads to remotes if the remotes are conditional in the corresponding read functions and thus may not be returned from via e.g. :remote, :static or whatever? Or can it do that somehow?

@jannis yes force sending just overrides everything

an alternative is to just parameterize the key yourself

and conditionalize in parse on that parameter

But the resulting queries don't include modifications to the AST that read might make if the condition for a remote is true?

@jannis I don’t understand what you’re asking.

my point is that read can’t easily know if something changed by itself

it can if you parameterize the key after the transaction

you can do whatever AST fu you want

Yes. We may be talking about two different aspects. When you read :foo/bar the usual way, its read function would be called and that might return :value plus perhaps :remote. The latter could be true or it could be a modified query/AST, right? Like in your static vs. dynamic example. Om would then use that modified query and send it to the remote. Actually, with force I assume read is still called and in there you can then detect that (via target?), bypass any conditional stuff and to get the same modified query even if the condition you'd normally use may be false?

@jannis I think I understand what you are trying to ask

whether quoting will allow you do something conditional instead of going some other way

the answer is yes you can in fact detect this

the AST fragment will have :target, normally it would not

Ok

@jannis so you can reuse whatever you’ve got going on

My confusion is mostly about what force/quoting actually does. Like: what would the resulting query look like compared to not forcing? How is it different from an unconditional remote in the corresponding read function? If read has (if <cond> {:remote ...modify AST...}), will the forced query have that modification if the cond is false. As for the latter, I'm guessing: only if the condition is aware of checking :target. Or if force is passed that same modified AST already.

I might be miles off with my assumptions about what it does...

sorry for not being clear

it only does ONE thing

add :target to the AST

so you can detect this case

for example

the first time you read you know you want to go remote because you cannot find it in the app state

after that you never want to go remote ...

unless someone forced a read and you know that happened because the AST now has :target

Ok, cool! :) So you can make it (if (or <cond> <has target?>) {:remote ...}) basically.

yeah there might be a bug here but that’s how it’s supposed to work

checking

Oooook, sorry it took me so long but I get it.

yeah there’s a bug fixing

and you can give it a shot

@jannis fixed in master, give it a spin

I'll try tomorrow. Put the laptop away because I needed a break (if only from hacking myself) 😉

@jannis haha, no rush

I was expecting more magic but looked at force and thought: "but how das that force anything?" So it doesn't per se but it makes forcing possible. Good, I can take a nap at ease now. 😉

Or does it also send the read over even if the local read function doesn't do anything / doesn't return anything for the target remote?

@jannis I just changed the behavior, previously it would go regardless, which was the wrong thing (that’s what I stated earlier)

you need to be able to control this, the new behavior doesn’t blindly go

You, agree.

@jannis yeah after you pointed out that you’d lose your AST modifications I realized the current way was wrong

:)

seriously everyone in the channel is amazing ❤️

thanks much for helping think through all of this stuff … there’s a lot of corners!

Well, we get your hard work, pace and incredibly amazing technology in return. It's worth for that alone but it's also great to learn so much as we discuss and be a really lively community through it. So thank you as well!