Fork me on GitHub
#untangled
<
2017-03-06
>
urbank11:03:36

why does IQuery (query [this]) get passed 'this'? In what case would it be used?

urbank12:03:07

Is it for dynamic queries? I read on the slack log some mention of dynamic queries, but I'm not sure what it means, and what is its use case

qqq12:03:20

@urbank: I don't know either. If you figure it out, please let me know.

qqq12:03:41

I do know that I often see [_] and it seems like it's main use is taht the (query ... ) can depend on the defui component

mitchelkuijpers13:03:52

@urbank @qqq You can use the instance of the component for dynamic queries indeed, but I never need this with untangled. And it adds a lot of complexity that I would stay away from.

qqq13:03:58

@mitchelkuijpers : someone else told me that was "impossible" becuase only the Root node makes queries

qqq13:03:16

so therefore, the only use is that it tells the Root what attributes it wants

qqq13:03:33

furthermore, iirc, this function is "static", so it's further evidence it can't actually depend on the value of this

mitchelkuijpers13:03:02

If you add a console.log in query you will see it get's called many times, but you are right that it needs to compose to root

qqq13:03:23

This is my understanding: when Root needs to render, root needs to get it's data to gets its data, it submits it query to get the query of Root, we call all the children of Root to get their queries (and this is when the query function gets called) == it is my understanding that besides the above, query is NEVER called is that correct ?

urbank13:03:09

@qqq That's my understanding as well. The component query gets called when its composed into the Root query.

urbank13:03:04

@mitchelkuijpers I'm still having a bit of a hard time reconciling generic components and not having dynamic queries which would tell the component which data it should care about.

qqq13:03:31

@urbank: I couldn't figure that out either, which is one of the reasons I switched over to datascript for my db.

urbank13:03:13

@qqq Yeah, datascript basically lets you do anything from anywhere, because it's literally just a bunch of datoms, so all you need to change some piece of data is the global unique id of that data. However, I found some issues with performance when 'benchmarking'. It's possible that I was just doing something very wrong. You've had no issues with performance?

urbank13:03:36

On the other hand, I'm pretty sure there's a solution to the generic component problem in untangled, I just don't understand it

qqq13:03:05

@urbank: what type of performance issues?

qqq13:03:20

for writes, I expect it to be 6x slower since it has to update every index

qqq13:03:46

for queries, as long as the ordering is not stupid, I expect it to be drastically faster since it doesnot have to linearly scan all objects

qqq13:03:28

@urbank: with no intention to offend -- have you worked through the first 4 sections of http://www.learndatalogtoday.org/ ? I found it very useful for mentally thikning about how the order of clauses affect the speed of a query

urbank13:03:51

None taken! I have actually gone through that. I think I just had a lot of queries. There was one q which got all the entity-ids that I wanted, and passed them to subcomponents where each of the subcomponents did a pull for the attributes it cared about

urbank13:03:30

I was also using Posh (if you've heard about it)

qqq13:03:11

Is Posh the one where it tries to infer which datascript queries need to be rerun automatically?

qqq13:03:19

That seemed way too magical for me.

urbank13:03:35

Yeah, that one

qqq13:03:43

I wonder if that's what's making it slow.

qqq13:03:09

And my strategy is: I write my own datascript querieis; I think very hard about how to make them fast, and I don't do any other optimizations.

qqq13:03:33

Basically, my goal is : queries = lookup 1 index, get items, then just minor filtering on them;

qqq13:03:09

I'm completely convinced that well organized queries should be as fast as hand written code

urbank13:03:01

My top level component had a query which got all entity ids of entities with some attribute [:find ?eid :where [?eid :attr]]

urbank13:03:31

this means that it had to rerun (according to posh) every time one of those entities changed

urbank13:03:15

triggering a ton of subsequent pull queries ... so in retrospect I suppose it's no wonder it was slow 😅

qqq13:03:15

The reason why I believe datascript can be faster in the generic case, is consdier todomvc where we want to filter on all "completed" tasks

qqq13:03:26

in a simple store, we have to (1) run through all todo items (2) check if it's "completed"

qqq13:03:48

in datascirpt, it's [:find ?eid :where [?eid :todo/status :completed]] and since there's an AVE index

qqq13:03:59

this is equiv to (get-in AVE [:todo/status :completed])

mitchelkuijpers13:03:13

Have you looked at union queries and the new routing in Untangled?

mitchelkuijpers13:03:33

I can tell you from experience that datascript is way slooooower then the normal app-db format

qqq13:03:55

how does union queries factor in when the fundamental issue is "data is indexed" vs "data is a list" ?

mitchelkuijpers13:03:49

If you add the correct idents you also index your data

qqq13:03:18

I seareched that page for "index" and got nothing

qqq13:03:21

where does it talk about indexing?

qqq13:03:31

I believe idents noramlize data, but that's very different from idnexing.

mitchelkuijpers13:03:16

If you have a user, and you create an ident which is [:user/by-id 1] where you store the user info

mitchelkuijpers13:03:32

Then I would say that is an index to get the user by id in your app-state

qqq13:03:05

suppose for each user, you also stored their age

qqq13:03:12

and I wanted "get me all users whose age is 25"

qqq13:03:21

can you do that without running through the list of all users?

qqq13:03:27

if not, I wouldn't call this indexing

mitchelkuijpers13:03:31

Then I would think in a real use case you go to the server

qqq13:03:22

I don't get what's "un"-real about the above scenario.

mitchelkuijpers13:03:12

You'l probably want pagination and stuff on users unless you know there will never be more than 1000 for example

mitchelkuijpers13:03:32

But I don't think you'll get much out of Untangled when you want to use Datascript

urbank13:03:16

Hm... I think that things like 'get me all users whose age is 25' definitely happen on the client. Well, at least in my specific use case. The data I get from the server is very general, and I have to transform it in to various views

mitchelkuijpers13:03:07

Ah for those cases we use post-mutations to build the right views for the same data. That is way to trigger a mutation after some data is loaded to create the right views for the data.

qqq13:03:50

... or just use a db query 🙂

mitchelkuijpers13:03:23

Yes or use Datascript (but then lose all the benefits from Untangled)

urbank13:03:53

Yeah, I'm still deciding between the two. But I don't think I understand untangled enough yet to make a call

qqq13:03:10

Untangled is what got me into trying Om in the first place, but at this point, I'm in love with datascript and can't see what Untangled offers me over just using plain transit.

mitchelkuijpers13:03:52

It offers you: getting data from the server, error handling, a lot plumbing, InitiallAppState goodies, Nice union query handling, A routing solution, Time traveling debugger, Testing tools, A generalized query solution, Server plumbing, i18n. You have to think if you need those things and if you are willing to build these yourself. And what do you do when you work with multiple developers and they have to use your homegrown framework.

qqq13:03:29

I'm using google app engine datastore on server side

qqq13:03:39

I'm using datascript on client side, and transit to communicate.

qqq13:03:04

I don't see all the plumbing, InitialAppState, union query, routing ...

qqq13:03:15

the replay debugger is nice, that I do miss

qqq13:03:15

I found that with transit/cljs-ajax this isn't nearly as big a deal as people claim to be.

qqq13:03:41

That's what people to me -- and I figured out I don't ahve to use om.next on the server side, I can just transit /cljs-ajax ... and write to my gae datastore.

qqq13:03:17

We should proably move to #off-topic

qqq13:03:36

It's probably not very polite of me to be bashing untangled in#untangled

mitchelkuijpers13:03:40

Yeah I was just trying to prove a point what Untangled fixes for you. I don't think you are bashing it. But I do want to make sure that you get what you are missing out on when you use om.next without untangled. (I started out with om.next and then later learned about untangled) ^^

qqq13:03:41

I thikn untangled makes this assumption that "you have to build on top of om.next for client/server", and I'm now convinced: I can just use datascript for client, gae datastore for server, and cljs-ajax/transit really isn't all that bad for communication.

qqq13:03:10

Also, om.next's query/db format seems very convoluted compared to datascript/datomic's eav store + datalog as query language.

mitchelkuijpers13:03:41

I would not recommend that to be honest, but you are free to do what works best for you.

urbank14:03:29

Anything in the above that would be a bad fit for untangled?

urbank14:03:44

I'm not quite clear how much of that derived data should actually have some form in the database, or how to easily swap generic components.

urbank14:03:56

@mitchelkuijpers Yes, but I have two unresolved questions. If there's no real structure to my data, would all the structure be given 'on the fly', with the component just getting a list of all the friends, or would some mutation create an entry in the database that would correspond to my example derived state.

urbank14:03:12

The second question pertains to generic components. And I probably don't know the answer because I don't quite understand those.

urbank14:03:59

If a multiselect is a generic component with a query [:ui-select/options]

urbank14:03:26

how does that map onto the :friends-id list? Where it isn't actually a list anywhere, it's just derived from the fact that Tom has these 3 friends

mitchelkuijpers14:03:33

Most Generic components don't have queries you probably don't need them for those. You can get the iinfo in a parent component and then give the generic component the :ui-select/options

mitchelkuijpers14:03:47

And the structure in your app-db will be given by using idents. When you load data from the server it will normalize that based on your idents. http://untangled-web.github.io/untangled/guide.html#!/untangled_devguide.D_Queries there are some examples of idents in this devcard

urbank14:03:28

I see. So I go to a different view of the flat data, would I create the structure on the fly, or create it in the app-db via some mutation? Also, in the untangled ui library, the calendar has a query, and Tony Kay indicated that this is useful for more complex components, because you can then affect them from anywhere in the app (click anywhere to hide some popup)

mitchelkuijpers14:03:37

The idents get created by creating the right queries with components. Have you followed the Untangled tutorials?

urbank14:03:00

I've seen the video tutorial, and read the untangled devguide where it pertains to the client-side. Maybe the relevant part went over my head. I'll definitely refer to it more.

mitchelkuijpers14:03:53

No problem at all just keep asking, I would recommend to build a small POC. You will miss stuff when you are just following the tutorial or a video

urbank14:03:16

Thanks! Yes I'm attempting a POC at the moment. Got stuck at generic components with queries, and merging their queries with something like an EditPerson component. EditPerson's query is composed into the Root query, and just specifies which attributes of a person to extract. So query: [:name :age :dark-secret]

urbank14:03:32

So if :dark-secret needs to be edited by some DarkSecretEditor, and that component has its own query, how would it be composed into the EditPerson query

urbank14:03:13

Since the data for generic components lives in top level tables (if I understand correctly)

urbank14:03:05

would this be a use case for links? ( [:dark-secret-editor/by-id '_] )

urbank14:03:07

what exactly would the DarkSecretEditor hold, since it's just editing the Person's :dark-secret?

urbank15:03:31

Or more concretely, since the calendar queries for :day :month :year, this means that there's a calendar table in the app-db, which holds these 3 values

urbank15:03:00

but what if these 3 values correspond to the birthday of some Person

urbank15:03:46

so Person has a :birthday field which is at some point controlled by Calendar

urbank15:03:52

how does this work?

mitchelkuijpers15:03:42

I have never used the calendar component to be honest

mitchelkuijpers15:03:19

Give me a second to see if I understand you correctly

mitchelkuijpers15:03:06

Ah so the basic thing is you have a person and you want to change his birthday with the calendar component right?

mitchelkuijpers15:03:45

But you want to change it when someone presses on save or submits a form, something along those lines am I correct?

mitchelkuijpers15:03:04

you could have a {:dark-secret-editor {:who [:person/by-id 1] :form/state {:secret "foo"}}}

mitchelkuijpers15:03:45

And when you change from which person you are editing you can change the :who and clear the :form/state

mitchelkuijpers15:03:05

And if you want to save them you merge the form state into the [:person/by-id 1]

urbank15:03:09

Oh, sorry I had to go somewhere for a moment. Yes, you understood correctly. It's an example though. Basically some generic component which has some state in the app-db is changing some other thing in the db

mitchelkuijpers15:03:26

No problem, and your dark secret editor could have a query of [{:who [:secret]} {:form/state [:secret]}}]

mitchelkuijpers15:03:35

Does this make sense?

urbank15:03:41

Right, that makes a lot of sense!

urbank15:03:21

And to make some more generic component (such as a calendar), which doesn't have a reference to what it's editing, you would just wrap it into another component that holds the reference, and the calendar's state?

mitchelkuijpers15:03:01

Yes, that is exactly what I would do

mitchelkuijpers15:03:23

I think you get it!

urbank15:03:48

Great, each time I pop over here and ask some questions it gets clearer. 🙂 Thanks very much for the help and your time!

tony.kay16:03:20

@qqq if you want to use Datascript, Untangled is not for you. Agreed.

gardnervickers16:03:13

I’m doing a data-fetch/load using an ident with the target being that same ident. Putting a watch on the state atom and observing the state transitions for the entity under the ident, I see it correctly change to a load marker, then nil, then the loaded state, but then it’s set back to nil again.

gardnervickers16:03:55

If anyone has any hints for where to look to resolve that, it would be appreciated. I’m not super familiar with the load marker lifecycle.

tony.kay16:03:41

So, target was really meant to target the insides of a node, not a top-level table entry. The query itself does that.

tony.kay16:03:11

I'm guessing you're using a link query and want a load marker as well?

tony.kay16:03:34

E.g. some component is doing a query of a specific table entry that you want to load dynamically?

gardnervickers16:03:46

We have a parent component that is trying to dynamically load a child by id.

tony.kay16:03:05

ok, so do it this way:

tony.kay16:03:36

1. In the parent's query, add some made-up keyword that points to the correct type: [{:the-child (om/get-query Child)}]

tony.kay16:03:50

2. Make sure the parent has an ident, so you know where it is

tony.kay16:03:53

3. Run a load with query params: (load [:child/by-id 42] Child {:target [:parent/by-id parent-id :the-child]})

tony.kay16:03:08

I think that should work

gardnervickers16:03:54

Should :target place the ident of the loaded data at the :target location?

tony.kay16:03:05

yes, that is what it is for

tony.kay16:03:13

normalization still applies

tony.kay16:03:58

(load-field this :the-child) is also a nice shortcut for this use-case, though then your server has to understand a query based on the parent's ident

tony.kay16:03:31

(e.g. what you'll get on the server is [{[:parent/by-id parent-id] [{:the-child (om/get-query Child)}]}]

gardnervickers16:03:03

load-field works well for us, however I’m not seeing df/load getting the ident of the loaded data and placing it at :target

tony.kay16:03:47

It is possible the ident-based load case has a bug. I was pretty sure it was tested in the cookbook load-samples, though

tony.kay16:03:01

could be wrong. Lots of combinations are possible

tony.kay16:03:49

It could be I only tested something like reloads, and never worked out the logic for an ident-based load that needed edge fixes in the graph

gardnervickers16:03:52

Ok great, I’ll start there.

tony.kay16:03:27

If you're seeing a problem on ident-based load, don't assume it is you.

tony.kay16:03:10

the load marker logic might not be right for that case

gardnervickers16:03:07

Ah, looks like when default-target here is an ident, it’s wrapped in another vector. Then we do get-in @state-atom default-target where default-target is [[component/by-id <id>]] which is nested too deep.

gardnervickers17:03:45

Here I conditionally wrap the return from data-query-key in a vector, covering the case where the query key is an ident instead of a keyword. https://github.com/untangled-web/untangled-client/pull/65

fz17:03:46

👋 Hello! I'm trying to wrap my head around queries and database structure

fz17:03:49

The Root has a query which evaluates to [{:people [:people/name]}]. That makes sense, given the database below it where :people is a vector of "person" maps.

fz17:03:53

What if I wanted :people to be an "Ident-able" table? Instead of {:people [{:id 1 …} {:id 2 …}]}, I'd have {:people { 1 {:id 1 …} 2 {:id 2 }}

fz17:03:11

Then the [{:people [:people/name]}] join would break

fz17:03:55

I'd like to get the same result as I'd get in the example, but also have :people be a "table" that can be accessed via Idents. Is that possible?

gardnervickers17:03:25

If you’re normalizing the “person” maps, you’d have a top-level key in your app-db that is something like :person/by-id.

gardnervickers17:03:03

Then you could query [:person/by-id] to get the raw table back.

fz17:03:39

So the app db would look like

{:people { 1 {:id 1 …} 2 {:id 2 … } }
 :person/by-id [{:id 1 …} {:id 2 …}]}
Or if not, what data would go under the :person/by-id key?

gardnervickers17:03:51

The app-db would look like this

{:person/by-id {1 {:id 1} 2 {:id 2}}
 :people [[:person/by-id 1] [:person/by-id 2]]}

gardnervickers17:03:57

ident’s are resolved using get-in, so if you have an ident that is [:person/by-id 1] there must be a :person/by-id top level key in your app-db, with it’s value being a map containing the key 1. The ident would then resolve to the value of the key 1.

fz17:03:20

If I wanted to set that up in initial-state, I presume I would do that on the Root component. Would I have to manually specify both the :person and :person/by-id values? I'm pretty sure there's a way to not have to manually keep the two sets of data in sync, but not clear how that works

darrellesh17:03:00

@tony.kay Is it possible to add a filtered-list-input to the possible input types supported with Forms. It would be nice to allow the user to key in a value and have the filtered list filter the results to the value entered and if there was no match in the filtered list then the input value would be the value persisted. Otherwise, the the id of the value which matches the filtered list item would be persisted?

tony.kay17:03:36

Any kind of form field you can imagine it easily added to form support

tony.kay17:03:58

how you render it is up to you, and how you interact with it is also up to you

tony.kay17:03:19

All forms care about is that you've declared some field of the entity to be form data. If it isn't a subform, you could render an alternate control (e.g. your filtered list) and use callbacks to populate the field.

tony.kay17:03:31

The "field" in this case is just a prop on your entity. Nothing special about it at all

darrellesh17:03:18

Right. That makes sense. I will render a filtered list but the text input would be under form support.

tony.kay17:03:27

Say you wanted to use the ui-calendar component to edit a date. The calendar itself has a query and mutations, and a callback. In that case you'd compose the calendar into your "form" and use the callback to update the date.

tony.kay17:03:51

The form support itself is about state management, with some nice add-ins for common cases.

tony.kay17:03:13

@gardnervickers Ah, so you're saying when you use an explicit target that is an ident, we get the wrong thing.

gardnervickers17:03:32

Actually no sorry

tony.kay17:03:01

I see the problem. I'm a little concerned about the solution. I'm still not convinced we should be "targeting" something that will already normalize to the correct place.

tony.kay17:03:17

I.e. relocate? should be false on an ident-based load

gardnervickers17:03:10

I misread the question. When I’m using a path like you suggested above, but loading from an ident, the default-target ends up being [[<ident>]] instead of [<ident>]

gardnervickers17:03:36

So (get-in state [[<ident>]]) always is going to be nil.

tony.kay17:03:36

right, but an ident load should not be "targeted"

tony.kay17:03:58

so the swap! should not go in that case, and you are right, it is going an screwing things up

gardnervickers17:03:00

I thought you were suggesting this above (load [:child/by-id 42] Child {:target [:parent/by-id parent-id :the-child]})

tony.kay17:03:54

still, isn't the right fix, since the dissoc should not run. Relocate is meant to move idents, not objects...you don't want to dissoc, though I guess it wouldn't hurt, since there is no dissoc-in

tony.kay17:03:48

I think this would be clearer, perhaps, as a cond that enumerates the cases.

gardnervickers17:03:03

Why dissoc ever? Wouldn’t you just want to overwrite the load-marker with the value in every case?

tony.kay17:03:18

dissoc is about loading into the root. It is simply a cleanup

tony.kay17:03:37

(load :my-thing Comp) adds a key to the app db :my-thing

tony.kay17:03:50

but you wanted the value to be elsewhere, so we don't leave it around.

gardnervickers17:03:43

and it works for me because (dissoc state [:thing/by-id <id>]) does not do anything since the ident is a path into app state, not a key.

tony.kay17:03:53

Yeah, loading a list of things, etc. Nice to send an abstract name of the query to the server, like (load :all-things Thing {:target [:thing/holder :things]})

tony.kay17:03:22

doesn't hurt, but the code is already more opaque than I'd like

tony.kay17:03:33

so a no-op dissoc makes it worse

tony.kay17:03:19

Did you verify that this does not break any tests?

gardnervickers17:03:40

Yup, I ran the firefox suite in addition to the lein tests.

tony.kay17:03:27

ok, I'm not going to worry about the dissoc for now. I'm doing a refactor, so I'll do some cleanup around this in a bit.

gardnervickers17:03:41

Fantastic, thanks for your help.

tony.kay17:03:22

on 0.8.1-SNAPSHOT on clojars

tony.kay17:03:34

be sure to upgrade Om to alpha48 and also latest cljs

tony.kay17:03:54

you can be a test subject for the new network stack 😉

gardnervickers18:03:15

Specifically for the loaded-callback during the remove-markers step.

tony.kay18:03:15

I thought we patched that the other day

tony.kay18:03:17

weird...did we talk about that one???

gardnervickers18:03:58

Yes but that fix didn’t actually work, probably because of the issue we addressed today.

gardnervickers18:03:16

Ahhh I see what you were saying. When data-query-key is an ident, we don’t want to (get-in st (data-query-key ...)) and relocate that because that’ll relocate the value, not the ident.

gardnervickers18:03:09

PR to set the data-path correctly for when a query is going off an ident, and retargeting loads to move the ident instead of the actual value. https://github.com/untangled-web/untangled-client/pull/66

gardnervickers18:03:25

Firefox + lein tests pass for me.

tony.kay18:03:22

couple of quick feedbacks

gardnervickers18:03:49

Hmm not seeing those on the PR

tony.kay18:03:07

forgot to submit

tony.kay19:03:39

@gardnervickers So, on load markers. I think we need to do a bit of refactoring, actually, and specify exactly what the behavior should be in all cases. There are not that many, and I'd like this to be cleaner

gardnervickers19:03:49

@tony.kay Thanks, fixed up the PR a bit.

tony.kay19:03:52

and more testable

tony.kay19:03:17

Seems there are the following cases: 1. Loading an entity by ident. We could place the data-fetch marker on an existing entity, if present (since it is a map). We could potentially generate said placeholder, but then the ident problem (nil id) is concerning. I guess we could derive the ID of the object from the query, but not the key. That would require some additional parameter on the load, which I'm a little loathe to add.

gardnervickers19:03:08

Loading by ident gives you the id though right?

tony.kay19:03:19

yeah, but we don't know what key it goes with

tony.kay19:03:55

but this is only a problem if you're using link queries

tony.kay19:03:06

(e.g. pulling the thing into some arbitrary component by ident)

tony.kay19:03:30

In 99% of good cases, you'd link the graph together and have a real UI component pointing to the loaded thing

gardnervickers19:03:40

The queries like [:toplevel/key ‘_]?

tony.kay19:03:44

in which case :target would be some prop on another obj

tony.kay19:03:14

[{[:obj 3] [:a :b]}]

tony.kay19:03:28

which you can do, but I would not recommend it

tony.kay19:03:39

I mean, you'd need dynamcic queries

tony.kay19:03:44

and the headaches that go with those

tony.kay19:03:09

So, in that case we never want to put a load marker in a table. I think we just make that a rule.

tony.kay19:03:26

you have to do your own marker if that's what you need

gardnervickers19:03:50

Oh I didn’t even know the client side parser supported that format of query.

tony.kay19:03:38

sure, works fine. As do dynamic queries. Standard Om fare, and we use db->tree from Om for reads, so it all works. Parameters are the thing you cannot use, but not because the syntax doesn't work

tony.kay19:03:54

just because we give you no hook to process them. Forced trade-off

tony.kay19:03:09

Write no read parser, get no client-side query params...but not in the dyn query sense

tony.kay19:03:28

ug. terminology

tony.kay19:03:52

[{:a ?subquery}] actually works in Untangled

tony.kay19:03:17

but [{(:a {:p 1})}] ignores the param

tony.kay19:03:30

and just reads prop :a

tony.kay19:03:53

the prior is a "query param", and the latter is a "parameter on a property in a query" (or query param for short 😜 )

tony.kay19:03:59

You can have a remote send the latter by messing with the AST in your mutation (common)...server can use them just fine

tony.kay19:03:10

back to the cases

tony.kay19:03:27

1. Loading via ident. If you give a target, that is where the load marker goes, else you don't get one.

tony.kay19:03:43

2. Loading via top-level key: load marker goes on top-level key. Again, all ok

tony.kay19:03:05

3. Loading via top-level key with target: marker goes at target

tony.kay19:03:09

am I missing any?

gardnervickers19:03:21

Does #2 get a marker if target is not specified?

tony.kay19:03:34

well, load-field is a utility auto-generates a query based on the parent's ident to populate a sub-field. So, it is technically (1), but with a query focused on the field.

tony.kay19:03:53

yes, 2 is ok, since it isn't a table, just a root graph edge

gardnervickers19:03:29

So load-field on [:something/by-id 5] on :field :foo puts it’s load marker where in app state?

gardnervickers19:03:45

at [:something/by-id 5 :foo]?

tony.kay19:03:14

I know I confused you there

tony.kay19:03:18

I was trying to remember

tony.kay19:03:41

(load-field this :comments): The :comments are on a subfield of this

tony.kay19:03:47

so, the query is [{ (om/ident this) (focus-the-query-of this :comments) }] The load marker shows up on the :comments field of whatever object "this" is

tony.kay19:03:07

For load-field, you're loading some additional bit of something you already have

tony.kay19:03:12

like a blog post

tony.kay19:03:26

but you only want to load some focused bit of it

tony.kay19:03:55

[{[:blog-post 3] [{:comments (om/get-query Comment)}]}]

gardnervickers19:03:59

But for that case, this could be an ident which resolves to a table entry, which we end up sticking a load marker into for :comments. Or does the “no load marker in a table” rule only apply to load markers being top-level elements in a table.

tony.kay19:03:14

no load marker as an entry in the table

tony.kay19:03:22

everything is in a table in a normalized database 😉

tony.kay19:03:39

except for the root edges, and those are keys -> idents

tony.kay19:03:53

and any scalar values you plop in the "root node"

tony.kay19:03:03

like :ui/react-key

tony.kay19:03:27

Oh, I guess the load-field case is a special one...since the load marker goes one level deeper than the others

tony.kay19:03:29

the ident we're using is not technically the thing we're loading...or should it be?

tony.kay19:03:53

conceptually, we're loading one or more of some children, so I've typically thought of that as loading the children. But you could also think of it as loading the "remainder" of the thing you've got.

tony.kay19:03:30

I think it is this use-case that makes most of the logic a mess 😕

tony.kay19:03:50

it's a really useful one, though

tony.kay19:03:04

and the one that motivated load markers to begin with

gardnervickers19:03:00

Yea, we most often have an entity partially loaded and want to load the rest of it. For example, when doing initial page load with a blank app-db, we can gather at least the desired entity id from the URL fragment. This gives us the entity id which we can run a mutation and put in our app-db.

gardnervickers19:03:15

Then we try and load the “remainder” of the entity with load.

tony.kay19:03:00

so you're treating it like a "refresh"

tony.kay19:03:05

instead of an initial load

gardnervickers19:03:23

Yes, for the ident load case.

gardnervickers19:03:40

For every other case, we want a list of things so we use a server property.

gardnervickers19:03:59

So it’s either loading [:user/by-id <id>] or :users/list-of.

tony.kay19:03:27

and your UI is querying via a link query?

tony.kay19:03:47

or do you have some component that has [{:current-user (om/get-query User)}]

gardnervickers19:03:00

[{:current-user (om/get-query User)}]

tony.kay19:03:09

ok, so that is your target 😉

gardnervickers19:03:23

But we’re still hitting the server with the query [{[:user/by-id <id>] (om/get-query User)}]

tony.kay19:03:26

You could even use load-field with that...it supports :params

tony.kay19:03:37

(load-field thing-with-current-user-prop :current-user :params { :user-id 5 })

gardnervickers19:03:43

And pass it the subquery as params?

tony.kay19:03:19

would send the query [({:current-user [props of user]} {:user-id 5})], and your server parser would see the params

tony.kay19:03:34

in the read keyed on :current-user

tony.kay19:03:00

there's another join in there

tony.kay19:03:14

so load-field might not be best

tony.kay19:03:21

since it would have the parent component composed in

tony.kay20:03:28

(load this :user User {:target [:parent :comp :current-user] :params {:id 5 }})

tony.kay20:03:09

where :user is now the join key the server sees. Just have to make sure it doesn't collide with any state of interest already in the root of app db

tony.kay20:03:41

and params can be used to pass the ID of interest

tony.kay20:03:47

now you'll get the load marker as you want it

gardnervickers20:03:57

That’s what we were doing originally

tony.kay20:03:14

why changed? Hopefully not because I told you 😜

gardnervickers20:03:16

Seeing the ident support in df/load made more sense. That way it was just (df/load this (om/ident this) (om/react-class this)) for refreshes

tony.kay20:03:02

so, I would be OK with putting in a load marker IFF the entity already looks to be in your db

tony.kay20:03:18

and in your case this would work, since you're pre-populating with something that looks like it

tony.kay20:03:30

actually, (df/refresh this) sounds like a nice util function with just that signature

tony.kay20:03:43

OK, so I'm going to re-enumerate the cases:

tony.kay20:03:53

1. Loading a field is a special case. Load marker goes in field of parent, and that parent's ident is what is used for the query. 2. Load via top-level key: Load marker goes at key (or target if specified) 3. Loading via ident: Understands that we're loading or refreshing an entry in a table. If there is already a map in the table, put the load marker on the object already in the table (and the graph will resolve it for the UI). Target is always ignored for this case. Do a post-mutation if you need to integrate the ident into app state. (or a pre-mutation if you want to see load markers for newly loading objects).

gardnervickers20:03:26

#3 works extremely well for us.

tony.kay20:03:47

k. patching that now

tobias21:03:11

I've found a strange behaviour change between untangled client 0.7.0 and 0.8.0 To do a login I'm using a google login that is checked on the server and then saving a jwt sent from the server in a cookie. I then do follow on reads in the transaction to set/check the client app state to show login and current user - my transaction looks like this:

#(om/transact! this
      `[(login/attempt {:jwt ~(om/tempid) :id_token ~(-> % .getAuthResponse .-id_token)})
        (untangled/load {:query [:login-sequence] ;; this is a dummy query to sequence the post-mutation
                         :post-mutation login/set-cookie})
        (untangled/load {:query [:logged-in? :current-user]
                         :post-mutation login/complete})])
I can see the cookie being set using goog.net.cookies and the correct value returned from the server in the app state from the om/tempid handling. The problem is that the last untangled/load call does not provide the cookie in the request using v0.8.0 whereas they did in v0.7.0. This is visible in the request headers in the browser network inspector.

tony.kay21:03:22

So, you're indicating that post-mutation did not run?

tobias21:03:11

The login/set-cookie post-mutation does run and the cookie is set, it appears like the next remote load (from the last untangled/load doesn't contain the cookie).

tony.kay21:03:42

oh, are you expecting the cookie to be set between these two calls?

tobias21:03:22

Yes - is that not the expected behaviour?

tony.kay21:03:49

hm. well, mutations go before reads, so your attempt will happen on a separate network request

tony.kay21:03:08

Then the loads are ganged together in a separate request (they go together on the net)

tony.kay21:03:21

I'm not sure why anything that changed would have affected something like this if it was already working 😕

tony.kay21:03:57

so , I would not have expected what you wrote to work because the post mutations always go after the queries have run, and both go at the same time...

tony.kay21:03:13

so no, I would never have expected that to work as described.

tony.kay21:03:51

reads have been combined since day one

tony.kay21:03:20

Can you have login/attempt set the cookie?

tobias21:03:10

I'll try that

tony.kay21:03:29

attempt is a remote mutation, right?

tony.kay21:03:51

and you have the value you need before it runs, or no?

tony.kay21:03:30

if no, then you could install a mutation-merge function at startup, and capture the attempt merge to handle a return value

tobias21:03:39

attempt is a remote read, and I generate my own jwt token on the server which is then placed in the app-state (using temp-id) hence the dummy read - I was attempting to follow the pattern at the bottom of M40 in the devguide

tobias21:03:52

sorry mutation not read

tony.kay21:03:53

I see. OK. So bullet 3, essentially.

tony.kay21:03:56

So, in a single transaction, the reads will go over the network together, and the post mutations will run (after they've returned)

tony.kay21:03:17

so it should have never worked to expect the last load to have the cookie if the cookie was set in the middle one

tony.kay21:03:26

The example you point to: The assumptions is that you already know who the user is. If the remap came from the server. Otherwise the server should throw an exception which will kill the transaction on the client.

tobias21:03:33

Ah, okay good to know about the ordering, I'll see if I can get something different to work

tony.kay21:03:36

clarification: the assumptions is you know what the user told you about themselves. The login mutation is trying to verify that (and using a tempid as a way to get a response).

tony.kay21:03:57

If you refuse to remap on the server, the the client will still have a tempid, which you could use to detect that something went wrong

tony.kay21:03:33

If you need identity info from the server after doing the login sequence (and that requires a cookie), then I'd probably use a mutation-merge and just return the info from the login

tony.kay21:03:15

The tempid remap WILL happen before the post mutations on the reads

tobias22:03:10

looking a bit further, I'm just trying to understand the ordering of things that happend with :post-mutation - I've setup a simple test case like this:

#(om/transact! this `[(untangled/load {:query [:something]
                                       :post-mutation test/pm1})
                      (untangled/load {:query [:something]
                                       :post-mutation test/pm2})
                      (untangled/load {:query [:something]
                                       :post-mutation test/pm3})])
Where test/pm1,test/pm2,test/pm3 just output a string to client consolt (pm1, pm2, pm3) - should I expect the output to be 'pm1','pm2','pm3' or is the ordering undefined?

tony.kay22:03:33

the order should be preserved

tony.kay22:03:56

but realize that the loads ALL run the networking part, and THEN run all 3 post mutations

tony.kay22:03:03

the networking is done before the first post mutation

tony.kay22:03:51

actually, you could confuse yourself with this particular example, since they all have the same query

tony.kay22:03:00

This is a special case

tony.kay22:03:54

The return value from the server has to be a map, and :something can only appear once in such a map, so Untangled should actually do these in perfect sequence as you were thinking (it must serialize them because otherwise they'd stomp on each others return values)

tony.kay22:03:13

change the queries to :a :b and :c, and you should only see one network requext

tony.kay22:03:02

@gardnervickers So, I've got a patch. I think this is right: :target really only applies on queries that load into the top-level (root)

tony.kay22:03:11

which simplified a bit of logic

tony.kay22:03:52

I added tests, and updated the relocation and marking code. I have not tested it on a live app, but I can push a snapshot

tony.kay22:03:05

I'll test live against that and would love to hear back from you

tony.kay22:03:58

0.8.1-SNAPSHOT on clojars. Testing for regressions against cookbook.

gardnervickers22:03:19

@tony.kay Fantastic, I’ll check that out on our app tonight.

tobias22:03:38

Okay, I now understand the network stuff and the way it's batched I think, the strange thing is that in 0.7.0 I see pm1,pm2,pm3 and in 0.8.0 I see pm3,pm2,pm1 - even with seperate queries BUT, I think I'm realising that in practice with the correct usage that this won't make a difference to the final outcome of the transaction

tony.kay23:03:11

Yeah, I was not aware the order changed. I did rework a bit on the network stack.

tony.kay23:03:34

I cannot think of what I did that would have revered them

tony.kay23:03:18

is that with out without the conflicts in naming?

tobias23:03:03

yes, reversed when the names are not conflicting

tony.kay23:03:48

hm, but not in 0.7.0, eh?

tony.kay23:03:02

it shouldn't matter, but I don't personally like things like that changing

tony.kay23:03:13

vectors imply order

tony.kay23:03:43

@gardnervickers It seems to check out for me. I also added a df/refresh! function for your specific case

tobias23:03:42

I'll run through the diffs and see if I can find anything

tony.kay23:03:15

could be in mark-loading

tony.kay23:03:20

I had to revamp that for the multiple remotes

tony.kay23:03:39

I you look at the actual network request, is it in the right order?

tony.kay23:03:54

i.e. is it just running the mutations in some arbitrary order?

tony.kay23:03:09

because the post-mutations will run in whatever order the responses are seen in, and the response is a map

tony.kay23:03:15

Your server code can affect that

tony.kay23:03:39

the post mutations go with the query. I don't think I changed anything. I think your response is causing the order of the post mutations

tobias23:03:08

@tony.kay git bisect says this is the first commit where it changes - bbae5ea568291c8589bf6dcae55863565b739a95 - Added support for multiple remotes - Updated defmutation to allow definition of multiple remotes interactions I don't have anything but a print on the server side, it's the ordering of the sends from the client that is being reversed

tony.kay23:03:39

thanks for doing the research. I'll look at it

tony.kay23:03:53

well, the item filtering down end up using a set, which doesnt have guaranteed order

tony.kay23:03:58

but that was how it was before

tony.kay23:03:59

the hashes of the data items could have changed, which would cause the set to reorder

tony.kay23:03:14

the data item markers in load tracking, that is

tony.kay23:03:17

which they would have

tony.kay23:03:53

So, technically this "broke" some time ago when the query collision avoidance was added