graphql

vlaaad 2021-08-30T08:42:44.005900Z

I’m integrating superlifter with lacinia, is it possible to add some hook to lacinia query execution engine that will trigger a fetch? What I’m doing is… solving N+1 problem with superlifter, i.e. given a query for some field of a sequence of items, I enqueue to superlifter a bunch of requests for individual items, and then use short timer to trigger the fetch. This looks wasteful on resources. Ideally, I want lacinia to do something like that: 1. run top-level query resolver that returns a list of N shallow items 2. run N resolvers for a field of each shallow item 3. run a callback provided by me to trigger the fetch 4. await N promises that it has from #2 Is it possible to achieve?

vlaaad 2021-08-31T07:21:48.008Z

@oliy I’m looking at your lacinia example, and one thing that concerns me is that resolver higher in hierarchy (resolve-pets) has to know about its children (resolve-pet-details).. what if I have many different children (e.g. resolve-pet-details, resolve-pet-age, resolve-pet-size) and whether they should be fetched or not depends on the query? It feels wrong to specify elastic bounds for all of the possible children..

vlaaad 2021-08-31T07:24:10.008200Z

@hlship any ideas where to add that in lacinia? I think it’s going to be much simpler if at some point in query execution I can just say to superlifter “go fetch everything”. I’m okay with maintaining a fork of lacinia, so it’s okay if the solution is “hackish”

2021-08-31T08:26:02.008500Z

you can use the lacinia introspecting to look at the child selections. what i'm not sure about is if you can know if those child selections have their own resolvers or not, i haven't had time to fully investigate. i have this issue https://github.com/oliyh/superlifter/issues/16 describing a neater solution - essentially asking lacinia "how many times is resolver X going to be called" and then sizing the elastic bucket appropriately

2021-08-31T08:33:28.008800Z

if you find a good way of doing this i'd be really interested to know, i'm happy to accept PRs for superlifter or to make additions

vlaaad 2021-08-31T09:17:58.009Z

I found another issue with elastic triggers that I’m not sure how to solve… Lacinia query execution is depth first. So, given a query like this:

query {
  concepts {
    related {
      type
    }
  }
}
where concepts and related are lists and type is batched with superlifter, execution will go like this: • resolve concepts list (e.g. 2 items) • resolve related list of 1st concept (e.g. 2 items) • resolve type of 1st related item of 1st concept (enqueue to superlifter) • resolve type of 2nd related item of 1st concept (enqueue to superlifter) — at this point superlifter will perform a fetch, because parent increased its bucket size to 2 and we enqueue 2nd item • resolve related list of 2nd concept (e.g. 2 more items) • resolve type of 1st related item of 2nd concept • resolve type of 2nd related item of 2nd concept — another fetch So this approach still has N+1 problem when dealing with nested collections queries…

vlaaad 2021-08-31T09:20:46.009200Z

this probably can be solved with some record keeping of how many parents have already been resolved..

2021-08-31T09:37:17.009400Z

i guess you have (N * M) + 2 here

2021-08-31T09:38:06.009600Z

you can increment the elastic bucket size on each resolving of concepts or related

2021-08-31T09:38:41.009800Z

there's a function update-trigger!

vlaaad 2021-08-31T09:52:06.010Z

yeah yeah, that’s what I’m figuring out..

2021-08-31T10:08:35.010200Z

it feels like this could all become boilerplate in a list resolver, to count the list size, look ahead at child selections that need a bucket, and set up/increment that bucket

2021-08-31T10:10:15.010400Z

or better, a helper in superlifter

hlship 2021-08-31T16:32:32.010700Z

I'm not sure how to make Lacinia generate the desired events without breaking APIs and/or affecting performance.

2021-08-30T11:34:54.006100Z

Hello, this is what the elastic trigger is for: 1. Your top level resolver resolves a lost of N shallow items, and creates an elastic bucket of size N 2. Your N child resolvers each put one item in the elastic bucket 3. When the elastic bucket has N items in it, the fetch will be triggered automatically by superlifter 4. Profit

2021-08-30T11:35:29.006300Z

Here is an example of this: https://github.com/oliyh/superlifter#lacinia-usage

2021-08-30T11:35:46.006600Z

This is the primary use case I wanted to solve with superlifter

vlaaad 2021-08-30T12:05:34.006800Z

woah, thanks!

vlaaad 2021-08-30T12:06:30.007Z

great, that’s exactly what I need

vlaaad 2021-08-30T12:06:41.007200Z

🙏

🙏 1
hlship 2021-08-30T21:45:37.007500Z

I have a vague idea about a relationship between Lacinia and ResolveResultPromise such that Lacinia can tell when processing is dead ended pending resolving of at least one promise; ideally there would be a way to notify a library such as Superlifter of this, so that it could automatically engage to resolve promises. The goal is to get some of the above behavior without knowing ahead of time what N is.

❤️ 1
hlship 2021-08-30T21:46:36.007700Z

I believe the RI and the DataLoader library can do this because it can tell when it gets to the end of one event loop dispatch with outstanding work, and can tell DL to go resolve some stuff.

2021-09-01T14:07:19.010900Z

i guess if the client code provides lacinia with a callback to be called after each layer of resolvers is done would be a way to maintain performance for those not using it, and only grow the api in a minimal way