Fork me on GitHub
#graphql
<
2021-08-30
>
vlaaad08:08:44

I’m integrating superlifter with lacinia, is it possible to add some hook to lacinia query execution engine that will trigger a fetch? What I’m doing is… solving N+1 problem with superlifter, i.e. given a query for some field of a sequence of items, I enqueue to superlifter a bunch of requests for individual items, and then use short timer to trigger the fetch. This looks wasteful on resources. Ideally, I want lacinia to do something like that: 1. run top-level query resolver that returns a list of N shallow items 2. run N resolvers for a field of each shallow item 3. run a callback provided by me to trigger the fetch 4. await N promises that it has from #2 Is it possible to achieve?

oliy11:08:54

Hello, this is what the elastic trigger is for: 1. Your top level resolver resolves a lost of N shallow items, and creates an elastic bucket of size N 2. Your N child resolvers each put one item in the elastic bucket 3. When the elastic bucket has N items in it, the fetch will be triggered automatically by superlifter 4. Profit

oliy11:08:46

This is the primary use case I wanted to solve with superlifter

vlaaad12:08:34

woah, thanks!

vlaaad12:08:30

great, that’s exactly what I need

vlaaad12:08:41

🙏

🙏 1
hlship21:08:37

I have a vague idea about a relationship between Lacinia and ResolveResultPromise such that Lacinia can tell when processing is dead ended pending resolving of at least one promise; ideally there would be a way to notify a library such as Superlifter of this, so that it could automatically engage to resolve promises. The goal is to get some of the above behavior without knowing ahead of time what N is.

❤️ 1
hlship21:08:36

I believe the RI and the DataLoader library can do this because it can tell when it gets to the end of one event loop dispatch with outstanding work, and can tell DL to go resolve some stuff.

vlaaad07:08:48

@U076R6N1L I’m looking at your lacinia example, and one thing that concerns me is that resolver higher in hierarchy (resolve-pets) has to know about its children (resolve-pet-details).. what if I have many different children (e.g. resolve-pet-details, resolve-pet-age, resolve-pet-size) and whether they should be fetched or not depends on the query? It feels wrong to specify elastic bounds for all of the possible children..

vlaaad07:08:10

@U04VDKC4G any ideas where to add that in lacinia? I think it’s going to be much simpler if at some point in query execution I can just say to superlifter “go fetch everything”. I’m okay with maintaining a fork of lacinia, so it’s okay if the solution is “hackish”

oliy08:08:02

you can use the lacinia introspecting to look at the child selections. what i'm not sure about is if you can know if those child selections have their own resolvers or not, i haven't had time to fully investigate. i have this issue https://github.com/oliyh/superlifter/issues/16 describing a neater solution - essentially asking lacinia "how many times is resolver X going to be called" and then sizing the elastic bucket appropriately

oliy08:08:28

if you find a good way of doing this i'd be really interested to know, i'm happy to accept PRs for superlifter or to make additions

vlaaad09:08:58

I found another issue with elastic triggers that I’m not sure how to solve… Lacinia query execution is depth first. So, given a query like this:

query {
  concepts {
    related {
      type
    }
  }
}
where concepts and related are lists and type is batched with superlifter, execution will go like this: • resolve concepts list (e.g. 2 items) • resolve related list of 1st concept (e.g. 2 items) • resolve type of 1st related item of 1st concept (enqueue to superlifter) • resolve type of 2nd related item of 1st concept (enqueue to superlifter) — at this point superlifter will perform a fetch, because parent increased its bucket size to 2 and we enqueue 2nd item • resolve related list of 2nd concept (e.g. 2 more items) • resolve type of 1st related item of 2nd concept • resolve type of 2nd related item of 2nd concept — another fetch So this approach still has N+1 problem when dealing with nested collections queries…

vlaaad09:08:46

this probably can be solved with some record keeping of how many parents have already been resolved..

oliy09:08:17

i guess you have (N * M) + 2 here

oliy09:08:06

you can increment the elastic bucket size on each resolving of concepts or related

oliy09:08:41

there's a function update-trigger!

vlaaad09:08:06

yeah yeah, that’s what I’m figuring out..

oliy10:08:35

it feels like this could all become boilerplate in a list resolver, to count the list size, look ahead at child selections that need a bucket, and set up/increment that bucket

oliy10:08:15

or better, a helper in superlifter

hlship16:08:32

I'm not sure how to make Lacinia generate the desired events without breaking APIs and/or affecting performance.

oliy14:09:19

i guess if the client code provides lacinia with a callback to be called after each layer of resolvers is done would be a way to maintain performance for those not using it, and only grow the api in a minimal way