Fork me on GitHub
#pathom
<
2020-02-06
>
ak-coram13:02:26

is there a way to limit concurrency for the parallel-parser? a large query I have immediately calls a single resolver 900 times. processing in the resolver is limited to a number of threads so most of the calls end up timing out.

kszabo14:02:02

you should probably write a batch resolver then

ak-coram14:02:05

it depends on what query I run though, so I'm not sure I want to make resolver adjustments for individual queries

ak-coram14:02:29

also it doesn't feel right to control the concurrency in individual resolvers

ak-coram14:02:49

another issue is that my resolver is already a batch resolver, it just gets called for 900 batches

ak-coram14:02:04

I've tried increasing ::pp/key-process-timeout , but individual resolver calls seem to time out regardless

wilkerlucio14:02:22

@ak407 is this related to the changes you did? can you send some smaller example that demonstrates the problem you are facing?

ak-coram15:02:16

@wilkerlucio I'm fairly sure it doesn't relate to my PR since it's also happening with non-batch resolvers. I'll try to create a small example, but I'll probably only get to it on the weekend.

ak-coram15:02:42

I think I have lots of batches because of my joins: a future parser could probably optimize this too and prepare a single batch across multiple levels of joins (this probably isn't always the right thing to do for performance though)

ak-coram16:02:32

maybe the resolver could even hint at some optimal batch size

kenny17:02:55

Does Pathom support returning an inputstream from a resolver?

kszabo17:02:58

the values of the attributes are up to you

wilkerlucio19:02:54

@kenny you would have a problem if you try to cross it on a boundary, but if you want to get it and use on the same process, it can work

wilkerlucio19:02:13

but that would just be a value as a input stream, pathom will not do anything special about it