Fork me on GitHub
#pathom
<
2019-01-01
>
Alex K18:01:53

I'm having a little issue with "optional" keys and computed keys in separate resolvers. I have entities in Datomic that sometimes never have an attribute, and entities that acquire one or more attributes later on when the user provides them. I've been querying for them anyway, using the elide-not-found plugin with a bit of conditional logic on the client, and everything was working ok. Then I added another resolver that takes an optional attribute as one of its keys. I was expecting it would only be called when this attribute existed in the parsing context (when the key's value is something other than ::p/not-found), but it's getting called anyway. The parser returns a load of reader errors because of "Insufficient resolver input". I saw in the docs a section about dependent attributes. Am I supposed to be using something like that in this situation? Interestingly, this also seems to be breaking something with the parallel parser/reader. Fulcro was complaining client-side about the Transit response (unexpected end of JSON or something like that), so I tapped the result of the parser to see what was going on. Some of the :com.wsscode.pathom.core/errors had a NullPointerException where there was exception information was for others. The number and position of these changed with each call. I replaced the parallel-parser with parser and parallel-reader with reader2, and the problem went away.

wilkerlucio18:01:00

hello, first lets talk about the transit error, the problem is encoding there, some error happened and was serialized as an error, you can fix that by supportign some default write handlers on the transit encoding, or changing how the error is processed, try adding to your environment: ::p/process-error p/error-str

wilkerlucio18:01:15

this should convert the error to text, less data but easier to transmit

Alex K18:01:34

Hmm. Adding that is giving Exception in thread "async-dispatch-6" clojure.lang.ArityException: Wrong number of args (2) passed to: com.wsscode.pathom.core/error-str

wilkerlucio18:01:42

about the optional thing, that's a feature on the roadmap, currently all inputs are always required, the idea will be to mark parts as optional, currently a work around is to prefill the data with nils, for example, if you are loading one person entity with the fields [:person/name :person/age], you can have a resolver like:

clojure
(pc/defresolver person-resolver [_ _]
  {::pc/input  #{:person/id}
   ::pc/output [:person/name :person/age]}
  (let [person (pull-person ...)]
    (merge
      {:person/name nil :person/age nil}
      person)))
I opened the issue to track the optional attributes: https://github.com/wilkerlucio/pathom/issues/70

wilkerlucio18:01:37

sorry. try as: ::p/process-error #(p/error-str %2)

Alex K18:01:19

Cool, that makes sense, thanks. Rich Hickey's latest Conj talk "Maybe Not" might be of interest. It seems he's changing spec with regards to optional attributes, breaking it out into "selectors" with a context. The gist is that attributes will be optional only at certain times, in certain contexts. So instead of required and optional keys in the specs, you use selectors to say what's needed at usage time. When I saw the slides for that, I immediately thought of EQL. Anyway, the correction to process-error worked, but unfortunately I'm still getting a broken Transit decode on the client. Unexpected end of JSON input.

wilkerlucio18:01:50

did you kept your parser change? not sure if you did one extra bit that is the parallel parser is async, so it returns a chan, the regular is sync, is that part ok?

wilkerlucio18:01:41

about the selectors thing, in pathom case the input already is the selector part, so the options would be mark on it or have a second selection to mark part of the original selection as optional

Alex K18:01:45

That makes sense. I switched the parser back to parallel-parser and parallel-reader. I'm running the parser in Pedestal, so I'm using a blocking take with the parallel-parser, and took that out when I tried with parser2. So yeah, the blocking take is back in to get the parser response from the chan.

wilkerlucio18:01:42

and still having the transit issue?

wilkerlucio18:01:20

had you tried to read the transit output to check if it looks ok?

Alex K18:01:15

Yeah, the transit output is definitely malformed. It seems to stop abruptly somewhere in the middle of the :com.wsscode.pathom.core/errors section. That's why I wondered if those NullPointerExceptions were responsible.

wilkerlucio18:01:03

thats quite strange, did you checked the data before encoding to see if there is something strange? how are you seeing those NullPointerExceptions?

Alex K18:01:27

I was using the new tap> functionality in Clojure 1.10 (with Puget as pretty printer) to dump the result of calling the parser. The NP exceptions were showing up there in the errors section of the parser output.

wilkerlucio19:01:52

can you share that part of the output with me? I'm trying to understand from where those are coming from

Alex K19:01:40

Yeah, no problem. I'm just trying to see if the cutting-off point in transit coincides with a NullPointerException.

Alex K19:01:51

Looking at the transit, I can't find any correlation between stop point and those null pointers (it's really hard to read), however it is only malformed when the NP exceptions are in the output. Otherwise it comes through fine.

Alex K19:01:21

If it would help I can try to put together a minimal working example of the error?

wilkerlucio19:01:15

sure, if you can do that I can take a look and try to figure with you

Alex K20:01:50

This is frustrating. I can't get it to reproduce with a minimal example. I could zip up my project and send it your way if you'd like to try to chase this down, but I'd hate to waste more of your time if this is down to something weird that I've done.

Alex K21:01:20

Uh. Thought I'd caught a break, but not quite. I just implemented the workaround you suggested to nil the optional attributes, and now every error is giving back a NullPointerException every time. However applying that to my minimal example, that works as it should. I must have done something odd, and I'm not seeing it. I'll come back to it tomorrow with fresh eyes and hopefully figure this thing out. At least this is more information. Thanks for your help!

Alex K16:01:33

Hi again. So today I've been looking into this more, and I think there's something funky going on in Pathom's async error handling code when using the parallel-parser/reader. I noticed if I remove p/error-handler-plugin from the parser, things work ok. It's only with that plugin enabled that those NullPointerExceptions show up. I've been through the Pathom code trying to figure out how all that stuff works. I think I have the gist of what's happening when my resolver is being called, but the details are over my head. I don't know how you managed to write all that, haha. I also figured out why nilling the optional attribute meant a reader-error every time. I forgot to adjust the second resolver to deal with the nil input from the first, so that was causing a second error. D'oh! So unfortunately I still don't know how to make a minimal example that demonstrates the problem, so my example case is still my whole project. Should it help any, I have traces from an entity with the required inputs (resolver works), and another from the broken case where the required input is missing and the resolver fails.