Fork me on GitHub
#yada
<
2017-08-30
>
stijn11:08:55

the documentation on large request bodies mentions an S3 bucket as a container https://juxt.pro/yada/manual/index.html#capturing-large-request-bodies

stijn11:08:15

does anyone have any example of this?

mccraigmccraig11:08:31

we stream request bodies to a local tempfile before sending to S3 @stijn iirc because the amazon S3 client makes it easier to deal with files

stijn11:08:55

@mccraigmccraig ok, that seems the easiest solution. Do you use multipart or binary data as the body?

mccraigmccraig11:08:04

i can show you our code for the tempfile handling if that would help ?

stijn11:08:55

I'd like to see the multipart handling yes. The docs don't have anything on that right?

mccraigmccraig11:08:03

ha, don't know now - it's been a while since i did it - i don't think there were any docs then

mccraigmccraig12:08:22

iirc you have to specify content-length for that - and we weren't always getting it, so it was easier to stream to a file then upload from there

stijn12:08:18

i think that's correct yes

stijn12:08:33

thanks, very interesting

aengelberg18:08:45

Is there a way to override the list of interceptors in a request so that I'm not using certain default Yada interceptors?

aengelberg18:08:12

I want to write a handler that does not automatically decode the input data when Content-Encoding is set to gzip

stijn18:08:55

@mccraigmccraig thanks for the hints. turns out it's easier for us on the client to upload every image separately as a binary. makes it a lot easier to process in yada too: https://gist.github.com/stijnopheide/7cf154f0fc7a7dd7ad7487a9ea0473e5

dominicm20:08:56

@aengelberg Yep. You can access the interceptors list for a resource under the :interceptors key after doing yada/resource on it.

danielcompton21:08:57

@aengelberg I have a few utils for doing stuff like that over many resources

(defn update-resources [routes f & args]
  (walk/postwalk
    (fn [x]
      (if (or (instance? Resource x)
              (instance? Handler x))
        (resource (apply f x args))
        x))
    routes))

danielcompton21:08:18

and then

(update-resources
            yada.handler/insert-interceptor i/invoke-method
            add-query
            add-command)

danielcompton21:08:47

I wrote remove-interceptors to pull stuff out which might be useful:

(defn remove-interceptors [res & interceptors]
  (let [interceptor-set (set interceptors)]
    (update res :interceptor-chain
            (partial remove (fn [i]
                              (contains? interceptor-set i))))))