Fork me on GitHub
#aws
<
2020-02-14
>
Burin16:02:16

Hi I trying to stream the result from aws s3 and process it. What I am trying is something like

(def s3-result (aws/invoke client {:op :GetObject 
                    :request {:Bucket "my-bucket"
                              :Key "path/to/some/big-file.csv"
                              :Range "bytes=0-30000" ;; just an example
                              }))

;; I want to process the file as it come in line-by-line without holding the things in memory

(-> s3-result 
    :Body)
;;=> this is of the type .BufferedInputStream
I know that the result is available in the :Body after the request. Now does anyone know how to do this lazily?

Burin16:02:45

@U0ENYLGTA Thanks for the clarification.

dchelimsky16:02:14

You're welcome.

dchelimsky16:02:28

We do plan to support "bring your own http-client" in the future, but there is no timeline for that.

👍 4
dchelimsky19:02:50

[ANN] Cognitect Labs' aws-api 0.8.437 is now available! https://groups.google.com/forum/#!topic/clojure/AIQzOkhUDBU

souenzzo20:02:43

!Ref "#off-topic"