Fork me on GitHub
#aws
<
2019-09-04
>
orestis07:09:47

Is there a trivial way to wrap an SQS queue with a core.async channel? There are various libraries that do that, but either are receive-only (squeedo) or are archived (fink-nottle).

orestis07:09:44

I'd like to use the Cognitect AWS API if possible but I believe it's HTTP client is synchronous, so I can't just use a go-loop, I need to think about threads and executors etc.

Joe Lane13:09:24

My understanding is that aws-api is both sync and async

ghadi13:09:53

that's right

jaihindhreddy14:09:30

There's no streaming AFAIK but aws-api has both sync and async.

orestis14:09:00

After further digging in the library I see that the sync client wraps the async one - but there’s no detail on how the underlying http library handles things. Is there a threadpool involved, is it truly async etc?

Joe Lane14:09:51

Can you define "truly async"?

orestis14:09:11

Async IO eg with using callbacks vs non-blocking IO where the IO takes place in a (new) background thread.

orestis14:09:29

Both have their issues when overloaded, eg if you try to put 10000 messages in a queue, you will either overload your CPU with pending callbacks etc or blow up your memory with 10000 threads.

orestis15:09:38

I’m trying to write to SQS which in theory has infinite capacity, so I don’t worry about overloading it, only my own system :)

Alex Miller (Clojure team)16:09:23

the http lib source is in the jar I believe