Fork me on GitHub

Is there a trivial way to wrap an SQS queue with a core.async channel? There are various libraries that do that, but either are receive-only (squeedo) or are archived (fink-nottle).


I'd like to use the Cognitect AWS API if possible but I believe it's HTTP client is synchronous, so I can't just use a go-loop, I need to think about threads and executors etc.

Joe Lane13:09:24

My understanding is that aws-api is both sync and async


that's right


There's no streaming AFAIK but aws-api has both sync and async.


After further digging in the library I see that the sync client wraps the async one - but there’s no detail on how the underlying http library handles things. Is there a threadpool involved, is it truly async etc?

Joe Lane14:09:51

Can you define "truly async"?


Async IO eg with using callbacks vs non-blocking IO where the IO takes place in a (new) background thread.


Both have their issues when overloaded, eg if you try to put 10000 messages in a queue, you will either overload your CPU with pending callbacks etc or blow up your memory with 10000 threads.


I’m trying to write to SQS which in theory has infinite capacity, so I don’t worry about overloading it, only my own system :)

Alex Miller (Clojure team)16:09:23

the http lib source is in the jar I believe