help me understand the relationship between core.async go blocks and threads

it’s my understanding that a go expression is immediately run on some internal core.async thread pool

yes?

and it will be parked and resumed as necessary but not on any particular thread

we’re actually trying to parallelize some Amazon S3 requests

@jcromartie: my understanding is that you should not use go blocks for blocking I/O, which I think S3 requests would qualify as; maybe use thread instead

yeah

it still feels a little naive to just blindly kick off a thread to do potentially long blocking I/O

we might have 2000 downloads

at a time

@jcromartie: afaics the AWS client supports asynchronous access - http://aws.amazon.com/articles/5496117154196801 - so you should be able to use it with core.async, or you might look into manifold which can straightforwardly adapt java futures

thanks

yeah the AWS client has some neat features

we want to go straight to memory not to disk though

but yeah

lots of good stuff there, thanks

core.async/thread doesn't spawn a thread

*necessarily. There is a pool

it just means try to run this function in the pool of threads made for real blocking

but that pool is not fixed size

so it will either spawn a thread or use one that someone else just finished with

which one is more suitable for handling db and request/response related operations? core.async, or pulsar? i heard someone argue that core.async is not built for IO read/write purposes, and that Pulsar is more suitable.

however i don’t know the underlying difference in how they achieve lightweight threads