Fork me on GitHub
#core-async
<
2020-11-15
>
holyjak08:11:42

Thanks a lot to everybody who advised me on improving my function to fetch blog posts from a paginated API. I have applied your idea to use a dedicated stop-signal instead of just closing the output dst channel, which simplified the code, and @hiredman’s excellent idea to factor out the looping by creating an async version of core iterate (it took me a while to understand it fully and I ended up writing a less flexible but easier for me to understand version of it). Here is the original async-fetch-all-postshttps://github.com/holyjak/clj_tumblr_summarizer/blob/c1ca0ea37d2a6749466059bbf957647f5f989004/src/clj_tumblr_summarizer/input.clj#L49 combining async/thread, try, loop, let, when & if and here is the refactored and renamed fetch-posts-async!https://github.com/holyjak/clj_tumblr_summarizer/blob/e03395a0cd22ad52130e22d7b550561c35250f14/src/clj_tumblr_summarizer/input.clj#L67 (Here https://github.com/holyjak/clj_tumblr_summarizer/blob/e03395a0cd22ad52130e22d7b550561c35250f14/src/clj_tumblr_summarizer/input.clj#L50 I start the process producing pages of data and here is the factored-out async-iteratehttps://github.com/holyjak/clj_tumblr_summarizer/blob/e03395a0cd22ad52130e22d7b550561c35250f14/src/clj_tumblr_summarizer/async_utils.clj#L20). If you have any comments, they would be of course most welcome. It is more code now but simpler (not easier :)), I think.