Fork me on GitHub
#core-async
<
2016-02-28
>
mccraigmccraig00:02:47

alet aka applicative-do syntax has this problem nailed... i haven't tried it @johanatan , but you should be able to use it with core.async via https://github.com/funcool/cats/blob/master/src/cats/labs/channel.cljc since the context implements the applicative protocol. promises (i.e. promesa) might be preferable though, since they have inbuilt error-handling, and the cats either-t transformer won't implement applicative even if the underlying context does

mccraigmccraig00:02:50

hmm - looks like http://dev.clojure.org/jira/browse/ASYNC-103 might work well with alet tho i think you would have to implement a new cats context for promise-chans if you want short-circuit behaviour on error

mccraigmccraig00:02:55

for those who haven't seen it - the alet macro identifies the dependency graph in a set of bind statements and executes statements in 'parallel' where it can - it's pretty neat

johanatan00:02:03

@mccraigmccraig: I'd like to do this with a single alet rather than an outer one with one inner one for each object. But the number of objects is dynamic. So, the alet bindings might have to be generated from a macro. Do you see any other way to handle a dynamic number of objects?

johanatan00:02:00

@hugesandwich: @jgdavey : there are of course probably ways to compose the components of core.async but a) it isn't immediately obvious how (hence my asking the question in the first place) and b) that's a lot messier/more code than just using alet.

johanatan00:02:39

so, alet (whether nested or a single one) gets my vote for most elegant way to approach this problem.

johanatan01:02:52

@mccraigmccraig: i think promesa will work best here as the operations are all existing procedures which only do some side-effects (i.e., make a web request, take the result and swap! some data into an atom). wrapping this in a promesa doing a simple resolve(true) on success or failure(whatever) on failure is easier than creating new channels etc.

hugesandwich02:02:30

@johanatan: alet sounds nice. I personally prefer to both be explicit and avoid non-core macros where possible. I think I'd use alet for smaller apps, one-offs, and quick things. My real-world experience with core.async is a lot of the out-of-the-box stuff, including other libraries and macros go out the window when you need a lot of control. It makes sense as no one can really envision your usage pattern. Of course, that's how you end up writing one of these libraries in the first place. More specifically, in my case I needed a lot of control over priorities and mimicing a lot of the behaviors in a pre-emptive multi-tasking scheduler + flow control + distributed coordination. Obviously core.async doesn't provide everything for me there, but it certainly helps for a lot of the glue and enforcing back pressure, etc. I'm doing a lot of this both server and client-side to control streams and multiple processes. I don't find it a lot of code to build most of it as the primitives on both the jvm and core.async are there. I find the easier my async code is, the less bugs I have. When you get too clever, subtle bugs tend to appear in a lot of code, which is one reason I like CSP.

hugesandwich02:02:18

some of what you describe sounds very close to what I've seen in the past in C#'s async libs and community libs, along with some things in go

hugesandwich02:02:16

https://msdn.microsoft.com/en-us/library/dd537609(v=vs.110).aspx among many other examples. It's been a few years since I was writing any code in that world, but most of these libraries make it very transparent and trivial. You could probably steal some ideas if you're stuck.

johanatan03:02:46

@hugesandwich: alet really is exactly what I was looking for and, if it hadn't existed, I would've created it or something like it (as I prefer nice, clean, high-level abstractions).

johanatan03:02:30

Thanks for sharing your opinion though.

mccraigmccraig11:02:38

@johanatan: have a look at fmap, fapply, join and alet itself in https://github.com/funcool/cats/blob/master/src/cats/core.cljc ... it's all based on the functor, applicative and monad concepts, which compose nicely

akiel16:02:42

Is there a way to easily convert a future or promise into a channel?

mccraigmccraig19:02:38

can you use a promise lib which supports callbacks @akiel , such as manifold ?

akiel19:02:38

@mccraigmccraig: thanks I will have a look

mccraigmccraig19:02:09

@akiel or indeed a promise-chan !

akiel19:02:39

No a promise-chan is just a chan which accepts only one value. I could use one with a callback but I don't see how I can provide a callback to a normal future

mccraigmccraig20:02:10

i meant skip the promise altogether and replace with a promise-chan, but i don't know what you are trying to achieve so that may be no use

mccraigmccraig20:02:11

iirc @akiel you can use manifold/chain with your future and it will coerce it in to a Deferred, presumably allocating a thread to block, and allowing the callback to be registered... https://github.com/ztellman/manifold/blob/master/docs/deferred.md

akiel20:02:59

@mccraigmccraig: I just try to park on Datomics transact-async. I don’t like to allocate a real thread for every transact-async call.

akiel20:02:00

just (async/thread @(d/transact-async conn …)) would do it but with a real thread.

akiel20:02:01

But I may even use real threads on the end if I find a way to constrain the number of concurrent transact-async calls.

val_waeselynck20:02:02

@akiel: transact-async returns datomic.ListenableFutures, which lets you register a call back on completion (you need to provide the Executor to manage that though)

akiel20:02:08

@val_waeselynck: Thanks. It works but I don’t see any documentation on the Clojure side.

akiel20:02:57

And it uses a thread anyway.

val_waeselynck20:02:31

@akiel: I don't think it blocks actually

akiel20:02:51

It doesn’t block but it uses a thread from the executors pool to block

akiel20:02:36

Anyway the real solution will be a constraint pool of transaction workers which are feeded by a channel. Maybe even with async/pipeline-blocking but it works in order and I’m not sure if the transactions return in order.

johanatan23:02:06

@mccraigmccraig: I'm finding that cats and promesa do not interoperate as the cats alet example would seem to indicate. Getting the error: Error: No context is set and it can not be automatically resolved.

johanatan23:02:20

was there something else required for cats to see promesa's promises as monadic contexts that it knows about?