Fork me on GitHub
#core-async
<
2016-10-05
>
josh.freckleton19:10:56

How can I achieve a sort of monadic async, with probably a reader monad? For ex: I have objects I want to put on the channel, and meta data about the object that I want to pass through a bunch of channels. I would like to not need "plumbing" on each of my functions that operate on the channels, eg stuff that understands the meta-data. I want those fns to only understand the object they're dealing with, and pass the meta-data unaffected (unless I choose to side-effect into the reader's "log")

josh.freckleton20:10:00

Does this make sense?

josh.freckleton20:10:35

My specific use case is a URL that comes from a DB, get's scraped, then css-selected, and then it's meta-data understands where in a DB it came from, and needs to go in another DB.

angusiguess21:10:36

@josh.freckleton This sounds like a job for

pipeline

angusiguess21:10:58

You write functions that operate on individual values, and then use them as a transducer in a pipeline.

josh.freckleton21:10:38

@angusiguess ah, of course there is more infrastructure that I should learn before cracking out my own crazy ideas 😉

josh.freckleton21:10:49

would you happen to have any resources you'd recomment I read?

josh.freckleton21:10:42

I'll enjoy working through this!

angusiguess21:10:01

You’re welcome, I hope it meets your needs!

andreas-thoelke21:10:01

Hi! I'm running into an issue using core.async/pipe. Based on the doc string it Will stop consuming the from channel if the to channel closes However, if I close the to-channel like this:

andreas-thoelke21:10:29

It seems the to-ch is still taking one final item from from-ch. So the problem is, that after a close of to-channel any next value put into from-chan will be lost.

val_waeselynck21:10:27

@andreas-thoelke I'm not surprised that an 'in-flight' element between from-ch and to-ch gets lost

val_waeselynck21:10:09

I think the only thing you can count on is that if an element is added after the call to close! it won't be consumed

val_waeselynck21:10:03

or is that what's happening ?

andreas-thoelke21:10:09

but it does! see above!

andreas-thoelke21:10:46

this really causes problems, I'll loose that value (42).

val_waeselynck21:10:24

for this sort of find-grained coordination, you'll probably want to write your own pipe-like algorithm then

angusiguess21:10:48

Yeah so it looks like pipe consumes from from but doesn’t bail until it can’t put on to.

val_waeselynck21:10:57

not sure if you're requirement is possible actually

val_waeselynck21:10:43

can't atomically check if to-chan is not closed, take from from-chan, and put to to-chan

angusiguess21:10:15

You could consume a value and put it back on.

andreas-thoelke21:10:19

@angusiguess yes, basically I have to do another put that just vanishes. it's strange behavior to me. can't see how this is helpful or in line with what the doc string says

angusiguess21:10:21

But ordering would get weird.

val_waeselynck21:10:09

to me the deep issue is the race condition around close!

val_waeselynck21:10:21

my way to go would be to have an isolated process that can either A- put an element in the destination channel or B- receive a command to change the destination channel

val_waeselynck21:10:36

like a kind of switch

angusiguess21:10:54

So we can sort of do that with alt

angusiguess21:10:36

But it’s not clear how to use alt without still consuming a value.

val_waeselynck21:10:12

if you need to ensure ordering of the add-element and stop-processing operations, I think these 2 will have to go in the same channel

val_waeselynck21:10:54

but you need to check that you do have this requirement - doesn't feel very queue-ish to me

andreas-thoelke22:10:43

Thank you for the tips @angusiguess and @val_waeselynck ! I will try to figure something out along those lines.