How can I achieve a sort of monadic async, with probably a reader monad? For ex: I have objects I want to put on the channel, and meta data about the object that I want to pass through a bunch of channels. I would like to not need "plumbing" on each of my functions that operate on the channels, eg stuff that understands the meta-data. I want those fns to only understand the object they're dealing with, and pass the meta-data unaffected (unless I choose to side-effect into the reader's "log")

Does this make sense?

My specific use case is a URL that comes from a DB, get's scraped, then css-selected, and then it's meta-data understands where in a DB it came from, and needs to go in another DB.

@josh.freckleton This sounds like a job for

pipeline

You write functions that operate on individual values, and then use them as a transducer in a pipeline.

@angusiguess ah, of course there is more infrastructure that I should learn before cracking out my own crazy ideas 😉

would you happen to have any resources you'd recomment I read?

@josh.freckleton Here’s a piece I wrote on pipelines a little while ago: http://blog.goose.haus/2016/04/30/you-would-be-downright-amazed-at-what-i-can-do-with-just-a-pipeline.html

thanks!

I'll enjoy working through this!

You’re welcome, I hope it meets your needs!

Hi! I'm running into an issue using core.async/pipe. Based on the doc string it Will stop consuming the from channel if the to channel closes However, if I close the to-channel like this:

It seems the to-ch is still taking one final item from from-ch. So the problem is, that after a close of to-channel any next value put into from-chan will be lost.

Ah

@andreas-thoelke I'm not surprised that an 'in-flight' element between from-ch and to-ch gets lost

I think the only thing you can count on is that if an element is added after the call to close! it won't be consumed

or is that what's happening ?

but it does! see above!

huh

this really causes problems, I'll loose that value (42).

for this sort of find-grained coordination, you'll probably want to write your own pipe-like algorithm then

hmm, good point

Yeah so it looks like pipe consumes from from but doesn’t bail until it can’t put on to.

not sure if you're requirement is possible actually

can't atomically check if to-chan is not closed, take from from-chan, and put to to-chan

You could consume a value and put it back on.

@angusiguess yes, basically I have to do another put that just vanishes. it's strange behavior to me. can't see how this is helpful or in line with what the doc string says

But ordering would get weird.

to me the deep issue is the race condition around close!

my way to go would be to have an isolated process that can either A- put an element in the destination channel or B- receive a command to change the destination channel

like a kind of switch

So we can sort of do that with alt

sure

But it’s not clear how to use alt without still consuming a value.

if you need to ensure ordering of the add-element and stop-processing operations, I think these 2 will have to go in the same channel

but you need to check that you do have this requirement - doesn't feel very queue-ish to me

Thank you for the tips @angusiguess and @val_waeselynck ! I will try to figure something out along those lines.