Fork me on GitHub
#core-async
<
2016-09-02
>
flipmokid09:09:31

Hi all, is there any way of telling pipeline how to distribute inputs inside the pipeline? Basically I have a pipeline where I want to make sure that items with a specific key always go to same thread/go block to make sure they're not processed out of order

Alex Miller (Clojure team)12:09:22

Other than using 1 thread :)

Alex Miller (Clojure team)12:09:55

The results are guaranteed to be in the same order as the inputs but that doesn't say anything abut the processing order

flipmokid12:09:05

Okay thanks for that Alex

hlolli20:09:38

Glad to know there's a special core-async group. A bit basic question, is it possible to do a parking take. There is to say, I will recur the loop when I've put something on a channel, but at another channel I will take from within the body, only if there's something to take. I tought about timeout, but in my case I can't really lose a millisecond, is this possible at all?

hlolli20:09:09

I dont have any example, I'm temted to use atom (dereference it in the go-loop body), but I'm afraid it will hit me back in the long run.

Alex Miller (Clojure team)20:09:45

have you looked at offer! and poll! ?

Alex Miller (Clojure team)20:09:06

they give you the “do if you can but return if you can’t” behavior

hiredman20:09:07

I don't entirely follow what you are trying to do, but the pair of <! and !> both park inside the go macro

hlolli20:09:08

no! complete foreign concepts to me.

hiredman20:09:33

!> being put, and '<!' being take

hlolli20:09:20

nice! offer rhimes with what I want. @hiredman doesnt go macro return a channel that I would in the end have to block to take out of?

hiredman20:09:34

the go macro returns a channel that will have the result of the expression put in it, but you don't have to use that channel at all

hlolli20:09:04

My idea was to use the recursion to take care of my state problem. I can see a go macro within a go-loop useful in case of side effects ie. dont care about the return value, but I may be wrong, just thinking out loud to be proven wrong 🙂

hiredman20:09:03

I think offer! makes what you asked a little more clear to me. you have a producer writing to a channel, and you have a consumer reading from the channel, and you don't want the producer to be blocked writing to the channel if the consumer is running behind the producer?

hlolli20:09:42

Good point I should have mentioned, the consumer cant be blocked (actually its a audio signal, if it would the audio could stop). So I basically want to send commands to audio signal in a go-loop (44,1k per second) without blocking even for a ms. Reading the clojure docs

poll! Takes a val from port if it's possible to do so immediately.
 Never blocks. Returns value if successful, nil otherwise.
sounds like what I want.

hiredman20:09:41

ah, so I have it backwards, the producer can be blocked, but the consumer cannot be

hlolli21:09:51

if a go-loop is running constantly for a long time, would it help performance wise in putting it in async/thread?

tanzoniteblack21:09:40

@hlolli the go-loop is running in a fixed thread pool (presuming clojure not clojurescript), the only time “performance”-wise it should matter is if you are actually having work trying to be assigned to all of the threads in that thread pool from other go blocks, since a long running task would have it take that thread and prevent other work from being done. If you do want your long running stuff to happen in addition to anything else being fired in a go block, than you can not use go-loop at all and just do a recur on a thread (created via async/thread or a future) and let the OS assign when to switch work from that thread to other things coming up

hlolli21:09:39

ah ok, I can see when that can be useful, as well as I can see that thread is not what I want for my usecase. thanks for a good reply @tanzoniteblack.

tanzoniteblack21:09:51

I’m actually kind of curious to see how what you’re working on goes; I’ve never considered using core.async for something that needs as low latency as audio processing

tanzoniteblack22:09:55

actually that’s more of a statement about clojure/jvm and low latency operations than core.async

tanzoniteblack22:09:26

though I know https://github.com/overtone/overtone/ has done pretty well, but I kind of assumed most of the actually sound code was in the supercollider side of things

hlolli22:09:13

yes, I started with supercollider then overtone and then I did this https://github.com/hlolli/panaeolus now I've worked full time as clojure developer for over half a year and I want to rewrite it again. Im actually only working with control signals not actually 44,1k per second.

tanzoniteblack22:09:16

🙂 I was just reading through that (useful that you re-use hlolli on github)

hlolli22:09:27

I imagine using more complex algorithms for each musical pattern, allowing the audio engine to do its job while each pattern calculates in async. As well as leaving this declarative programming.

hlolli22:09:52

ok, what an unlikely thing that you would read trough that, haven't "marketed" it at all.

tanzoniteblack22:09:30

not unlikely at all! I went and searched for it explicitly based off this conversation. Like I said, I was curious

hlolli22:09:28

ahh ok I see, you looked up hlolli, thought you had seen it before, understand.