Fork me on GitHub
#core-async
<
2016-10-10
>
josh.freckleton03:10:35

i'm running this in emacs using CIDER

josh.freckleton03:10:33

if I run the relevant code in a lein figwheel REPL, it works fine. Just... not in cider's nrepl.

hiredman04:10:22

According to google the rhino js engine doesn't include the setTimeout function (like the error says), which is what core.async uses to run tasks

tbaldridge16:10:28

Yeah, IIRC we never tested core.async in Rhino. Wouldn't be hard to add support we'd just need some sort of function like setTimeout.

josh.freckleton16:10:06

@hiredman @tbaldridge thanks! So, I prob just won't test CLJS from within emacs for now 🙂

tbaldridge16:10:32

yeah figwheel is so nice I never really made Rhino much of a priority.

tbaldridge16:10:43

You can jack into Figwheel from emacs though right?

bcbradley17:10:57

I have this thing called "cache" that is a caching abstraction. When I tell the cache to return something, if it can't find it inside itself it will ask a server for it. If i tell the cache to write something to itself, it will actually tell a server to write something and wait for an "ok" response before commit it to itself. Writes generally take a while, but reads could return super quickly or they could also take awhile. So would it be inappropriate to use go blocks here? Should I use threads? And why?

angusiguess17:10:43

@bcbradley If you perform a blocking operation in a go block, you’ll consume one of the threads in the core.async thread pool for the duration of that operation.

bcbradley17:10:07

if you perform a blocking operation on a real thread, you'll consume a real thread though won't you?

bcbradley17:10:25

and isn't the thread pool backed by real threads?

angusiguess17:10:16

Yes, you’ll consume a real thread.

angusiguess17:10:33

The default thread pool size for core.async is 8.

bcbradley17:10:48

i suppose if you have 8 real threads and only back the thread pool with 4 of them, then by using thread instead of go you'll be using threads you wouldn't otherwise use, and you won't slow down your thread pool

bcbradley17:10:58

but what if you have 8 real threads and your thread pool is backed by all of them

bcbradley17:10:15

doesn't it degenerate to the same thing?

angusiguess17:10:17

And if you tie up that pool with blocking operations, pretty much all core.async operations are blocked.

bcbradley17:10:37

you can't really tie up the whole pool can you?

bcbradley17:10:42

i mean unless you block on every thread that backs it

bcbradley17:10:34

i don't even see how you can block on even a single thread that backs it if you use go'

bcbradley17:10:37

won't they just park?

angusiguess17:10:03

blocking i/o won’t just park, unfort.

bcbradley17:10:20

i'm sorry if i'm being combative i'm just trying to understand when and why to use thread vs go

angusiguess17:10:33

Oh no worries! I’m just putting my thoughts together here 🙂

val_waeselynck17:10:57

@bcbradley if it's asynchornous (non-blocking / event-driven / etc.) use go. If it blocks, use thread.

hiredman17:10:22

even that can be troublesome

hiredman17:10:58

if you are block on anything besides reading or writing to a channel, then don't use go

angusiguess17:10:14

So java threads don’t map one-to-one to cpu threads, to start.

hiredman17:10:31

that is not true

angusiguess17:10:46

But you can have more java threads than cpu threads, can you not?

angusiguess17:10:49

Not active all at once.

bcbradley17:10:49

here is the overview of my architecture: you use http RESTful things like GET POST DELETE etc on an abstraction called a "cache". If the cache finds what you are looking for inside of it (via GET), it will just return that. If it can't do the operation on itself because it can't find it it will call the server. If it can't do the operation because it is a mutation it will call the server. In any case, these commands are "fire and forget", the cache is responsible for updating itself via the server in response to commands.

hiredman17:10:00

by cpu threads you mean cores?

bcbradley17:10:08

In order to program in this model, the user makes "callbacks" that fire when the cache changes in a specific way

bcbradley17:10:22

for instance, if "val A" is cached --> fire this

hiredman17:10:22

sorry I meant os threads

angusiguess17:10:41

I suppose I meant cores in this instance.

bcbradley17:10:50

I guess that means what i'm doing is asynchronous right?

tbaldridge17:10:08

Why not make the whole stack async?

tbaldridge17:10:41

Use go blocks for everything, and when you need to make a call to a remote server use async http (or whatever your protocol is).

hiredman17:10:51

or just burn a thread

tbaldridge17:10:05

yeah, that too, depends on your use case.

hiredman17:10:30

I mean, the jvm has threads, they work pretty well

tbaldridge17:10:58

also true. Most times people need async...they don't really

bcbradley17:10:30

so let me get this straight-- if you perform long blocking operations in a go block, it will park one of the tasks that the thread pool is responsible for executing. They will continue to execute other tasks, but presumably come back periodically to "check up" on the status of the long blocking task. If you do this for lots of tasks in that thread pool, a lot of the work being done by the thread pool will be "check up" work, reducing efficiency

bcbradley17:10:33

Is that right?

tbaldridge17:10:20

If you perform IO inside a go block it will block the thread executing the go block. Do that more than the number of go block threads (defaults to 8) and you can deadlock your program.

tbaldridge17:10:30

Don't do IO inside a go block.

hiredman17:10:57

go blocks only "park" when reading or writing to a channel, and parking is the only thing that causes a go block to yield so other go blocks can execute on the threadpool go blocks execute on

bcbradley17:10:39

Alright that last bit helps me understand a lot @hiredman

bcbradley17:10:18

So i should probably use threads not go blocks

bcbradley17:10:29

since the cache could potentially block

tbaldridge17:10:01

Possibly...but I would consider if your cache needs to block or not.

bcbradley17:10:15

it must block for write operations

bcbradley17:10:31

if it is doing anything useful at all being there, it won't block most of the time for read operations

bcbradley17:10:55

maybe threads for write and go blocks for read?

tbaldridge17:10:10

Well, what are you writing to? Many things these days support async IO

bcbradley17:10:26

i'm making a clojure api for discord

bcbradley17:10:07

i'm writing to a server somewhere far away

tbaldridge17:10:21

Right, and that's via HTTP?

tbaldridge17:10:17

If you use an async HTTP client library you can make the whole stack async. And that may integrate cleaner if the user of your library is also using an async stack (like a Pedestal web server).

tbaldridge17:10:01

But it all depends on the number of API calls you expect, overall throughput, system resource constraints, etc.

bcbradley17:10:45

@tbaldridge thanks for the advice! After looking for an asynchronous library I think http-kit is probably the best fit (was using clj-http)

josh.freckleton17:10:42

@tbaldridge > You can jack into Figwheel from emacs though right? so from a command line, I do lein figwheel, and then emacs can jack in to that? That sounds incredible, could you link to a resource or something please, that explains how to do that? (i'm a bit new)

tbaldridge18:10:19

sure, @josh.freckleton IIRC firstly you need to add :nrepl-port 7002 to your figwheel config. Then you can (from within emacs) run cider localhost:7002 to connect to the CLJS repl.

tbaldridge18:10:53

It is possible, sadly I haven't used Figwheel in about a year and cider in about 2 years, so I may have some of the configs wrong.

josh.freckleton18:10:23

thanks! I'm sure, with this, i can get it going.

josh.freckleton18:10:59

so, you don't use cider? Do you use something else? I mean, the quick feedback loop is a big reason i love clj, i'm wondering if there's an alternative that's on the rise...

tbaldridge18:10:12

@josh.freckleton I use Cursive via InteliJ, it uses nRepl, but it's not cider.