Fork me on GitHub
#immutant
<
2016-03-03
>
jcrossley314:03:12

can you post those traces for @tcrawley please?

yogthos14:03:18

sure thing

jcrossley314:03:58

@tcrawley: he's using inc 714

jcrossley314:03:57

@yogthos: you see no failed requests from the client, right? just these exceptions on the server?

tcrawley14:03:07

the latter exception is ignorable I think. The former is a problem

yogthos14:03:12

the client seems fine in the browser at least

tcrawley14:03:21

what does the code look like that triggers the first exception?

yogthos14:03:27

but when ab runs it might get some bad responses?

jcrossley314:03:25

@yogthos: make sure you see the correct port in your log, please simple_smile

tcrawley14:03:51

this same code ran w/o error on 2.1.2?

yogthos14:03:11

let me flip the dependency back to make sure

yogthos14:03:04

I believe it only happens when run functions are chained

yogthos14:03:38

ok yeah I'm not seeing errors without the incremental build

tcrawley14:03:36

do you see errors on every request to /json, or just some?

tcrawley14:03:18

I must be doing something wrong. if I lein run the app, then git localhost:3000/json, I get a 404

tcrawley14:03:56

hitting / gives me "Hello, World!"

yogthos14:03:03

oh go to `/io/json

yogthos14:03:10

cause that's served by the nested handler

tcrawley14:03:07

and you don't see the error every request? only intermittently? only under load?

jcrossley314:03:34

@tcrawley: pretty sure only load. he didn't see it in the browser

yogthos14:03:04

yeah it's definitely not for every request

yogthos14:03:15

I ran ab -c 10 -n 1000

yogthos14:03:24

and only saw a few exceptions

tcrawley14:03:08

ok, I'll try the same here and see if I can recreate, then maybe add some debug output to collect more data

tcrawley15:03:33

@yogthos: do you see some errors every ab run? or only some? I've ran it several times, and tried bumping up -c and -n, with no errors

tcrawley15:03:49

what OS/java version are you on?

yogthos15:03:07

I don't see errors on every run, and I'm on os x, jdk 8

yogthos15:03:06

hmm and now I'm not able to reproduce that either

jcrossley315:03:36

fwiw, i'm unable to replicate as well

tcrawley15:03:02

that's troubling - the only way I can see for this to happen is if an exchange gets reused somehow, but I can't see what could cause that

tcrawley15:03:17

it's troubling that it was there but isn't now, that is

jcrossley315:03:37

i'm beginning to feel better about undertow not being the bottleneck, though.

tcrawley15:03:05

the good news is, with ab -c 50 -n 100000 , I see 2.7k req/s with 2.1.2, and 13.2k req/s with 714

jcrossley315:03:32

@yogthos: is there a simple way to only omit the formats middleware?

jcrossley315:03:43

@tcrawley: is that an apples-to-apples test without being able to set :dispatch? false?

tcrawley15:03:15

ah, pb. those 2.7k requests would all be errors

jcrossley315:03:31

you don't see those on stdout?

tcrawley15:03:54

oh, I do. I just didn't look at std out on the 2.1.2 run :)

jcrossley315:03:46

@tcrawley: more troubling to me is that i actually see a little better perf from 2.1.2 with dispatch true

jcrossley315:03:01

than i do with 714 and dispatch false

tcrawley15:03:06

with :dispatch? true, I see 12.8k req/seq vs. 13.2k on 714

tcrawley15:03:17

with :dispatch? false on the latter

jcrossley315:03:39

i'm running that

tcrawley15:03:11

at some point, you'll see contention between ab and the jvm

yogthos15:03:03

so it sounds like I'll remove the middleware for the techempower benchmark, and we should see very nice performance simple_smile

yogthos15:03:52

sounds like this would be a reasonable default for luminus as well? :worker-threads 200 :io-threads (* 2 (.availableProcessors (Runtime/getRuntime))))

jcrossley315:03:40

@yogthos: it really depends on your hardware

jcrossley315:03:43

and your app

jcrossley315:03:06

@yogthos: the immutant defaults are 1xProcessors for io and 8xIO for workers

jcrossley315:03:32

you might breach someone's ulimit with 200 workers

jcrossley315:03:00

and the immutant defaults are undertow's so i kinda yield to their expertise

jcrossley315:03:59

regardless, as long as you make it clear in your template where to adjust them, i don't think it matters much what you default them to

jcrossley315:03:15

@tcrawley: i see 12k r/s with 714/false and 15k r/s with 2.1.2/true

yogthos15:03:16

yeah that sounds reasonable

yogthos15:03:36

I think I'm going to add a "server tuning" section in the docs and discuss different strategies there

jcrossley315:03:59

@yogthos: good idea. happy to review when you have something.

yogthos15:03:14

fantastic I'll definitely take you up on that simple_smile

jcrossley315:03:16

@tcrawley: i still see slightly better perf with 2.1.2/true at 30 clients

jcrossley315:03:03

@tcrawley: it may be undertow, though. forcing 2.1.2 to use 1.3.18 slows down to 10k r/s

jcrossley315:03:10

but that's still dispatching

jcrossley315:03:36

let's try 714 with b9

jcrossley316:03:30

yeah, that's 16k r/s with 714/false/b9

jcrossley316:03:34

so undertow got slower

jcrossley316:03:45

now i feel stupid for bumping wboss up off b9 😞

tcrawley16:03:30

no harm in going back!

jcrossley316:03:56

13.5k r/s with 714/true/b9 which makes sense