Fork me on GitHub
#clojure
<
2015-11-11
>
cfleming00:11:24

Man, I really wish fns could accept metadata

bronsa00:11:06

? fns are IMetas

bronsa00:11:28

yeah, try (with-meta #() {})

cfleming00:11:20

Is that because the anonymous instances implement it explicitly? Unless I’m blind, AFn only implements IFn, and IFn doesn’t extend IMeta

bronsa00:11:36

AFunction implements IWithMeta IIRC

bronsa00:11:00

yeah AFunction implements IObj

cfleming00:11:08

Aha, so it does - I thought fns were AFns, I didn’t know about AFunction

bronsa00:11:12

every fn is compiled into an AFunction

bronsa00:11:29

(not only "function literals")

cfleming00:11:50

Awesome, thanks @bronsa

cfleming00:11:01

I miss the (inc bronsa) from IRC

cfleming00:11:17

Life isn’t the same without fake internet points.

bronsa00:11:30

don't worry, I'm keeping a RL counter

bronsa00:11:20

(one small gotcha with attaching metadata to functions is that you won't be able to return primitives anymore)

cfleming00:11:38

That’s ok in this case

emil0r09:11:32

how would you flatten a list of lists being produced by a transducer as a part of the transformation process?

mpenet09:11:39

(mapcat identity)?

mbertheau12:11:38

Hmm, I miss not<=.

swizzard13:11:47

does flatten not have a transducer arity?

dialelo13:11:50

@emil0r: you can use the cat transducer directly

Alex Miller (Clojure team)14:11:13

your code needs more cats anyways

bostonaholic14:11:02

this explains so much

dm314:11:28

is there a combinator in Specter that can collect the path visited by walker? E.g. (select [(collect-walker symbol?)] {:x {:y 'a}}) -> ([:x :y 'a])

max15:11:23

My web app seems to be leaking threads. I see that the thread count climb from 100 to 4000 steadily over the past month, which (I think) caused my system to crash. Can anyone recommend a profiler that will make hunting this down easy? I am looking at jconsole, visualvm, and yourkit, but I haven’t used any of them...

dm315:11:44

yourkit is the best IMO, but it costs money. JVisualVM is a newer version of JConsole, so that's what I use mostly.

pbostrom15:11:00

Java Flight Recorder is included in JDK8, run jmc

bcambel16:11:05

max, how do you manage threads in your application? or is it only the web server threads ? jconsole will help you.

max16:11:48

bcambel: I’m using http-kit for the webserver, I also use carmine to do some backround tasks backed by a redis queue, and have some jobs happen periodically using quartzite. I’m not sure what’s leaking threads (though I suspect its a web endpoint).

max16:11:57

I just don’t know how to approach the problem yet simple_smile

bcambel16:11:32

I would highly suggest you to use something like https://github.com/TheClimateCorporation/claypoole I don't think http-kit will be the problem but who knows.

deas16:11:51

@max: You know what's spawning them, right?

max16:11:18

deas: I don’t actually

bcambel16:11:24

also one way to figure out is to separate background tasks from the main process, and see if threads are growing still

max16:11:51

that is, I don’t know how http-kit manages threads per request.

deas16:11:19

@max The thread names might give you a clue.

bcambel16:11:49

do you have nginx or anything in front of the webserver ?

max16:11:54

bcambel: yes

max16:11:11

is it possible to do a flight recording off production to analyze offline without restarting the app?

max16:11:36

part of the problem is I haven’t been able to reproduce the issue locally as well as I’d like.

bcambel16:11:02

yeah this problems won't happen in local environments simple_smile

deas16:11:07

@max Do you see where they are stuck and why they do not terminate? "kill -QUIT <pid>" might give you an idea.

pbostrom16:11:29

@max: is it jdk8, and can you set up X forwarding? if so then flight recorder should work

max16:11:02

@pbostrom: damn it the server is java version "1.7.0_79" OpenJDK Runtime Environment (IcedTea 2.5.6) (7u79-2.5.6-0ubuntu1.14.04.1) OpenJDK 64-Bit Server VM (build 24.79-b02, mixed mode)

bcambel16:11:15

@max divide & conquer simple_smile keep the webserver as light as possible

max16:11:24

bcambel: are you suggesting splitting other aspects of the application into separate apps/processes?

bcambel16:11:05

@max yes, do all the tasks outside of the webserver process if possible

pbostrom16:11:44

@max: I think it's possible with jdk7 but you will have to restart the java process

max16:11:53

fair enough. I am pro-microservices, although I am not sure how clojure people do it. If I have 12 diff processes do I spin up 12 JVMs for it?

bcambel16:11:27

no, one or two for webserver, one for all the other background tasks

max16:11:41

Yeah. The goal is to get there, but as you know it often makes sense to start with a monolith when you’re trying to figure out what you’re building simple_smile

bcambel16:11:20

sure I understand the simplicity aspect of it, but you got the first signal, so better to warm up your hands. It's not that hard to manage 2 processes. ( I am also big fan of Supervisor )

bcambel16:11:40

I also need to implement very soon background jobs for http://github.com/bcambel/hackersome

max16:11:57

bcambel: definetly. I would like to learn more about best practices for clojure web stacks. Even deciding whether to use a servlet container or not was difficult. And I will have to re-evaluate that as I split up the monolith.

bcambel16:11:56

it is always difficult . For example Puppetlabs has these kind of base services. checkout https://github.com/puppetlabs/trapperkeeper

bcambel16:11:47

I am not saying use them, but the ideas behind are good learning points

dm316:11:12

@max, kill -3 PID and looking at the thread dump should tell you whose threads are hanging. Most of the libraries name their thread, e.g. nrepl-worker-..., http-.... Splitting the app just because you have one problem is a bit too extreme.

max17:11:17

dm3: Agreed, splitting the app is a future plan, not the first step of debugging. Is sigquit supposed to dump something?

dm317:11:18

java process will print thread dump to stdout

max17:11:53

dm3: that didn’t work (maybe an openjdk thing?), but jstack did. And I found the cause of the error. Thank you all!