Fork me on GitHub
#clojure
<
2015-06-29
>
sander09:06:27

i'm considering to use clj+ring+jetty+jvm to run multiple websites on my vps, slowly replacing rails, django, php and node. i'm just worried about ram usage: my jvm instances seem to take a lot more than the scripted servers i'm familiar with. is there a common set-up for having just one jvm running, with multiple web servers from separate jar files?

sander09:06:21

i could package them all up in one jar file but would prefer not, the sites have separate owners and purposes

tcrayford09:06:41

sander: you could always just buy more memory ;) ram is so cheap these days ;)

tcrayford09:06:12

sander: but yeah, typically the jvm wants at least 512mb of ram, and it's more happy with 1gb

sander09:06:41

@tcrayford: on linode going from 2gb to 4gb is $20 extra per month, too much for my hobbyish state right now

tcrayford09:06:03

Sure, I guess. I mean contracting for 2 hours or whatever would pay for that for 6 months (assuming a fairly normal hourly rate of $120/h)

sander09:06:45

my server is more about personal hobby sites and building small sites friends' companies for 1/3th of that simple_smile

sander09:06:20

still a student

tcrayford09:06:24

@sander: another option is to tune down the heap size, but that'll start hurting the GC a lot. I don't have any advice on multiple jars in one process sadly :(

tcrayford09:06:36

@sander: charge more ;)

sander09:06:38

i suspect that .war files and servlets might provide what i need, not sure if that fits the clj/ring type of websites

tcrayford09:06:15

Ring can work just fine under servlets and war stuff iirc. Never seemed like anything but headaches to get setup though :(

tcrayford09:06:33

Sorry I can't help more :(

sander09:06:36

@tcrayford: no problem and thanks, good to consider charging more and investing in ram instead of hours from time to time

goodwind8909:06:38

@sander: I am in the process of trimming my RAM usage to fit into the 2GB box on DigitalOcean

goodwind8909:06:23

not easy to troubleshoot heap limit exceeded errors with Clojure

goodwind8909:06:48

so if you can spend on a bigger box, go ahead and save you some development time

sander09:06:26

@goodwind89: are optimising your own code or looking for an ideal configuration? my current solution by the way is to use cljs/node instead, not sure if it's more efficient but i feel a bit more confident there (and don't need most of the jvm)

goodwind8909:06:04

@sander: that's my own code. I think the platform overhead is not that much (50MB for heap dump as I start repl with some minimal work). Just that my code loads quite a bit of data. Managing memory with lazy structures is going to be a headache.

stig09:06:31

@sander: perhaps Heroku’s new “free hobby tier” will work for you?

sander09:06:45

@stig: looks interesting, thanks!

ifesdjeen11:06:57

if anyone is interested / needs help on #C06UUKHV2 libraries, join #C06UUKHV2 simple_smile I guess it makes sense to migrate our IRC channel here

jwm11:06:23

no more irc? simple_smile

mitchelkuijpers12:06:22

Does anyone know how to allow extra keys in a map in a prismatic schema?

jellea12:06:09

http://clojure.java.io/resource question. I’ve included core.async to my deps. Whats the path to the async.cljs file then? (io/resource "core.async/async.cljs”)?

ifesdjeen12:06:19

nah, we’ll stick to IRC too

ifesdjeen12:06:27

for the transition period at least

ifesdjeen12:06:54

@mitchelkuijpers: you can use schema with “regular” maps, and only validate them when needed

ifesdjeen12:06:15

e.g. not their record instances

mitchelkuijpers12:06:34

The problem was this I got a map with some keys I do care about and the rest I don't care about but they can change

mitchelkuijpers12:06:38

fixed it by doing

mitchelkuijpers12:06:43

(schema/defschema InstallBody
  {:clientKey String
   :baseUrl String
   :sharedSecret String
   :productType String
   :publicKey String
   :serverVersion String
   schema/Keyword String})

Lambda/Sierra12:06:50

@jellea: find the path within the core.async JAR (use jar -tf file.jar to list files). io/resource will use the same path.

jellea12:06:22

stuartsierra: Great, found it! thanks!

meow13:06:36

@richhickey: Welcome to slack! Nice to see you here. simple_smile

estsauver13:06:18

Did Yesql lose support for :result-set-fn?

hmadelaine13:06:58

@mitchelkuijpers did you try

{ :my-key s/Str  s/Any s/Any} 

mitchelkuijpers13:06:36

@hmadelaine: Ye pretty much the same but i used

schema/Keyword String

mitchelkuijpers13:06:43

but any might be smarter just in case

estsauver14:06:06

@mitchelkuijpers: if you think you might ever need interop with cljs, you can use s/Str which will resolve to java.lang.String in clojure and the appropriate clojurescript schema also.

estsauver14:06:17

schema/Keyword s/Str

mitchelkuijpers14:06:36

@estsauver: Yeah already changed that, I like to have that option

jrychter14:06:18

@sander: I'm running two JVMs on a Hetzner VX11 vServer, which is 2GB, with no problems. 12EUR/month.

Busy escaping the comfort zone14:06:20

@sander: Digital ocean is pretty cheap on the low end side

estsauver14:06:08

OVH has another brand for really cheap dedicated boxes. https://www.kimsufi.com/en/

goodwind8915:06:00

@estsauver: thanks for the OVH suggestion. I have been looking for some non-SSD offering for a while.

estsauver15:06:50

Yah, as a heads up, the “120 second” provisioning time can be quite a bit more, and they frequently do often have trouble delivering enough servers (at least on their kimsufi/soyoucanstart.com lines.) I’d be happy to give a more thorough rundown of dedicated providers off the main channel.

noisesmith16:06:13

@michaeldrogalis: "a peer can only be working at least most one job at any given time" http://onyx-platform.gitbooks.io/onyx/content/doc/user-guide/concepts.html I assume "least most" should be "most" ?

michaeldrogalis16:06:34

@noisesmith: Yes. Should say "at most". Will fix, thanks

michaeldrogalis16:06:56

Fixed it in the docs, should update on Gitbook in a few mins.

noisesmith17:06:15

cool - I'm doing my first dip into onyx, looks like it will work out for our team so far

andrew17:06:38

Any experienced folks here want to post their exercism handle? Otherwise, it seems users are stuck looking at recent fellow beginners' submissions which, while useful, does of course have its limits. The #C053AK3F9 have started a clojurian exercism team. I don't think joining would require any extra time on your part since we can learn just by inspecting your past submissions...

andrew17:06:10

(I have made a feature request of sorts in exercism/support in http://gitter.im so maybe they'll make it easier to discover well-regarded users in a particular language)

michaeldrogalis17:06:39

@noisesmith: Great to hear. Hit me up if you need anything.

noisesmith17:06:30

michaeldrogalis: cool, I'm checking out the source in order to make some PRs against the gh-pages - in some cases these will be speculative (I think I know what is meant, but am not sure...)

noisesmith18:06:17

@michaeldrogalis: I'm confused about this gitbooks thing - I had assumed the docs I was reading was the gh-pages branch of your repo, but how would I clone and pr against the gitbook?

michaeldrogalis18:06:46

@noisesmith: Gitbook is built out of the /doc/userguide directory of the repo automatically

michaeldrogalis18:06:52

gh-pages are the code level APIs from Codox

michaeldrogalis18:06:17

Fork Onyx, edit /doc/user-guide, send a PR. It will update in GitBook by itself

noisesmith18:06:28

thanks again

noisesmith18:06:06

michaeldrogalis: d'oh, a simple git log could have led me to the answer on that one

malcolmsparks18:06:04

when you parse a form (application/x-www-form-urlencoded) in Clojure, is it usually to parse field names as strings or keywords (can't find anything that will do the latter). Asking for a friend 😉

donaldball19:06:53

It seems to be idiomatic to use keywords, but there's no guarantee that the field names will actually be valid keywords

noisesmith19:06:08

malcolmsparks: there's the wrap-keyword-params middleware for ring

donaldball19:06:20

But since the keyword fn doesn't validate the domain of its input, in practice most folks don't care

noisesmith19:06:36

also, generating keywords is not free. I think making them is a slightly controversial idiom (though it is idiomatic)

gtrak20:06:28

When doing filtering of transitive AOT (using maven-jar-plugin, in this case), I'm getting 'ClassNotFoundException' in a protocol provided by a third-party dependency. Re-evaluating my calling namespace in a running process fixes the problem. Am I doing something obviously wrong?

arrdem20:06:17

gtrak: my gut is that your AOT'd code refers to an AOT'd class which is regenerated when you boot the app. Your code is then calling or otherwise reaching a stale class or stale class instance which will give you a ClassNotFoundException. Recompiling/reloading your code refreshes the class reference to the new one from the old one and then the JVM can link correctly and everything's OK.

arrdem20:06:27

but that's just a guess from seeing similar stuff in Oxcart

gtrak20:06:46

@arrdem: right, it's referring to a class that doesn't exist. But, I get the error when the function runs, not when the namespace loads.

arohner20:06:46

@gtrak: it’s not that surprising to me this is a runtime error, esp. when using reflection

arohner20:06:10

@gtrak: at some point, there’s a (.bar (Class/forName “Foo”) 3) in your function

gtrak20:06:24

I'm calling a protocol method

gtrak20:06:34

that might explain it?

gtrak20:06:32

yea... hm.

gtrak20:06:11

I was considering classloaders as well, the code is loaded up from java via the clojure.java.api class.

gtrak20:06:56

I am using 1.7RC2, is it not a problem in 1.6? I can try 1.6 and see if it's still a problem.

gtrak20:06:22

still an issue in 1.6

Alex Miller (Clojure team)20:06:40

1.7 fixes some real issues with AOT and reloading (CLJ-979) but unfortunately that unmasks other issues (like CLJ-1741) - there is more work to be done. I'm not sure I actually understand what you're doing or the error well enough to say exactly. There are bigger issues that need to be addressed re transitive AOT (CLJ-322).

gtrak20:06:44

thought it might've gotten easier since a few years ago simple_smile, I'm just stripping class files post-AOT and limiting it to my namespaces.

gtrak20:06:42

Got around it by manually including the offending protocol interfaces. In practice, I think this means that until 322 is fixed, apps should avoid directly calling lib protocol methods, and libs should wrap protocol methods in standard functions, so they'll be referenced at the time the lib namespace loads (which I think happens a lot anyway, to work around lack of varargs).

michaeldrogalis21:06:49

@noisesmith: Unintentional. Used to be 3, now 2. Fixing

csd23:06:19

Hi can anyone recommend some links discussing best practices for doing Java => CLJ interop? E.g. advice on how to best wrap Java objects, etc