Fork me on GitHub
#clojure
<
2015-11-23
>
pesterhazy11:11:26

How to people come up with good values for -Xmx/-Xms on servers? I'm running a JVM on AWS's m3.medium with 3764 MB total RAM.

pesterhazy11:11:30

With -Xmx3400m -Xms3400m, I see eventually see mmap errors: Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007fab0a3fd000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)

pesterhazy11:11:18

(no swap on these machines)

tcrayford11:11:03

@pesterhazy: give the OS ~ 1GB. Then it depends on your app (e.g. does it rely on the page cache etc)

pesterhazy11:11:45

@tcrayford: so -Xmx(TOTAL_MEM - 1G)? interesting, I wouldn't have assumed you need that much leeway

tcrayford11:11:54

@pesterhazy: you can probably get away with less leeway if and only if you promise to never run anything else on the machine

tcrayford11:11:17

e.g. you never login to it and do anything memory intensive, you don't run nginx or anything else like that etc etc

pesterhazy11:11:24

yeah, these machines are pretty much jvm-only

pesterhazy11:11:51

but you're right, I do ssh in sometimes and do stuff, so it's better to err on the safe side

tcrayford11:11:22

e.g. if you pipe a large log file into vim on the machine rather than scping it off, you don't want linux to OOM kill your JVM app whilst doing so

roberto13:11:50

Has anyone a web app that uses component library to heroku?

roberto13:11:38

I’m having some issues, it won’t connect to the database, but it works locally. I added some log statements, and it seems like the start method in the db component isn’t even called.

roberto13:11:33

this is how my db compnent looks:

(defrecord Database [mongo-uri]
  component/Lifecycle
  (start [this]
    (log/info "Trying to connect to " mongo-uri)
    (let [{:keys [conn db]} (mg/connect-via-uri mongo-uri)]
      (log/info "Connected to mongo.")
      (assoc this :db db :conn conn)))
  
  (stop [this]
    (when-let [conn (:conn this)]
      (mg/disconnect conn))
    (log/info "Disconnected from mongo.")
    (dissoc this :db :conn)))

(defn start-database [mongo-uri]
  (log/info "Will attempt to connect to " mongo-uri)
  (log/info "Connected: " (mg/connect-via-uri mongo-uri))
  (map->Database {:mongo-uri mongo-uri}))

apviitanen13:11:26

@roberto: how are you starting your system in Heroku? Does it work locally if you build an uberjar and run it with java -jar uberjarname.jar ?

roberto13:11:10

yeah, it starts locally. Okay, now it is connecting to the db, but I’m getting a NPE, which I don’t get locally.

roberto13:11:13

(defrecord HttpServer [db-component api-routes html-routes]
  component/Lifecycle
  (start [this]
    (let [port          (env :server-port)
          auth-handlers (-> (routes (:handlers api-routes) (:handlers html-routes))
                            (friend/authenticate {:allow-annon? true
                                                  :workflows    [(auth-config/workflow auth-config/client-config (partial credential-fn (:db db-component)))]}))
          server        (run-server (wrap-defaults auth-handlers 
                                                   (assoc-in site-defaults [:security :anti-forgery] false)) 
                                    {:port port :join? false})]
      (log/info (str "Started server on port " port))
      (assoc this :server server)))
  
  (stop [this]
    (stop-server (:server this))
    (assoc this :server nil)))

(defn create-system
  [{mongo-uri :mongo-uri :as config}]
  (component/system-map
   :html-routes (start-html-routes)
   :db-component (db/start-database mongo-uri)
   :api-routes (component/using
                (map->ApiRoutes {})
                [:db-component])
   :app (component/using
         (map->HttpServer {})
         [:db-component :api-routes :html-routes])))

(defn -main
  [& args]
  (component/start (create-system {:mongo-uri (env :mongo-uri)})))

roberto13:11:30

it says the NPE is where I’m starting the server:

server        (run-server (wrap-defaults auth-handlers 
                                                   (assoc-in site-defaults [:security :anti-forgery] false)) 
                                    {:port port :join? false})

roberto13:11:35

will dig some more

roberto13:11:26

ah, i’m so stupid

roberto13:11:45

had a typo in one of the env variables

davebryand14:11:47

hi all--very excited to be beginning my journey into clojure

roberto15:11:31

welcome to clojure @davebryand . I hope you enjoy your journey.

fergalbyrne15:11:53

hi everyone. Anyone know about how to pass a Java 8 method reference eg FileSensor::create in Clojure?

fergalbyrne15:11:08

Can't find anything useful.

danboykis15:11:21

@fergalbyrne: Are you passing it from Java to Clojure or vice versa?

fergalbyrne15:11:39

From Clojure to Java.

danboykis15:11:01

@fergalbyrne: I haven't tried this, but my first thought is to try and reify java.util.function.UnaryOperator something along the lines of (reify java.util.function.UnaryOperator (apply [_] (FileSensor/create)))

danboykis15:11:16

and pass that to the java code

fergalbyrne15:11:51

Cool, I'll try that. Thanks

danboykis15:11:29

let me know how it works out for you

fergalbyrne16:11:57

OK, that was on the right track, ta very much. This at least compiles: (def fs-creator (reify org.numenta.nupic.network.sensor.SensorFactory (create [_ params] (FileSensor/create params))))

fergalbyrne16:11:40

(the Java method accepts a SensorFactory or a reference to a method, so this is a workaround).

roberto17:11:26

has anyone had issues with immuconf not identifying ~ as the home directory?

roberto17:11:10

ah, it seems to be an issue with slurp

roberto17:11:25

how can you get the HOME directory in clojure?

mpenet17:11:30

@roberto: (System/getenv "HOME")

roberto17:11:10

I used (System/getProperty “user.home”)

mpenet18:11:27

roberto: yours is probably better

jstew18:11:12

I like using environ for that sort of thing. A nice abstraction over system properties.

roberto18:11:59

I don’t like environ for deploying to prod.

roberto18:11:17

i find immuconf much more flexible

jstew18:11:59

I like being here. I always learn about new libraries I've never heard of simple_smile

Lambda/Sierra18:11:06

~ as home dir is a Unix convention; Java-based tools generally don't support it.

hectorqlucero18:11:18

Are there any up to date tutorials for luminus?

gtrak18:11:30

@bronsa: I think I figured out my AOT bug from the other day: http://stackoverflow.com/questions/19594360/preserving-timestamps-on-clojure-clj-files-when-building-shaded-jar-via-maven-s It only happens on the shaded jar, not when I run with maven directly.

gtrak18:11:38

turns out timestamps affect AOT

hectorqlucero18:11:35

Any good tutorials for web development with clojure

jstew18:11:28

@hectorqlucero: The docs page is pretty much a full tutorial http://www.luminusweb.net/docs

robert-stuttaford19:11:33

i feel like there’s a core function for what’s going on in the anon fn here: (map #(or (:some-kw %) “default"))

samflores19:11:43

(get map key not-found)

mpenet19:11:57

you can even skip the "get" (:some-kw % "default-val")

robert-stuttaford19:11:12

now that i didn’t know!

mpenet19:11:13

buy yeah, readability :

robert-stuttaford19:11:39

i think i prefer the or, tbh

mpenet19:11:53

it reads better

jr19:11:17

beware there is a subtle difference:

user=> (get {:key nil} :key  "default")
nil
user=> (or (:key {:key nil}) "default")
“default”

robert-stuttaford19:11:58

your first example is wrong, jr

robert-stuttaford19:11:11

you’re trying to read the key “default” from the map

jr19:11:38

same result

robert-stuttaford19:11:30

then or is definitely my friend here

davebryand19:11:53

can anyone help me understand why this doesn’t work?

eng-dacom.repl=> (def a "11111111-1111-1111-1111-111111111111")
#'eng-dacom.repl/a
eng-dacom.repl=> #uuid a
AssertionError Assert failed: (string? form)  clojure.uuid/default-uuid-reader (uuid.clj:11)

samflores19:11:19

#uuid is a reader macro. it requires a string literal and you are giving it a symbol

davebryand19:11:20

What I’d like to do is to pass a string into my function and then tag that with #uuid to forward on to datomic. Something like this. Do you know how to accomplish this?

(def get-person [uuid]
  (ffirst (q get-person-query
             (db @conn)
             [:entity/uuid #uuid uuid])))

samflores19:11:15

I guess you'll need the same Java interop used by the reader

samflores19:11:40

(java.util.UUID/fromString uuid)

samflores19:11:48

the reader macro is syntactic sugar for literal values, just like #inst for dates. you string uses java.util.Date to create them at runtime

davebryand19:11:54

good call—I’m new to all this and I guess at the end of the day the whole point of the tagging is to just get the reader to get me a UUID obj

bronsa19:11:22

@gtrak: yeah, timestamps affect how code is loaded

bronsa19:11:42

cool that you figured that out

gtrak20:11:12

I shot in the dark long enough simple_smile

sdroadie20:11:53

transit is really awesome. just in case anyone didn't know.

borkdude20:11:23

What is the most convenient way of working with clojure datastructures from another JVM languages nowadays?

borkdude20:11:48

I'm asking for a friend (really) who is interacting with Datomic from Java

jgdavey20:11:56

Most Clojure datastructures implement java.util.Collection

gtrak21:11:07

happy to report clojure 1.8's working fine on our codebase, embedded in a spring/dropwizard app with elasticsearch/elastisch, core.match/analyzer/reader and clj-antlr, notably.

Tim22:11:40

does clojure not allow you to use any reader macros?

davebryand23:11:19

can anyone weigh in for a noob on whether or not the way that the conn is managed as an atom in the dacom skeleton app is still considered best practice? https://github.com/bellkev/dacom/blob/master/src/dacom/server.clj#L14

seancorfield23:11:04

Question about a former behavior of Clojure: back in the day (pre-1.4 I think), it used to wrap exceptions in RuntimeException in some cases. I modified clojure.java.jdbc to unwind RTEs to expose the underlying exception (so you could easily catch SQLException) but now I can’t figure out exactly how to repro that wrapping.

seancorfield23:11:13

I’d like to remove the whole throw-non-rte thing from the code, so it doesn’t unwrap ex-info and other things, but I need a test that verifies the wrapping behavior to see which Clojure versions will no longer work.

seancorfield23:11:54

It seems related to http://dev.clojure.org/jira/browse/CLJ-855 but I can’t figure out a simple test case that runs into that...

seancorfield23:11:19

NM, I ended up pulling the FileNotFoundException example from the mailing list thread linked in CLJ-855 and that verifies the behavior was only there for Clojure 1.3.0 so I think I’m just going to drop support for that in the next version of clojure.java.jdbc (which will therefore be 0.5.0).