This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-09-12
Channels
- # beginners (63)
- # boot (3)
- # braveandtrue (153)
- # cider (19)
- # cljdoc (2)
- # clojure (80)
- # clojure-dev (25)
- # clojure-italy (73)
- # clojure-losangeles (1)
- # clojure-nl (4)
- # clojure-spec (67)
- # clojure-sweden (1)
- # clojure-uk (83)
- # clojurescript (56)
- # clojutre (11)
- # core-logic (37)
- # cursive (18)
- # datomic (14)
- # editors (10)
- # emacs (13)
- # figwheel-main (11)
- # fulcro (62)
- # graphql (11)
- # jobs (3)
- # klipse (1)
- # leiningen (6)
- # off-topic (91)
- # onyx (7)
- # pedestal (3)
- # portkey (5)
- # re-frame (14)
- # reagent (13)
- # remote-jobs (1)
- # shadow-cljs (111)
- # tools-deps (4)
- # yada (10)
lein test :only
is a good query as well (as it will point to the ns which is what I primarily seek)
Hello we have a clj back end with a 400 MB size. This back end uses 1.1GB RAM on our Heroku server, so we need a large dyno tu run the app. Has anybody faced a similar problem? Are there known ways to reduce the RAM burden on Heroku of clj apps? thanks for any answer!
so if I git clone
the repo it will weigh 400MB? seems fishy (or an incorrect measurement)
it also includes all git data?
Did you try to limit the maximum heap size?
> -Xmx size in bytes Sets the maximum size to which the Java heap can grow. The default size is 64M. (The -server flag increases the default size to 128M.) The maximum heap limit is about 2 GB (2048MB).Aug 16, 2018
Hi @kirill.salykin i will ask our developer
im not a technical person, so i just know that this app creates a heavy RAM burden on heroku
I actually do not think there is a direct correlation between repo size and ram usage
but we compared repo of same size and different languages (example node.js) and the RAM consumption of our CLJ app was 5 times
@sufaber not an expert on optimization/etc, but I'd say clojure is simply memory-hungry. It easily takes 4GB+ RAM for development (not so much in production fortunately) that said 1.1GB is nothing (considering how production workloads can take terabytes of RAM these days), so if you're being charged too much for 1.1, consider yourself being ripped off 🙂 instead of spending eng effort in RAM optimization, switching languages, etc I'd switch from Heroku -> AWS Beanstalk, it gives you a similar experience for a lower cost (Heroku uses AWS after all)
Not to be too annoying, but beware of the sunk cost fallacy 🙂 with a bit of luck AWS Beanstalk could be adopted in say, one day, and the investment could last years. Has been the case for me
You should try -Xmx
, but use a JVM profiler either way. Usual first suspects for ram blowup are (faulty) memoization and holding the head of lazy sequences.
Some JVMs in some runtime environments with some runtime settings try to grab a big chunk of available RAM. You need to understand your workload to tune the GC subsystem to balance RAM usage against your performance needs.
You should try -Xmx
, but use a JVM profiler either way. Usual first suspects for ram blowup are (faulty) memoization and holding the head of lazy sequences.
Is a write/read roundtrip possible with re-pattern? (edn/read-string (pr-str #".*Apache License, Version 2\.0.*"))
yields an error: RuntimeException No dispatch macro for: " clojure.lang.Util.runtimeException (Util.java:221)
something like that: (clojure.edn/read-string {:readers {'app/rx re-pattern}} "#app/rx \".*\"" )
@sufaber I guess most people developing on the JVM have experienced memory hungry applications. The solution to this is to profile your application and see which objects use that much memory and try to optimize that. There is not much more to say here as this is usually an individual solution. Also having 400 MB in your repo sounds like your storing your dependencies in git, which is not idiomatic at all, but thats just a guess. Correlating between the repo size and the memory usage on the other hand is just plain wrong, this is not a useful metric in any way.
I could create you easily a repo containing a few kb that will use up all the ram in the world.
https://github.com/clojure-goes-fast/clj-memory-meter might be helpful
hello, is there a built in way to ns-qualify an expression while not qualifying what should not be qualified (like fn argv or let left bindings) ?
(fn [a]
(let [b c]
(d e b a)))
;; =>
(clojure.core/fn
[a]
(clojure.core/let
[b me/c]
(me/d me/e b a)))
user=> `(fn [~'a] (let [b ~'c] (~'d ~'e b a)))
(clojure.core/fn [a] (clojure.core/let [user/b c] (d e user/b user/a)))
@U3L6TFEJF thank you, i know how to do it by hand, i would like a generic way
resolve
might also be useful to you
user=> (resolve 'foo)
nil
user=> (resolve '+)
#'clojure.core/+
@alexmiller thank you, i will play with it and see what i can do
if you want to do this in a generic way that works for all clojure code you’ve got quite some work to do, but if you just want to handle special cases (like fn/let) it should be doable
https://clojure.github.io/clojure/clojure.walk-api.html and/or https://github.com/ztellman/riddley might be useful
@U3L6TFEJF thank you it will help i'm sure, also @alexmiller advice of using resolve seems to be powerful! it seems to do the trick 🙂
A javac tool & AOT tool for tools.deps
https://github.com/gfZeng/tools.deps.task.compile
please add to https://github.com/clojure/tools.deps.alpha/wiki/Tools !
Ah, sorry that's kind of what I meant. You'd have to make your own deep learning algos/etc.
+ Bayadera 🙂
Is there documentation that lists all of the options that can be included in a deps.edn
file for use with the clj
and clojure
cli tools? In particular, i'm trying to write a more advanced :alias
field, and there seems to be some hidden features in here that I would love to try out
in alias, all of the supported options are doc’ed on the reference page https://clojure.org/reference/deps_and_cli
that set being :extra-deps, :override-deps, :default-deps, :extra-paths, :classpath-overrides, :jvm-opts, and :main-opts
Ah ok, I wasn't reading it correctly, I had a warped idea on what it could be used for. So it provides more general aliasing of everything
there is one undocumented, unsupported, use at your own risk, I do not promise it will continue to exist or do what it does now option which is :verbose
there are definitely some under or undocumented keys in coordinates and top-level config
eventually I will get around to fixing that
speaking of :jvm-opts... -add-modules disabled in Java10?
@alexmiller is there something equivalent to leiningen's do
command, where you can run several things at once, or are you encouraged to write your own scripts to handle each case?
wouldn't you be outside the scope of the clj tool if you are doing anything other than resolving deps and starting a repl?
no - clj is about a) making classpaths and b) running programs. Any Clojure code can be a program and that program might do anything!
you run scripts (pass the .clj path) or mains (-m) or use stdin (-)
OK, that's fair
in other words, builds are programs (boot got that right), but dependencies are data (lein did that better)
Interesting point, it would be interesting to see developments on some sort of build tool like gulp
to use clj
under the hood
I was just following the figwheel.main
tutorial, and it seems to use clj directly, so I assumed the direction of the tool was something akin to gulp
for clojure
you might find what you want here https://github.com/clojure/tools.deps.alpha/wiki/Tools