Fork me on GitHub
#clojure
<
2018-09-12
>
ivana00:09:39

I usually search words 'fail' and 'error' in terminal search

ivana00:09:25

it jumps me directly to errors without scrolling

vemv00:09:02

seems a good-enough solution, thanks!

vemv00:09:25

lein test :only is a good query as well (as it will point to the ns which is what I primarily seek)

sufaber07:09:53

Hello we have a clj back end with a 400 MB size. This back end uses 1.1GB RAM on our Heroku server, so we need a large dyno tu run the app. Has anybody faced a similar problem? Are there known ways to reduce the RAM burden on Heroku of clj apps? thanks for any answer!

vemv07:09:30

What does 400MB represent? RAM, jar size?

sufaber07:09:58

@vemv is the size of the codebase in github

vemv07:09:00

so if I git clone the repo it will weigh 400MB? seems fishy (or an incorrect measurement)

sufaber07:09:27

thats the only info i have now…

sufaber07:09:04

where are you based?

vemv07:09:15

Barcelona

sufaber08:09:57

nice to meet and thanks

sufaber08:09:09

freelancer?

👍 8
kirill.salykin07:09:21

it also includes all git data?

kirill.salykin07:09:41

Did you try to limit the maximum heap size?

kirill.salykin07:09:48

> -Xmx size in bytes Sets the maximum size to which the Java heap can grow. The default size is 64M. (The -server flag increases the default size to 128M.) The maximum heap limit is about 2 GB (2048MB).Aug 16, 2018

sufaber07:09:22

Hi @kirill.salykin i will ask our developer

sufaber07:09:50

im not a technical person, so i just know that this app creates a heavy RAM burden on heroku

kirill.salykin07:09:51

I actually do not think there is a direct correlation between repo size and ram usage

sufaber07:09:28

yes thats probably right

sufaber07:09:59

but we compared repo of same size and different languages (example node.js) and the RAM consumption of our CLJ app was 5 times

sufaber07:09:07

stretching heroku server

vemv07:09:59

@sufaber not an expert on optimization/etc, but I'd say clojure is simply memory-hungry. It easily takes 4GB+ RAM for development (not so much in production fortunately) that said 1.1GB is nothing (considering how production workloads can take terabytes of RAM these days), so if you're being charged too much for 1.1, consider yourself being ripped off 🙂 instead of spending eng effort in RAM optimization, switching languages, etc I'd switch from Heroku -> AWS Beanstalk, it gives you a similar experience for a lower cost (Heroku uses AWS after all)

sufaber07:09:08

@vemv thanks, unfortunately we have already invested time in Heroku(

vemv07:09:05

Not to be too annoying, but beware of the sunk cost fallacy 🙂 with a bit of luck AWS Beanstalk could be adopted in say, one day, and the investment could last years. Has been the case for me

sufaber07:09:36

heroku has dyno at 50 usd month ( 1 GB) and next step is 4GB RAM (250$ month)…

sufaber07:09:55

this is the reason why im trying to understand if anyone had the same issue before

aisamu12:09:08

You should try -Xmx, but use a JVM profiler either way. Usual first suspects for ram blowup are (faulty) memoization and holding the head of lazy sequences.

rmprescott01:09:33

Some JVMs in some runtime environments with some runtime settings try to grab a big chunk of available RAM. You need to understand your workload to tune the GC subsystem to balance RAM usage against your performance needs.

aisamu12:09:08

You should try -Xmx, but use a JVM profiler either way. Usual first suspects for ram blowup are (faulty) memoization and holding the head of lazy sequences.

gv12:09:26

Is a write/read roundtrip possible with re-pattern? (edn/read-string (pr-str #".*Apache License, Version 2\.0.*")) yields an error: RuntimeException No dispatch macro for: " clojure.lang.Util.runtimeException (Util.java:221)

mpenet12:09:35

it's not part of the spec, you can create a tagged element to wrap it tho

mpenet12:09:40

#yourapp/rx "...." and have it return the value passed though re-pattern

mpenet12:09:03

something like that: (clojure.edn/read-string {:readers {'app/rx re-pattern}} "#app/rx \".*\"" )

sveri14:09:57

@sufaber I guess most people developing on the JVM have experienced memory hungry applications. The solution to this is to profile your application and see which objects use that much memory and try to optimize that. There is not much more to say here as this is usually an individual solution. Also having 400 MB in your repo sounds like your storing your dependencies in git, which is not idiomatic at all, but thats just a guess. Correlating between the repo size and the memory usage on the other hand is just plain wrong, this is not a useful metric in any way.

👍 4
sveri14:09:21

I could create you easily a repo containing a few kb that will use up all the ram in the world.

greywolve14:09:00

Wow this looks cool, thanks for sharing

👍 4
pbaille15:09:27

hello, is there a built in way to ns-qualify an expression while not qualifying what should not be qualified (like fn argv or let left bindings) ?

pbaille15:09:11

(fn [a]
  (let [b c]
    (d e b a)))

;; =>

(clojure.core/fn
  [a]
  (clojure.core/let
    [b me/c]
    (me/d me/e b a)))

schmee15:09:48

sounds like syntax quoting to me!

schmee15:09:43

user=> `(fn [~'a] (let [b ~'c] (~'d ~'e b a)))
(clojure.core/fn [a] (clojure.core/let [user/b c] (d e user/b user/a)))

schmee15:09:23

you have to manually un-quote the things that you don’t want namespaced though

pbaille15:09:55

@U3L6TFEJF thank you, i know how to do it by hand, i would like a generic way

Alex Miller (Clojure team)15:09:21

resolve might also be useful to you

Alex Miller (Clojure team)15:09:04

user=> (resolve 'foo)
nil
user=> (resolve '+)
#'clojure.core/+

pbaille15:09:31

@alexmiller thank you, i will play with it and see what i can do

schmee15:09:16

if you want to do this in a generic way that works for all clojure code you’ve got quite some work to do, but if you just want to handle special cases (like fn/let) it should be doable

pbaille15:09:03

@U3L6TFEJF thank you it will help i'm sure, also @alexmiller advice of using resolve seems to be powerful! it seems to do the trick 🙂

benzap17:09:02

is there an equivalent tool for the clj cli tool for lein-less?

deliciousowl18:09:23

Ah, sorry that's kind of what I meant. You'd have to make your own deep learning algos/etc.

🍹 4
deliciousowl18:09:41

neanderthal for what it is, is damn amazing

🍺 4
deliciousowl18:09:00

+ Bayadera 🙂

benzap18:09:48

Is there documentation that lists all of the options that can be included in a deps.edn file for use with the clj and clojure cli tools? In particular, i'm trying to write a more advanced :alias field, and there seems to be some hidden features in here that I would love to try out

Alex Miller (Clojure team)18:09:12

in alias, all of the supported options are doc’ed on the reference page https://clojure.org/reference/deps_and_cli

Alex Miller (Clojure team)18:09:32

that set being :extra-deps, :override-deps, :default-deps, :extra-paths, :classpath-overrides, :jvm-opts, and :main-opts

benzap18:09:27

Ah ok, I wasn't reading it correctly, I had a warped idea on what it could be used for. So it provides more general aliasing of everything

Alex Miller (Clojure team)18:09:49

there is one undocumented, unsupported, use at your own risk, I do not promise it will continue to exist or do what it does now option which is :verbose

benzap18:09:48

I guess that's for more output, so far i've been liking the direction of the tool

Alex Miller (Clojure team)18:09:03

there are definitely some under or undocumented keys in coordinates and top-level config

Alex Miller (Clojure team)18:09:18

eventually I will get around to fixing that

deliciousowl18:09:43

speaking of :jvm-opts... -add-modules disabled in Java10?

benzap18:09:15

@alexmiller is there something equivalent to leiningen's do command, where you can run several things at once, or are you encouraged to write your own scripts to handle each case?

noisesmith18:09:09

wouldn't you be outside the scope of the clj tool if you are doing anything other than resolving deps and starting a repl?

Alex Miller (Clojure team)19:09:45

no - clj is about a) making classpaths and b) running programs. Any Clojure code can be a program and that program might do anything!

Alex Miller (Clojure team)19:09:16

you run scripts (pass the .clj path) or mains (-m) or use stdin (-)

noisesmith19:09:47

OK, that's fair

Alex Miller (Clojure team)19:09:15

in other words, builds are programs (boot got that right), but dependencies are data (lein did that better)

benzap19:09:25

Interesting point, it would be interesting to see developments on some sort of build tool like gulp to use clj under the hood

benzap19:09:59

I was just following the figwheel.main tutorial, and it seems to use clj directly, so I assumed the direction of the tool was something akin to gulp for clojure

benzap19:09:23

oh wow, there's already a ton of tools

bhauman19:09:09

@benzap also don’t forget that you can just use -e “(user/start-server)” which would go pretty darn far

bhauman19:09:30

and of course there is --init script_name.clj

benzap19:09:24

nice! I'll give it a shot

benzap19:09:17

Understanding this tool more, it seems like a good direction, since you're just wrapping scripts and progressively configuring your build. Leads to a lot less confusion

👍 4