Fork me on GitHub
#clojure
<
2019-12-21
>
quadron02:12:57

God I hate javafx with all my soul. what am I doing wrong? even the lightest pure text gui eats up ALL (15 GB) the memory until the whole computer grinds to a halt and requires hard reset. I have tried the following options too in leiningen project.clj :

:jvm-opts ["-XX:+UseG1GC" "-Xms2g" "-Xmx2g"]
and its not just my applications. it's the same for nightcode, scenebuilder, anything javafx. what gives? should I just switch to electron? is swing any better? what other choices do I have for building a gui?

andy.fingerhut03:12:16

If the JVM process is actually running with those command line options, I am surprised if it is consuming more than 2GBytes of memory. What measurement are you taking that leads you to conclude it is going over that?

jumpnbrownweasel04:12:03

I don't use javafx but I am curious about this, since I know that DirectBuffers are often used for graphics work and these are allocated outside of the Java heap. I found a couple of related issues but they may not apply to your case. https://bugs.openjdk.java.net/browse/JDK-8192056 https://stackoverflow.com/questions/26247746/bizarre-javafx8-outofmemoryerror-fadetransition/26250775

jumpnbrownweasel04:12:05

Probably worth asking in a Javafx forum.

didibus06:12:21

I think by default Xmx also sets the max allowed off heap memory

didibus06:12:35

So my guess here is you're not properly setting Xmx flag maybe?

didibus06:12:30

Also, default Xmx is supposed to be 1/4 of your physical memory. So this is puzzling

quadron06:12:37

thanks for the hints. I guess I have to mess around with jvm-opts and monitor the memory consumption of different fx apps more closely.

didibus06:12:01

At a minimum, it shouldn't max out your ram, you should be abke to limit Java's memory usage and it should throw an outofmemory error if it runs out

quadron06:12:13

how am i supposed to pass these flags to a java app (like nightcode) from cli?

quadron06:12:11

do i have to set these options globally?

didibus06:12:21

Hum, for night code I don't know. It probably has a launcher script, that calls java, and that must be setting those (or not setting them)

didibus06:12:40

What if you call less on the CLI? It could be a bash script

didibus06:12:06

But I don't even think night code is a Java Fx app? Is it?

andy.fingerhut06:12:27

Cognitect REBL uses Java FX, and while I haven't used it long, I know plenty of people who do, and haven't heard complaints about memory usage.

didibus06:12:50

Oh, night code seems to use javapackager to create a bundle. I don't know if its possible to then change the JVM options

andy.fingerhut06:12:11

On which OS are. you running Nightcode?

quadron06:12:08

linux. passing the jvm options directly to the nightcode bin seemed to have worked.

quadron06:12:15

hmmm, i'll just play with it for awhile to see what happens. left to their own configurations the javafx apps eat all the memory until the system freezes.

didibus06:12:54

Oh ya, it is a javafx app

didibus06:12:32

Someone said online some JVMs now default to using 75% of your ram. I think especially server ones, which is likely what the ones that come with Linux distros are going to be targetting

didibus06:12:38

With Xmx being 75% of your total ram. It doesn't leave a lot of room. So if the other apps use more than 25% of your Ram your OS would start to trash and Linux isn't able to recover from trashing

didibus07:12:55

Try running

didibus07:12:11

java -XX:+PrintFlagsFinal -version | grep -iE 'HeapSize|PermSize|ThreadStackSize'

didibus07:12:30

It'll show you the defaults for your JVM

quadron07:12:41

does this look normal to you?! size_t MaxMetaspaceSize = 18446744073709547520 {product} {default}

didibus07:12:33

Mine is: uintx MaxMetaspaceSize = 18446744073709547520 {product}

didibus07:12:50

What's your MaxHeapSize ?

quadron07:12:16

size_t MaxHeapSize = 5263851520 {product} {ergonomic}

quadron07:12:34

uintx MaxHeapFreeRatio = 70 {manageable} {default} uintx MaxMetaspaceFreeRatio = 70 {product} {default}

didibus07:12:14

5263851520, that's 5GB I think

didibus07:12:05

Do you have 5GB of free RAM ?

didibus07:12:11

Before starting the Java application ?

didibus07:12:55

In fact its 5.25 GB

didibus07:12:17

Well, so it seems if you start a java app, which does not set Xmx, you'd default to having a Xmx of 5.25GB

quadron07:12:49

by the way, htop shows 87.5G VIRT memory for the nightcode

quadron07:12:12

I'm not sure what it means, but its way higher than any other running process

didibus07:12:17

Is AggressiveHeap false?

quadron07:12:53

the other processes all consume less than 2G VIRT memory

didibus07:12:15

Woa, 87G !!

didibus07:12:26

That's very weird

didibus07:12:48

What is USED and RET ?

quadron07:12:12

I just killed nightcode (which was launched with nightcode -XX:+UseG1GC -Xms1g -Xmx1g ). right after death, about 9G memory was released.

quadron07:12:56

now I just launched it again and top shows 102G VIRT !!!

didibus07:12:18

Ya something is a bit crazy there

didibus07:12:34

VIRT includes memory mapped files, as well as swap and real

didibus07:12:02

Let me try getting nightcode on my computer

quadron07:12:43

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 8395 veixq5 20 0 102,6g 449016 116936 S 3,2 2,2 0:29.37 java

didibus07:12:08

PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                            
31905 ------    20   0  6.048g 0.439g 0.094g S 4.983 11.78   1:02.39 Nightcode 

didibus07:12:42

This is running on my linux laptop, which has 4GB of RAM

quadron07:12:44

looks reasonable to me

quadron07:12:23

is the VIRT high regardless, with respect to other processes? on your system?

didibus07:12:55

A little, firefox is my second most memory consuming process, and it has 4g VIRT

didibus07:12:09

With 0.300g RES

quadron07:12:18

thanks for your help. I guess I might try running these apps in another OS through virtualbox.

didibus07:12:19

Might be VIRT is not a big deal

didibus07:12:26

I think RES is what matters

quadron07:12:50

I agree, but 102G is very exorbitant.

didibus07:12:49

Doesn't really make it clear, but since to indicate it is neither the amount of physical or swap memory used, but just the amount of address space allocated. Dunno what that means

andy.fingerhut07:12:28

I believe RES memory matters much more than VIRT when it comes to thrashing

👍 4
andy.fingerhut08:12:23

I see crazy large numbers for VIRT all of the time, and have never found a reason to pay attention to them.

quadron08:12:26

in any case, tweaking the jvm heap size and GC seems to have been effective. I'll leave nightcode open for a few hours.

didibus08:12:52

Whats your JVM ?

quadron08:12:54

fingers crossed.

didibus08:12:57

Just curious

didibus08:12:17

OH, well, nevermind for nightcoce its the one bundled with it anyways

quadron08:12:18

four different ones

quadron08:12:21

tried all of them

quadron08:12:57

java-11-openjdk java-13-openjdk (default) java-8-jdk java-8-openjdk

quadron08:12:54

nightcode is spawned by java command though

quadron08:12:00

java -jar ...

didibus08:12:50

Well the nightcode bin is packaged with javapackager

didibus08:12:56

Unless you start it differently?

didibus08:12:09

And javapackager packages a JVM with it

didibus08:12:20

So my nightcode won't be using java. but I used the RPM from the website

quadron08:12:08

not sure but it looks like it is using the system

quadron08:12:09

I have had the same issue with scenebuilder too. gonna try REBL too.

didibus08:12:21

Oh, there is one thing though

didibus08:12:33

Nightcode actually starts a Clojure REPL as well, that one will show as java

didibus08:12:40

But you should also see an entry labeled Nightcode

didibus08:12:17

I believe Nightcode itself runs using its own bundled JVM, but the REPL it launches uses your system's JVM

Jeremy11:12:41

Sigh. Are there any tricks for using lein uberjar when you (:require [aleph.http])? It just seems to hang for me and I suspect it's because netty's running on a thread and preventing completion.

Jeremy11:12:13

Hmm. aleph gives some netty log messages (when used in combination with taoensso/timbre and slf4j adapters) but doesn't prevent uberjar. yada gives a reflection warning but doesn't prevent uberjar.Must be something else!

Vasilii Avdiuhskin13:12:40

aleph/start-http usually starts in the background AFAIK. I remember jetty adapter had this parameter called join? which by default blocked the current thread, but I never experienced that with aleph. Do you provide any additional options when starting the server?

Jeremy22:12:51

Thanks Vasilii actually I worked out aleph is fine. The simple act of requiring cljfx was the obstruction!

borkdude13:12:39

How do I get this working with lein?

:compile-java {:jvm-opts ["-Xlint:unchecked"]
                            :java-source-paths ["src-java"]}
lein with-profiles +compile-java do clean, javac
Unrecognized option: -Xlint:unchecked

borkdude13:12:08

Got it: :javac-options ["-Xlint:unchecked"]

vemv13:12:04

TIL about javac-options :)

Hi14:12:31

how to read lazy sequence of text line by line?

Hi18:12:27

What is the best way to parse a string into sensible peaces? For example this one: Dec 21 17:38:43 hostname script.sh[483]: 19-12-21 17:38:43 hostname INFO [namespace.function:216] - logstring . How to get from this string tockens like date, loglevel, namespace/function/line , logstring, etc...

darwin18:12:03

in general regular expressions, that is not clojure related, here are some examples how to use them in clojure: https://purelyfunctional.tv/mini-guide/regexes-in-clojure/

vemv21:12:35

If under your control, I'd try hard to favor data over text https://clubhouse.io/blog/structured-logging/

didibus22:12:39

And for more complicated parsing, Instaparse and Antlr can be used, though I don't think you need these in your case. Regex or Spec should do

👍 4