hi everyone I'm profiling memory usage on my datomic transactor (1.0.7277)
jcmd 1 GC.class_stats | sort -k12 -n -r
and I'm seeing that a lot of memory is consumed by com.amazonaws.services.s3.AmazonS3Client
Index Super InstBytes KlassBytes annotations CpAll MethodCount Bytecodes MethodAll ROAll RWAll Total ClassName
10374 10238 0 4672 0 103800 312 25861 99320 82872 128544 211416 com.amazonaws.services.s3.AmazonS3Client
I'm not using any s3 related functionality for transactor, is it possible to exclude this class from the build to minimize memory consumption? (200+ Мb is a lot)In general, do not mess with transactor code. Also are you sure that’s not bytes, not kb?
ah, yeah, my bad
KlassBytes just reflected in my brain as K ... Bytes = kilobytes 🤦
thanks @ghadi
I was curious about the Klass term, and then I found an https://openjdk.org/groups/hotspot/docs/StorageManagement.html#:~:text=A%20central%20data,couldn%27t%20use%20that.). You probably use an OpenJdk implementation. So it literally means "Bytes of a Class" > A central data structure of the HotSpot virtual machine is the “klass hierarchy”: the data structure that describes objects in the managed storage of the virtual machine, and provides all the operations on those objects. (We use “klass” with a “k” because “class” is a keyword in C++ so we couldn't use that.)
yeah, that's interesting
I’m still interested in helping you with transactor memory usage. Can you say more about why you’re looking at the transactor?
I was just fine tuning limits for the transactor pod in k8s and decided to understand what memory is used for, I gave it 6Gi limit, it has 4Gb xms=xmx, and at the moment usage is 4765Mi so I was curious what that +700Mb over heap is, but seems like there's nothing serious to worry about
At Nu we always give some spare headroom in k8s for things like offheap memory or direct memory. I think it’s an extra 20%, though that number is likely specific to our setup and you may need less headroom. Still irritating though.
aha, gotcha, thanks @joe.lane 🙏