Fork me on GitHub
#cljs-dev
<
2019-10-18
>
lilactown07:10:58

Is there a way to measure how much memory a CLJS data structure takes up?

Roman Liutikov08:10:05

In Chrome DevTools this can be measured by comparing two heap snapshots, where first doesn't have anything and second one includes a value of interest assigned to a global variable so it doesn't get GCed

Roman Liutikov08:10:39

Comparing two snapshots shows allocations diff

Filipe Silva11:10:39

there's some caveats to this approach though

Filipe Silva11:10:00

depending on the data structure and the way it is operated upon, it can suffer either optimizations/deoptimizations

Roman Liutikov11:10:57

You mean different runtimes right? That’s true. I imagine numbers can be different between browser JS VMs as well

Filipe Silva11:10:41

in that case, using the same runtime but changing a for loop slightly reduced memory usage by 30 to 70%

Filipe Silva11:10:14

it didn't affect node 10 but did affect node 12

Filipe Silva11:10:18

this was the change

Filipe Silva11:10:32

for (let prop in obj) { -> for (let prop of Object.keys(obj)) {

Filipe Silva11:10:10

I haven't seen it, but this sounds like it's just up my alley 😄

Roman Liutikov11:10:14

there’s a funny moment he describes how having a shorter user name on his machine caused GCC to produce 100x faster code 😄

Filipe Silva11:10:02

you know what, that doesn't surprise me at all

Filipe Silva11:10:19

webpack had a similar problem around two years ago

Filipe Silva11:10:45

it's similar insofar as names and paths would be concatenated indefinitely

Filipe Silva11:10:06

so the longer your file paths were, the quicker you'd hit the memory limit

lilactown16:10:16

This is awesome. Thank you both! What I need this for is to measure the relative size as a data structure grows, so I think Roman's suggestions should suffice.

👍 4
borkdude09:10:14

I have a toJS function exposed here to make working with CLJS values easier from JS. It also converts MetaFns to normal functions so they are easier to work with: https://github.com/borkdude/sci/blob/e895c4524a1a43568e4919315364978ea61df607/src/sci/impl/js.cljs#L36 Does it make sense to add that part to clj->js proper?

borkdude09:10:22

Maybe the use case is too niche.

thheller09:10:17

you can already do that via the protocols clj->js uses

thheller09:10:04

(defprotocol IEncodeJS
  (-clj->js [x] "Recursively transforms clj values to JavaScript")
  (-key->js [x] "Transforms map keys to valid JavaScript keys. Arbitrary keys are
  encoded to their string representation via (pr-str x)"))

borkdude09:10:31

@thheller I'm interested in the bundle size visualisation to see how much of several parts of sci occupy in the final bundle. I remember vaguely you had something for this in shadow-cljs. Any tutorial how to set this up just for the visualization? I have almost 0 experience with shadow.

souenzzo13:10:44

@U04V15CAJ you can generate source maps and inspect with any bundler inspector from npm

borkdude13:10:40

thanks! @thheller already guided me to a solution with shadow-cljs which works quite nicely

👍 4
borkdude09:10:43

Thanks for the protocol pointer.

thheller09:10:23

do you build for the browser? the build reports currently only work for browser builds

borkdude09:10:46

it's a library that can be used in both environments, so also for the browser

thheller09:10:39

basically you just create a simple build config and run the build report normally

thheller09:10:11

:builds {:sci {:target :browser :modules {:sci {:entries [sci.core]}}}}

thheller09:10:22

you may need to setup an actual example use though

thheller09:10:39

if the library is written "correctly" all the code will be removed as dead code otherwise

thheller09:10:50

since nothing is actually used if you just import the ns

borkdude09:10:44

Right. Also a lot of the core functions are pulled in by the lib, but might already be used the the app otherwise, so it wouldn't be 100% on the lib

borkdude09:10:11

Just to get going for now, I'm getting: Can't find 'shadow.cljs.devtools.cli' as .class or .clj for lein run: please check the spelling. I've set :lein true in shadow-cljs.edn. Should I add a dep to project.clj?

thheller09:10:28

yes. include the dep

thheller09:10:00

the build report shows how much your code contributed

thheller09:10:11

so cljs.core code is shown separately

thheller09:10:26

thats likely the (def clojure-core ... etc which can never be removed

thheller09:10:03

acceptable size though. so probably best to set up an actual example with the common uses

borkdude09:10:58

yeah, it was the point of def clojure-core to hold on to all those functions, because you don't know up front which ones the user is going to use in their program string. When I compile sci to an npm library (advanced compiled JS) I end up with ~350KB unzipped

Filipe Silva11:10:51

is it meant for npm consumption by js clients?

Filipe Silva11:10:42

350kb is a fair bit but for nodejs consumers it's not the worst thing in the world

Filipe Silva11:10:47

especially if it's a single lib

Filipe Silva11:10:32

typescript, for instance, is 7mb in a single file

Filipe Silva11:10:11

and TS ships several versions of the compiler, to cater for different consumers

borkdude11:10:00

yeah. I guess TS is just a dev dependency though

Filipe Silva11:10:38

in your case you want to interpret cljs

Filipe Silva11:10:49

but there are plenty of libs that want to interpret ts

borkdude11:10:06

oh, then I'm fine I guess 🙂

Filipe Silva11:10:11

or that require TS to be compiled with, like Angular

borkdude11:10:19

I was more worried about front-end bundle size

Filipe Silva11:10:20

TS is a total of 47mb on a clean install

borkdude11:10:37

it's about 80kb zipped

borkdude11:10:50

which is fine maybe, trade-off

dnolen13:10:20

@alexmiller hey I built a Google Closure Library artifact today - could we get a release?

Alex Miller (Clojure team)13:10:02

Sure, do I just need to press the button on the build box?

dnolen13:10:19

no you have to do those sonatype steps

dnolen13:10:30

we talked about this last time - this could be automated

dnolen13:10:58

the build has the instructions at the top

Alex Miller (Clojure team)13:10:06

that dimly rings a bell

Alex Miller (Clojure team)13:10:41

so you did the build, I just need to do the sonatype part?

Alex Miller (Clojure team)13:10:43

I see two staging repos, one for google-closure-library-third-party and one for google-closure-library. I assume these both need to be released?

Alex Miller (Clojure team)13:10:42

released both, I assume will take a bit to show up

Alex Miller (Clojure team)13:10:33

google-closure-library 0.0-20191016-6ae1f72f and org/clojure/google-closure-library-third-party 0.0-20191016-6ae1f72f are out there now

Alex Miller (Clojure team)13:10:47

the third party jar is ~50% the size of the last release. not sure if that's weird

dnolen13:10:15

@alexmiller yeah both need to be released

thheller15:10:33

oh nice, thanks for the new releases!