This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
- # announcements (7)
- # babashka (1)
- # beginners (125)
- # bristol-clojurians (4)
- # calva (49)
- # cider (10)
- # cljs-dev (3)
- # clojure (102)
- # clojure-berlin (3)
- # clojure-czech (4)
- # clojure-europe (84)
- # clojure-france (29)
- # clojure-italy (2)
- # clojure-nl (26)
- # clojure-norway (5)
- # clojure-spec (4)
- # clojure-sweden (4)
- # clojure-uk (50)
- # clojurescript (3)
- # community-development (2)
- # conjure (19)
- # cursive (9)
- # datascript (4)
- # datomic (33)
- # fulcro (4)
- # off-topic (26)
- # parinfer (10)
- # pedestal (4)
- # quil (8)
- # re-frame (1)
- # reagent (20)
- # remote-jobs (5)
- # reveal (53)
- # sci (10)
- # shadow-cljs (37)
- # spacemacs (6)
- # sql (25)
- # tools-deps (24)
- # vrac (2)
- # xtdb (10)
httpkit also compiles with graalvm so you could run this natively. See also here: https://github.com/kloimhardt/bb-web
I read that graalvm doesn't really use less memory at runtime. I haven't confirmed myself.
Let's put it to the test.
$ time clj -J-Xmx5m -e "(+ 1 2 3)" 6 clj -J-Xmx5m -e "(+ 1 2 3)" 2.88s user 0.17s system 232% cpu 1.309 total avg shared (code): 0 KB avg unshared (data/stack): 0 KB total (sum): 0 KB max memory: 108348 MB
$ time bb -Xmx5m -e "(+ 1 2 3)" 6 bb -Xmx5m -e "(+ 1 2 3)" 0.03s user 0.01s system 93% cpu 0.049 total avg shared (code): 0 KB avg unshared (data/stack): 0 KB total (sum): 0 KB max memory: 33488 MB
I think the comment I saw did not figure out how to set max memory on a graal binary. I'd love to test this myself.
This is really intriguing for me because it's a path to sneak in some clj at the dayjob without risking much for the company.
In my situation we run everything as some kind of container workload either via
docker or some other cgroupy thing - wonder if I could get graal to respect that in some way?
yes, graalvm native images respect this: e.g.:
$ bb -Dmy.prop=123 -e '(System/getProperty "my.prop")' "123"
$ bb -XX:+UseCGroupMemoryLimitForHeap error: Could not find option 'UseCGroupMemoryLimitForHeap'. Use -XX:PrintFlags= to list all available options.
I think native can go smaller because the program isn't allocated to memory by java anymore, so there's a little extra space
If you want to know how GraalVM wins in terms of memory, there’s a podcast with the main author
actually I listened to this one: https://airhacks.fm/#episode_78 but the above one might also be interesting
Preliminary testing: openjdk is the fastest when you have a healthy amount of heap (eg 128M). At lower quantities (8/9/10) native dominates, even when using a fallback binary. However, I'm yet to determine if the cost gets eaten into the binary, so the overall memory usage is potentially the same.
Fallback native image is 340M memory with 5M heap, so I'll have to dig in I think :)
@U09LZR36F Recommended flags:
"-H:+ReportExceptionStackTraces" "-J-Dclojure.spec.skip-macros=true" "-J-Dclojure.compiler.direct-linking=true" "-H:ReflectionConfigurationFiles=reflection.json" "--initialize-at-run-time=java.lang.Math\$RandomNumberGeneratorHolder" "--initialize-at-build-time" "-H:Log=registerResource:" "-H:EnableURLProtocols=http,https,jar" "--enable-all-security-services" "-H:+JNI" "--verbose" "--no-fallback" "--no-server" "--report-unsupported-elements-at-runtime"
Reporting back, the RSS is now closer to ~15M, now that I've solved the fallback issue.
On low amounts of memory, native image without fallback completely falls apart. 133 req/s with http-kit under 10M of memory for native, but under OpenJDK that's 8708 req/s. However, the fact is that it depends on how you're hosting. On a shared java host, you might prefer OpenJDK as the heap is more important than the libraries that are brought in. On a VM (ec2, openvz, kvm, etc.) you're more interested in overall memory, so you can give native-image an extra 20M and be fine.
Having said that, Linux only virtually loads the whole java binary, so in theory you might get away with it (albeit with a performance hit on actual low-memory machines where you end up in swap or back on disk).
I only need to serve a few requests per second and the IO is mostly waiting for other service calls. The payloads are tiny
I guess the other possibility would be ClojureScript, compile to JS and target Node perhaps?
There is also a #graalvm channel where folks work on compiling Clojure to native binaries that I believe have smaller requirements than running on the JVM, but I don't know how they compare to ClojureScript. Worth asking there.
@gbson you could make an exceptionally small single-file runtime using clojurescript on node with e.g. shadow-cljs and then compiling to a single binary with https://github.com/zeit/ncc - the one requirement would be that the end user has a node binary on their system. if that's not a blocker for you, i have done similar things before so let me know if you want any tips on setting it up.
and if you don't want to require a node binary and you know the target architecture you can
pkg to bundle your script + node together: https://medium.com/@tech_girl/deploy-your-node-js-application-as-a-single-executable-4103a2508dd7
liveview - wondering if there’s something like that in clojure, please? seems like this project has stalled https://github.com/prepor/liveview-clj
@U04V4KLKC looking at the examples, it seems like it’s sending the whole page html thru the websocket on every event. does that work for you the same way in production?
@U04V4KLKC hmm is there a reason for that? isn’t one of the main points of live view that it’s sending just the diff?
on client side it is using simple morphdom library to rerender dom nodes that needs to change
diff is tricky. you have to take in account the time between send action from client + render + send diff back. during that range client’s state can be changed bu user.
hmm. liveview actually designates dynamic “slots” in the template and then is sending just the values for these slots basically. not full html.
it’s also using morphdom library, just minimizing the payload to almost nothing
yeah, I know about difference between phoenix’s liveview. the goal for that library was simplicity and predictability not minimization of payload. Also it was written in few days )
also you could check my PR — https://github.com/prepor/liveview-clj/pull/1/files it adds ability to register event handlers on the client to react on changes coming from backend
this “feature” theoretically should give you ability to send diffs but without any helpers. If you will need something — let me know, will try to help
@gbson you could also consider fennel-lang, a single binary with the whole lua interpreter + fennel bundled would hover around 400Kb + size of your code (once "compiled" to lua)
then sure, you'll need to add some dependencies to that, and it's definitely not clojure, even if it has some ressemblance
@gbson Someone is experimenting with adding httpkit server + reitit to babashka here:
It can run with low memory (like 5-30mb). bb supports JVM flags like
Not sure if this will be in bb mainline, but it could be added under feature flags, or it could appear as its own bb spin-off. Either way, in that repo there are some downloads available.
One thing I am considering is adding httpkit server to babashka for the next release, but not sure yet about a routing lib.
There's a build of babashka with httpkit server available here: https://github.com/borkdude/babashka/issues/556. You'd have to do your own routing with this. Please leave a note in this issue if you're interested.
So I have a CLI app that I’m running as an uberjar. Is there anything I can do, apart from
:aot :all, to make it start faster?
FWIW my long-term plan is to compile it using GraalVM, but I’m looking for a simpler solution that I can use for the time being
Well, not really… I sometimes run other tasks in the same project, and those do have a ton of dependencies, which do end up in my uberjar. But the parts I use for the CLI only depend on some libraries I’ve written myself, which in turn have no essential dependencies apart from Clojure itself
@U04V15CAJ any suggestions on how to generate the --classpath argument for a lein project?
yeah, I tried that but it turns out I have a few dependencies that do need to be loaded (e.g. clojure.match)
anyway, now that I’ve build an uberjar, I’m getting:
----- Error -------------------------------------------------------------------- Type: clojure.lang.ExceptionInfo Message: Could not resolve symbol: definterface Location: clojure/core/match/protocols.clj:39:2 Phase: analysis ----- Context ------------------------------------------------------------------ 35: (syntax-tag [this])) 36: 37: ;; markers 38: 39: (definterface IExistentialPattern) ^--- Could not resolve symbol: definterface 40: 41: (definterface IPseudoPattern)
so I’m not really using core.match, I just happen to require an ns that requires it (for a fn I don’t use) — is there a way to work around this issue?
@U15RYEQPJ one way: bb supports .cljc +
:bb so you can exclude parts using reader conditionals.
I think direct linking helps with startup time. Also I have been playing around with openJ9 and shared classes and it significantly helps startup times.