This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-02-21
Channels
- # announcements (39)
- # architecture (7)
- # aws (9)
- # babashka (111)
- # beginners (139)
- # bristol-clojurians (1)
- # calva (47)
- # chlorine-clover (5)
- # cider (17)
- # clj-kondo (26)
- # clojars (25)
- # clojure (251)
- # clojure-berlin (1)
- # clojure-dev (5)
- # clojure-europe (22)
- # clojure-france (1)
- # clojure-hungary (6)
- # clojure-losangeles (8)
- # clojure-nl (18)
- # clojure-spec (3)
- # clojure-uk (68)
- # clojured (32)
- # clojurescript (32)
- # core-async (10)
- # core-typed (120)
- # cursive (8)
- # datascript (10)
- # datomic (11)
- # docker (2)
- # emacs (6)
- # figwheel-main (4)
- # fulcro (10)
- # graalvm (92)
- # hoplon (2)
- # instaparse (9)
- # jobs (3)
- # jobs-discuss (31)
- # joker (2)
- # kaocha (1)
- # lambdaisland (5)
- # leiningen (10)
- # luminus (1)
- # lumo (14)
- # meander (30)
- # mid-cities-meetup (1)
- # midje (1)
- # off-topic (46)
- # pathom (22)
- # perun (2)
- # re-frame (10)
- # reitit (1)
- # remote-jobs (8)
- # shadow-cljs (71)
- # spacemacs (7)
- # sql (40)
- # tools-deps (31)
- # tree-sitter (11)
- # vim (14)
- # vscode (2)
- # xtdb (5)
I'm trying to build my project (https://git.itwont.work/samples) with graalvm and it just runs for a long time (like, half an hour) then runs out of memory
@nihilazo Instead of starting with a big project, it's often better to start with smaller parts and then add things bit by bit to see what's causing this
for reference, all my own graal projects compile within 2 minutes on CircleCI. if it takes longer, I find that suspicious
jvm11 works, but I've ran into some issues with it (unlikely that you will encounter them with hello world)
@nihilazo Are you trying to graalvm compile datahike? That's slightly more advanced than hello world I would say?
I tried that before, but that didn't work for me. that's why I ended up using transit instead of datahike in clj-kondo (I wasn't even able to make a performance benchmark because I never got that far)
if you want to have a fast CLI solution with a database you could consider babashka + the sqlite-cli: https://github.com/borkdude/babashka/blob/master/examples/sqlite.clj
I'm not sure I even really need a database? The queries are super simple (literally just checking if something has a tag, and checking if it doesn't) but there is potentially a reasonable amount of things being checked
actually, that said, I wonder if datahike is actually worse performance-wise than doing something like that manually
@nihilazo What I did for clj-kondo is split the data up into partitions so I can choose to only read from disk what I need. you can tailor this to your problem to make it performant
ah, for this in every case I want to search all the data, and it's stored in a text format on disk already, I might try doing it without any DB at all and see how it goes
might even be able to do everything I was doing with datahike using a couple of filter
s
@nihilazo JUXT published a post today where they plowed through some text files with babashka: https://juxt.pro/blog/posts/babashka.html
if you can make the search faster by pre-organizing it into an indexed way first, that can help
compiling the new datahike-less version results in some unsupported things, but that's progress!
the only thing I'm using outside of my own code is http://clojure.java.io and clojure.string
check out the options that I'm using here: https://github.com/borkdude/jet/blob/master/script/compile
btw, using http://clojure.java.io without a require is bad practice: https://git.itwont.work/samples/tree/src/samples/core.clj?h=no-datahike#n61
I don't mean it to be offending, the reason it's "bad" is that you should not trust other tools to load http://clojure.java.io for you. If you use a linter like clj-kondo, it will warn you about such practices.
I'm not sure. your project doesn't seem to be doing anything weird. does it work with clojure 1.9?
when I try a graal build with 1.9, I get a different error, Error: No instances of
yeah, 20 just got out and has a couple of issues. But reporting those issues definitely helps and they will be fixed.
gotta love arch linux for having the latest everything, even when the latest doesn't work
can graalvm not access environment variables? It builds perfectly with the older graalvm and older clojure but I use System/getenv and that doesn't seem to be working, seems to fail
@nihilazo It depends if you're doing that at compile time or run time. When you execute that call at compile time, the call is made too early
even bb can do this:
$ bb '(System/getenv "PWD")'
"/Users/borkdude/dre/DocSearch/app"
so surely your custom graalvm clojure app can.yeah, so the problem here is that you're doing it top-level: https://git.itwont.work/samples/tree/src/samples/core.clj?h=no-datahike#n15
that won't work, since that captures the value of that environment variable when you compile it, not when you run it
anyway, this is a problem I've had before and the solution is to make reading the environment variable a call at runtime, i.e. in the execution of -main, not top-level/before it.
everything is now working on graalVM! Thank you for helping me out and answering my stupid questions
if you can make a tiny repro of the: > No instances of http://java.io.FilePermission are allowed in the image heap as this class should be initialized at image runtime that would be helpful for fixing it in v20
usually posting your AOT-ed uberjar + native-image build instructions is sufficient for the graalvm devs to reproduce it
well, that's up for them to find out I think. just saying: this works with graalvm 19.1.3 jdk 11 on macOS/linux/Windows but not on v20 .... is sufficient, provided you give them the uberjar + build arguments of native-image
the repo is here: https://github.com/oracle/graal