This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-08-17
Channels
- # announcements (8)
- # atom-editor (8)
- # aws (1)
- # babashka (96)
- # beginners (128)
- # calva (7)
- # cider (12)
- # cljsrn (1)
- # clojure (75)
- # clojure-europe (28)
- # clojure-hamburg (2)
- # clojure-italy (7)
- # clojure-nl (7)
- # clojure-norway (3)
- # clojure-uk (13)
- # clojurescript (26)
- # conjure (2)
- # cursive (18)
- # data-science (7)
- # datalog (21)
- # datomic (9)
- # duct (15)
- # expound (29)
- # figwheel-main (14)
- # fulcro (59)
- # helix (4)
- # jobs (2)
- # kaocha (19)
- # leiningen (15)
- # luminus (4)
- # malli (57)
- # meander (2)
- # off-topic (2)
- # pathom (12)
- # pedestal (8)
- # re-frame (53)
- # reitit (9)
- # remote-jobs (1)
- # shadow-cljs (64)
- # spacemacs (1)
- # specter (2)
- # tools-deps (12)
- # tree-sitter (2)
- # vim (11)
- # xtdb (17)
i've got a question about expected behavior. lets say i have a jar with a static resource foo.edn
in resources
folder. when i add this jar to my classpath in babashka, i should be able to access this resource by simply (io/resource "foo.edn")
, right? this works in a standard clojure context, but i keep getting nil when i try this with babashka. am i missing something?
:man-facepalming:yup its right there in the issues. didnt check. thanks for the quick response. ill check out master branch and see if it solves mu usecase
Uberjar in action on Windows: https://twitter.com/borkdude/status/1295282956513300483
Nice 🙂
What kind of use cases do you see? Is it being able to run something from both bb and java? Is it a replacement for depstar?
The use case is if you want to package up a project and distribute it over multiple servers for example. Or just distribute it to users: here's the uberjar, run it with bb, no need to clone a project.
ah yeah i understand now. Thanks!
It uses a fork of depstar, but it's not a replacement. Babashka won't be able to do AOT of clojure sources.
So it is some kind of uber-uberscript :)
It's an alternative to uberscript. E.g. when your project uses io/resource
it will still work with uberjar, but not with uberscript. For simple scripts uberscript is still a nice option
I agree. Great addition 🙂
I have a project which runs fine under bb
, and which i can build a graalvm image from, but I can’t get it to uberscript. When running bb --uberscript
it throws a Could not resolve symbol
exception when loading one of the namespaces. The namespace is being required by a ns
form that is three levels deep. The symbol is aero/read-config
, but I don’t think that is important, it is just the first external symbol referenced in the namespace. Does uberscript
affect how require
works?
During the uberscript build, it only looks at the top-level requires and the rest is effectively a no-op
Or we can document it as a caveat / known limitation and recommend the uberjar approach as an alternative
@hugod I notice --uberscript
ignores a (require ...)
form that is not in an ns
form, I think that can still be fixed, if that require uses a literal symbol and is top-level
I’m just using ns
forms. I added a prn
in load-fn
and see it loading all the required namespaces. I’m guessing it just doesn’t evaluate them.
That's right. uberscript doesn't evaluate things, since it would depend on the runtime script arguments which namespaces would be loaded, so it's kind of a form of static analysis. It still uses the interpreter to run through the ns
forms, but turns off function evaluation basically
umm, so why is the defn
form it contains getting evaluated? (and causing the unresolved symbol exception)
Run deps.exe -m makejack.main uberscript
in https://github.com/hugoduncan/makejack/tree/wip
(the project is very much an outline at the moment)
That's something that should not occur. I'll take a look at this later this week, if that's ok
Say I turned on evaluation during the uberscript and I ran the invocation that was printed, I'm getting something like:
{:options {}, :arguments [], :summary -h, --help, :errors nil}
Failed to read makejack file mj.edn: clojure.lang.ExceptionInfo: Config error on line null {:line nil}
clojure.lang.ExceptionInfo: Config error on line null [at line 33, column 7]
That would occur if aero failed to read mj.edn
, or we failed to merge the defaults It should be fine with an empty map in mj.edn
so it prints bb -cp … -m makejack.main --uberscript bin/mj
- is that what you’re running in the makejack project root? makejack.main
is makejack’s main namespace. which is what it tries to uberscript. makejack runs the command after printing it.
yes. that's what I tried to run. I got rid of a couple of checks to that prevents eval during uberscript, but then I got into this error
I seem to be missing something. It didn’t work with the mj.edn
in the tree? https://github.com/hugoduncan/makejack/blob/wip/mj.edn
Within the root. Never mind, I hadn't noticed that there was an mj.edn in the root already
@hugod Found it. The issue is that the uberscript doesn't go beyond this:
https://github.com/juxt/aero/blob/743e9bc495425b4a4a7c780f5e4b09f6680b4e7a/src/aero/core.cljc#L27
because the entire body is wrapped effectively in one big when
:
https://github.com/juxt/aero/blob/743e9bc495425b4a4a7c780f5e4b09f6680b4e7a/src/aero/impl/macro.cljc#L9
So this is the root cause of the problem, I'll have to ponder a little bit about a solution. Suggestions welcome
Maybe the best solution is to just ignore these kinds of errors during the uberscript build
@hugod You can try the uberscript-ignore-unresolved branch of babashka. This will ignore unresolved symbols during analysis.
I'm considering making uberscript even more "accepting": https://github.com/borkdude/babashka/issues/538 It doesn't need to do all the analysis, just skip over the irrelevant forms
@hugod A'ight. I now made uberscript as accepting as possible. It will only inspect ns forms and ignore all the rest in the code.
(So potentially this could even be used for uberscripting JVM Clojure libs that don't even run with bb, as a side effect).
OK, now also top-level require
is supported. I added this to the README:
> Caveat: building uberscripts works by running top-level ns
and require
> forms. The rest of the code is not evaluated. Code that relies on dynamic
> requires may not work in an uberscript.
Hello 🙂 I am relatively new here and playing around a little. I would like to use babashka to get some clojure into my work life. I want to build a cli that connects to a datahike db and to different web apis. Is it currently possible to use babashka to connect to datahike? I tested it by setting the classpath and it did not seem to work, but this could also be because of my inexperience. Currently there exists no pod for datahike, maybe I need to wait for it (or build it). I also appreciate any resources about cli development in babashka / clojure that you came across and found to be helpful 🙂 Any help is much appreciated 🙃
Hi! Datahike is a pretty complex project. I've tried to compile it with GraalVM once but that didn't work. So a pod won't work because of that reason.
Here are some example projects: https://github.com/borkdude/babashka/blob/master/doc/libraries.md#projects
I think I am going to play around a little with datascript. My data-persistence-needs are so minimal, that it would be okay if I could just write/read the whole db to a file and replace it after an update.
OK, in that case you'll need to compile babashka yourself with the appropriate feature flag: https://github.com/borkdude/babashka/blob/master/doc/build.md#feature-flags Datascript could alternatively be turned into a pod, which might be a nice project if you're in for it
I am executing script/compile. I did not explicitly set the max heap size. The process is now running for about 15 minutes, is that normal, or should I have set the max heap size?
yes. DataScript requires a lot of memory, so setting BABASHKA_XMX
to -J-Xmx8g
or more is recommended
Usually with such dependencies there are a few specific things which trigger a lot of memory usage. It could be worthwhile figuring out which parts that are by starting small and adding more incrementally.
It took some time and with the heap to 6500m it worked. Now I am getting an error in some of the datascript examples. When I try to execute the following with babashka:
(d/q '[:find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)]]
{:a [1 7], :b [2 4]}
range)
I get this error:
java.lang.IllegalArgumentException: Class clojure.lang.Keyword[] is instantiated reflectively but was never registered. Register the class by using org.graalvm.nativeimage.hosted.RuntimeReflection [at...
While other examples work fine. Did you encounter similar errors before? (Google is not much of a help)Does the GraalVM error message also point to some location in the source of DataScript?
no, this is the full error message:
java.lang.IllegalArgumentException: Class clojure.lang.Keyword[] is instantiated reflectively but was never registered. Register the class by using org.graalvm.nativeimage.hosted.RuntimeReflection [at /Users/tim/projects/todo-2/src/my/core.clj, line 32, column 1]
contents of the file core.clj:
(require '[datascript.core :as d])
(d/q '[:find ?k ?x
:in [[?k [?min ?max]] ...] ?range
:where [(?range ?min ?max) [?x ...]]
[(even? ?x)]]
{:a [1 7], :b [2 4]}
range)
(and a bunch of commented codewhereas this example works without error:
(require '[datascript.core :as d])
(let [schema {:aka {:db/cardinality :db.cardinality/many}}
conn (d/create-conn schema)]
(d/transact! conn [{:db/id -1
:name "Maksim"
:age 45
:aka ["Max Otto von Stierlitz", "Jack Ryan"]}])
(d/q '[:find ?n ?a
:where [?e :age 45]
[?e :name ?n]
[?e :aka ?a]]
@conn))
The problem is not so much your code, but how GraalVM works. If Clojure code needs reflection then GraalVM can't analyze this ahead of time and you need to add some reflection config. See https://github.com/borkdude/babashka/blob/master/src/babashka/impl/classes.clj. So there are two solutions: Either find out where the reflection happens in DataScript and prevent it or add the appropriate config.
Preventing reflection would be the better solution as it results in smaller more efficient binaries.
Note that GraalVM wants config for the array, not just the keyword class. There are a few examples in that file for other array types.
Could you give me a pointer on where I can find configuration in the file for other array types? If the keyword class is only inserted when features/datascript? -> true, then the standard binary size would not be affected.