This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-10-22
Channels
- # announcements (9)
- # asami (52)
- # aws (1)
- # babashka (7)
- # babashka-sci-dev (12)
- # beginners (72)
- # calva (24)
- # cider (9)
- # clj-kondo (76)
- # cljs-dev (15)
- # clojure (19)
- # clojure-australia (4)
- # clojure-europe (33)
- # clojure-france (9)
- # clojure-gamedev (17)
- # clojure-nl (6)
- # clojure-portugal (5)
- # clojure-uk (5)
- # clojurescript (61)
- # clojureverse-ops (4)
- # code-reviews (23)
- # conjure (1)
- # data-science (2)
- # datalevin (6)
- # datomic (49)
- # gratitude (1)
- # helix (24)
- # holy-lambda (14)
- # jobs (3)
- # lsp (92)
- # malli (7)
- # missionary (8)
- # pathom (12)
- # proletarian (3)
- # re-frame (4)
- # remote-jobs (1)
- # shadow-cljs (4)
- # spacemacs (3)
- # sql (9)
- # tools-build (90)
- # vim (1)
- # xtdb (11)
fairly minor dev-time thing, I'm curious what other people's experiences are:
as I understand things build.clj
must exist in the top level of the repo.
the top-level directory of a repo is usually not on the classpath so when developing/iterating the commands in build.clj
I cannot use a REPL as I normally would.
Does anyone else run into this and how do you deal? (What I'm doing now is just developing "blindly" - using edit, run from command-line, loop).
You can put it anywhere you like
It's just a namespace resolved from the paths, which defaults to :paths [“.”]
You can rely on that root and put it in a namespace, or change the paths, or both
There is a gap here with making it easier to get a -T esque classpath and I am going to work on that at some point
As a workaround you can basically use :replace-deps and :replace-paths to simulate the same thing to start a build repl
@danvingo When I'm developing a build.clj
script, I start a REPL with -A:build
to get the dependencies, open up build.clj
and do whatever is your editor's "load file" command. In most editors, a file doesn't need to be on the classpath as long as all the namespaces and libraries it depends on are on the classpath (or already loaded).
You can always load an arbitrary Clojure file into the REPL, as long as its dependencies are accessible.
At least in cursive, if you namespace isn't on the classpath it cannot be loaded into the REPL
Really? That's weird. Clojure has a load-file
function built-in that lets you read in an arbitrary file -- that's what all the editors I've used do.
But then I hear of all sorts of other weird restrictions with Cursive... It doesn't work well with Polylith because it doesn't support :local/root
deps properly (I think that's the issue?) and it doesn't currently support the whole tools/build thing very well (again, because of classpath issues).
I think most of those are due to IntelliJ's model of the world being very Java-centric, based on what I've seen Colin say about his frustrations with trying to support this stuff.
aha yea I was referring to cursive's "load current namespace in REPL" feature. For sure, you could use load-file
if you wanted. - This is all about ergonomics so wanted the convenient thing to be available.
That sounds about right re IntelliJ issues
Thanks for the info - I haven't done the requisite reading up on the -T/tools feature in depth yet so I think that will clarify things for me.
What is the correct way to pass :java-opts
along to an uberjar build? Ive got the java-command
fn and :java-opts
being passed into the uber task, tried to add :cp
with the file path as well. Im trying to pass a production config edn file into the uberjar process to load prod specific vars. We use :jvm-opts ["-Dconf=dev-config.edn"]
in the :run
alias for dev mode, just trying to reflect that for prod in our uberjars. What am I missing here? Do I need a compile step before running uber to utilize the :java-opts
correctly? I put a prod-config.edn
in both env/prod/resources
and at the top project level next to build.clj
.
I am using @seancorfield’s build-clj lib. Currently my uberjar function looks like this:
(defn uber "Build the uberjar." [opts]
(-> opts
(assoc :lib lib :main main
:command-args (b/java-command {:basis basis
:main main
:cp ["env/prod/resources/"]
:java-opts ["-Dconf=./prod-config.edn"]}))
(bb/clean)
(bb/uber)))
what code is supposed to be affected by these jvm-opts?
you can't "set" jvm opts in an uberjar - that has to be done on the java command line that uses it
Were using the cprop lib thats loading some system configs in at startup. It looks for a file at compile time to load from, and fails to compile without it. Hence trying to pass it into the process at compile time, and not just after the uberjar is built when I run the jar. Since its failing to build the jar when it cant find a config file to load.
If you do end up looking at another config library, I'd recommend looking at juxt/aero
. They recommend just one config file that contains both dev
and prod
(and any other) profiles. I've transitioned from cprop
to aero
myself.
Will definitely look into that! Thanks! In the market for any lib to handle various configs throughout our system. Were using a monorepo with Polylith as well which makes things an extra layer of interesting.
I have surveyed config libs three times so far. First time, I ended up with environ
, the next time with cprop
and now have been happy with aero
for a few years now. perhaps the progression reflects the sophistication of the projects I've been working on at the times I did the surveys.
Might just switch to a different lib for dev/prod config loading, or modify the way cprop is handling this so it can at least compile without a config. But was curious if there was a way to handle this current case.
Just gave that a shot but no luck. Wasnt sure how/if :jvm-opts
was being utilized by tools.build
Its okay if there isnt a solution for this the way we've got it set up, I really just wanted to make sure I wasnt missing something obvious in the usage of tools.build. I understand why/how it doesnt function this way with uberjars. Seems like we need to look into other config libs, or at least re-evaluate our usage of cprop
Blocked by TBUILD-18 in tools.build
itself.
sounds like grossness on the use of cprop, it recommends a top level side effecting form (load-config)
which of course means it tries to do stuff when loading the code for compilation
Malli is another library that relies on a JVM property at compile-time 😐
if you replace (load-config)
with (delay (load-config))
and wrap a (force ...)
where needed then you'll at least side step that
Thanks for the info @seancorfield! Sounds like it might be taken care of in the future. Might look into the juxt/aero
lib, or try @hiredman’s suggestion in the meantime
(assuming of course you don't have other top level side effects that would force the config)
For Mono repos with local/roots and git dependencies how do you set it up too compile to an uberjar with .class files for those deps? Currently it appears to just be moving the source code of these dependencies into the uberjar but not compiling dependencies to java byte code?
The issue I'm hitting (and maybe theirs a work around) is running java -jar <my_uberjar>
fails with java.lang.ClassNotFoundException for my local dependency presumably because the libs source not java bytecode is in the jar?
a class not found exception can happen for a number of reasons, and clojure source can be loaded just fine from jars
clojure's aot compilation is transitive, so if you are are aot compiling your source, anything it loads will also be aot compiled
the best thing to do might be to start from the class not found exception, does that class exist in the uberjar, and what is on the stack that is triggering the class not found exception
The error I was getting was Caused by: java.lang.ClassNotFoundException: gen_fhi.fhir_client.protocols.Client
When I ran jar tf target/workspace-0.0.313-standalone.jar | grep gen_fhi/fhir_client
I can see the libs source is included but not a .class file (it's referenced as a local dependency in my deps.edn where I'm running tools.build
$ jar tf target/workspace-0.0.313-standalone.jar | grep gen_fhi/fhir_client
gen_fhi/fhir_client/
gen_fhi/fhir_client/utils.cljc
gen_fhi/fhir_client/protocols.cljc
So the clj source is in the uberjar but no .class file.you are using the interface generated by the protocol somewhere, but not actually loading the code that creates the protocol
somewhere you using gen_fhi.fhir_client.protocols.Client
(the java interface that is generated from the protocol definition) without requiring gen-fhi.fhir-client.protocols
(the clojure namespace that defines the protocol)
In the stack trace it's coming from
at clojure.lang.RT.classForName(RT.java:2212)
at clojure.lang.RT.classForName(RT.java:2221)
at gen_fhi.workspace.db.postgres.core$fn__52985.<clinit>(core.clj:41)
at gen_fhi.workspace.db.postgres.core__init.load(Unknown Source)
at gen_fhi.workspace.db.postgres.core__init.<clinit>(Unknown Source)
Which I am requiring it
(ns gen-fhi.workspace.db.postgres.core
(:require [integrant.core :as ig]
[next.jdbc :as jdbc]
[hugsql.core :as hugsql]
[clj-json-patch.core :as patch]
[clojure.core.match :refer [match]]
[gen-fhi.operation-outcome.core :as oo]
[gen-fhi.workspace.db.postgres.extensions]
[gen-fhi.workspace.ws.protocols :refer [WSDataHandler get-patches-after]]
[gen-fhi.fhir-client.protocols :refer [Client] :as fdb]))
Line 41 is a long defrecord where I use the protocol
(defrecord PGFHIRDatabase [source]
WSDataHandler
(get-patches-after [{:keys [source] :as this} {:keys [workspace] :as ctx}]
(jdbc/with-transaction [tx source]
(let [res (get-latest-version-id tx {:workspace workspace})]
(get-patches-after this ctx (- (:version_id res) 1)))))
(get-patches-after [{:keys [source]} {:keys [workspace]} version-id]
(let [version-id (if (string? version-id) (Integer/parseInt version-id) version-id)]
(get-patches-and-prev-version source {:workspace workspace
:version-id version-id
:limit 10})))
Client
(metadata [this ctx]
(throw (ex-message "Not Implemented")))
(invoke [this ctx op parameters]
(throw (ex-message "Not Implemented")))
(invoke [this ctx type op parameters]
(throw (ex-message "Not Implemented")))
(invoke [this ctx type id op parameters]
(throw (ex-message "Not Implemented")))
(search [{:keys [source]} {:keys [workspace]} parameters]
(let [res (search-system source {:workspace workspace
:cols ["resource"]
:where (when-let [where-clause (parameters->where-clause parameters)]
(where-snip
where-clause))})]
is gen-fhi.workspace.db.postgres.core being loaded possibly as part of some kind of plugin system?
It shouldn't be. When I look at the jar it has .class files for it
gen_fhi/workspace/db/postgres/core$fn__53038.class
gen_fhi/workspace/db/postgres/extensions$__GT_pgobject.class
gen_fhi/workspace/db/postgres/core$fn__53034.class
gen_fhi/workspace/db/postgres/core$loading__6737__auto____52859.class
gen_fhi/workspace/db/postgres/core$eval52915.class
gen_fhi/workspace/db/postgres/core$property_search.class
gen_fhi/workspace/db/postgres/core__init.class
$ jar tf target/workspace-0.0.313-standalone.jar | grep workspace/db/postgres.core
gen_fhi/workspace/db/postgres/core$fn__52985.class
gen_fhi/workspace/db/postgres/core$parameters__GT_where_clause.class
gen_fhi/workspace/db/postgres/core$fn__52867.class
gen_fhi/workspace/db/postgres/core$fn__52947.class
gen_fhi/workspace/db/postgres/core/
gen_fhi/workspace/db/postgres/core$parameters__GT_where_clause$fn__52955$fn__52960.class
gen_fhi/workspace/db/postgres/core$fn__52985$map__GT_PGFHIRDatabase__53030.class
gen_fhi/workspace/db/postgres/core$fn__52861.class
gen_fhi/workspace/db/postgres/core$eval52885.class
gen_fhi/workspace/db/postgres/core$fn__52903.class
gen_fhi/workspace/db/postgres/core$parameters__GT_where_clause$fn__52955.class
gen_fhi/workspace/db/postgres/core.clj
gen_fhi/workspace/db/postgres/core$fn__52985$__GT_PGFHIRDatabase__53028.class
gen_fhi/workspace/db/postgres/core$fn__53038.class
gen_fhi/workspace/db/postgres/core$fn__53034.class
gen_fhi/workspace/db/postgres/core$loading__6737__auto____52859.class
gen_fhi/workspace/db/postgres/core$eval52915.class
gen_fhi/workspace/db/postgres/core$property_search.class
gen_fhi/workspace/db/postgres/core__init.class
gen_fhi/workspace/db/postgres/core$property_search$fn__52951.class
gen_fhi/workspace/db/postgres/core$fn__52911.class
gen_fhi/workspace/db/postgres/core$eval52871.class
gen_fhi/workspace/db/postgres/core$eval52929.class
gen_fhi/workspace/db/postgres/core/PGFHIRDatabase$fn__53010.class
gen_fhi/workspace/db/postgres/core/PGFHIRDatabase$fn__53014.class
gen_fhi/workspace/db/postgres/core/PGFHIRDatabase$fn__52994.class
gen_fhi/workspace/db/postgres/core/PGFHIRDatabase$reify__52989.class
gen_fhi/workspace/db/postgres/core/PGFHIRDatabase.class
$
the aot compilation of gen-fhi.workspace.db.postgres.core should cause aot compilation of gen-fhi.fhir-client.protocols then
there were some bugs in the aot compilation stuff (the order in which it aot compiled could lead to problems), I don't see how that could cause this, but you never know
This was my build script:
(ns build
(:require [clojure.tools.build.api :as b]))
(def workspace-lib 'gen-fhi/workspace)
(def version (format "0.0.%s" (b/git-count-revs nil)))
(def class-dir "target/classes")
(def basis (b/create-basis))
(def uber-file (format "target/%s-%s-standalone.jar" (name workspace-lib) version))
(def src-dirs ["clj" "cljc" "test" "migrate" "resources" "cli"])
(defn clean [_]
(b/delete {:path "target"}))
(defn figwheel []
(b/process {:command-args ["clojure" "-m" "figwheel.main" "-O" "advanced" "-bo" "dev"]}))
(defn uber [_]
(clean nil)
(figwheel)
(b/copy-dir {:src-dirs src-dirs
:target-dir class-dir})
(b/compile-clj {:basis basis
:src-dirs src-dirs
:class-dir class-dir})
(b/write-pom {:class-dir class-dir
:lib workspace-lib
:version version
:basis basis
:src-dirs src-dirs})
(b/uber {:class-dir class-dir
:uber-file uber-file
:basis basis
:main 'gen-fhi.workspace.core}))
It's mostly copied from the reference material with modification to source dirs.
Tools.build I'm on the latest sha from the repo.It should delete the directory containing it "target" I can also delete that dir specifically as well for sanity.
Should have immediatlly included this but this is the entire stack trace when I run java -jar
have you looked in target/classes after running uber, are is there a class file from gen-fhi.fhir-client.protocols
in there?
No their isn't .class files in there for gen-fhi.fhir-client.protocols. The only files that exist there appear to be compiled assets and resources from the project I'm running tools.build. I do have user.clj file
$ jar tf target/workspace-0.0.313-standalone.jar | grep user
user$start_server.class
user$fn__52521.class
user__init.class
user$reset.class
user.clj
user$loading__6737__auto____52519.class
so it is being loaded before aot compilation happens, which is causing anything it loads not to be properly aot compiled
and then it is being loaded again as your uberjar loads clojure before anything else is loaded
if you take your user.clj file and comment everything out then rebuild that will likely fix everything
Ah so this is happening when I'm executing the jar itself it's running the user.class without having AOT compiled for the deps in there?
because clojure will load the first user.clj it finds some tutorials use it for setting up dev tooling
I'm assuming this would only affect my local project or would dependencies user.clj also present issues?
but it is kind of brittle, and you need to make sure it isn't included when aot compiling or jaring
the first user.clj clojure finds, clojure itself just pulls stuff from the classpath
Yea I think that solved it. I'm getting a different issue now but its unrelated and probably from some generated code I have
Execution error (IndexOutOfBoundsException) at clojure.asm.MethodWriter/computeMethodInfoSize (MethodWriter.java:2061).
Method code too large!
Full report at:
/var/folders/7m/3hkb8jgx3bnfcx4dpm1hqfqr0000gn/T/clojure-18076644096480273651.edn
Execution error (ExceptionInfo) at clojure.tools.build.tasks.compile-clj/compile-clj (compile_clj.clj:89).
Clojure compilation failed
Thanks @hiredman I would not have figured this out on my own.The fact that you got class files for a namespace, but not for namespaces it depended on is kind of the tell
It generally means something was causing those depended on namespaces to be loaded before compilation
@chesslunatic You probably want to ensure user.clj
is in a separate folder (e.g., dev
) that is only added to your classpath via an alias (e.g,. :dev {:extra-paths ["dev"]}
) so that it won't interfere with JAR building.
:thumbsup: Yep I'll be doing that in the future thanks @seancorfield