This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-03-14
Channels
- # announcements (10)
- # architecture (3)
- # atom-editor (1)
- # babashka (53)
- # babashka-sci-dev (118)
- # beginners (74)
- # biff (10)
- # calva (13)
- # clara (13)
- # clerk (20)
- # clj-commons (17)
- # clj-kondo (6)
- # cljdoc (19)
- # cljs-dev (3)
- # clojars (2)
- # clojure (63)
- # clojure-art (2)
- # clojure-europe (68)
- # clojure-nl (1)
- # clojure-norway (6)
- # clojure-uk (3)
- # clojured (19)
- # clojurescript (34)
- # clr (1)
- # cursive (11)
- # emacs (12)
- # fulcro (3)
- # helix (2)
- # holy-lambda (2)
- # honeysql (27)
- # hyperfiddle (36)
- # malli (2)
- # off-topic (72)
- # polylith (4)
- # rdf (20)
- # re-frame (20)
- # reitit (4)
- # rewrite-clj (14)
- # shadow-cljs (17)
- # slack-help (2)
- # tools-deps (36)
- # xtdb (3)
I'm a bit puzzled with running a jar file created using tools.build
.
When running the app with REPL the logging works just fine.
But. I created a jar file using tools.build
. And when running the standalone jar file I get: java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
I provide some more information in the:thread:not to pollute this main level with long listings.
deps.edn:
{:paths ["resources"]
:aliases {:dev {:extra-paths ["dev-resources" "classes"]
:extra-deps {org.clojure/clojure {:mvn/version "1.11.1"}
thheller/shadow-cljs {:mvn/version "2.20.20"}
...
:backend {:extra-paths ["src/clj"]
:extra-deps {metosin/ring-http-response {:mvn/version "0.9.3"}
...
org.clojure/tools.logging {:mvn/version "1.2.4"}
commons-logging/commons-logging {:mvn/version "1.2"}
org.slf4j/slf4j-api {:mvn/version "2.0.6"}
ch.qos.logback/logback-classic {:mvn/version "1.4.5"}
...
:build {:deps {io.github.clojure/tools.build {:mvn/version "0.9.4"}}
:ns-default build}
build.clj:
(ns build
(:refer-clojure :exclude [test])
(:require [clojure.tools.build.api :as b]))
(def lib 'karimarttila/webstore)
(def main 'backend.core)
(def class-dir "target/classes")
(defn- uber-opts [opts]
(assoc opts
:lib lib
:main main
:uber-file (format "target/%s-standalone.jar" lib)
:basis (b/create-basis {:project "deps.edn"
:aliases [:common :backend :frontend]
}
)
:class-dir class-dir
:src-dirs ["src/clj" "src/cljc"]
:ns-compile [main]))
(defn uber [opts]
(println "***** Building uberjar *****")
(println "Cleaning...")
(b/delete {:path "target"})
(let [opts (uber-opts opts)]
(println "Copying files...")
(b/copy-dir {:src-dirs ["resources" "src/clj" "src/cljc"]
:target-dir class-dir})
(println "Compiling files...")
(b/compile-clj opts)
(println "Creating uberjar...")
(b/uber opts)))
Build uberjar:
clj -T:build uber
Run uberjar:
java -jar target/karimarttila/webstore-standalone.jar
=> java.lang.NoClassDefFoundError: clojure/tools/logging/impl/LoggerFactory
When I unzip the uberjar and search for log / factory related classes:
λ> fd | grep -i log | grep -i factory
./org/slf4j/LoggerFactoryFriend.class
./org/slf4j/ILoggerFactory.class
./org/slf4j/helpers/SubstituteLoggerFactory.class
./org/slf4j/helpers/NOPLoggerFactory.class
./org/slf4j/LoggerFactory.class
./org/slf4j/spi/LoggerFactoryBinder.class
./org/apache/commons/logging/impl/LogFactoryImpl.class
./org/apache/commons/logging/impl/LogFactoryImpl$1.class
./org/apache/commons/logging/impl/LogFactoryImpl$3.class
./org/apache/commons/logging/impl/LogFactoryImpl$2.class
./org/apache/commons/logging/LogFactory$3.class
./org/apache/commons/logging/LogFactory$1.class
./org/apache/commons/logging/LogFactory$6.class
./org/apache/commons/logging/LogFactory$5.class
./org/apache/commons/logging/LogFactory.class
./org/apache/commons/logging/LogFactory$2.class
./org/apache/commons/logging/LogFactory$4.class
./ch/qos/logback/core/sift/AppenderFactoryUsingSiftModel$1.class
./ch/qos/logback/core/sift/AppenderFactoryUsingSiftModel.class
./ch/qos/logback/core/sift/AppenderFactory.class
./ch/qos/logback/core/net/ObjectWriterFactory.class
./ch/qos/logback/core/net/QueueFactory.class
./ch/qos/logback/core/net/ssl/SSLContextFactoryBean.class
./ch/qos/logback/core/net/ssl/KeyStoreFactoryBean.class
./ch/qos/logback/core/net/ssl/TrustManagerFactoryFactoryBean.class
./ch/qos/logback/core/net/ssl/SecureRandomFactoryBean.class
./ch/qos/logback/core/net/ssl/ConfigurableSSLServerSocketFactory.class
./ch/qos/logback/core/net/ssl/ConfigurableSSLSocketFactory.class
./ch/qos/logback/core/net/ssl/KeyManagerFactoryFactoryBean.class
./ch/qos/logback/core/testUtil/MockInitialContextFactory.class
./ch/qos/logback/classic/util/StatusViaSLF4JLoggerFactory.class
./ch/qos/logback/core/model/ModelHandlerFactoryMethod.class
./ch/qos/logback/core/joran/util/beans/BeanDescriptionFactory.class
And the main entry point of the app starts with:
(ns backend.core
(:require
[clojure.tools.logging :as log]
...
(:gen-class))
...
Actually. Now investigating this a bit further, I realized that the clojure/tools/logging package is not compiled:
λ> fd -HI | grep -i clojure | grep -i logging
./clojure/tools/logging.clj
./clojure/tools/logging
./clojure/tools/logging/test.clj
./clojure/tools/logging/impl.clj
./clojure/tools/logging/readable.clj
...
(but no .class files found)
You probably need to set :src-dirs and :class-dir for compile-clj
I thought I have already done that:
(def class-dir "target/classes")
...
(defn- uber-opts [opts]
(assoc opts
...
:class-dir class-dir ;;; ======> HERE
:src-dirs ["src/clj" "src/cljc"] ;;; ======> HERE
:ns-compile [main]))
(defn uber [opts]
...
(let [opts (uber-opts opts)]
(b/copy-dir {:src-dirs ["resources" "src/clj" "src/cljc"]
:target-dir class-dir})
(b/compile-clj opts)
Sorry, lot of stuff to read
No problem. 🙂
There is no hurry regarding this issue. I think I will investigate this a bit more in the weekend.
You’re not getting the class-dir in your uberjar
Or at least that’s the thing to check - are the classes in target/classes
λ> ll target/classes/
total 36
drwxrwxr-x 5 kari kari 4096 Mar 14 15:11 ./
drwxrwxr-x 3 kari kari 4096 Mar 14 15:11 ../
drwxrwxr-x 3 kari kari 4096 Mar 14 15:11 backend/
-rw-rw-r-- 1 kari kari 1099 Feb 20 20:18 config.edn
drwxrwxr-x 2 kari kari 4096 Mar 14 15:11 data/
-rw-rw-r-- 1 kari kari 3292 Feb 16 15:02 logback.xml
drwxrwxr-x 2 kari kari 4096 Mar 14 15:11 public/
-rw-rw-r-- 1 kari kari 218 Feb 13 19:33 README.txt
-rw-rw-r-- 1 kari kari 769 Mar 14 12:18 user.clj
=> should I have at this point all dependencies as class files here as well?> should I have at this point all dependencies as class files here as well? no
but are there classes under backend/ ?
λ> ll target/classes/backend/
...
-rw-rw-r-- 1 kari kari 1100 Mar 14 15:11 'core$create_users.class'
-rw-rw-r-- 1 kari kari 1315 Mar 14 15:11 'core$env_value.class'
...
-rw-rw-r-- 1 kari kari 2492 Mar 14 15:11 'core$fn__20803.class'
-rw-rw-r-- 1 kari kari 3267 Mar 14 15:11 'core$loading__6789__auto____20754.class'
-rw-rw-r-- 1 kari kari 3167 Mar 14 15:11 'core$_main.class'
-rw-rw-r-- 1 kari kari 2118 Mar 14 15:11 'core$read_config.class'
-rw-rw-r-- 1 kari kari 3152 Mar 14 15:11 'core$system_config.class'
-rw-rw-r-- 1 kari kari 851 Mar 14 15:11 'core$system_config_start.class'
-rw-rw-r-- 1 kari kari 1956 Mar 14 15:11 core.class
-rw-rw-r-- 1 kari kari 3664 Mar 14 13:10 core.clj
-rw-rw-r-- 1 kari kari 5650 Mar 14 15:11 core__init.class
drwxrwxr-x 2 kari kari 4096 Mar 14 15:11 db/
-rw-rw-r-- 1 kari kari 13234 Mar 14 11:14 webserver.clj
so revising my answer above, you shouldn't have dependent jars here but you should have any needed compiled classes from those dependencies. in particular, tools.logging will detect (at runtime) a logging factory to load and because of that, you're probably not getting that logging factory to compile just from loading backend.core (which pulls in tools.logging)
I'm not sure which logging factory you intend to use - seems like there are a bunch of loggers available in your deps
I think maybe just explicitly including the logging impl to the namespaces to compile would fix the issue though :ns-compile [main clojure.tools.logging.impl]
Ok. Thanks! I will try! 🙂
or actually, maybe better to swap the order of those
You have user.clj
in clj/src
-- that is breaking things.
If you move that to a folder that is only added in :dev
, your uberjar will work.
Ah! Let's try!
I was curious about it because we use tools.logging
and we don't need to explicitly AOT compile the impl
ns -- just the app's main ns -- so I cloned your repo and rebuilt it without user.clj
and, sure enough -- compiled tools.logging
stuff:
> jar tvf target/karimarttila/webstore-standalone.jar |fgrep tools/logging
1421 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging$find_factory.class
2469 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging$loading__6789__auto____142.class
5145 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging$logp.class
1856 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging$fatalf.class
0 Tue Mar 14 09:40:02 PDT 2023 clojure/tools/logging/
657 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging$log_stream$fn__291$fn__292.class
1949 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging$fn__299$log_uncapture_BANG___304.class
18496 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging__init.class
...
767 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging/impl$fn__192$G__188__195.class
13861 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging/impl__init.class
6668 Tue Jan 04 14:42:06 PST 2022 clojure/tools/logging/readable.clj
1232 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging/impl$reify__216.class
1356 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging/impl$reify__218.class
1909 Tue Mar 14 09:39:51 PDT 2023 clojure/tools/logging/impl$find_factory.class
...
Thank you so much! 🙂
Yep! It works now! Thank you all so much!
Now I just have to add frontend to the standalone jar. I believe I can do that. 🙂
User strikes again, maybe I should detect and warn about that
Damn. The frontend step was easy.
Maybe I write a blog post "Creating a production build for a full-stack Clojure app"?
I mostly write these blog posts for myself - what I have learned. But maybe someone might benefit also regarding a blog post like that.
I see so many people get tripped up with user.clj
-- it seems like a real foot-gun feature of Clojure these days... what was the original idea behind why it was added?
At least for now on, I remember to keep it in:
:aliases {:dev {:extra-paths ["dev-resources" "dev-src"]
=> dev-src
🙂This is a great community. I'm pretty sure I couldn´t have figured that out myself.
I avoid the auto-load altogether by having an explicit file -- not called user.clj
-- in a dev-only path and a dev-only alias that starts my REPL and explicitly loads that file -- so there's no chance that Clojure will start loading code on its own. Color me paranoid 🙂
user.clj is very old, added well before I was even using Clojure, so not sure what the original thinking was
"user.clj considered harmful" ? 🙂
I'm certainly in that camp. I've never used user.clj
in... nearly 13 years of Clojure programming...
I write a blog post about this building story - and add a chapter called "user.clj considered harmful" 🙂
https://github.com/clojure/clojure/commit/dff8bca0286f2af46ab5522cf9d86165ab83eaa6 ?
@U064X3EF3 Interesting. That adds a load of user.clj
but doesn't remove the current machinery that auto-loads it? and that machinery has changed since to use maybeLoadResourceScript()
🙂 (nice to see the DCL for the root classpath -- that will address socket REPL etc, yes?)
that's a very old commit from when user.clj was added
Oh, I misread the stuff at the top of that page since I saw 1.12-alpha1
I looked at the google group and irc log from around that time, which was when all the ns/require/use stuff was percolating (grep for jewel.clj or lib.clj for some ancient discussions there), but didn't actually find any discussion specifically of user.clj
"committed on Apr 20, 2008" 🙂 My bad!
I'd be interested to hear if Rich remembers why it was decided to auto-load user.clj
😉
https://www.karimarttila.fi/clojure/2023/03/14/creating-uberjar-for-clojure-fullstack-app.html
I am a bit puzzled by the following definition in book: "Everything Is a Sequence Every aggregate data structure in Clojure can be viewed as a sequence." from Programming Clojure, 3rd Edition by Alex Miller, Stuart Halloway, Aaron Bedra. link https://www.oreilly.com/library/view/programming-clojure-3rd/9781680505719/f_0028.xhtml 1. "Everything Is a Sequance" I don't think so i.e. number 1 is not a sequance. So we can't say "Everything Is a Sequance". 2. "Every aggregate data structure in Clojure can be viewed as a sequence." This term "aggregate data" is only once used in book and there is no definition of it. I've found a definiton in my notes (but I don't remember the source) 'data type that holds more than one thing'. I was thinkig about it and I think it should be 'data type that can hold more than one thing' because list with one or zero elements is also a 'aggregate data' and can be viewed a sequence. What do you think?
1. Of course there are primitive elements like 1
and true
that can't easily be interpreted as sequences. As the next phrase indicates, the title doesn't try to to hint at such elements. But the aggregate data structures
can be thought of as sequences.
2. Think of aggregate data structure
as just your typical data structure. The qualifying word aggregate
just seems to be included to clarify that indeed it's those typical data structures that they're referring to here. https://www.merriam-webster.com/dictionary/aggregate
"Everything is a sequence" is a section title, not an assertion. I think it's just there to theme the idea that collections can be viewed as sequences.
This section is very important for Clojure, partly because it discerns it from other Lisps that you might know.
Much of its core library and the runtime deal in sequences, which is an abstraction over concrete data structures like vectors, maps, lists, sets and lazy collections generated by functions.
The key idea here is that you're programming against this abstraction if you are dealing with anything that has a concrete collection behind it.
This section wants you to think of all of these things as sequences in order to see why all of the sequence functions work on these concrete data structures.
Note that later you'll be introduced to transducers, which take this a step further. If you think of sequence transformations as reducing them with conj
, producing a new sequence, then that conj
can be parametrized or pulled out so to speak in order for you to compose generic step transformations that can be applied to sequences, channels, streams maybe websocket messages or whatever can be seen as doing work on individual steps in a series of steps.
So ultimately the "Everything is a Sequence" statement is very contextual. A more precise statement would be "All of those things are sequences and here's what you can do with them".
Oh and in a sense the statement is very true, since Clojure is a Lisp, it is defined in terms of its own data structures. In that sense everything is really a sequence. You might get error messages from say your REPL or from a linter when some form isn't proper. That's because your code is just a data structure in a certain shape. You can produce these forms and transform them with sequence functions. So everything kind of is a sequence.
In the web app am working on, am defining spec in one namespace and using it in another. I use the spec in my namespace like this :path-to-where-spec-resides/name-of-spec-definition
When I run the application, I dont get any errors and app runs as expected. But when I compile using leinuberjar and run the jar I get the following error.
Started app
Execution error (AssertionError) at spec-tools.core/create-spec (core.cljc:536).
Assert failed: Unable to resolve spec: :path-to-where-spec-resides/name-of-spec-definition
(get-spec spec)
Does anyone have an idea what this might be and how it can be solved 🙂Specs are in a global registry and not the ns where you def them with s/def
which might be the issue
I think that is the issue. So if not with the namespace then how can one call it ?
When I give just the spec name like :name-of-spec-definition
it give Unable to resolve spec:
error.
is the namespace which defines your spec in your uberjar? And do you require that namespace before using the spec?
No I did not require it. Also I don't know how to find out if it is in the uberjar (am not familiar with java 😐)
This is the spec definition in the spec NS (s/def ::spec-test (s/keys :req-un [::post-id]))
And am using it in my route file like {:body :path-to-spec-file/spec-test}
Previously I was defining the spec in the route file and using it like this and it works.
{:body ::spec-test}
I want to focus on the namespace that defines the specs, not any particular namespace of the keyword of the spec if that makes sense to you. Which namespace includes the s/def
form defining the spec?
app_name/spec/request.clj
is where spec is defined.
app_name/domain-name/routes.clj
is where that spec is used.
do you require app-name.spec.request
anywhere in another namespace? I suspect the code isn’t getting loaded
I was not aware that spec has to be required, I thought referring them with the fully qualified keyword alone is enough.
For anyone checking this in the future, it works after I require them like (:require [app_name.spec.request :as spec-request])
and use it like ::spec-request/name-of-spec
Thank you @U11BV7MTK for pointing this out !! 🙌