This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-06-23
Channels
- # announcements (11)
- # babashka (35)
- # beginners (110)
- # calva (2)
- # cider (46)
- # clara (5)
- # clj-kondo (29)
- # cljdoc (4)
- # cljsrn (59)
- # clojure (163)
- # clojure-brasil (3)
- # clojure-europe (24)
- # clojure-italy (2)
- # clojure-nl (10)
- # clojure-sweden (1)
- # clojure-uk (36)
- # clojurescript (58)
- # conjure (24)
- # cursive (19)
- # data-science (14)
- # datascript (1)
- # datomic (10)
- # docker (3)
- # expound (6)
- # figwheel-main (17)
- # fulcro (16)
- # graalvm (1)
- # leiningen (9)
- # malli (11)
- # off-topic (22)
- # parinfer (1)
- # pathom (1)
- # re-frame (18)
- # reagent (18)
- # reitit (3)
- # ring (3)
- # shadow-cljs (8)
- # spacemacs (3)
- # specter (79)
- # sql (20)
- # tools-deps (25)
- # vim (4)
- # xtdb (8)
when you create a jar file for a Clojure application and then run the jar file with java, can you still get a repl to that clojure application?
java -Dclojure.server.repl="{:port,44444,:accept,clojure.core.server/repl}" -jar helloworld.jar
❯ telnet localhost 44444
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
user=> (+ 1 2)
3
user=>
I always create my jars with clojure.main as their main, so I can do java -jar myjar.jar
to get a REPL, or java -jar myjar.jar -m myapp.main
to run the app.
you could always do java -cp myjar.jar clojure.main
didn't set clojure.main as your main class
Doesn’t the reverse make more sense? Having java -jar myjar.jar
to boot the app, and using java -cp myjar.jar -m clojure.main
to start a REPL? The intention of packaging an app is to normally run the app not a repl right?!
I already know how clojure.main works, having to learn each project's special main when I'm trying to debug a production issue is just pain waiting to happen.
"oh, we use -m for migrations. -repl is for a repl, yeah, like in go. Barry used to do Go, he said it's the posix standard now"
Sorry I meant to say java -cp myjar.jar clojure.main
Sure but doesn’t your way also depend on creating the jar in a particular format too?
How can I get deftype
to use interface default implementations?
I’m trying to implement https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/serialization/Serializer.java via
(deftype Foo []
Serializer (serialize [...] ...))
which should suffice because all the other methods have default implementations, yet I’m getting this error:
java.lang.AbstractMethodError: Receiver class my-ns.Foo does not define or inherit an implementation of the resolved method 'void configure(java.util.Map, boolean)' of interface org.apache.kafka.common.serialization.Serializer.
…or should I be using something other than deftype for this? I need to be able to resolve the class by name afterwards.
in order to use concrete inheritance with a predictably named class, you need gen-class
afaik
mind you proxy
does create a class with a name, just not a predictable one you can hard code from outside the app
hmm okay
I do seem to get a predictable class name with deftype
but maybe that’s a coincidence?
Either way, my question was mainly how I can get Clojure to indicate that the default interface methods should be used
deftype provides a predictable class name, and canot do concrete inheritance. proxy provides concrete inheritance but does not offer a predictable class name
you could combine the two (using deftype as a wrapper) or use gen-class, or write a java shim
I wasn't as clear originally as I intended to be, though I thought I was answering the question I skipped some details
oh, the other (implicit) detail - abstract class default implementations are just inheritance by a subclass plus delegation to a super as far as the bytecode is concerned
which is why deftype, defrecord etc. can't do it
got it, thanks
can someone help on why the macro below throws an error -
(defmacro make-defs
[coll]
~@(map (fn [[var-name# v#]]
`(def ~var-name# ~v#))
~coll))
(make-defs [[n1 1] [n2 1]])
I want the macro to generate -
(def n1 1)
(def n2 2)
@murtaza52 Try wrapping the body in a do:
`(do ~@(map ...))
@borkdude it still throws the same error - Attempting to call unbound fn: #'clojure.core/unquote
(defmacro make-defs
[coll]
`(do ~@(map (fn [[var-name# v#]]
`(def ~var-name# ~v#))
coll)))
~
is working only inside of (...)`
@delaguardo thanks that works !
I am having problems with import, when I am trying to import org.apache.commons.lang3.StringUtils in lein repl, it works, but when I do it in my program it says
Could not locate org/apache/commons/lang3/StringUtils__init.class, org/apache/commons/lang3/StringUtils.clj or org/apache/commons/lang3/StringUtils.cljc on classpath.
@U2J6U9D1R Can you share the relevant piece of code from your program?
@U0P0TMEFJ If it works in the REPL, they must have the dependency.
It sounds like you're trying to reference org.apache.commons.lang3.StringUtils
as if it were a Clojure namespace...
yeah, that error looks like an error message from require
or use
, which won't work on a class
okie dokie
(ns bob
(:require [org.apache.commons.lang3.StringUtils :as StringUtils]))
this is what I have in my program
and I import it in repl differently
(import org.apache.commons.lang3.StringUtils)
Yeah, I think they're saying it should be (ns bob (:import [org.apache.commons.lang3 StringUtils]))
then I get this error
Syntax error macroexpanding clojure.core/ns at (bob.clj:1:1).
((:import [org.apache.commons.lang3.StringUtils])) - failed: Extra input spec: :clojure.core.specs.alpha/ns-form
and I haven't added anything to the project.clj
it looks like this
(defproject bob "0.1.0-SNAPSHOT"
:description "bob exercise."
:url ""
:dependencies [[org.clojure/clojure "1.10.0"]])
based on exercism
you have extra parens on :import
((:import ...))
is wrong, it should be (:import ...)
- parens are not free in clojure
so in the actual code it's
(ns bob
(:import [org.apache.commons.lang3.StringUtils]))
OK, there's two versions of import, one with extra coll delimiters one without
Also, you might want to focus on the #beginners channel if you're running into these sorts of problems and can't figure them out yourself -- that channel has folks who have opted in to spending time helping folks who are very new to Clojure and are making straightforward mistakes like this.
(both myself and @U051SS2EU are in that channel, so we'll still help you there!)
you either want (:import foo.bar.Baz)
or (:import (foo.bar Baz))
yes @U04V70XH6, should I replicate it there?
or for the future?
For the future.
I invited you (back) into #beginners so it should be easy to find in your Slack sidebar.
thank you!
finally - the class you want from apache commons should not be available if you haven't added it to project deps
Yup. Not sure how you got it working in lein repl
with it being in your :dependencies
.
maybe it's a dep pulled in by a plugin(?)
Unless Leiningen adds that as a dependency behind the scenes?
it won't do that normally, but plugins can do anything to your deps before launching the target vm
Looks like lein
brings that dep in without any plugins
(! 611)-> lein repl
nREPL server started on port 55265 on host 127.0.0.1 -
REPL-y 0.4.4, nREPL 0.6.0
Clojure 1.10.0
OpenJDK 64-Bit Server VM 14+36-1461
Docs: (doc function-name-here)
(find-doc "part-of-name-here")
Source: (source function-name-here)
Javadoc: (javadoc java-object-or-class-here)
Exit: Control+D or (exit) or (quit)
Results: Stored in vars *1, *2, *3, an exception in *e
user=> (ns bob
#_=> (:import (org.apache.commons.lang3 StringUtils)))
nil
oh wow, thats new to me
might be an nrepl thing?
You would need to add [org.apache.commons/commons-lang3 "3.7"]
to your :dependencies
vector in project.clj
to be able to access that class...
yeah, I suspect it comes in via reply, anyway the right thing to do is to explicitly declare deps for classes you use directly in your code
(! 634)-> cat project.clj
(defproject bob "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url ""
:license {:name "EPL-2.0 OR GPL-2.0-or-later WITH Classpath-exception-2.0"
:url " "}
:dependencies [[org.clojure/clojure "1.10.1"]
[org.apache.commons/commons-lang3 "3.7"]]
:main ^:skip-aot bob.core
:target-path "target/%s"
:profiles {:uberjar {:aot :all}})
(! 635)-> lein run
Hello, World!
(! 636)-> cat src/bob/core.clj
(ns bob.core
(:import (org.apache.commons.lang3 StringUtils))
(:gen-class))
(defn -main
"I don't do a whole lot ... yet."
[& args]
(println "Hello, World!"))
it's working now!
Does it mean I need to install stringutils on my machine?
feels like I'm missing something very basic
(import org.apache.commons.lang3.StringUtils)
so it's different way of importing it
anyhow, it fixed now! thank you for help
it's been a decade since import became a macro and accepted unquoted symbols, you'd think I'd be on top of that
What is the purpose of the symbol @
in the code below ?
(def a [1 2])
@(ns-resolve *ns* (symbol "a"))
it's a reader macro for deref
user=> '@a
(clojure.core/deref a)
resolve returns a var, a var is a container, you can use deref to get the value inside a var
check out (doc deref)
for what deref does on vars, agents, atoms, delays, futures, etc
I have a macro which is creating defs. It works when called in the same ns, however when I require it and call it from another ns it fails -
(defmacro make-def
[coll]
(let [coll# (if (symbol? coll) @(ns-resolve *ns* coll) coll)]
`(def a ~coll#)))
(def b [1 2])
(make-def b)
I get the following error when called from a different ns -
Cant create defs outside of current ns
. How do I write the above macro sot hat it can be called from a different ns ?the a
in the macro body gets turned into a namespaced symbol by `, and creates an invalid def
btw resolve
implicitly uses the bindings of *ns*
and ns-resolve
is only needed if looking up symbols as if you were in another ns, you can just use
@(resolve coll)
also, that error with a
getting namespaced should occur no matter where you call the macro from
@UMNSSKZL3 I have some data for which I need to generate defs. So instead of writing 50+ defs by hand, I decided to write a macro that will generate it.
@U051SS2EU the error does not occur when I call the macro in the same ns it has been defined, it only happens when I call it from another ns, which is what I need.
you said that before, but I find it hard to beleive - maybe because a
already exists in that ns, so it accidentally works...
hmm - def must have a special case for the current ns
anyway, the solution is to not use a namespaced symbol - ` is doing nothing in your code
(well, not really - the only thing it's doing is creating a namespaced symbol, which causes this error outside the ns where it was defined)
one question what is pro / con of not using ` and using just ' . Is there a disadvantage of not having namespaced symbol ?
I think you should look into macros a bit more (no offense!). In you can "unquote" (
~`) some symbols, with ' you cant. Regarding your question: It differs. When def
ing, you dont want namespaced symbols, in other cases you want them.
Can you provide an example, (with example input and output) what code the macro should create?
is for templating, so it's usually used in macros (but you can write macros without
and you can use outside macros)
namespaces symbols to prevent a common sort of error in macros - accidentally capturing symbols in the use context@UMNSSKZL3 yes my macro understanding is pretty rudimentary, I need to read up more. I am using the fulcro UI framework, and it uses a certain def to define UI elements. There is a large form I need to create, and instead of typing all the elements, I am using the macro to generate them.
@U051SS2EU thanks for the explanation - that is what I was looking for - accidentally capturing symbols.
a paper on the problem of symbol capture https://www.csee.umbc.edu/courses/undergraduate/331/resources/lisp/onLisp/09variableCapture.pdf - I guess maybe "variable capture" is the preferred name?
@UMNSSKZL3 any reading recommendations ?
@U051SS2EU yup thanks
@murtaza52 I like the book "Clojure for the Brave and True", its lightweight and fun to read. Also the author lets you read it online for free. For macros in particular https://www.braveclojure.com/read-and-eval/ and the following chapter. Have fun!
@UMNSSKZL3 thanks let me go through it
I run app like this on localhost and it works perfect
docker run --rm -it -p 80:80 -m 128M --memory-swap 0 --memory-swappiness 0 foo
I run the same image on cloud run
(kubernetes) and I get Memory limit of 128M exceeded with 333M used. Consider increasing the memory limit
Even if I use -Xms
and -Xmx
, check memory limits and usage by Java functions and by bash it shows me I use 60-80 MB. Only set to 512MB fix the memory issue, but I literally see in all possible way app use around 60-80 MB.
Any hints? I tried literally everything. How is it even possible the same docker image works on my localhost but not in google cloud run.
so the app even not start, the memory is exceeded when starting and even first print from main
functions is not processed
two things: cloud run has hard memory limits while docker doesn't and heap is isn't the only memory allocated
"official" is kind of a loaded term, they are the official images from docker as far as I know, not official from the people who make clojure
> while docker doesn’t
Do you mean docker run --rm -it -p 80:80 -m 128M --memory-swap 0 --memory swappiness 0 foo
doesn’t limit memory in the same way?
(let [free (/ (.freeMemory (Runtime/getRuntime)) 1048576.)
total (/ (.totalMemory (Runtime/getRuntime)) 1048576.)
used (- total free)]
(println "max memory" (/ (.maxMemory (Runtime/getRuntime)) 1048576.))
(println "free memory" free)
(println "total memory" total)
(println "used memory" used))
this tooyou can try using -XX:InitialRAMPercentage
and -XX:MaxRAMPercentage
to set the heap size as percentage of available limits within the container
@U0JEFEZH6 what values do you recommend for clojure apps?
IIRC starts at 128mb and goes up to whatever it can, MaxRAMPercentage I believe has no default or it's 20%
Depends - we use fargate which has its own limits, so it does require some tuning (instance resources -> docker resources -> jvm resources). If you co-locate many applications on one host that's a different challenge altogether - I'd say it's 20% learning, 80% experimentation
your operating system will have some way to get the used memory size for a process (interpreting that can be tricky as well)
but the point is when process start I see it use very low memory (I also printed bash commands to logs). The issue about memory is happening only when start the container.
I cut this code of the app to so simple, almost hello world. This is so strange. It looks like I can cut more random code and it start to work. But add a few lines and it consume +hundreds of MB. Really doesn’t make sense.
As I understand this docker run --rm -it -p 80:80 -m 128M --memory-swap 0 --memory swappiness 0 foo
should make it fail also on my computer
Cloud run gives you runtime monitoring - so it should report memory/cpu usage for your containerized process - that would be a starting point. In my experience you always need more resources in environments like these (e.g. my lambda functions do work under 128mb but run faster in 515mb envs). Lastly, do you see the same problem running in docker: as in app starts, but eventually gets killed by the OOM while receiving the same traffic as in cloud run? Also you will always get better results with a uberjar + fine tuned jvm flags
I am not familiar with the memory limiting options for docker, but in general I have found things that deal with linux cgroups to be very complex, and sometimes (due to running on old versions where certain limits don't exist, etc) fail to actually limit things
yeah, the memory resource controller is very hairy in my experience, especially with the JVM, i tend to over provision by default
> so it should report memory/cpu usage for your containerized process it does but it doesn’t help debug information for this condition practically doesn’t exist > Lastly, do you see the same problem running in docker: as in app starts, but eventually gets killed by the OOM while receiving the same traffic as in cloud run? no, when it start it is fine and showing very low memory usage. never exceed. so it looks like this is only starting app issue. I guess. Even more saying. Loading Java issue, because even first line of the code is not processed. > Also you will always get better results with a uberjar + fine tuned jvm flags I was trying many things. I use uberjar. What other jvm flags can I use?
There's a #docker channel and a #kubernetes channel which this should be in. This is not about Clojure.
Or at least use a thread please instead of cluttering up the main channel with a lot of non-Clojure questions and speculation. Thank you.
Sorry. It looks like it is about clojure / java running app, but maybe docker issue. I don’t know.
Is it possible clojure run app have a peak of memory usage when loading the app? Did you have such condition before?
We don't run code in small containers so we don't "care". The problem you are having is specific to the Docker/Kubernetes environment you are running in, which is why the conversation doesn't belong here.