This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-07-16
Channels
- # aleph (2)
- # announcements (1)
- # beginners (162)
- # calva (16)
- # cider (37)
- # cljdoc (9)
- # cljs-dev (2)
- # cljsrn (3)
- # clojure (86)
- # clojure-dev (17)
- # clojure-europe (3)
- # clojure-houston (1)
- # clojure-italy (6)
- # clojure-nl (3)
- # clojure-spec (10)
- # clojure-uk (20)
- # clojuredesign-podcast (15)
- # clojurescript (7)
- # data-science (14)
- # datascript (1)
- # datomic (5)
- # emacs (8)
- # figwheel-main (8)
- # fulcro (25)
- # graalvm (1)
- # jobs (10)
- # jobs-discuss (4)
- # keechma (14)
- # leiningen (2)
- # off-topic (31)
- # onyx (1)
- # other-languages (4)
- # pathom (4)
- # pedestal (1)
- # re-frame (20)
- # remote-jobs (4)
- # shadow-cljs (25)
- # sql (6)
- # tools-deps (15)
- # vim (18)
- # xtdb (9)
#_ is comment-form
it prevents an entire (balanced) form from being compiled
user=> #_(anything goes here, clojure will not try to compile it)
42 ; to get a prompt
42
it just skipped the whole list as if it hadn't been typed in
didn't even think it needed to print a new prompt
you can still get reader errors inside #_ though
also, it stacks
#_#_
is a useful idiom for commenting out a k/v pair in an inline hash-map
user=> {:a 0 #_#_ :b 1}
{:a 0}
I tend to use #_
quite a bit - I was just reminded today that I want to change cljfmt
so it allows #_
on a line by itself (to comment out a definition without adding whitespace noise in git history)
yeah - like that (also it's a handy thing for taking out a part of a function during debugging)
that's useful too - but most people use (comment ...)
in order to have a distinct block
A Rich Comment Form 🙂
instead of a bunch of forms that each start with a #_
also, if you define a series of zero-arg functions, you don't need to comment them out (though your editor might not make this as convenient, it's straightforward via a repl)
A Rich Comment Form 🙂
As seen here, for example https://github.com/clojure/clojure/blob/master/src/clj/clojure/set.clj#L158
It's just the default namespace created when a REPL starts up. It has some clojure.repl
stuff referred in for convenience.
https://github.com/clojure/clojure/blob/master/src/clj/clojure/main.clj#L355-L359
That's Clojure doing that.
No, just because the default Clojure REPL works that way.
It creates a user
namespace and runs those requires.
I'm just talking about the default clojure.main
behavior when you start a REPL.
(! 666)-> clj -e "(ns foo.bar)" -r
foo.bar=> (->> (ns-refers *ns*) vals (map symbol) (filter #(not= "clojure.core" (namespace %))))
(clojure.repl/doc clojure.pprint/pprint clojure.java.javadoc/javadoc clojure.repl/find-doc clojure.repl/dir clojure.repl/pst clojure.pprint/pp clojure.repl/source clojure.repl/apropos)
foo.bar=>
Here I tell Clojure to start a REPL and evaluate (ns foo.bar)
first -- and then it runs those requires above, referring in those symbols.Leiningen uses the Reply library but it's also dealing with nREPL client/server stuff and all sorts of middleware etc etc. "It's complicated" pretty much by definition -- and if the behavior re user
and the referred symbols is documented, it'll be somewhere in Leiningen's documentation I expect. Or Reply's docs. Or maybe nREPL's docs. I don't know -- I don't use any of that stuff.
At work we switched from lein
to boot
back in 2015 and then we switched from boot
to the Clojure CLI/`deps.edn` stuff last year.
lein
is the most commonly used tool in tutorials and books -- but I do think it's important for beginners to look at the "standard" Clojure CLI stuff.
CIDER supports the Clojure CLI stuff (but it will still inject nREPL as a dependency to start a client/server REPL).
I expect, over time, tutorials and books will probably shift over to the Clojure CLI for introducing people to Clojure. Editor tooling has been adding support for deps.edn
over the last year or so.
I wouldn't say starting with Leiningen is a bad recommendation. It does have stuff under the hood that usually helps you, but if you want to understand all of the pieces, there are more parts under the hood.
^ This, definitely. I was able to find the code for exactly what the Clojure REPL does pretty quickly, but I still haven't managed to chase down exactly where in the Leiningen/Reply/nREPL stack the "same" thing happens! Lots of magic involved 🙂
Part of why I tend to steer beginners to clj
and deps.edn
is just because there's a lot less "magic" involved -- fewer dependencies, no unpleasant surprises caused by ~/.lein/profiles.clj
containing plugins that you forgot you added, etc.
(the number one cause of beginner problems with Leiningen tends to be something they added to that profiles.clj
file because something they read somewhere suggested a plugin to "make life easier" but then it conflicts with some other project they're trying to work their way through)
And a relatively common root of Leiningen problems for experienced developers, too, but sometimes difficult to remember to rename ~/.lein/profiles.clj as a troubleshooting step if you haven't done so in a while.
Heh, the only thing in my ~/.lein/profiles.clj
file is
:socket {:jvm-opts ["-Dclojure.server.lein={:port 55555 :accept clojure.core.server/repl}"]}
so I can do lein with-profile +socket ...
so I can then connect to whatever starts up via Chlorine/Atom so I can avoid nREPL 🙂@U06BE1L6T I specifically need to add that profile (and I have to use +
to add the socket
profile to whatever lein
command I'm running).
I was referring to the "rename ~/.lein/profiles.clj" tip - using the base profile is an alternative to this I think
Ah, good to know.
The clojure repl has its default requires at https://github.com/clojure/clojure/blob/master/src/clj/clojure/main.clj#L355-L359
I think nrepl actually works from that same var, iirc
actually, I guess it doesn't https://github.com/trptcolin/reply/blob/master/src/clj/reply/initialization.clj#L172
Wow... that's... a lot of custom stuff!
Hello, I'm working on my first clojure app. Mostly it's been great, but I'm running into an issue with my ring server hanging non-deterministicly while processing a request. The failure results in the entire JVM (jdk-11) locking up indefinitely, which I am only able to stop by sending a kill signal from another process. Thought I'd ask if anyone has ran into this? I'm wondering if it may be related to the fact that I am making JNI calls to opencv in the handler, although the issue doesn't arise in independent testing of the code path (not through the server).
I don't know of any examples of what might cause that off hand, but if you can get access to a terminal where the JVM process was started when this happens, typing Ctrl-\ will show information about all threads that might show if they are executing some blocking call.
Perhaps there is a way to do that remotely without terminal access, too. Not sure.
Thanks, so far I can't get the terminal process to respond to any key presses. I'll look into whether it can be done remotely
Maybe more info here that can help figure out a way to do it remotely: https://docs.oracle.com/javase/8/docs/technotes/guides/troubleshoot/tooldescr019.html
I'll take a look, thanks for the help!
Hello, newbie here. How do you guys run your db migration? Currently I do it every time my application starts (I'm using component and ragtime). I'm trying to mimic micro-services for the sake of learning, so my application is split into a different github project. But unfortunately ragtime complains about the conflicting migration after the first project. Is this actually a bad approach to do migration (splitting it into a multiple projects)? Should I create another project which contains all the migration scripts that has a job to do just that?
Put all your migrations in your project repo, but run them out of band. That is, not a part of your main server runtime. Run them as part of your deployment process.
you upgrade 'prod' from v0.1.2 to v0.1.3... you run the migrations on 'prod' db as part of that step
you might even bundle them into the uberjar, but you probably want to launch them with a different entry point (run a separate command on the shell using the jar at deployment time)
also microservices... why do that to yourself. If you are beginning, just build a monolith.
Thank you for the answers. Yes, I've read several github project and they differentiate between running the db migration process and the actual app. For example:
(defn -main
[& args]
(if (= (first args) "migrate")
(run-migration!)
(run-app!)))
And about the microservices thing, I would like to know any difficulties I'm dealing with while build it. If this is a monolith project, this wouldn't become an issue for me. So to clarify things, it's alright for a project to have it's own migration script right? I was just wondering if this is correct, since ragtime complains about it, I wonder if it's a ragtime limitation or the author enforcing a "good" workflow.After some reading, ragtime complains after the first migration because the second and so on migration would contain different migration scripts with no overlap, for example: first migration runs [01.foobar.up.sql, 02.foobar.up.sql]
from the first project, the second one runs [01.barfoo.sql]
and it conflicts since it can't find the [01.foobar.up.sql, 02.foobar.up.sql]
in the folder.
@UL449EM4K Look at Migratus. I think it overcomes the purely linear approach in Ragtime.
@U04V70XH6 Thank you. I'll definitely look at it!
Mangaged to to get the JVM to dump threads. Does seem like some kind of deadlock, but hard to interpret.
No deadlock, the only thread doing anything is the middle of transit encoding some lazy seq
If it is infinite you should see the heap usage grow until you oom, or you'll get a stack overflow error
If it is just very large or slow to materialize if you make two http requests in parallel you should see two identical or close to stacktraces but no oom/stack overflow
Hm, thing is the same request will usually succeed more or less instantly. Only once every 10 requests or so will it hang. And I'm delaying the requests so there's no contention.
Even stranger, it seems to happen at a random point of execusion
No, I'm logging the code path extensively and it's always running. Random relative to my preception, haha
But my guess is you are logging function calls, but not logging as your lazy results are realized, so you are missing most of what is happening
When you say hang, have you tried increasing the timeout on the http clone you are using?
It'll vary from the very first line logged to the last. Anywhere in here:
clojure
(defn get-route-data [route-hash]
(let [route (first (query (-> (h/select :*)
(h/from :routes)
(h/where [:= :route_hash route-hash])
(h/limit 1)
(hsql/format))))
_ (info "route-hash:" (:routes/routeHash route))
bdl (bundle/to-map (download-route-bundle route))
_ (info "Downloaded route bundle.")
msg (bdl "brainos.nav.static_map")
tensor (.getData msg)
tf (.getTransformation msg)
tf-mat (utils/tensor->mat tf)
static-map-mat (utils/tensor->mat tensor)
_ (Imgproc/cvtColor static-map-mat static-map-mat Imgproc/COLOR_GRAY2BGR)
_ (info "Colored")
static-path-msg (bdl "brainos.nav.static_path")
static-path-tensor (.getPathData static-path-msg)
static-path-data (.asByteBuffer (.getData static-path-tensor))
static-path-mat (utils/tensor->mat static-path-tensor)
^MatOfPoint path-indices (utils/to-img-indices static-path-mat tf-mat)
_ (info "Creating path indices array.")
path-indices-array (.toArray path-indices)
_ (info "Creating path-points.")
path-points (loop [idx 0 ret (transient [])]
(if (= idx (count path-indices-array))
(persistent! ret)
(let [^Point p (aget path-indices-array idx)]
(recur (inc idx) (conj! ret (vector (.-x p) (.-y p)))))))
img-buf (MatOfByte.)
_ (info "IM encoding")
_ (Imgcodecs/imencode ".png" static-map-mat img-buf)
img-bytes (.toArray img-buf)
_ (info "Writing img str.")
img-str (.encodeToString (Base64/getEncoder) img-bytes)
_ (info "Creating static-path")
static-path (vec path-points)
_ (info "Creating static map")
static-map-transform (vec (utils/tensor->array tf))]
(info "Return Route Bundle.")
#:routeBundles
{:height (.rows static-map-mat)
:width (.cols static-map-mat)
:staticMapImage img-str
:staticPath static-path
:staticMapTransform static-map-transform}))
Sadly slack doesn't seem to understand clojure code.
So you have work happening after your last logging, because you are constructing lazy seqs and not forcing them
vec
will eagerly construct a vector right?
Hm, I specifically tried to get rid of lazy seqs with that weird loop, but haven't had luck.
Yeah I'm currently doing a dirty recursive call to query
There's no stackoverflow situation here, but I should go ahead and fix that
It is executing the query function, and in the middle of that somewhere transit encoding some thing
You might be onto something there
I did try switching between transit and prn-str
and it didn't change anything. prn-str
actually had some other, serisouly nutty bugs.
So that stack trace was from before I removed this lazy seq:
(map #(vector (.-x %) (.-y %)) (seq (.toArray path-indices)))
and replaced it with the loop.
@collins you said "the entire JVM (jdk-11) locking up indefinitely" - how does this manifest? Is really the whole process "frozen"? Or is it just your request thread?
The entire vm. Only way I can kill the process is with kill -9 <pid>
form a different terminal. Been trying different opencv versions, including building from source. If I remove the opencv calls it works, but with opencv calls present the process will hang at seemingly a random point. Anyway, I'm done for tonight, thanks for all the help.
when its frozen, check the processes state, with ps
or top
. Is the state of the process D
?
Hey all. I tried to build an uberjar with deps.edn and it kept packaging until I ran out of space (a 200+gb jar). I used the following library and followed the instructions here (https://github.com/tonsky/uberdeps). Anyone know why this might have happened ?
Hey please report this as a issue
If possible, a minimal deps.edn
to reproduce this behavior.
Did you include an apache commons library?
But srsly, does printing out your dependency tree uncover any pathologies?
@donaldball not really. It is a similar set of dependencies that was working with lein!
If the jar isn’t corrupt, you could unzip -l
on it and see what hilarity it contains
I’m a bit confused with my workflow using clj and deps.edn. Everything was so magic with lein, I never had to actually understand what was going on. 🙂 no such luxury/laziness here.
I have an app that has both an API server and a clojurescript frontend. The server is using http-kit.
In my deps.edn I have {:aliases {:run {:main-opts [ "-m" "alexandria.core" ] } }
— in alexandria.core I initialize the db, start the http-kit server etc.
In my global deps.edn, I have the alias :nrepl {:extra-deps {nrepl/nrepl {:mvn/version "0.6.0"}}}
What I would like to do is start the server, and have an nrepl session start at the same time, preferably with both a repl prompt and a server to connect to from the editor (I use fireplace/vim)
Right now I’m starting the server with clj -A:run
, which doesn’t start a repl as near as I can tell
and in a different terminal clj -R:nrepl -m nrepl.cmdline
, which starts a nrepl server
and in a different terminal just clj
to get a repl prompt
and in a different terminal shadow-cljs watch alexandria
to get clojurescript compiling plus a clojurescript repl going
I’m sure this is not the Right Way to do things.
Have you tried running clj -A:run:nrepl
? I have successfully used multiple aliases that way.
That should use both of the aliases, but I am ignorant of nrepl enough that I don't know whether it will start an nrepl server, too. It should add the dependency, but I don't know whether adding the dependency alone is enough.
I would suggest having your own dev-time entrypoint namespace that called whatever you need to call
how are processing
and change-email
defined? I assume they get stuck in an unhelpful state
also this isn't beginner friendly cljs code (I assume you are asking here because you are new on a project and someone else wrote this)
I forget precisely how reaction works (haven't used cljs in over a year) but I'd look into the value of that change-email value as a starting place (and remember that ""
is truthy, so it shows up as true in a test - perhaps (not-empty @change-email)
would change things, or maybe not)
or since that's only a test, and not using the value, just (seq @change-email)
oh never mind it's already checking for empty
but if you have figwheel set up, check out the values of change-email and processing?, or otherwise set them up to be logged, I think that will be informative
Hi guys, Do you knows books about stuart component ?
It's covered a little in Clojure Applied
thank you @U064X3EF3
ya. I almost suggested an id change earlier.
figwheel makes this straightforward - one trick is attaching add-watch
to an atom / reaction to print when it changes
(that trick doesn't require figwheel)
I cannot seem to get my work environment going. I’m not sure what I’m doing wrong or even what channel to ask about it. I’m using fireplace to connect to cider-nrepl through vim, with a clj based project. Weird things have started happening. I’ve tried a bunch of things but the state right now is that I start the repl server with
clj -Sdeps '{:deps {cider/cider-nrepl {:mvn/version "0.21.1"} }}' -m nrepl.cmdline --middleware "[cider.nrepl/cider-middleware]"
as instructed on the cider-nrepl site. I connect with vim using :Connect.
The first time I try to connect, I get an error: Vim(return):E15: Invalid Expression:
. The second time I try, it seems to work.
After I connect, I try to evaluate a simple expression - (+ 1 2)
. The terminal with the nrepl server prints a stack trace:nREPL server started on port 52195 on host localhost -
Exception in thread "nRepl-session-588832b7-43c8-4260-8324-0a14383dac6c" java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:116)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at nrepl.transport$bencode$fn__427.invoke(transport.clj:116)
at nrepl.transport.FnTransport.send(transport.clj:41)
at nrepl.middleware.print$send_nonstreamed.invokeStatic(print.clj:159)
at nrepl.middleware.print$send_nonstreamed.invoke(print.clj:138)
at nrepl.middleware.print$printing_transport$reify__818.send(print.clj:174)
at cider.nrepl.middleware.track_state$make_transport$reify__3988.send(track_state.clj:214)
at nrepl.middleware.caught$caught_transport$reify__853.send(caught.clj:58)
at nrepl.middleware.interruptible_eval$evaluate$fn__909.invoke(interruptible_eval.clj:123)
at clojure.main$repl$fn__9095.invoke(main.clj:460)
at clojure.main$repl.invokeStatic(main.clj:458)
at clojure.main$repl.doInvoke(main.clj:368)
at clojure.lang.RestFn.invoke(RestFn.java:1523)
at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:79)
at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:55)
at nrepl.middleware.interruptible_eval$interruptible_eval$fn__916$fn__920.invoke(interruptible_eval.clj:142)
at clojure.lang.AFn.run(AFn.java:22)
at nrepl.middleware.session$session_exec$main_loop__1017$fn__1021.invoke(session.clj:171)
at nrepl.middleware.session$session_exec$main_loop__1017.invoke(session.clj:170)
at clojure.lang.AFn.run(AFn.java:22)
at java.lang.Thread.run(Thread.java:745)
After that though, I can evaluate the simple expression just fine, and the result prints in the editor.
It feels like I spend 3 days struggling with tooling for every 1 day writing code. 😞
there's a vim plugin that uses a socket repl instead of cider (it has less features but is more likely to just work)
also, I'm not sure if fireplace is up to date to use the newest cider changes
@noisesmith Are you thinking of https://github.com/Olical/conjure that works with neovim?
I was thinking of https://github.com/jebberjeb/clojure-socketrepl.nvim