This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2019-08-13
Channels
- # announcements (27)
- # beginners (184)
- # boot (4)
- # cider (9)
- # cljdoc (1)
- # cljsrn (2)
- # clojure (208)
- # clojure-austin (1)
- # clojure-conj (4)
- # clojure-dev (20)
- # clojure-europe (15)
- # clojure-italy (66)
- # clojure-losangeles (2)
- # clojure-nl (32)
- # clojure-spec (64)
- # clojure-uk (80)
- # clojurescript (50)
- # cursive (2)
- # data-science (3)
- # datomic (17)
- # emacs (1)
- # events (6)
- # fulcro (3)
- # jobs (15)
- # juxt (5)
- # klipse (2)
- # leiningen (31)
- # nyc (3)
- # off-topic (34)
- # re-frame (2)
- # reagent (9)
- # schema (1)
- # shadow-cljs (52)
- # specter (5)
- # sql (3)
Is there any kind of option to clojure.test to cause every deftest to print a message when it starts executing? I know I can simply add a println anywhere I want to show progress -- just curious if there was something built in already that can be enabled for that.
@andy.fingerhut Yes, you can redefine do-report
for :begin-test-var
I think to log that...?
thx I will look for those in the code or other projects.
user=> (require '[clojure.test :refer :all])
nil
user=> (deftest foo (is (= 42 13)))
#'user/foo
user=> (defmethod clojure.test/report :begin-test-var [m] (println 'running (:var m)))
#object[clojure.lang.MultiFn 0x30814f43 "clojure.lang.MultiFn@30814f43"]
user=> (run-tests)
Testing user
running #'user/foo
FAIL in (foo) (NO_SOURCE_FILE:1)
expected: (= 42 13)
actual: (not (= 42 13))
report
not do-report
@andy.fingerhut Like so ^
These extension points are "blank" by default but you can report on all of them https://github.com/clojure/clojure/blob/master/src/clj/clojure/test.clj#L407-L410
is there any way for a macro to know the line/file information of where the macro is invocated? 🙂 could one hack it with (def temp-var nil) (meta temp-var)
?
i think this might be relevant to you: https://stackoverflow.com/a/30813410
oh awesome, thanks! the local docs didn't mention the metadata: https://clojure.org/reference/macros
there's no way to "force a number to be primitive", the long
function returns a primitive long
, but it depends on the callsite where that value is used whether it will be boxed or not
Hi. I have a test that connects to a websocket. The connect function takes a callback and I want to test that something is sent back. Now the problem is that clojure.test does not recognize is
expressions in the callback similar to this:
(g/connect (str ws-base-url "ws")
:on-receive #(is (= 400 %))
My idea would be to have an atom that is set in the on-receive callback and checked after a given time while the Thread is sleeping.
Does someone have a better idea? Something more idiomatic?via some third party tool, deps.edn is not a packager or build tool
excel sheets and clojure in 2019: docjure or something else? I've used docjure some years ago and it was nice
I've used docjure in 2019 and it works. I've been tempted to try out something like this though https://youtu.be/qnJs79W0BDo
I've packaged the add-lib
feature from tools.deps into an uberjar for an app that simulates a REPL using eval. However, upon running add-lib
it errors w/ Context classloader is not a DynamicClassLoader
what is that about?
add-lib relies on hooking the current thread’s context classloader but it needs to find a Clojure DCL to be able to dynamically inject libs
And it didn’t find that in your environment
thanks @alexmiller, is a clojure DCL inside of a uberjar possible?
@denik Unless you AOT the entire app, I would not expect to see much difference between clj -m ...
running the -main
from source and java -cp the.jar -m clojure.main ...
running your -main
inside the uberjar.
What impacts startup time is mostly whether the code needs to be compiled as it is loaded vs it being already compiled to .class
files -- i.e., AOT.
is one gen-class for the main file enough or does this mean I need gen-classes everywhere?
But for any long-running process, the difference is mostly irrelevant, especially for relatively small app.
If you want your -main
to be a callable entry point, instead of going through clojure.main
, then :gen-class
is needed and you need to AOT compile that file before you make the uberjar (and the .class
files need to be on your path, which some tooling takes care of and some needs a hint).
When you AOT compile your -main
namespace, it will also transitively compile other namespaces that are directly referenced in that namespace. That behavior is considered a bug by many people.
At work we do not AOT compile anything.
We build uberjar files from source, and run them via java -cp path/to/the.jar clojure.main -m entry.point
(to run entry.point/-main
).
The new CLI/`deps.edn` machinery has no support for AOT compilation -- it's intended just for running Clojure.
You can manually compile Clojure code (from the REPL, from the command line, etc), and if those .class
files are on the classpath when you run something like depstar
via clj
, then they'll be added to the JAR file.
My recommendation would be: ignore AOT compilation completely unless start up time of your app is critical. And if that is the case, do AOT as one last step before creating a production deployment artifact -- don't try to work with AOT while you're developing.
thanks! I'm actually using your seancorfield/depstar
so adding gen-class by itself will and calling clojure -A:depstar -m hf.depstar.uberjar myApp.jar is not sufficient?
depstar
does no AOT by design. I do not think Clojure's AOT is a good model in general.
I'm building a desktop app, electron frontend, clojure backend. startup time is somewhat crucial, since it affects the delay between clicking the app icon and having the frontend ready
The Clojure backend is going to run locally? So you're starting up two things? An Electron app and a Clojure/Java process?
Sounds like a complicated and fragile way to build a desktop app to me...
I could also start the electron app simultaneously and have a loading screen. still, currently it's around 10s of startup for the backend. I'd want this to be as fast as possible
Ten seconds of startup sounds long to me. Most of our servers spin up a lot faster than that (when run without New Relic instrumentation).
the thing is when the user clicks and app an expects it to open, any delay feels like forever
Have you considered writing it all as an Electron app instead of client/server?
(Electron apps aren't exactly nimble when starting up...)
possible to do this later. however the app literally contains a clojure repl so that needs to be startup up at some point
Ah, OK.
If you reduce the static dependencies in your -main
namespace (and then dynamically require the rest of the app in a background thread), it will load much quicker (still without AOT) but you'd have to optimize the path to the code being able to accept some requests from the front end. Doable but fiddly.
If I definitely needed AOT of the whole app, I'd write a small Clojure utility to do that as its own -main
and run that via clj
before building the uberjar.
Or you could just use lein
to build the uberjar with :aot :all
🙂
Uberjar is just packaging. It’s totally independent of the classloader hierarchy
@alexmiller so something must be different between running the app using clj
and java
as a uberjar
clj just calls java with the classpath you built
is there a stable DAG library that people like to use? seems like ubergraph is a good all-in-one option.
I have used ubergraph recently, and found it to be a pleasure to use.
If you want multiple parallel edges between two nodes as a possibility, ubergraph can do that, loom cannot.
I have a little bit of code I hope to submit back to ubergraph that implements strongly connected components using a faster algorithm than the one ubergraph currently provides, which it inherits from loom.
multiple parallel edges offers an interesting twist that was going to make me have to split out my graph into way more nodes/edges than I necessarily wanted (in this case, a Kubernetes pod spec plus all of it's spec items as separate nodes)
It was very useful for my application to have parallel edges, too, and glad I didn't have to work around that wish, nor write my own lib.
ubergraph also allows you to have an arbitrary Clojure map of attributes for every node and/or edge, so if you are wishing for parallel edges because of attributes on edges with different keys, you do not need parallel edges for that.
this is going to be 1000x more expressive than the Go version of this controller. excellent.
right, clj also causes the classbuilder error
@hiredman I didn't actually and didn't mean that. It works fine in the REPL but the Context classloader is not a DynamicClassLoader
appears both during clj
and java
with the uberjar
an uberjar tool should let you decide whether to aot witha switch
if you look at clojure/main.clj's repl function you can see the first thing it does is set the current thread's context classloader
Anyone mind tossing their 2¢ in on the following? https://gist.github.com/devn/32652a23a32e0874175a6c52c65b5e5f
Also, I'd share my code, but it contains a ton of work-related stuff I'd have to clean out. This is the essence of the transform.
@devn looks like (->> tree-seq filter map) to me
oh, that doesn't address the name_path thing
so more like (->> prewalk tree-seq filter map) - where prewalk injects the name-path based on parent, tree-seq gets you branches, filter prunes to the branches that you want, and map removes the :children key
@hiredman would you care to sketch just a bit further? I'm not sure I follow. m is the initial map, and then...?
@noisesmith sort of a similar ask for you, if you don't mind
https://clojurians.slack.com/archives/C03S1KBA2/p1565717490419100?thread_ts=1565716868.407600&cid=C03S1KBA2
the next issue is "Could not locate net/cgrand/enlive_html__init.class, net/cgrand/enlive_html.clj or net/cgrand/enlive_html.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name."
when using add-lib
with the added classloader
You verified that this works OK in a regular REPL? (just to eliminate typos etc)
(it works for me)
(! 754)-> clj -A:deps
Cloning:
Checking out: at d0b33e0d346736aa985c150145b332f97b92135e
Clojure 1.10.1
user=> (require '[clojure.tools.deps.alpha :as t])
nil
user=> (require '[clojure.tools.deps.alpha.repl :refer [add-lib]])
nil
user=> (add-lib 'enlive {:mvn/version "1.1.6"})
Downloading: org/clojure/clojure/1.2.0/clojure-1.2.0.jar from
true
user=> (require '[net.cgrand.enlive-html :as h])
nil
user=> ^D
Does loading other libs and then requiring them work in your DCL in your JAR, or is it just this one that fails?
OK, just wanted to check. Have you managed to get any library to add/load into the DCL? Or is this just the first one you've tried?
Interesting, it works with
(require '[clj-http.client :as client])
(:status (client/get ""))
OK, so some libraries can be added and loaded (in the JAR) but some others cannot?
Hmm, not sure what to suggest at this point. Sorry.
thanks @U04V70XH6, @alexmiller if you can think of a path forward, I'd appreciate it
sorry, not sure. If you wanted to look into it further, my suspicion is that add-lib is not tracking the libs the same way in the two scenarios. the clj version leverages the libs determined by tools.deps.alpha, which is passed via a system property iirc. If it can't find that, then it falls back to parsing the java classpath, but if you have an uberjar, that would hide all the deps from it.
nearly all of what I just said is implemented here: https://github.com/clojure/tools.deps.alpha/blob/add-lib/src/main/clojure/clojure/tools/deps/alpha/libmap.clj if you want to look at it more
here's how unrepl does it https://github.com/Unrepl/unrepl/blob/ea92a94bbcc7355a8db1671f385026cbe98d3342/src/unrepl/repl.clj#L396
@denik Do you have (with-bindings {clojure.lang.Compiler/LOADER slcl} ...
in your code?
(as an aside, one of the reasons add-lib is still sitting on a branch is working through some of these ergonomics)
@U04V70XH6 @U5H74UNSF setting the classloader helped resolving the Context classloader is not a DynamicClassLoader
error. However it seems now that some libs don't end up on the classpath as @alexmiller suggested. thanks for the help, everyone. I might move forward with a temporary and naive approach of bundling the libraries into the app that fail through add-lib (since some do work) until add-lib
makes it into deps proper.
the reason some other deps like clj-http worked is not because of add-lib
but because they're transitive deps of other libs in the deps.edn
@alexmiller after add-lib enlive
made it into (clojure.tools.deps.alpha.libmap/lib-map)
however, it doesn't get picked up by require
Lib-map is bookkeeping. To be loaded, it has to be accessible via the DCL
@noisesmith i don't really follow how you are creating and tracking and creating IDs and parent IDs with prewalk and tree-seq
((fn f [path item]
(cons (assoc item :path path)
(for [child (:children item)
i (f (conj path (:name item))
child)]
i)))
[nil]
{:name "root",
:description "root",
:children
[{:name "Foo",
:description "foo",
:children
[{:name "Bar",
:description "bar",}
{:name "Baz",
:description "Baz",
:children [{:name "Qux",
:description "qux"}]}]}]})
if you can live with random ids(uuids, gensyms) instead of numbers it would be a lot easier
I'm doing this exact xform for a library right now, in a reduced way: just passing down the parent name to qualify the child name
with a stateful (id)
function you can apply it to the leaves as they are visited - that does make it trickier and I didn't see it at first
right - I was doing that in prewalk (more precisely, updating children of each node (if any) as the parent is updated)
a queue / loop might be more natural than a prewalk updating children before visited, yeah
(in my case the context
didn't exactly correspond to parent nodes, but was altered at some parent nodes, but the idea is the same)
(loop [acc [] q (conj clojure.lang.PersistentQueue/EMPTY [root nil])]
;; process, add self to acc
;; add children to queue
the loop, the queue, the prewalk whatever, just like, do a recursive for, it is great
I actually had something sort of similar to the (for [child (:children item)] ...)
, but I was calling f
in the body, which ultimately meant I needed to concat the initial processed row with the result of for
and flatten
, which is always a smell IMO
something like:
(defn make-row [parent row]
(let [{:keys [name path]} row]
{:name name
:path path
:name_path (if parent (str (:name_path parent) (:name row) "::") "::")}))
(defn unroll-hierarchy [m & [parent]]
(let [curr-path (if parent (str (:path parent) id "::") "::")
row (assoc (make-row parent m) :id id :path curr-path)
next-parent (assoc row :id id :path path)]
(flatten [row (for [child (:children m)] (unroll-hierarchy child next-parent))])))
couldn't that flatten just be concat?
maybe with the help of an apply and [] around row
i've left out the id generation bit, rest assured it is actually worse than what you see there
what i had/have is something gross where i pass an argument id-space into unroll-hierarchy
, and then remove the bits that i've used, but that is actually broken for cases where there are multiple children under a parent-- their id's are the same
You could do a zipper based approach.
(defn decorate-element [counter node]
(zip/edit node
assoc
:id (swap! counter inc)
:parent_id (when-let [parent (zip/up node)]
(:id (zip/node parent)))
:path "TODO"
:name_path "TODO"))
;; removed bad code here
(See the rest below) Fill in the path and name path and it produces what you are looking for.
Better version
(defn traverse-zipper [f zipper]
(loop [node zipper]
(if (zip/end? node)
(zip/node node)
(recur (zip/next (f node))))))
(let [counter (atom 0)]
(->> (zip/zipper map? :children (fn [r c] (assoc r :children c)) x)
(traverse-zipper (partial decorate-element counter))
(tree-seq map? :children)
(map #(dissoc % :children))))
I have discovered a truly marvelous solution, which this channel is too narrow to contain.
@bbloom gave a very enjoyable talk called "Dendrology" a few years ago, covering different styles of tree walking
sometimes you have to pass information down the tree, sometimes you pass up the tree, sometimes both directions....
(defn walk-and-chew [root]
(letfn [(mp [p x] (if p (str p x "::") "::")) ; quadratic string building :-/
(wn [nodes parent id cont]
(lazy-seq
(if-some [[{:keys [children name] :as node}] (seq nodes)]
(let [more-nodes (rest nodes)
node (-> node
(dissoc :children)
(assoc
:id id
:parent_id (:id parent)
:path (mp (:path parent) id)
:name_path (mp (:name_path parent) name)))]
(cons node
(wn children node (inc id)
#(wn more-nodes parent % cont))))
(cont id))))]
(wn [root] nil 1 (constantly nil))))
I don't know if this is possible, but I really wish Clojure fns implemented the java.util.function interfaces.
As I'm interoperating with CompletableFuture
s, I realized two things: a lot has changed in Java (I thought futures were the latest and greatest, had no idea there is a whole framework for asynchronous processing!) and — I really wish Rich would appear at a conference and give a talk explaining how to deal with exceptions/errors in async code.
we' been talking about the function interfaces off and on for a couple years and we would definitely like to make Clojure a better player in this regard. the question is how to do it in a way that derives full advantage.
It seems extremely complex: for it to be flawless, we'd need to support all the variants (Consumer, Function, BiFunction, etc).
there's only about 4 essential ones
https://ask.clojure.org/index.php/767/integration-with-java-util-function-interfaces if you want to vote/comment
As to exceptions, I feel like I'm using core.async channels to poorly reimplement some of what CompletableFuture
provides. There must be a better way than inventing a protocol for sending exceptions on metadata channels.
there’s also https://github.com/ajoberstar/ike.cljj which can help a bit when dealing with functional interfaces
Interesting… but I'm writing a library (interfacing with FoundationDB), so I'd rather reduce dependencies to an absolute minimum.
@jrychter one thing to decide is whether to support the interfaces within j.u.function directly (Function, Supplier, etc), or to support a mechanism what works for all Java lambdas
@alexmiller is it OK if I click "Answer" on http://ask.clojure.org to provide context information on how this integration would be useful to me as a user/developer? I'm not sure what is considered good form there. And I don't have meaningful solutions ("answers") to contribute.
absolutely
these "answers" are a little weird in that they were imported from the related jira comments
but go for it
Thanks. I did. But the site is really unhappy with backquotes in the example macros I provided. (correction: got it to work, I thought markdown is supported with code blocks)
@ghadi: I realize that. I'm prematurely optimizing. And these will be used only in a limited number of places.
But you are definitely right and I do plan to revisit this and check if it makes any sense to have these as macros.
what's even more challenging about this interop: It looks like FoundationDB takes a bunch of Function<Transaction> as args:
if you do
(reify Function
(apply [_ ^Transaction tx] .....))
as you'd expect, it won't compile(reify java.util.function.Function
(apply [_ ^String s] (str "foo" s) ))
Syntax error (IllegalArgumentException) compiling reify* at (REPL:1:1).
Can't find matching method: apply, leave off hints for auto match.