This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-04-09
Channels
- # beginners (41)
- # boot (4)
- # cider (36)
- # cljsrn (9)
- # clojure (365)
- # clojure-dev (1)
- # clojure-dusseldorf (1)
- # clojure-nl (1)
- # clojure-russia (3)
- # clojure-spain (1)
- # clojure-spec (19)
- # clojure-uk (1)
- # clojurescript (159)
- # code-reviews (7)
- # core-async (51)
- # cursive (2)
- # datascript (1)
- # datomic (1)
- # emacs (5)
- # figwheel (3)
- # hoplon (18)
- # incanter (1)
- # lein-figwheel (1)
- # leiningen (3)
- # lumo (145)
- # off-topic (26)
- # onyx (21)
- # re-frame (2)
- # reagent (45)
- # rum (4)
- # uncomplicate (10)
- # untangled (23)
- # yada (6)
I'm using cassandra + cassaforte-3.0.0-rc1. I get an error of the form "failed to lod calss org.slf4j.impl.StaticLoggerBinder" and "ClassNotFoundException" "com.google.common.util.concurent.FutureFallback"
@U050SC7SV: would you like to declare a conflict of interest? 🙂
well, it's already better than cassaforte in that I can connect to my local cassandra server w/o throwing an exception
I tried googling these but couldn't find anything useful; anyone know how to fix this? I'm on ubuntu 16.04 with cassandra 3.10
the sl4j related one you can fix by adding an sl4j compatible logger to your project
that's a decent option with clojure if you use timbre, yes
there's also clj-logging-config
@qqq the code you are using demands to be able to log, but they are nice enough not to force you to use a specific logger
clj-logging-config should wrap this up pretty simply?
okay, so you're saying:
(1) add to build.boot clj-logging-config "1.9.13"
(2) rerun everything?
you might also need some specific sl4j thing - I forget if it provides it - but yes
it seems alot of logging libraries choose slf4j as its first choice, but none of thema ctually provides it
that's intentional - you are supposed to be able to pick your own sl4j compatible logger
it's complicated and bad and I don't like it, but it's what libs from java expect
I just forgot if clj-logging-config on its own provided a logger, or just a friendlier clojure ui to controlling the logging - clearly it's the latter
@noisesmith : by following the 2nd answer on http://stackoverflow.com/questions/7421612/slf4j-failed-to-load-class-org-slf4j-impl-staticloggerbinder ; we now get a different error:
this is what clj-logging-config is for, setting up appenders and such
@noisesmith : all logging issues resolved; thanks for your help!
What's the idiomatic way to do update-in and map over the value? Bonus points for not requiring spectre.
Here's what I have:
(defn map-over
[f & args]
(let [apply-f (fn [x] (apply f x args))]
(fn [x] (into (empty x) (map apply-f x)))))
(defn reset-errors [form]
(update form :fields (map-over dissoc :errors)))
I guess I'm looking for something elegant/readable
since you are already using into, you can move a close paren and use the map transducer
also be aware that reverses lists (that may be what you want)
Do you mean (into (empty x) (map apply-f) x)
? Not sure where the reversing comes in. (but thanks)
because into a list reverses the input
,(into (empty '(1 2 2 3)) (map inc) '(1 2 2 3))
Ah, gotcha. List vs vector.
right, and sets of course all bets are off
Thanks for the tip.
the advantage to the map transducer is that it isolates the transformation from the data source (and it avoids creating a bunch of garbage which tends to be great for performance)
What is a normal amount of GC to see in VisualVM when processing a large lazy sequence? I'm seeing spikes that claim to have been 7,000,000µs (7s) but that doesn't seem right! (These are also Minor GC pauses, not Major)
7s might not be much if the whole processing takes 1 hour, but it might be much if it takes 30seconds at all
My test expression: (let [mfn (apply juxt (repeatedly 20 #(comp str rand-uuid)))] (doseq [x (map mfn (range 50000000))] nil))
@dominicm looking at visual vm while that thing is running you can see it constantly creates char[] arrays. like ca. 15 000 000 instances in 3 seconds, then throws them away, and again
But I suspect it is not due to the doseq
function, but due to the str
and rand-uuid ID function, in combination
No sure, I'm not looking for an answer necessarily. Just an understanding of what GC should look like here (because I don't understand at all). A lot of the java GC stuff is arcane to start with, and Clojure's lack of objects makes it harder to "track" causes.
But you do understand what a GC's job in general is? I hope this question does not sound to hard 🙂
@sveri to jump in an sweep up the unused objects (after references to them are dropped). I'd expected a much lower number throughout though. I figured a GC > 2s would be considered terrible.
It looks like I might be missing some columns from my sampler table. Damn HiDPI display.
what watch (I don't care android or apple) is easiest to program in clj or cljs? I don't care about the brand of the watch -- I just want a write computation device programmable in clj/cljs
@dominicm the instances count goes from 3 Million to 25 Million in 1 - 2 seconds constantly and then drops back
Oh okay, that is right then. I know I've seen an allocations/per second before, but couldn't see it for memory. The grey above the scrollbar confused me 🙂. Yeah, I can see that now.
If you just run: (doseq [i (take 50000000 (repeatedly (fn [] 1)))] nil)
it will finish in a few seconds, at least on my machine
Part of it seems to be that juxt
, when I use the juxt
to generate 20 items, I suppose that's turning 50mil to a billion.
I just wonder why so many char[] instances are created, its just, that my little one came onto my lap right now and he is not that interested in inspecting arbitrary clojure functions
Part of the problem is the randomUUID function, if you leave that out, it will not allocate as much strings, but anyway
Or I would say, generating a random UUID is slow in general and if you take that out it finishes much faster
@sveri Not code in production, interesting in learning some more about how to debug these kinds of things though (for prod later). I was trying to generate a random string really (something sufficiently big) so that it would be representative of e.g. db columns.
@dominicm thats the point, what you were measuring, was the generation of random UUIDs, not the doseq of a lazy seq.
Performance optimization in general is pretty straight forward. Take the code you want to optimize, run it in a loop, connect a profiler at it, watch which code paths take the most time, fix that.
Where fix that = pick the most appropriate solution to the specific kind of problem in terms of speed.
Although I have used yourkit at work most of the times, lately I switched to flight recorder and it is sufficient so far
I'm starting to think that I've been looking at the steady GC & confusing it for "high" GC, when we're actually being CPU limited somewhere
@sveri I'll check out Flight. Is YourKit significantly better than VisualVM (& in what ways, approximately?)
https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/run.htm#JFRUH176 is this flight?
@dominicm yourkit is more exhaustive in regards of features and offers some more stuff, like Hot Spots, common errors etc.
No, sure. I'll take a look at getting a YourKit license, we're 100% Clojure, so it's likely a worthwhile investment
If I were to pay for yourkit myself, right now I would pick flight recorder instead. I never used one of the specific yourkit features.
I am not sure if the runtime has to be the oracle JDK. You might want to try to download it, start the flight recorder from the bin path of the oracle JDK and connect to Open JDK instance.
I guess it might work, but dont know about any licensing issues you might have. There is a reason it is only shipped with the Oracle JDK.
hey mates, one quick and probably simple question reading some clojure articles/books , almost every text says reading from global variable is a kind of side-effect ( make my function impure )
(def constant "some value here")
(defn my-func [x y]
(str x y constant))
does my-func is considered “impure” ?
the same code in haskell is considered purewe don't really have the kind of strict pure/impure distinction that haskell does, but unlike haskell our vars (namespace level values) are mutable containers
that said, anyone that changes vars at runtime is a fool
The definition I've normally seen is if the function's output is not produced deterministically by its input, it's impure. Since the var could change, this function would be impure. Practically speaking if the var will NEVER change, then I suppose it isn't, but you'd have to see the whole code base to know nothing will ever change it.
yeah, good point @noisesmith , thx
with def, or alter-var-root
oh, intern would do it too
there's a few approaches available 😄
no - def creates a var which in your namespace is pointed to by the symbol 'constant
it's the namespace that links the symbol to the value, the symbol knows nothing - it's just a special kind of string
you can alter the namespace to map 'constant
to a different var, or you could alter the var contents without touching the namespace
@john const doesn't do what people usually want and if you aren't defining a primitive, isn't useful
my vote is that the above function should be called "pure" because that is the intent of the code.
But as a matter of practice, if you wanted to be pedantically pure, you could pass the constant
into the function as well.
I've found myself having to rewrite functions that 'were' pure, but then ended up not being so pure when I had to change the semantics elsewhere in my program. Which caused me to design my functions defensively pure - only in and out, minimizing references to values outside the parameters as much as possible.
hmm... regarding ^:const
- I was always told it wasn't for constants and it is only useful for primitives, but in a simple repl usage it really does prevent the new value from being looked up (but it doesn't prevent redefinition)
+user=> (def ^:const foo "hello")
#'user/foo
+user=> (defn bar [] foo)
#'user/bar
+user=> (bar)
"hello"
+user=> (def foo "bye")
#'user/foo
+user=> (bar)
"hello"
+user=> foo
"bye"
this is pure. whether it's a static value, or a function, is irrelevant -- it's referentially transparent
@lmergen but anther namespace could call alter-var-root or intern and make it no longer referentially transparent
yes, and that's only because clojure allows that -- clojure the language is impure, and this is the consequence
@john > But as a matter of practice, if you wanted to be pedantically pure, you could pass the constant
into the function as well.
yeah, it is not referential transparent. if I pass all the arguments , probably it would be considered pure. I was comparing with haskell that consider that kind of code “pure”
would you consider it pure if constant was a function that always returned the same value ?
yeah, the bar function is “pure” https://clojurians.slack.com/archives/C03S1KBA2/p1491754990636139
Even Haskell isn't 100% pure
Allocating a closure mutates the GC in almost every language
@oliv hence my point of view, due to referential transparency, you can replace values with functions, and as such this is pure
So imo it's about your definition of purity
Clojure's purity is half structural and half really good convention. I think you have to accept some amount of convention in your definition of what appears to be "pure"
yes, also, a classic Haskell argument is that the computer getting hot is also a side effect, etc etc -- at some point, you have to be pragmatic :)
Purê languages are useless... the point is to be practical. Take Erlang as an example: it's a functional language, there is immutability, but something's are not immutable: FIle IO, and stuff..
well, in a certain sense, planet earth is a pure function, to the extent that it doesn't mutate alpha centauri
Right, so in that vein vars are mutable to enable reply driven development. But mutating them a runtime is a bad idea.
@oliv you didn't specifically ask, but I want to point this out in case anyone else is reading: in your case may want to create a version of the function that takes the constant
as a parameter, maybe make that function private, and then (def bar (partial bar* constant))
. This way your bar*
is pure, but you can easily use bar
from anywhere. I find this to be a good pattern 🙂
Direct linking does the same thing.
hey @tbaldridge I’m not so aware of clojure . what you mean by direct linking ? ( going to google it now 🙂 )
the way top level functions are linked to each other in the general case in clojure is via vars at runtime
g
has a reference to the var #'f
and when you invoke g
it derefs #'f
and invokes the value
direct linking is more or less doing the deref step at compile time instead of at runtime
so g
is directly linked with whatever the value of #'f
is when g
is compiled, as opposed to getting the current value of #'f
whenever g
is invoked
the var linkage is the general case, direct linking is an exception, and there are others for things like primitive arguments and direct protocol function calls
@hiredman a google search for "clojure direct linkage" turns up noise for me. do you have any links to good articles on it?
def symbol points to var that points to value ( kind of pointer to pointer in c/c++) and direct linking will dereference in compile time
namespace maps symbol to var
local names (let or function arguments) are treated differently and don't get vars, because they don't really make sense there
the compilation unit for clojure is a single top level form, and vars are there to be a (re-linkable in the repl) link between compilation units
in a let form, all the name usages are in the same compilation unit as the binding of the name, so no vars
aot compilation works a namespace at a time, but in the general case, clojure compiles a single top level form in to bytecode at a time
the way vars are looked up at compile time depends on ns, when the compilation and evaluation of previous forms can effect
in order to re-run individual definitions in your namespace, you need to do lookup on usage of namespace level definitions (vars)
a form is compiled and executed, the execution effects the state of the system, then another form
*ns*
is part of the namespace system, if you compile something that doesn't need to figure out names, it doesn't need to be looked at
there's always an ns, it's user if nothing else sets it
at least in most cljs repls, if a ns doesn't exist it will try to create one for the user
if you execute code outside of the repl and don't fiddle with *ns*
the value is usually clojure.core
people often think of namespaces as some kind of execution context, and code running in a namespace, which is entirely incorrect
hiredman: this is making me nervous. in fact i do think of an ns as an execution environment. clojure's metaprogramming facilities allow us to control that env. you can change the meaning of a form by evaluating it in a different ns. if i understand your comment
@hiredman citation for ns ever defaulting to clojure.core? I've seen implicit ns be user
often, I've never accidentally ended up in a clojure.core context
*ns*
defaults to clojure.core
, and clojure.main sets it to user
if you have some other entry point (an aot compiled class for example) when you run that the value of *ns*
will be the namespace named clojure.core
@hiredman @oliv sorry i was responding to oliv's question what referential transparency meant, not about linking
@hiredman now your complectifying me. i just solved a macro problem by `(binding [ns ~ns].. ) which looks like code running in an ns. what am i missing.
@mobileink you are introducing a bug, because *ns*
at runtime has no relationship to *ns*
at compile time
your ns form at the top of the file sets the namespace used when compiling the forms in the file, but when those forms are executed (if they functions when they are invoked) *ns*
could be anything
also, *ns*
is a namespace object, not a symbol, I am not sure I would recommend embedding that directly in code (via macroexpansion)
fwiw, my use case is meta-programming - turning clojure syntax into another syntax. specifically Polymer (html+js+css) i think maybe different best practice rules apply.
see the defmacro for lambda at https://github.com/miraj-project/polymer-dom/blob/master/src/main/clj/miraj/polymer/dom.clj where i use (binding [*ns* ~*ns*]...)
sample code that uses it is at https://github.com/miraj-project/polymer-dom/blob/master/test/system/clj/miraj/polymer/dom/binding_test.clj.
https://clojuredocs.org/clojure.core/with-local-vars <-- for cases where I can use this, this should be faster than vars, faster than atoms, and basicaly faster than anything else, since (1) it's thread local and (2) it's local context so (3) clojure compiler can optimize the sh*t out of this right?
volatiles are faster than local vars
afaik there's no good use for local vars
feel free to benchmark
it takes a while
+user=> (crit/bench (with-local-vars [a 0] (dotimes [i 1000] (var-set a (inc (var-get a))))))
Evaluation count : 593280 in 60 samples of 9888 calls.
Execution time mean : 106.803212 µs
Execution time std-deviation : 1.569357 µs
Execution time lower quantile : 104.391640 µs ( 2.5%)
Execution time upper quantile : 109.464807 µs (97.5%)
Overhead used : 1.636004 ns
nil
+user=> (crit/bench (let [a (volatile! 0)] (dotimes [i 1000] (vswap! a inc))))
Evaluation count : 6498900 in 60 samples of 108315 calls.
Execution time mean : 9.429217 µs
Execution time std-deviation : 114.956656 ns
Execution time lower quantile : 9.226158 µs ( 2.5%)
Execution time upper quantile : 9.628630 µs (97.5%)
Overhead used : 1.636004 ns
nil
volatile was ~10 times faster
yup - feel free
and maybe your specific usage is somehow different? dunno
yeah - often not even mutating is the winner 😄
(def cnt 100000)
(defn via-recur []
(loop [s 0
lst (range cnt)]
(if (empty? lst) s
(recur (+ s (first lst))
(rest lst)))))
(defn via-volatile []
(let [s (volatile! 0)]
(doseq [i (range cnt)]
(vswap! s + i))))
(defn via-locals []
(with-local-vars [s 0]
(doseq [i (range cnt)]
(var-set s (+ i (var-get s))))))
(defn via-reduce []
(reduce + (range cnt)))
(quick-bench (via-recur) :verbose) ;; 6.49 ms
(quick-bench (via-volatile) :verbose) ;; 3.15 ms
(quick-bench (via-locals) :verbose) ;; 13ms
(quick-bench (via-reduce) :verbose) ;; 1.03 ms
because range is optimized for reduce
no - why would that help?
reduce recognizes a range, and knows how to use it via a fast code path in the range implementation
it's supposed to do that
oh, in that case, sure - or use (dotimes [i 1000] ... i ...) to just access the numbers directly
(reduce could work, but wouldn't in that case)
the doall doesn't change the type
and reduce's optimization is a property of the type returned by range
maybe (list* (range 100000))
no, list* doesn't do it... (into [] (range ...))
alright, retrying with (def my-list (into [] (range cnt))) / even though this is now a vector
oh yeah, that would do it
and range always returns a specific number type, allowing for further optimization, right?
reducing a range is literally a for loop from start to end (at least, in the bounded, integer step case)
see https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/LongRange.java#L218-L226
alright, that's enough for today; thanks for insights on criterium/bench , volatile!, and reduce
Your kit is expensive at $500.00 https://www.yourkit.com/purchase/
visualvm works OK and it's free
Any function that can change hash-map to list. {:a 1 :b 2} => '(:a 1 : b 2) ?
With into
Im getting vector pairs, understandibly, but then I feel like a newb trying to use flatten
to solve that.
solved it (reverse (reduce #(into %1 %2) '() {:a 1 :b 2}))
just had to ask to get the answer from the rubber duck.
other options include (apply concat ...) and (into [] cat ...)
this is why I love asking on this channel, one can always find more beautiful solutins in Clojure.
Let m1, m2 be maps. Does (into m1 m2) use transients? Looking for more than a yes/no answer. Can someone point me at the line of code where this is resolved?
@qqq yes, read the output of (source into)
in your repl
hopefully some day merge will use transients, but for now into is the better option
been looking at that ticket @noisesmith for merge + transients
oh, cool
i want to take a new approach with its development. 1) write the benchmarks 2) need a really good fspec for the current impl of merge
nice - two forms enter, one form leaves
the fspec will help ensure that an implementation doesn't break the contract -- which currently has a lot of "extras"
^ above is my "favorite" merge behavior
it also acts like identity for single arguments other than nil or false (but many functions do that)
@ghadi another case that will be a pain in the ass when reimplementing is (merge [1 2] 3 4 5) - totally incorrect, but accidentally works
or worse yet (def my-reverse (partial merge ()))
yeah- that's fair - I know I've seen code at work that misused merge in similar ways
but totally legit to break it
Considering Rich's presentations on compatibility, is it? https://clojurians.slack.com/archives/C03S1KBA2/p1491780329036554
well, i wouldn't think anybody could get angry if (merge [1 2] 3 4)
stopped working tbh
merge's docstring explicitely says that it only works on maps so everything else is GIGO
people come asking for help with code where they are incidentally calling merge on vectors with surprising regularity - I have no idea why it's so common
I'd assume they'd invoke it like (merge [1 2] [3 4])
rather than (merge [1 2] 3 4)
tho
@noisesmith what has appeared in the wild was (merge {} []) where the entry is not in the first position...
the former is understandable but doesn't work anyway, the latter just doesn't make any sense and I can't imagine anybody using it
but (merge [1 2] [3 4]) returns [1 2 [3 4]] so they realize (merge [1 2] 3 4) works - iteration in practice
sucess-typing -- doesn't have problem with dynamic types; and any type errors are guaranteed to be type errors
or a spec
for me, it's (1) look at example on clojuredocs, (2) try random stuff, (3) when it breaks, ask #clojurians/slack, (4) get told to read docstring, and (5) finally read docstring
if you don't read the docstring then you can't complain if you're using a function wrong and your usage eventually breaks
@noisesmith right, a spec for merge based on the contract from the docstring would accept map args (and I guess nils too) but every other input is just garbage according to the docstring
the other day I was working on some code and found the following: (defn queue [] (clojure.lang.PersistentQueue/EMPTY))
there's two layers in wat there for me - it makes me sad that the parens work
and it makes me sad that anyone thought the function indirection was needed
I switched it to (def queue clojure.lang.PersistentQueue/EMPTY)
and deleted some parens of course
Considering the lousy documentation of clojure.lang.PersistentQueue, I am not surprised. I think it is kind of ok, because at least it documents the arity, which def doesn’t (and most clojure devs I’ve met do not know PersistentQueue) - but of course the opportunity to add a docstring was missed, which is not ok 🙂
user=> (macroexpand '(Integer/MAX_VALUE))
(. Integer MAX_VALUE)
here's why that works btwif we were strict about .- ...
if method calls were only (. Klass (method))
rather than also (. Klass method)
there would be no ambiguity between field access & no arg method calls
it's never going to change ofc because of backwards compatibility but it's one of the things that annoy me most about .
tangentially, the queue example reminds me of one of my favorite bugs I've found - someone creates a PesistentQueue and then proceeds to use first and rest on it instead of peek / pop, thus turning their fifo into a lifo
maybe if they'd read the doc string for rest, haha