Fork me on GitHub
#clojure
<
2017-04-09
>
qqq00:04:59

I'm using cassandra + cassaforte-3.0.0-rc1. I get an error of the form "failed to lod calss org.slf4j.impl.StaticLoggerBinder" and "ClassNotFoundException" "com.google.common.util.concurent.FutureFallback"

mpenet01:04:16

qqq: use alia instead of cassaforte. Cassaforte is not really maintained anymore

mpenet01:04:00

Its original author actually advises to use alia nowadays

qqq01:04:15

@U050SC7SV: would you like to declare a conflict of interest? 🙂

mpenet01:04:05

Not really, as I said the creator of cassaforte says the same ;)

mpenet01:04:22

But yes, I might be a bit biaised

qqq01:04:25

well, it's already better than cassaforte in that I can connect to my local cassandra server w/o throwing an exception

qqq01:04:26

alia wins

mpenet01:04:59

It's much better in many ways ;)

qqq00:04:40

I tried googling these but couldn't find anything useful; anyone know how to fix this? I'm on ubuntu 16.04 with cassandra 3.10

noisesmith00:04:57

the sl4j related one you can fix by adding an sl4j compatible logger to your project

qqq00:04:39

I tried googling that, and I got slf4j-timbre -- is that what I want ?

qqq00:04:55

(I've never cared about SLF4J until now)

noisesmith00:04:03

that's a decent option with clojure if you use timbre, yes

qqq00:04:16

I'm not using timbre.

qqq00:04:24

I just want to get rid of this error. but I am willing to install timbre.

noisesmith00:04:25

there's also clj-logging-config

qqq00:04:31

What do you recommend? 🙂

noisesmith00:04:49

@qqq the code you are using demands to be able to log, but they are nice enough not to force you to use a specific logger

noisesmith00:04:58

clj-logging-config should wrap this up pretty simply?

qqq00:04:49

okay, so you're saying: (1) add to build.boot clj-logging-config "1.9.13" (2) rerun everything?

noisesmith00:04:03

you might also need some specific sl4j thing - I forget if it provides it - but yes

qqq00:04:28

same error

qqq00:04:34

okay, I need some slf4j specific thing

qqq00:04:00

it seems alot of logging libraries choose slf4j as its first choice, but none of thema ctually provides it

noisesmith00:04:32

that's intentional - you are supposed to be able to pick your own sl4j compatible logger

noisesmith00:04:45

it's complicated and bad and I don't like it, but it's what libs from java expect

noisesmith00:04:30

I just forgot if clj-logging-config on its own provided a logger, or just a friendlier clojure ui to controlling the logging - clearly it's the latter

qqq00:04:18

"No appenders could be found for .... " "Please initialize the log4j system propertly."

qqq00:04:25

This is progress. 🙂

noisesmith00:04:40

this is what clj-logging-config is for, setting up appenders and such

qqq01:04:25

@noisesmith : all logging issues resolved; thanks for your help!

Oliver George01:04:48

What's the idiomatic way to do update-in and map over the value? Bonus points for not requiring spectre.

Oliver George01:04:38

Here's what I have:

(defn map-over
  [f & args]
  (let [apply-f (fn [x] (apply f x args))]
    (fn [x] (into (empty x) (map apply-f x)))))

(defn reset-errors [form]
  (update form :fields (map-over dissoc :errors)))

Oliver George01:04:56

I guess I'm looking for something elegant/readable

noisesmith01:04:04

since you are already using into, you can move a close paren and use the map transducer

noisesmith01:04:20

also be aware that reverses lists (that may be what you want)

Oliver George01:04:44

Do you mean (into (empty x) (map apply-f) x) ? Not sure where the reversing comes in. (but thanks)

noisesmith01:04:56

because into a list reverses the input

noisesmith01:04:27

,(into (empty '(1 2 2 3)) (map inc) '(1 2 2 3))

Oliver George01:04:29

Ah, gotcha. List vs vector.

noisesmith01:04:54

right, and sets of course all bets are off

Oliver George01:04:04

Thanks for the tip.

noisesmith01:04:03

the advantage to the map transducer is that it isolates the transformation from the data source (and it avoids creating a bunch of garbage which tends to be great for performance)

dominicm09:04:18

What is a normal amount of GC to see in VisualVM when processing a large lazy sequence? I'm seeing spikes that claim to have been 7,000,000µs (7s) but that doesn't seem right! (These are also Minor GC pauses, not Major)

sveri10:04:49

@dominicm How long does the processing take?

dominicm10:04:26

@sveri My current testing is just a doseq with a nil body

dominicm10:04:35

(Warming up my house)

sveri10:04:27

@dominicm ok, then how large is the lazy sequence?

dominicm10:04:48

@sveri 50 million is my test number.

sveri10:04:52

7s might not be much if the whole processing takes 1 hour, but it might be much if it takes 30seconds at all

dominicm10:04:40

@sveri total processing time was 15m with the nil body.

dominicm10:04:08

The GC is more or less constant.

dominicm10:04:29

My test expression: (let [mfn (apply juxt (repeatedly 20 #(comp str rand-uuid)))] (doseq [x (map mfn (range 50000000))] nil))

dominicm10:04:52

rand-uuid is just a function that always returns a (UUID/randomUUID)

sveri10:04:59

@dominicm looking at visual vm while that thing is running you can see it constantly creates char[] arrays. like ca. 15 000 000 instances in 3 seconds, then throws them away, and again

sveri10:04:09

so yes, from a first glance, a lot of GC activity looks reasonable

sveri10:04:36

But I suspect it is not due to the doseqfunction, but due to the strand rand-uuid ID function, in combination

sveri10:04:44

but thats just my first guess after a quick look

dominicm10:04:11

No sure, I'm not looking for an answer necessarily. Just an understanding of what GC should look like here (because I don't understand at all). A lot of the java GC stuff is arcane to start with, and Clojure's lack of objects makes it harder to "track" causes.

sveri10:04:09

But you do understand what a GC's job in general is? I hope this question does not sound to hard 🙂

sveri10:04:15

while running the function, click on the sampler tab and sample the memory

sveri10:04:27

you can watch a lot of objects get created and destroyed

dominicm10:04:09

@sveri to jump in an sweep up the unused objects (after references to them are dropped). I'd expected a much lower number throughout though. I figured a GC > 2s would be considered terrible.

dominicm10:04:27

It looks like I might be missing some columns from my sampler table. Damn HiDPI display.

dominicm10:04:50

@sveri Are the allocations/s the column after "Instances"?

qqq10:04:05

what watch (I don't care android or apple) is easiest to program in clj or cljs? I don't care about the brand of the watch -- I just want a write computation device programmable in clj/cljs

sveri10:04:08

@dominicm the instances count goes from 3 Million to 25 Million in 1 - 2 seconds constantly and then drops back

dominicm10:04:30

Oh okay, that is right then. I know I've seen an allocations/per second before, but couldn't see it for memory. The grey above the scrollbar confused me 🙂. Yeah, I can see that now.

dominicm10:04:35

Why is it allowed to go so high?

sveri10:04:53

I think "allowed" is the wrong word. It just does, what the function does

dominicm10:04:07

Yes, you're right 🙂

sveri10:04:59

If you just run: (doseq [i (take 50000000 (repeatedly (fn [] 1)))] nil) it will finish in a few seconds, at least on my machine

sveri10:04:48

Thats an easy way to make sure its not doseq that consumes all the memory

dominicm10:04:54

Part of it seems to be that juxt, when I use the juxt to generate 20 items, I suppose that's turning 50mil to a billion.

sveri10:04:05

I just wonder why so many char[] instances are created, its just, that my little one came onto my lap right now and he is not that interested in inspecting arbitrary clojure functions

dominicm10:04:28

Don't worry about it 😛. Family > Everything else.

sveri10:04:04

Part of the problem is the randomUUID function, if you leave that out, it will not allocate as much strings, but anyway

sveri10:04:15

Is that code the same that runs in production?

sveri10:04:24

Or are you just playing around to learn some stuff?

sveri10:04:00

Or I would say, generating a random UUID is slow in general and if you take that out it finishes much faster

dominicm11:04:33

@sveri Not code in production, interesting in learning some more about how to debug these kinds of things though (for prod later). I was trying to generate a random string really (something sufficiently big) so that it would be representative of e.g. db columns.

sveri11:04:56

@dominicm thats the point, what you were measuring, was the generation of random UUIDs, not the doseq of a lazy seq.

dominicm11:04:59

Yeah, I see that now 😛

sveri11:04:28

Performance optimization in general is pretty straight forward. Take the code you want to optimize, run it in a loop, connect a profiler at it, watch which code paths take the most time, fix that.

sveri11:04:46

Where fix that = pick the most appropriate solution to the specific kind of problem in terms of speed.

sveri11:04:56

or memory, if that is your concern, or both 😄

sveri11:04:58

Although I have used yourkit at work most of the times, lately I switched to flight recorder and it is sufficient so far

dominicm12:04:30

I'm starting to think that I've been looking at the steady GC & confusing it for "high" GC, when we're actually being CPU limited somewhere

dominicm12:04:25

@sveri I'll check out Flight. Is YourKit significantly better than VisualVM (& in what ways, approximately?)

sveri12:04:26

@dominicm yourkit is more exhaustive in regards of features and offers some more stuff, like Hot Spots, common errors etc.

sveri12:04:33

Yes, thats it, its part of the oracle JDK

sveri12:04:48

but does not come with Open JDK if you use that instead.

dominicm12:04:17

No, sure. I'll take a look at getting a YourKit license, we're 100% Clojure, so it's likely a worthwhile investment

sveri12:04:03

If I were to pay for yourkit myself, right now I would pick flight recorder instead. I never used one of the specific yourkit features.

sveri12:04:14

but, thats just me, your mileage may vary, of course.

dominicm12:04:05

Oh okay, then maybe we'll just use that (though that means switching to oracle jdk)

sveri12:04:49

I am not sure if the runtime has to be the oracle JDK. You might want to try to download it, start the flight recorder from the bin path of the oracle JDK and connect to Open JDK instance.

sveri12:04:20

I guess it might work, but dont know about any licensing issues you might have. There is a reason it is only shipped with the Oracle JDK.

sveri12:04:18

so it only matters if you want to use it in production

dominicm12:04:15

I see the rand-uuid being the actual bottleneck now.. hehe

dominicm14:04:56

@sveri This helps a lot, thanks for the help 🙂.

sveri14:04:50

@dominicm np, your welcome

tdantas16:04:20

hey mates, one quick and probably simple question reading some clojure articles/books , almost every text says reading from global variable is a kind of side-effect ( make my function impure )

(def constant "some value here")
(defn my-func [x y]
  (str x y constant))
does my-func is considered “impure” ? the same code in haskell is considered pure

noisesmith16:04:41

we don't really have the kind of strict pure/impure distinction that haskell does, but unlike haskell our vars (namespace level values) are mutable containers

noisesmith16:04:27

that said, anyone that changes vars at runtime is a fool

elena.poot16:04:22

The definition I've normally seen is if the function's output is not produced deterministically by its input, it's impure. Since the var could change, this function would be impure. Practically speaking if the var will NEVER change, then I suppose it isn't, but you'd have to see the whole code base to know nothing will ever change it.

tdantas16:04:43

yeah, good point @noisesmith , thx

tdantas16:04:01

how would we change the constant above ?

noisesmith16:04:11

with def, or alter-var-root

noisesmith16:04:41

oh, intern would do it too

noisesmith16:04:52

there's a few approaches available 😄

tdantas16:04:15

yeah, make sense

tdantas16:04:46

the constant points to a var and the var contains the value, right ?

tdantas16:04:57

we could change the pointer of constant to another var

tdantas16:04:01

that is correct ^^ ?

noisesmith16:04:33

no - def creates a var which in your namespace is pointed to by the symbol 'constant

noisesmith16:04:01

it's the namespace that links the symbol to the value, the symbol knows nothing - it's just a special kind of string

john16:04:29

(def ^:const constant ??

noisesmith16:04:38

you can alter the namespace to map 'constant to a different var, or you could alter the var contents without touching the namespace

noisesmith16:04:06

@john const doesn't do what people usually want and if you aren't defining a primitive, isn't useful

john16:04:37

my vote is that the above function should be called "pure" because that is the intent of the code.

john16:04:23

But as a matter of practice, if you wanted to be pedantically pure, you could pass the constant into the function as well.

john16:04:27

I've found myself having to rewrite functions that 'were' pure, but then ended up not being so pure when I had to change the semantics elsewhere in my program. Which caused me to design my functions defensively pure - only in and out, minimizing references to values outside the parameters as much as possible.

noisesmith16:04:10

hmm... regarding ^:const - I was always told it wasn't for constants and it is only useful for primitives, but in a simple repl usage it really does prevent the new value from being looked up (but it doesn't prevent redefinition)

+user=> (def ^:const foo "hello")
#'user/foo
+user=> (defn bar [] foo)
#'user/bar
+user=> (bar)
"hello"
+user=> (def foo "bye")
#'user/foo
+user=> (bar)
"hello"
+user=> foo
"bye"

lmergen16:04:19

this is pure. whether it's a static value, or a function, is irrelevant -- it's referentially transparent

noisesmith16:04:03

@lmergen but anther namespace could call alter-var-root or intern and make it no longer referentially transparent

lmergen16:04:40

yes, and that's only because clojure allows that -- clojure the language is impure, and this is the consequence

tdantas16:04:30

@john > But as a matter of practice, if you wanted to be pedantically pure, you could pass the constant into the function as well. yeah, it is not referential transparent. if I pass all the arguments , probably it would be considered pure. I was comparing with haskell that consider that kind of code “pure”

lmergen16:04:26

would you consider it pure if constant was a function that always returned the same value ?

tbaldridge16:04:09

Even Haskell isn't 100% pure

tdantas16:04:15

@lmergen good point, but I would say YES

tdantas16:04:38

all the side-effects are inside the ‘monads IO’

tbaldridge16:04:41

Allocating a closure mutates the GC in almost every language

lmergen16:04:56

@oliv hence my point of view, due to referential transparency, you can replace values with functions, and as such this is pure

tbaldridge16:04:23

So imo it's about your definition of purity

john16:04:42

Clojure's purity is half structural and half really good convention. I think you have to accept some amount of convention in your definition of what appears to be "pure"

lmergen16:04:28

yes, also, a classic Haskell argument is that the computer getting hot is also a side effect, etc etc -- at some point, you have to be pragmatic :)

lmergen16:04:04

gamma rays make my code impure as well :)

imetallica16:04:24

Purê languages are useless... the point is to be practical. Take Erlang as an example: it's a functional language, there is immutability, but something's are not immutable: FIle IO, and stuff..

john16:04:31

well, in a certain sense, planet earth is a pure function, to the extent that it doesn't mutate alpha centauri

john16:04:26

this is why functional programmers end up becoming philosophers 😉

lmergen16:04:59

.. or insane

tbaldridge16:04:00

Right, so in that vein vars are mutable to enable reply driven development. But mutating them a runtime is a bad idea.

john16:04:12

tomato tomato

kauko16:04:53

@oliv you didn't specifically ask, but I want to point this out in case anyone else is reading: in your case may want to create a version of the function that takes the constant as a parameter, maybe make that function private, and then (def bar (partial bar* constant)). This way your bar* is pure, but you can easily use bar from anywhere. I find this to be a good pattern 🙂

tdantas16:04:20

@lmergen what is your interpretation regarding referentially transparent ?

tdantas16:04:29

yeah, we are freezing the ‘constant’ in the partial application .

tbaldridge16:04:47

Direct linking does the same thing.

tdantas16:04:06

hey @tbaldridge I’m not so aware of clojure . what you mean by direct linking ? ( going to google it now 🙂 )

lmergen17:04:09

@oliv it means that a function can be replaced with the value it returns

hiredman17:04:35

That is not correct

hiredman18:04:18

the way top level functions are linked to each other in the general case in clojure is via vars at runtime

hiredman18:04:05

you def a function f and then def a function g that invokes f somewhere in its body

hiredman18:04:14

g has a reference to the var #'f and when you invoke g it derefs #'f and invokes the value

hiredman18:04:05

direct linking is more or less doing the deref step at compile time instead of at runtime

hiredman18:04:00

so g is directly linked with whatever the value of #'f is when g is compiled, as opposed to getting the current value of #'f whenever g is invoked

hiredman18:04:50

it is not some kind of constant folding

hiredman18:04:50

the var linkage is the general case, direct linking is an exception, and there are others for things like primitive arguments and direct protocol function calls

tdantas18:04:06

great @hiredman really thx

mobileink18:04:40

@hiredman a google search for "clojure direct linkage" turns up noise for me. do you have any links to good articles on it?

tdantas18:04:57

def symbol points to var that points to value ( kind of pointer to pointer in c/c++) and direct linking will dereference in compile time

noisesmith18:04:20

namespace maps symbol to var

hiredman18:04:26

more or less

hiredman18:04:14

local names (let or function arguments) are treated differently and don't get vars, because they don't really make sense there

hiredman18:04:17

the compilation unit for clojure is a single top level form, and vars are there to be a (re-linkable in the repl) link between compilation units

hiredman18:04:07

in a let form, all the name usages are in the same compilation unit as the binding of the name, so no vars

mobileink18:04:49

compilation unit being a namespace?

mobileink18:04:09

not a file, yeh?

hiredman18:04:32

a compilation unit is a single top level form

hiredman18:04:37

like what you type in to a repl

hiredman18:04:35

aot compilation works a namespace at a time, but in the general case, clojure compiles a single top level form in to bytecode at a time

mobileink18:04:35

ok, but "link between conpilation units" - that depends on ns, no?

hiredman18:04:37

the way vars are looked up at compile time depends on ns, when the compilation and evaluation of previous forms can effect

noisesmith18:04:08

in order to re-run individual definitions in your namespace, you need to do lookup on usage of namespace level definitions (vars)

mobileink18:04:09

maybe: compilation units, but always in a compilation context (ns)?

hiredman18:04:37

the compilation context is everything that was previously compiled and run

hiredman18:04:56

macro definitions, var definitions, fiddling with whatever

mobileink18:04:48

majes sense. but ns is universal, right? no ns, no compile?

hiredman18:04:54

a form is compiled and executed, the execution effects the state of the system, then another form

hiredman18:04:49

*ns* is part of the namespace system, if you compile something that doesn't need to figure out names, it doesn't need to be looked at

mobileink18:04:50

and there is no null or global ns?

noisesmith18:04:58

there's always an ns, it's user if nothing else sets it

hiredman18:04:38

the repl actually sets the namespace to *ns*

john18:04:01

at least in most cljs repls, if a ns doesn't exist it will try to create one for the user

hiredman18:04:04

if you execute code outside of the repl and don't fiddle with *ns* the value is usually clojure.core

hiredman18:04:32

*ns* is only used for figuring out how to map names when compiling

hiredman18:04:04

people often think of namespaces as some kind of execution context, and code running in a namespace, which is entirely incorrect

mobileink19:04:35

hiredman: this is making me nervous. in fact i do think of an ns as an execution environment. clojure's metaprogramming facilities allow us to control that env. you can change the meaning of a form by evaluating it in a different ns. if i understand your comment

mobileink19:04:33

you think an ns is not an execution context; what do you mean?

noisesmith18:04:50

@hiredman citation for ns ever defaulting to clojure.core? I've seen implicit ns be user often, I've never accidentally ended up in a clojure.core context

hiredman18:04:52

*ns* defaults to clojure.core, and clojure.main sets it to user if you have some other entry point (an aot compiled class for example) when you run that the value of *ns* will be the namespace named clojure.core

lmergen18:04:58

@hiredman @oliv sorry i was responding to oliv's question what referential transparency meant, not about linking

mobileink18:04:40

@hiredman now your complectifying me. i just solved a macro problem by `(binding [ns ~ns].. ) which looks like code running in an ns. what am i missing.

mobileink19:04:13

sorry make that [\ns\ ~ns]

hiredman19:04:28

@mobileink you are introducing a bug, because *ns* at runtime has no relationship to *ns* at compile time

hiredman19:04:15

your ns form at the top of the file sets the namespace used when compiling the forms in the file, but when those forms are executed (if they functions when they are invoked) *ns* could be anything

mobileink19:04:21

what's with the middle finger?

lmergen19:04:49

i meant it as a "look up"

hiredman19:04:06

also, *ns* is a namespace object, not a symbol, I am not sure I would recommend embedding that directly in code (via macroexpansion)

hiredman19:04:48

(find-ns (ns-name *ns*))

hiredman19:04:59

I forget if the compiler actually supports embedding namespace objects

mobileink19:04:00

hmm, thanks, the code works, but now i'm worried it only seems to work. simple_smile

mobileink19:04:59

i'll be posting it pretty soon so consider yourself a reviewer! 😉

mobileink19:04:37

macros... can't live with 'em, can't really f*ck up without 'em.

mobileink19:04:51

fwiw, my use case is meta-programming - turning clojure syntax into another syntax. specifically Polymer (html+js+css) i think maybe different best practice rules apply.

qqq20:04:24

https://clojuredocs.org/clojure.core/with-local-vars <-- for cases where I can use this, this should be faster than vars, faster than atoms, and basicaly faster than anything else, since (1) it's thread local and (2) it's local context so (3) clojure compiler can optimize the sh*t out of this right?

bronsa20:04:45

the clojure compiler doesn't really do any optimizations besides basic inlining

noisesmith20:04:45

volatiles are faster than local vars

qqq20:04:01

why? don't volatiles have to be written out to RAM ?

noisesmith20:04:03

afaik there's no good use for local vars

qqq20:04:10

whereas local vars can live in caches

noisesmith20:04:22

feel free to benchmark

qqq20:04:54

alright, what's the easiest benchmark function in clojure?

qqq20:04:02

I have a var setup already; I just need to import a benchmark library

bronsa20:04:05

use criterium

qqq20:04:30

how long is a (bench ...) supposed to take ?

noisesmith20:04:02

it takes a while

qqq20:04:09

yeah, apaprently it decided to call it 150,000 + times

noisesmith20:04:32

+user=> (crit/bench (with-local-vars [a 0] (dotimes [i 1000] (var-set a (inc (var-get a))))))
Evaluation count : 593280 in 60 samples of 9888 calls.
             Execution time mean : 106.803212 µs
    Execution time std-deviation : 1.569357 µs
   Execution time lower quantile : 104.391640 µs ( 2.5%)
   Execution time upper quantile : 109.464807 µs (97.5%)
                   Overhead used : 1.636004 ns
nil
+user=> (crit/bench (let [a (volatile! 0)] (dotimes [i 1000] (vswap! a inc))))
Evaluation count : 6498900 in 60 samples of 108315 calls.
             Execution time mean : 9.429217 µs
    Execution time std-deviation : 114.956656 ns
   Execution time lower quantile : 9.226158 µs ( 2.5%)
   Execution time upper quantile : 9.628630 µs (97.5%)
                   Overhead used : 1.636004 ns
nil

noisesmith20:04:44

volatile was ~10 times faster

qqq20:04:31

I'm going to replicate these numbers myself just to get the hang of bench/quick-bench

noisesmith20:04:41

yup - feel free

noisesmith20:04:19

and maybe your specific usage is somehow different? dunno

qqq20:04:46

hmm, so I added up the numbers in (range 100000)

qqq20:04:01

loop-recur: 6.49 ms volatile: 3.15 ms locals: 13ms

qqq20:04:10

so volatile wins

rauh20:04:46

@qqq let's see the code!

qqq20:04:03

actually, reduce is 1.03 ms

qqq20:04:04

so reduce wins

qqq20:04:16

sure, let me pull out the code

noisesmith20:04:19

yeah - often not even mutating is the winner 😄

qqq20:04:10

(def cnt 100000)

(defn via-recur []
  (loop [s 0
         lst (range cnt)]
    (if (empty? lst) s
        (recur (+ s (first lst))
               (rest lst)))))


(defn via-volatile []
  (let [s (volatile! 0)]
    (doseq [i (range cnt)]
      (vswap! s + i))))


(defn via-locals []
  (with-local-vars [s 0]
    (doseq [i (range cnt)]
      (var-set s (+ i (var-get s))))))


(defn via-reduce []
  (reduce + (range cnt)))





(quick-bench (via-recur) :verbose) ;; 6.49 ms

(quick-bench (via-volatile) :verbose) ;; 3.15 ms

(quick-bench (via-locals) :verbose) ;; 13ms

(quick-bench (via-reduce) :verbose) ;; 1.03 ms

qqq20:04:35

why is loop-recur so much slower than reduce

noisesmith20:04:47

because range is optimized for reduce

qqq20:04:05

should I "force the range" first ?

qqq20:04:22

i.e. (def my-lst (doall (range cnt)))

noisesmith20:04:27

no - why would that help?

qqq20:04:42

to avoid any range/reduce interplay

noisesmith20:04:42

reduce recognizes a range, and knows how to use it via a fast code path in the range implementation

noisesmith20:04:51

it's supposed to do that

qqq20:04:05

yeah, but my generic code is not summing over a range

qqq20:04:11

otherwise I'd just use n*(n+1)/2

noisesmith20:04:29

oh, in that case, sure - or use (dotimes [i 1000] ... i ...) to just access the numbers directly

noisesmith20:04:48

(reduce could work, but wouldn't in that case)

qqq20:04:19

wtf, so after a (define my-list (doall (range 100000))), and re-bench, I get:

qqq20:04:27

(drum roll)

qqq20:04:44

recur: 3.22 ms volatile: 2.7 ms locals: 13ms reduce: 1 ms

qqq20:04:49

so reduce is still fastest

noisesmith20:04:01

the doall doesn't change the type

noisesmith20:04:10

and reduce's optimization is a property of the type returned by range

noisesmith20:04:35

maybe (list* (range 100000))

noisesmith20:04:17

no, list* doesn't do it... (into [] (range ...))

bronsa20:04:47

(take n (iterate inc 0))

qqq20:04:53

alright, retrying with (def my-list (into [] (range cnt))) / even though this is now a vector

noisesmith20:04:53

oh yeah, that would do it

bronsa20:04:58

vectors still have optimizations for reduce

bronsa20:04:10

they get reduced in a chunked way

qqq20:04:51

if reduce has all these optimizations, maybe I should just use reduce 🙂

qqq20:04:47

(take n (iterate inc 0)) slows everything down, probably due to it beinga list

bronsa20:04:34

if you want to iterate over collections, reduce is usually the fastest way

bronsa20:04:06

it understands reducibles, chunked seqs and used the fastest path available

bronsa20:04:29

doseq only special cases chunked seqs, loop/recur is up to you

john20:04:44

and range always returns a specific number type, allowing for further optimization, right?

bronsa20:04:02

reducing a range doesn't even materialize a collection at all

bronsa20:04:46

reducing a range is literally a for loop from start to end (at least, in the bounded, integer step case)

qqq20:04:57

alright, that's enough for today; thanks for insights on criterium/bench , volatile!, and reduce

qqq20:04:22

Your kit is expensive at $500.00 https://www.yourkit.com/purchase/

qqq20:04:25

Is there any decent alternative ?

noisesmith20:04:38

visualvm works OK and it's free

bronsa20:04:15

you can get free yourkit licenses for open source projects

hlolli21:04:16

Any function that can change hash-map to list. {:a 1 :b 2} => '(:a 1 : b 2) ? With intoIm getting vector pairs, understandibly, but then I feel like a newb trying to use flattento solve that.

hlolli21:04:04

solved it (reverse (reduce #(into %1 %2) '() {:a 1 :b 2})) just had to ask to get the answer from the rubber duck.

bronsa21:04:25

(mapcat identity {:a 1 :b 2})

hlolli21:04:54

niiiice @bronsa thanks!

noisesmith21:04:06

other options include (apply concat ...) and (into [] cat ...)

hlolli21:04:57

this is why I love asking on this channel, one can always find more beautiful solutins in Clojure.

qqq22:04:03

Let m1, m2 be maps. Does (into m1 m2) use transients? Looking for more than a yes/no answer. Can someone point me at the line of code where this is resolved?

noisesmith22:04:35

@qqq yes, read the output of (source into) in your repl

qqq22:04:56

ah, I see the transient and persistent!

noisesmith22:04:29

hopefully some day merge will use transients, but for now into is the better option

ghadi22:04:57

been looking at that ticket @noisesmith for merge + transients

ghadi22:04:48

i want to take a new approach with its development. 1) write the benchmarks 2) need a really good fspec for the current impl of merge

ghadi22:04:01

3) Fight to the death on implementations

noisesmith22:04:13

nice - two forms enter, one form leaves

ghadi22:04:36

the fspec will help ensure that an implementation doesn't break the contract -- which currently has a lot of "extras"

ghadi22:04:58

For example, you can merge maps, vectors, and mapentries

ghadi22:04:11

because it calls conj underneath, and conj for map can do that

noisesmith22:04:32

^ above is my "favorite" merge behavior

ghadi22:04:36

wait, wha?

ghadi22:04:48

oh... i know why

ghadi22:04:59

there's an (or m {}) in the impl

noisesmith22:04:51

it also acts like identity for single arguments other than nil or false (but many functions do that)

qqq23:04:32

imho (merge false {:a 0}) isn't too bad

qqq23:04:40

sometimes, I wnat to grab something (get ...) then merge on the answer

qqq23:04:44

and the get might return nil

qqq23:04:06

so then having it be (or m {}) seems reasonable

noisesmith23:04:53

@ghadi another case that will be a pain in the ass when reimplementing is (merge [1 2] 3 4 5) - totally incorrect, but accidentally works

noisesmith23:04:24

or worse yet (def my-reverse (partial merge ()))

bronsa23:04:07

is making those work actually a goal?

bronsa23:04:23

to me they all look like garbage input that happen to work

noisesmith23:04:23

yeah- that's fair - I know I've seen code at work that misused merge in similar ways

noisesmith23:04:29

but totally legit to break it

ghadi23:04:58

Considering Rich's presentations on compatibility, is it? https://clojurians.slack.com/archives/C03S1KBA2/p1491780329036554

bronsa23:04:23

well, i wouldn't think anybody could get angry if (merge [1 2] 3 4) stopped working tbh

bronsa23:04:49

merge's docstring explicitely says that it only works on maps so everything else is GIGO

bronsa23:04:14

same as if (deliver inc 1) stopped working

noisesmith23:04:23

people come asking for help with code where they are incidentally calling merge on vectors with surprising regularity - I have no idea why it's so common

qqq23:04:39

based on the english definition of merge, (merge [1 2] 3 4) seems reasonable

bronsa23:04:43

I'd assume they'd invoke it like (merge [1 2] [3 4]) rather than (merge [1 2] 3 4) tho

ghadi01:04:17

@noisesmith what has appeared in the wild was (merge {} []) where the entry is not in the first position...

bronsa09:04:20

surely that's an error and people should be using conj instead

bronsa23:04:32

the former is understandable but doesn't work anyway, the latter just doesn't make any sense and I can't imagine anybody using it

qqq23:04:48

you know what would solve this?; erlang's dialyzer type checker ported to clojure

noisesmith23:04:58

but (merge [1 2] [3 4]) returns [1 2 [3 4]] so they realize (merge [1 2] 3 4) works - iteration in practice

qqq23:04:02

sucess-typing -- doesn't have problem with dynamic types; and any type errors are guaranteed to be type errors

bronsa23:04:05

or reading merge's docstring

qqq23:04:50

i actually don't read doc string al that often

qqq23:04:20

for me, it's (1) look at example on clojuredocs, (2) try random stuff, (3) when it breaks, ask #clojurians/slack, (4) get told to read docstring, and (5) finally read docstring

bronsa23:04:27

if you don't read the docstring then you can't complain if you're using a function wrong and your usage eventually breaks

bronsa23:04:09

@noisesmith right, a spec for merge based on the contract from the docstring would accept map args (and I guess nils too) but every other input is just garbage according to the docstring

noisesmith23:04:54

the other day I was working on some code and found the following: (defn queue [] (clojure.lang.PersistentQueue/EMPTY))

noisesmith23:04:10

there's two layers in wat there for me - it makes me sad that the parens work

bronsa23:04:22

yeah that's bothered me for a long time

noisesmith23:04:23

and it makes me sad that anyone thought the function indirection was needed

noisesmith23:04:13

I switched it to (def queue clojure.lang.PersistentQueue/EMPTY) and deleted some parens of course

clojureman05:04:42

Considering the lousy documentation of clojure.lang.PersistentQueue, I am not surprised. I think it is kind of ok, because at least it documents the arity, which def doesn’t (and most clojure devs I’ve met do not know PersistentQueue) - but of course the opportunity to add a docstring was missed, which is not ok 🙂

bronsa23:04:47

user=> (macroexpand '(Integer/MAX_VALUE))
(. Integer MAX_VALUE)
here's why that works btw

bronsa23:04:08

i hate how ambiguous . is

noisesmith23:04:15

if we were strict about .- ...

bronsa23:04:24

not even that

bronsa23:04:50

if method calls were only (. Klass (method)) rather than also (. Klass method) there would be no ambiguity between field access & no arg method calls

bronsa23:04:46

ATM both (. Klass meth arg) and (. Klass (meth arg)) work

bronsa23:04:44

it's never going to change ofc because of backwards compatibility but it's one of the things that annoy me most about .

noisesmith23:04:04

tangentially, the queue example reminds me of one of my favorite bugs I've found - someone creates a PesistentQueue and then proceeds to use first and rest on it instead of peek / pop, thus turning their fifo into a lifo

noisesmith23:04:15

maybe if they'd read the doc string for rest, haha

john23:04:17

I got used to the field access / method call semantics for . in clj and then cljs' requirement for .- annoyed the heck out of me. But I guess it lessens ambiguity? Y'all just don't think the . should be able to get functions and/or values?