Fork me on GitHub
#clojure
<
2017-06-28
>
seancorfield00:06:08

@masztal’s question was posted at 4:08am Pacific: “Hi, I’m optimizing my clojure app. Here are results from jvisualmvm. Could you tell me, when clojure.lang.util.hash is invoking? Maybe it should be cached?” /cc @alexmiller

Alex Miller (Clojure team)00:06:49

Hash is called when using a hashed data structure (either hash map or hash set). In some cases it is cached, depends on what is being hashed.

Alex Miller (Clojure team)00:06:18

Both keys and vals are hashed, so consider what types end up in keys and vals of hash maps

seancorfield00:06:59

(thanks Alex — hopefully they’ll see that tomorrow when they come back online!)

Andy05:06:38

Hi, How to understand * notation from https://clojure.org/reference/java_interop, specifically (. Classname-symbol (method-symbol args))* or (. Classname-symbol method-symbol args)* vs:

user=> (. clojure.lang.RT contains #{1} 1)
true
user=> (. clojure.lang.RT (contains #{1} 1))
true

Andy05:06:33

I am not sure about placement of * , it would seam that (. Classname-symbol (method-symbol args*)) or (. Classname-symbol method-symbol args*) is more natural ...

seancorfield06:06:58

@happy.lisper Looks like typos -- The section above has the * in the correct place.

Andy14:06:59

seancorfield: ty for confirmation. Should I send a pull request?

joshjones14:06:52

I think this would qualify as a "small change" so see the relevant section here: https://clojure.org/community/contributing_site @happy.lisper

Andy15:06:51

TY. The edited source reads *(_._ instance-expr (method-symbol args*))* - so the * is there ... just not shown or we need to quote it 🙂 ** will help

Andy13:06:00

@joshjones @U04V70XH6 due to non technical reasons, my PR https://github.com/clojure/clojure-site/pull/194 did not make it. If anybody could adopt it, it would be great morning

matan07:06:33

timeline rumors about 1.9 anyone? 😅 can't wait 🙂

souenzzo12:06:32

Feel like a Debian release

qqq14:06:05

given the keys are 0-n, is transient-vector significantly fastedr than transient-hash-map ?

joshjones14:06:03

@qqq significantly faster at what, specifically?

noisesmith14:06:32

one would assume conj!, since that's what transients are for

noisesmith14:06:47

seems like an easy thing to test with criterium

noisesmith14:06:35

+user=> (crit/bench (let [v (transient [])] (reduce conj! v (range 1000))))
Evaluation count : 3435960 in 60 samples of 57266 calls.
             Execution time mean : 17.803731 µs
    Execution time std-deviation : 319.760540 ns
   Execution time lower quantile : 17.305732 µs ( 2.5%)
   Execution time upper quantile : 18.382921 µs (97.5%)
                   Overhead used : 1.572942 ns

Found 1 outliers in 60 samples (1.6667 %)
        low-severe       1 (1.6667 %)
 Variance from outliers : 7.7727 % Variance is slightly inflated by outliers
nil
+user=> (crit/bench (let [m (transient {})] (reduce #(conj! % [%2 %2]) m (range 1000))))
Evaluation count : 292260 in 60 samples of 4871 calls.
             Execution time mean : 204.650986 µs
    Execution time std-deviation : 11.916923 µs
   Execution time lower quantile : 197.407687 µs ( 2.5%)
   Execution time upper quantile : 223.127690 µs (97.5%)
                   Overhead used : 1.572942 ns

Found 3 outliers in 60 samples (5.0000 %)
        low-severe       1 (1.6667 %)
        low-mild         2 (3.3333 %)
 Variance from outliers : 43.4448 % Variance is moderately inflated by outliers
nil

noisesmith14:06:08

so the map with number keys version is ~ 10x slower

mpenet14:06:43

for a size 1000 <thing>

noisesmith14:06:09

perhaps a smaller input would better represent real use cases

mpenet14:06:17

my guess is that for vectors it will be O(1) no matter the size

mpenet14:06:02

well it's a bit of a generalisation, but you get what I mean

joshjones14:06:12

"real use cases" depends on the use case. that's why when someone asks "which is faster, x or y?" it's important to understand the use case

joshjones15:06:27

also @noisesmith , you are creating a persistent vector in your conj! for maps, and it's faster for your test scenario to use assoc! instead

(c/bench (let [m (transient {})] (reduce #(conj! % [%2 %2]) m (range 1000))))
Evaluation count : 385800 in 60 samples of 6430 calls.
             Execution time mean : 157.793112 µs
    Execution time std-deviation : 3.521488 µs
   Execution time lower quantile : 154.508093 µs ( 2.5%)
   Execution time upper quantile : 165.003217 µs (97.5%)
                   Overhead used : 1.312303 ns

Found 3 outliers in 60 samples (5.0000 %)
	low-severe	 2 (3.3333 %)
	low-mild	 1 (1.6667 %)
 Variance from outliers : 10.9550 % Variance is moderately inflated by outliers
=> nil
(c/bench (let [m (transient {})] (reduce #(assoc! % %2 %2) m (range 1000))))
Evaluation count : 515160 in 60 samples of 8586 calls.
             Execution time mean : 118.954362 µs
    Execution time std-deviation : 4.101911 µs
   Execution time lower quantile : 115.324719 µs ( 2.5%)
   Execution time upper quantile : 130.422397 µs (97.5%)
                   Overhead used : 1.312303 ns

Found 4 outliers in 60 samples (6.6667 %)
	low-severe	 4 (6.6667 %)
 Variance from outliers : 20.6386 % Variance is moderately inflated by outliers
=> nil

noisesmith15:06:21

I'm redoing the test creating the input outside the benchmark for that reason

noisesmith15:06:45

every result, regardless of pre-making vectors, or size of input, is showing vectors significantly faster than number keyed maps though

noisesmith15:06:10

and the ratios are consistent across input sizes, number keyed hash-maps take 8x as long, for both 20 element and 1000 element inputs

noisesmith15:06:33

(8x as long using assoc! of coure, 10x using conj!)

roberto16:06:22

I have a test assertion like this:

(not (nil? pin-config))
Where pin-config is a non-empty map. But the test is failing with:
expected: (not (nil? pin-config))
 actual: (not (not true))

roberto16:06:19

It looks like (not (nil? pin-config)) is transformed to (not (not true) but then this is not evaluated to (not false) and then to true

roberto16:06:31

Any suggestions on what I might be doing wrong?

jstokes16:06:59

roberto: unrelated, but (not (nil? x)) == (some? x)

roberto16:06:49

ah, thanks, I switched to (complement nil?)

roberto16:06:53

switching to some? 🙂

a1316:06:26

Is thery any kind of "memoization" library that supports an external in-memory storage backend?

jcromartie16:06:06

what does "external in-memory storage" mean? like Redis?

a1316:06:45

Yes, Redis is fine.

a1316:06:27

Thanks! Exactly what I look for!

Pablo Fernandez16:06:25

Anybody knows how to make figwheel load all builds or at least a specific one, when calling (start-figwheel) ?

jcromartie16:06:10

sort of a general design question, but I'm trying to figure out how fine-grained certain events in my system should be

jcromartie16:06:20

we have records that move through various states

jcromartie16:06:34

and I'm trying to decide if I should have events for each possible transition, which would be a lot

jcromartie16:06:53

or just one event that records the new status as a parameter

jcromartie16:06:37

so (defrecord ProjectApprovedEvent [project-id]) vs (defrecord ProjectStatusChangedEvent [project-id new-status])

jcromartie17:06:08

or maybe the difference between (defn approve! [project-id] ...) (defn reject! ...) vs. (defn update-status [project-id new-status] ...)

ibarrick17:06:43

I have a Ring application that works perfectly fine running on jetty when compiled with lein ring uberwar and runs locally with lein ring server-headless but when I compile with lein ring uberjar and use nginx-clojure 0.4.5 locally, it complains that it can't find medley in the classpath.

ibarrick17:06:40

Googling the medley classpath error led to a comment that said that it was likely the result of using an old clojure version but I'm running 1.8 (nginx-clojure runs on 1.5.1 but I don't think that should matter)

ibarrick17:06:45

Can anyone help?

bja17:06:13

@ibarrick have you tried lein clean (this can clean up stale classes and resolve weird problems from time to time)

bja17:06:30

lein clean && lein ring uberjar

ibarrick17:06:11

I have and I get the same error

ibarrick17:06:12

@bja I can use the same jar in EC2 with same version of nginx-clojure and it will run 😕

ibarrick17:06:40

There must be something different between my environment locally and the remote one but I can't figure out what it would be

bja17:06:58

I'm not familiar with nginx-clojure, does it setup a classpath for you (and possibly include a different version of clojure)? In the past, I've witnessed systems like that (Storm and Hadoop come to mind) that cause weird dependency issues

bja17:06:31

oh, yeah it does

bja17:06:58

can you try to add a :provided profile to your project.clj and put your clojure dependency on that

ibarrick17:06:20

I'll try that now

bja17:06:34

the example for nginx-clojure seems to indicate that v1.5.1+ is okay (and uses 1.7.0 specifically)

ibarrick17:06:29

@bja I got the same issue after adding my clojure dependency to :provided 😕

swizzard17:06:25

is the #quil channel dead?

seancorfield18:06:22

@swizzard I’d say that it’s just quiet

lxsameer18:06:37

how can I get the list of protocols which a type implements ?

joshjones19:06:02

@lxsameer ancestors will include protocols implemented, along with superclasses

(defprotocol Foo
  (bar [this]))
=> Foo
(deftype Baz [a b c]
  Foo
  (bar [this] nil))
=> user.Baz
(ancestors (class (->Baz 3 4 5)))
=> #{user.Foo clojure.lang.IType java.lang.Object}

lxsameer19:06:47

@joshjones nice, is there any way to get the methods for each interface or protocol ?

caio19:06:08

is there a version of run! to use with transducers? I mean, some fn run2! that you use like (transduce xf run2! init coll)

joshjones19:06:36

(into [] (.getDeclaredMethods SomeClassHere))
@lxsameer

joshjones19:06:30

you can .getName on those for just the name, you can also call them since you have reflected and have the Method objects

rauh19:06:03

@caio (run! #(prn %) (eduction identity [0 1 2]))

noisesmith19:06:45

if you look at the source of run!, I think (transduce xf #(f %2) nil coll) is the most parsimonious thing

user=> (source run!)
(defn run!
  "Runs the supplied procedure (via reduce), for purposes of side
  effects, on successive items in the collection. Returns nil"
  {:added "1.7"}
  [proc coll]
  (reduce #(proc %2) nil coll)
  nil)

caio19:06:18

yeah... I thought so too @noisesmith . and that's what I'm doing @rauh . I just wanted to know if there was some other fn to use directly, but thanks

sgerguri20:06:01

I’ve been doing a bit of reading about functors recently and it just occurred to me that transducers might be exactly that. Only touched the tip of the iceberg so are they really the same thing or is there a counterexample of a transducer that cannot be classified as a functor?

noisesmith20:06:45

(map f) can't be mapped over, so it isn't a functor

noisesmith20:06:58

in fact I can't think of any transducer that you can map over...

ghadi20:06:35

transducers exist outside of collections

sgerguri20:06:21

functors are really type classes so they’re not tied to concrete types from the get go either

noisesmith20:06:09

right, but functors are things you can map over, and though you can map over a function, I don't know how I would apply that to a transducing function

noisesmith20:06:14

perhaps that's my ignorance though?

sgerguri20:06:29

(Or at least in their Haskell implementation they are. After years of Erlang and Clojure I feel a little bit like Alice in Wonderland reading about all that stuff.)

noisesmith20:06:18

doing some reading to refresh my knowledge of this stuff, supposedly mapping over a function is composing it, and transducers are functions that when called on another function return a function that will compose their transducing actions when applied in a transducing context

noisesmith20:06:34

but I don't know how to translate "transducing context" here

sgerguri20:06:43

^ yeah, that’s the angle I was coming from

noisesmith20:06:00

because looking at typical haskell examples (+) 3 doesn't get applied to (*) 2 to return a new function, it gets composed to make something you can apply

noisesmith20:06:05

which isn't what transducers do...

noisesmith20:06:34

in fact they are inside out of that right? you apply them to get something that composes the two

noisesmith20:06:02

I feel like I'm over my head with all of this though honestly

sgerguri20:06:19

thinking this through

dpsutton20:06:29

apparently functors must preserve the identity mapping. ie identity in Category C maps to identity of Category D. If a transducer returns a non-empty list when given an empty list then it could not be a functor. (the categories being the collection of lists of the underlying set). This is my thinking.

noisesmith20:06:58

so does this mean transducing mapcat, filter, remove, etc. are not functors but transducing map is?

dpsutton20:06:47

asking the wrong person. got undergrad and graduate background in math but no idea about this. but start see if it makes sense by definition. ie, are the domain and codomain categories to begin with? If not the term has no meaning. To be categories we need the "arrows" on them and make sure they compose. blah blah. All I was looking for was a single counter example where a transducer would not behave according to the definition of a functor.

hcarvalhoaves20:06:37

there's also state (https://github.com/matthiasn/talk-transcripts/blob/master/Hickey_Rich/Transducers/00.36.36.jpg) which suggests to me transducers are it's own thing (hence why a new name)

hcarvalhoaves20:06:01

((or maybe multiple things under the same umbrela name))

noisesmith20:06:02

oh, right - functors can't be stateful like that

lwhorton21:06:42

with defmethod is there an idiomatic way to handle “dispatch on anything in this set” instead of a singular dispatch value?

lwhorton21:06:05

(defmulti foo :my-type)
(demethod foo #{:a :b :c} [_] …)
instead of having to do
(demethod foo :a [_] …)
(demethod foo :b [_] …)
etc

noisesmith21:06:39

you could check for that set in your dispatch function and have that return ::foo

noisesmith21:06:59

remember your dispatch function doesn't have to just return the thing

lwhorton21:06:15

ah i see, then wire in a defmethod like :a-set and :b-set

noisesmith21:06:56

(defmulti foo #(if (contains #{...} (:my-type %)) ::foo (:my-type %)))

noisesmith21:06:05

then (defmethod foo ::foo ...)

noisesmith21:06:32

I think that is the simplest choice (with a better dispatch function of course, that's just the basic concept)

lwhorton21:06:42

great, that’s much nicer than what I was planning to do

lwhorton21:06:53

any particular reason you use ::foo .. are defmulti’s global or something?

noisesmith21:06:26

because who knows what value you would find, and namespaced things are safer? you can use any placeholder you like of course

noisesmith21:06:21

@lwhorton part of my instinct to use ::foo there is to indicate "this keyword is something my namespace cares about and maybe not generally relevant in other contexts"

lwhorton21:06:42

i should probably be doing more of that. thanks again for the help

wei23:06:38

I use emacs and my cofounder uses Cursive. Is there a way to reconcile the formatting styles between the two so our code is consistent?

noisesmith23:06:22

my team uses the cljfmt plugin

hcarvalhoaves23:06:44

best thing would be a linter as a post-commit hook

danielcompton23:06:07

@wei Cursive is fairly customisable for formatting settings. You can probably match most things that Emacs does

cfleming23:06:56

@wei If you have specific issues, let me know over in #cursive and I’ll help you.

noisesmith23:06:26

with cljfmt I can even account for people that never remember to auto-format

noisesmith23:06:44

before we started using it, we had repeated problems where person x would indent code totally wrong, and person y had parinfer turned on and moved all the parens around based on the indentation - there were two human failures that kept happening but just making sure cljfmt got applied to the repo was easier than fixing the bad code practices

wei23:06:40

I think it’s actually Emacs whose behavior seems to have more special cases. wondering if there’s a premade setting for either Cursive or Emacs? (surely I’m not the first one to try to reconcile the two)

dpsutton23:06:05

from what i understand emacs will be more rigid and stubborn than cursive

noisesmith23:06:07

that's why cljfmt is so great, it doesn't care what your editor is

dpsutton23:06:30

and by rigid i mean less adaptable

cfleming23:06:31

@noisesmith You still really need your editor to agree with it though, or you will suffer pain.

wei23:06:39

^ @cfleming agree. and x2 with your partner’s code. the worst is when Emacs indentation breaks parinfer on Cursive- that actually changes the program’s behavior

dpsutton23:06:54

i'll bet your easier path is making cursive behave like emacs

noisesmith23:06:10

@cfleming just running it before I check my code in catches little stuff, and my editor is smart enough to catch changed files

noisesmith23:06:57

always running it before commit, and always running it after a pull or merge is a pretty smooth workflow, even when some people forget to run their editor's indentor at all

noisesmith23:06:19

to be clear I'm not just running cljfmt, I'm using the optional fix arg that edits all the files in place

wei23:06:24

@noisesmith cljfmt seems like a good first step, thanks for the suggestion. @cfleming gonna give cursive formatting settings a shot

cfleming23:06:50

@wei I think the main one that’s required to make Cursive line up with Emacs’ initial settings is Settings-&gt;Editor-&gt;Code Style-&gt;Clojure-&gt;General-&gt;One space list indent