Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2020-08-04

Channels

#clojure
<
2020-08-04
>
firstclassfunc01:08:53

Hello all, Does anyone have experience with Criterium benchmark-round-robin* I am trying to construct the exprs map with generated macro but can't get the data structure without calling the expr itself.

noisesmith01:08:56

also it looks like benchmark-round-robin* is an implementation detail of benchmark-round-robin (this naming is common, adding a * for an internal detail which is exposed) - that takes a series of forms and constructs the hash-maps that run-benchmarks-round-robin uses

JoshLemer17:08:30

I wonder if the folks who steward the Persistent Vector in the standard library (looks like mostly @alexmiller, Stuart Halloway, Rich Hickey) have taken notice of the rewrite that happened to Scala’s Vectors (which were originally pretty much copied from clojure) that was merged in March, and results in many operations being order(s)-of-magnitude faster? Definitely at least worth a look to see if any of the improvements can be ported https://github.com/scala/scala/pull/8534

Alex Miller (Clojure team)17:08:35

were those changes related to the CHAMP stuff?

Alex Miller (Clojure team)17:08:45

I had not seen this, so thanks for the link, will look when I get a chance

JoshLemer17:08:23

Not related to the CHAMP stuff, nope

Alex Miller (Clojure team)17:08:51

at a glance, a few of the array opts seem like they are things already being done but need to read it more closely

ghadi18:08:06

Benchmarks assume monomorphism, but impl introduces megamorphism

Ben Sless18:08:02

can give a brief explanation of mono/megamorphism? I've seen it brought up before in similar contexts but never came across an explanation and how it affects jvm optimizations

ghadi18:08:03

if the JVM can see the same class at a particular callsite, it assumes that the class will show up and avoids doing method dispatch

ghadi18:08:13

and it can inline method implementations

ghadi18:08:16

that's monomorphism

ghadi18:08:24

'mono' = 1

ghadi18:08:53

bimorphism == one of 2 classes seen, JVM can add a conditional

ghadi18:08:12

megamorphic == many classes seen at a call site, have to do pessimistic dispatch

ghadi18:08:23

no inlining, worst case

ghadi18:08:02

there is only one commonly used persistent vector class in clojure, and 2 commonly used maps (PersistentArrayMap, and PersistentHashMap)

Ben Sless18:08:06

class at call site -> the class whose method is invoked? Also, how does the jvm dispatch? hash table?

ghadi18:08:30

a virtual invoke of some sort

ghadi18:08:58

if IBar is an interface, and class Foo implements IBar, I'm talking about x.bar() where some x is a Foo

ghadi18:08:28

if the JVM knows there's only one implementation of IBar in the whole class hierarchy, it can just ignore the dispatch and turn it into a static method essentially

Ben Sless18:08:08

I see, thanks for the explanation 🙂

ghadi18:08:17

we've tried specializing maps for small sizes

ghadi18:08:25

Map0 -> Map6

ghadi18:08:31

it makes all the call sites go megamorphic

ghadi18:08:55

but microbenchmarks don't reveal this, only running real world programs

Ben Sless18:08:14

And the micro benchmarks showed good results but didn't work "in the wild"

ghadi18:08:30

maybe it was vectors that we specialized, but yeah it "looks good on paper"

ghadi18:08:53

my explanation is better 🙂

Ben Sless18:08:52

I think so, too

ghadi18:08:02

I am highly skeptical, at first glance

Alex Miller (Clojure team)18:08:48

I have seen that movie before :)

3
ghadi18:08:50

I wish Scala the best

miikka18:08:51

What about CHAMP? Anyone looked into that?

ghadi18:08:21

(IMHO) one thing worth exploring is the map merge operation. Scala's maps exploit the structure of the HAMT while merging

ghadi18:08:00

(from the same series of papers that contain CHAMP)

JoshLemer18:08:16

by merging do you mean map “concatenation” / ++ ?

ghadi18:08:50

I mean clojure.core/merge

ghadi18:08:55

dunno what Scala calls that

JoshLemer18:08:26

ah, yeah, that’s concat / ++

miikka18:08:42

Hmm, yeah, that'd be interesting, would be great if merge was faster

Alex Miller (Clojure team)18:08:03

btw, this whole convo is prob best in #clojure-dev in the future

markaddleman21:08:20

I'm writing a regression test suite that compares entity map result sets against expected entity maps. The entity maps can contain doubles or floats and I only care if the results of those numbers match to a particular precision. I vaguely remember something in the clojure.test library that provides control over the precision of equality. Am I making that up?

wcohen17:08:42

https://github.com/microsoft/same-ish works if you want it to be actually quite close in terms of precision, more recently i just have been using https://github.com/clojure-expectations/expectations/blob/f12f710d49cfd1db5af6cba5fcbc8f798cadaa2c/src/cljc/expectations.cljc#L732. Since expectations is only clj, you may have to copy the function itself if you need it in cljs too.

Alex Miller (Clojure team)21:08:00

I think you're making that up

😂 6
Alex Miller (Clojure team)21:08:13

or if not, I have learned something new

markaddleman21:08:41

Alright. Thanks 🙂

Lennart Buit21:08:28

There may exist a matcher for that in matcher-combinators.

Lennart Buit21:08:48

I am personally pretty fond of that lib, but the usual ‘don’t add libraries randomly’ disclaimers apply

noisesmith22:08:14

@markaddleman Math/nextAfter might help - finds the next double closest in the direction of a second arg 

org.noisesmith.hammurabi-test=> (Math/nextAfter 1.0 -1.0)
0.9999999999999999
org.noisesmith.hammurabi-test=> (Math/nextAfter 1.0 2.0)
1.0000000000000002
of course, you might just want to make your own testing function based on - if the precision of doubles themselves isn't your limitation

markaddleman22:08:46

Cool. I didn't know that function existed

dpsutton22:08:59

I wouldn’t expect hammurabi to be so lenient

noisesmith22:08:43

@dpsutton haha - the project name comes from the fact that his code was famously brutal to offenders (the project domain is contiguous / off heap low level data, with no reified java objects)

noisesmith22:08:16

which means you can literally have pointer math errors that make you read from the wrong "thing", since the data is just a big pile of bytes

dpsutton22:08:05

Sounds interesting. Not sure if you ever write or blog but I would definitely read it

noisesmith22:08:20

also, fun fact, doing some background reading I found out that the brutal punishments only applied to "freedmen" AKA not nobles and not slaves nobles just paid fines, slaves were executed, it was the freedmen who go the eye-for-an-eye treatment

noisesmith22:08:08

@dpsutton I would blog if I had more self discipline, I really think I should haha

noisesmith22:08:22

but this project in particular would be a great blog topic

dpsutton22:08:33

Yeah I never do and it was fun to throw together one about cache recently

dpsutton22:08:56

Yeah I don’t see much off heap bit stuff in clojure so I’m already interested

dpsutton22:08:07

And ensuring speed in a boxed object world must be tough

noisesmith22:08:26

well, with off-heap there's no such thing as boxes, so that's a great start

noisesmith22:08:37

there's also no such thing as methods...

noisesmith22:08:48

long term plan is to also combine this with ASM.java, and this is all being done in parallel with learning arm64 assembly

noisesmith22:08:20

domain task is doing as much DSP calculation within a given timespan as you can, with a fun language

dpsutton22:08:09

could be a good candidate for assert-expr

noisesmith22:08:58

as someone who came to a project using the assert-expr multimethod, I'd say that the annoyance factor of a reader knowing what the hell they are reading outweighs the convenience factor of extending the syntax of is

noisesmith22:08:28

I see code like (is (foo x y)), so I'm like "OK I guess I'll go look up foo" - no such thing

dpsutton22:08:47

fair. i was just looking at implementing this and i noped out

dpsutton22:08:57

need to make sure you do-report correctly, etc

noisesmith22:08:28

it does very little that foo wouldn't do as a function, and has the detriment that you need to remember a special case that doesn't follow any of clojure's idiomatic scoping rules

noisesmith22:08:14

the amazing thing is that other people somehow survive using languages where this is commonplace

noisesmith22:08:50

the sudden absence of sane scoping / clarity about where definitions come from really makes me realize how lucky we have it the rest of the time :D

dpsutton22:08:31

it is nice that it can craft a message though. but i agree otherwise

noisesmith22:08:58

you can pass a formatted string to is, including runtime data like actual values

dpsutton22:08:11

> You can extend the behavior of the "is" macro by defining new methods for the "assert-expr" multimethod. These methods are called during expansion of the "is" macro, so they should return quoted forms to be evaluated.

dpsutton22:08:39

(is (epsilonish 3.00001 3))

Ludger Solbach22:08:40

(defn about-equal "Tests if the actual and expected values are equal in the given error margin." ([actual expected] (about-equal actual expected 0.0001)) ([actual expected error-margin] (<= (abs (- actual expected)) error-margin)))

Ludger Solbach22:08:51

=> (conj [1 2 3] (conj [1 2 3] [[1 2 3]])) [1 2 3 [1 2 3 [[1 2 3]]]] => (cons [1 2 3] (cons [1 2 3] [[1 2 3]])) ([1 2 3] [1 2 3] [1 2 3])

noisesmith22:08:57

I'd suggest (- actual (Math/nextAfter actual expected)) is a better default than 0.0001

Ludger Solbach22:08:23

why is the behaviour of conj different than that of cons regarding the collection nesting?

noisesmith22:08:36

because the collection and item args are reversed

noisesmith22:08:44

cons takes item first, conj takes coll first

dpsutton22:08:55

Depends on how much computation is involved. If many steps each computation is a chance for loss of precision

Ludger Solbach22:08:37

and depends on the precision of the values to compare against.

noisesmith22:08:58

right, which is my motivation in using nextAfter to calculate a default

noisesmith22:08:33

not the precision as much as the magnitude

Ludger Solbach22:08:15

I use my function to compare calculated values against text book values given with just 3-4 digits after the point.

Ludger Solbach22:08:41

the calculations give more fractional digits of course. But if all you have is a few fractional digits to compare against, you have to work in that precision (or magnitude).

noisesmith22:08:43

ahh, that makes sense

isak23:08:24

Proposal: Implement IFn on clojure.lang.Atom, and it would just reset! them. I.e., 

(def foo (atom nil))
(foo :bar)
@foo ; => :bar
Am I wrong?

noisesmith23:08:38

refs and vars (also mutable containers) implement IFn, and delegate to the function they contain

noisesmith23:08:22

=> ((ref +) 1 2 3)
6

isak23:08:15

Ah, I guess it would be confusing then.

phronmophobic23:08:22

additionally, you really want mutation to stand out. making mutation look like any other function call would obscure the mutation

Ludger Solbach23:08:40

and swap! is preferable to reset!

Ludger Solbach23:08:57

because swap! is a function of the old value.

noisesmith23:08:02

I mean, if what you want is to unconditionally set a value, and you don't care about the existing value, reset! is for exactly that, and better than a swap! / constantly combo

noisesmith23:08:37

but yeah, if your primary action is on atoms is reset!, maybe they aren't the construct you actually want? a var would do fine for that

3