Fork me on GitHub
#clojure-dev
<
2023-01-20
>
JoshLemer18:01:54

For contrib libraries, where does an unsolicited (no pre-existing jira ticket asking for it) performance improvement patch go? First create the http://ask.clojure.org -> contrib owner creates the jira -> attach the patch? Or would non-owner contributors open a Jira directly and attach the patch?

hiredman18:01:19

you would create an http://ask.clojure.org, then someone will decide if it is something that actually needs addressing

hiredman18:01:48

then a jira issue would be created, and patches if and when they exist would be attached there

👍 2
hiredman18:01:38

you need to have signed a contributor agreement before you can submit patches (or get a jira account)

JoshLemer18:01:53

I have both signed the CLA and a jira account now

hiredman18:01:19

depending it might be a very up hill climb

hiredman18:01:56

my experience is rich really doesn't like it when issues coming with a patch attached

Noah Bogart18:01:40

is rich still hands-on with the contrib libraries?

hiredman19:01:41

it depends, I think he has delegated to an owner for most of them

👍 2
seancorfield19:01:57

@UEH6VEQQJ Very much depends on the Contrib lib maintainer -- which lib(s) are you thinking about?

JoshLemer19:01:11

data.int-map

seancorfield19:01:57

@U064X3EF3 has mostly been taking care of that recently so he can probably provide more specific guidance on how he'd like work on that approached...?

Alex Miller (Clojure team)19:01:40

hey @UEH6VEQQJ for contrib, if you're working tightly with a project, it's ok in this case for you to create the jira directly

👍 2
Alex Miller (Clojure team)19:01:39

just as an fyi, Fogus and I try to cycle through and look at various contrib projects on Fridays so if you submit some stuff and don't see activity for a bit, that's normal. but feel free to ping me here if you're looking for feedback on something or if it's been a while etc

JoshLemer20:01:03

Thanks Alex!

borkdude21:01:09

I think get-in could short circuit (for performance) if the intermediate lookup thingie is nil - has this ever been considered?

Ben Sless21:01:09

Let's start with using reduce over reduce1:upside_down_face:

😆 4
Alex Miller (Clojure team)21:01:41

don't remember it being considered

borkdude22:01:24

;; original get-in: 357ms
(time (dotimes [_ 10000000]
        (get-in {:a {}} [:a :b :c :d :e])))

;; get-in + reduce1 -> reduce: 265ms
(time (dotimes [_ 10000000]
        (get-in-old {:a {}} [:a :b :c :d :e])))

;; get-in + reduce + short circuit on nil: 134ms
(time (dotimes [_ 10000000]
        (get-in-new {:a {}} [:a :b :c :d :e])))

😍 2
borkdude22:01:22

Let me know if there's interest in an issue.

borkdude22:01:53

;; 94ms
(time (dotimes [_ 10000000]
        (some-> {:a {}} :a :b :c :d :e)))

Ben Sless22:01:35

For comparison, can you check it against non-empty maps? Also, what about the not-found case?

borkdude22:01:06

non-empty map with get-in + reduce:

;; 300 ms
(time (dotimes [_ 10000000]
        (get-in-old {:a {:b {:c {:d {:e 1}}}}} [:a :b :c :d :e])))

borkdude22:01:27

non-empty map with get-in + reduce + short circuit:

;; 224ms
(time (dotimes [_ 10000000]
        (get-in-new {:a {:b {:c {:d {:e 1}}}}} [:a :b :c :d :e])))

borkdude22:01:27

I can't explain why the short-circuit version seems to be faster with a non-empty map, but I get it consistently

Ben Sless22:01:49

are you comparing it to the version which uses reduce1 ?

borkdude22:01:08

no, comparing to get-in + reduce like I wrote

Ben Sless22:01:01

:thinking_face:

borkdude22:01:01

Here's my messy file, feel free to play...

borkdude22:01:37

I haven't changed the not-found case, but similar things can be done there

Alex Miller (Clojure team)22:01:10

1 2 you know what to do

Alex Miller (Clojure team)22:01:23

3 4 ask clojure

😂 16
🤯 4
borkdude22:01:17

I'll make an "ask" tomorrow, thanks ;)

Ben Sless06:01:05

Could the short circuit case be faster because the compiler can prove get doesn't get called on nil

Ben Sless06:01:49

I hope I'll have time, I can try testing this with jmh

Ben Sless11:01:38

Jmh running Hopefully I configured it somewhat correctly

borkdude21:01:27

Or is the concern that someone could extend nil to ILookup (imo not a good thing to do, but ...)

🥴 2
Noah Bogart21:01:01

that's such a horrible devilish idea, i am in awe

Alex Miller (Clojure team)21:01:26

ILookup is an interface, you can't "extend" it to nil

souenzzo12:01:01

Say that for JVM.

borkdude12:01:43

Yeah, but even then, the clojure implementation could keep track of nil being extended to ILookup and invalidate that assumption. It's bad practice anyway to extend types you don't own, so one could also say: you're not supposed to do that

👍 2
JoshLemer21:01:00

If I'm not mistaken, I think that in java an inner (non-static) class will retain a reference to its outer class instance, and stop it from being GC'd, even if the class never actually accesses anything from the enclosing instance, is that right? Namely, IntSet.BitSetContainer and IntSet.SingleContainer are inner, non-static classes of IntSet, even though they can/do outlive their original IntSet, and don't actually access their enclosing instance. So it could be the case that they end up blocking GC of whatever IntSet they originated from. https://github.com/clojure/data.int-map/blob/master/src/main/java/clojure/data/int_map/IntSet.java#L18

JoshLemer22:01:24

(as well as taking up extra memory by storing a reference to the enclosing instance)

Alex Miller (Clojure team)22:01:38

if there's no path back to a gc root, the whole thing will get gc'ed, so maybe not a big deal, but certainly if the inner class does not use the outer class field state, it should just be static

👍 4
andy.fingerhut02:01:13

JVM garbage collectors CAN collect dead data even with cycles of references, unlike reference counting garbage collectors.

JoshLemer02:01:38

Not talking about dead data here, rather an instance of the inner class can outlive its outer instance, holding onto its reference

JoshLemer02:01:47

IE the child is still reachable

andy.fingerhut04:01:23

That clarifies your concern for me. Sorry, I misunderstood.

🙂 2