what does the ^not-native type hint actually do? I’m trying to optimize some paths that use a lot of a particular protocol, which is extended to native JS types (string, number, nil) as well as CLJS/user types like Keyword, PersistentVector, and Object

it is an optimization that'll make the compiler skip checking if something implements a particular protocol fn

(defn foo [^not-native x] (bar x)) -> x.some$ns$TheProtocol$bar$artity$1() vs x.some$ns$TheProtocol$bar$artity$1 ? x.some$ns$TheProtocol$bar$artity$1() : some.ns.bar(x)

something like that

no noticeable speed-up but I might be using it wrong

speedup is 3-10% in isolated micro benchmarks measuring basically nothing else

if you are doing other stuff the overhead is not noticable indeed because pretty much everything else is more expensive

it used to make a bigger difference - not sure anymore these days

@lilactown but in your case you can't use it

^not-native means the protocol will never be invoked on a native type

protocol invoke on native types won't be fast

if you use ^not-native incorrectly you will get broken code

my advice - don't use it 🙂

Here are notes I've accumulated on that subject: satisfies? vs. implements? and native-satisfies? ---- satisfies? Normal, when you extend a protocol to a deftype or defrecord implements? like satisfies? but does not check for natives that extend the protocol Often used in conjunction with ^not-native native-satisfies? works if you extend to, say, object, or some native JS type ^not-native type hint: In the presence of this type hint all protocol fns on the hinted symbol will be directly dispatched under advanced compilation. Frequent pattern in core:

cond
  implements?  (with ^not-native)
  …
  native-satisfies?

Oh, and one impression I get with all of the above is that it is really meant for internal use in the std lib. External code is probably best off avoiding ^not-native and the other predicates, and simply sticking with satisfies?

thanks. that’s helpful

I’m struggling with how to approach optimizing my library further. at this point I’m investigating trying to get rid of any calls to seq, but that’s proving very difficult. I’m not even sure I’m going in the right direction

I have a lot of places that need to do ex. (-parse-element el (first args) (rest args))

that, and I have a memoization I’m doing and doing the deref + lookup seems to take quite some time

@lilactown well I've found that the profiles are useful to identity actual bottlenecks

I'm assuming your memoization is actually speeding things up?

if deref + lookup takes longer than remove the memoization

I believe so: with memoize: hx x 212,037 ops/sec ±1.75% (58 runs sampled) without memoize: hx x 143,112 ops/sec ±1.95% (59 runs sampled)

I actually specialized it with a memoize1 since I didn’t need the varargs

k, are these profiles coming from Chrome or something else?

those are benchmarks, and I’ve been staring at the chrome profile of the benchmark runs for the past 2 days

yeah interpreting them isn't easy

but they do tell you what's wrong

that's how I optimized ClojureScript bootstrapped, the reader, core.async

@lilactown I see I sucked you into the benchmark black hole 😛

it's all your fault @thheller 😂

but I got hx up to about 2/3 of reagent, which is a lot better than where it started!

nice

@lilactown what is your top line item in self time? seq?

honestly it’s mostly my code

it’s the confusing part of the profiles. my code is pretty much just calling other fns.

the only thing that the functions with high self-time really do is process the args into (first arg) and (rest arg) and then call another fn

so I’m kind of left guessing. maybe the calls to first/seq are getting inlined by the VM and it’s not reporting them as calls? I can’t really tell

more likely they are so cheap they just don't show up

hah, that could be it too 😛 like I said, i'm kind of shooting in the dark

this is bad if you want a hint https://github.com/Lokeh/hx/blob/master/src/hx/react.cljs#L26-L30

first step iterates the map to find a key, then maybe updates it

second step does the same again

third again

then finally convert it to JS obj

this benchmark doesn’t even hit that path, but that is on my list of things

could be done all in one iteration

apply itself is also costly

btw here is the current iteration: https://github.com/Lokeh/hx/blob/benchmark/src/hx/react.cljs

If it's any consolation, I've never had a profiler give me actionable information (python, java, js so far). I always have to make a semi-realistic benchmark workload and try things.

profiler is a bit more useful with :simple or :advanced+ pseudo-names sometimes

since it removes some of the extra IIFE that CLJS generates for certain stuff

yeah I’ve been using advanced + psuedo-names

I think profilers are just bad at identifying death-by-many-cuts perf problems, especially sampling ones where the papercuts may not even show up

it sounds like you're already trying to optimize the "ligaments" of your code rather than the muscle

i.e. the stuff that happens invisibly in expanded code

"in between" your real work

yeah. I think I may have squeezed out all there is to optimize what I have; I need to find a way to do what I’m doing, less

I'll just say that I disagree w/ the above profiler claim

all the perf benefits in ClojureScript compile time runtime were all done via YourKit and Chrome

interpreting the results is non-trivial - but they do not lie

I’ve also found https://github.com/clojure-goes-fast/clj-async-profiler useful for JVM profiling

@lilactown another trick I've learned is that you have to squint a bit at profile results - esp self-time - it might not be the very top-line but there's no doubt it something in the top 30 items

I've often found it necessary to go through each item one by one and read the source body closely

this might not lead to the answer but it might hone your intuition about what's going wrong

The above is basically every time I use a profiler

it does also help to have some intuition about the kinds of optimizations that the JVM and JS engines do

many thing are non-obvious esp. for JS, like in the past the body of a try/catch would not be optimized

and for ClojureScript there's lot of things to avoid for the fastest code

no native types in your protocol dispatch, avoid var ags w/o arity unrolling, no apply, etc.

once you remove the perf ruiners

then you still have to do another pass for stuff like truth tests

and unlifted closures

the last thing is actually pretty killer in tight loops

transducers/reduce over their lazy seq counterparts also make a difference

and easy mistake to make, the right hand side of a binding form cannot be too complex

hmmm

for the last case

use :simple, if you see that your fn has a bunch of IIFEs you're dead

they will not get removed in :advanced

also note that this can lead too

"my function is not doing anything but my self-time is big" judgements

fascinating

stuff like that can often been discovered by comparing to GC activity

if the GC activity seems far higher than the work - could be just a mess of closures

interestingly if I switch from protocols to just a big cond I see pretty much no difference in the benchmark

I’m going to look at closures next

not unexpected

if every type is going through a protocol the call site will be megamorphic

it does reduce the size of the call tree substantially though 😛

which looks nicer to me 😂

it can also be faster

a cond which tests some common cases and then protocol

(reduce the size when I use cond I mean)

yeah I think I might switch to cond for native types

exactly