Fork me on GitHub
#clojure
<
2022-08-28
>
wombawomba08:08:27

Is there a clean way to have a macro produce a namespace-qualified symbol? The following works, but it feels a bit convoluted:

(defmacro ns-qualify [x] `(symbol (str ~*ns*) ~(name x)))
(ns-qualify foo) ; => my.ns/foo

p-himik08:08:27

Seems alright to me. Although, I'd make it a function and not a macro.

wombawomba09:08:40

Hmm.. I actually realized this doesn't seem to work in CLJS.

wombawomba09:08:45

It's giving me clojure.lang.Namespace is not a valid ClojureScript constant.

wombawomba09:08:28

(defmacro ns-qualify [x] `(symbol ~(str *ns*) ~(name x)))
does work though

wombawomba09:08:00

Anyway, thanks. I figured there might be some sort of magic quoting trick I could do to have the compiler do this work instead of me, but perhaps not.

Drew Verlee10:08:30

I'm also curious why this needs to be a macro.

p-himik10:08:57

Given that it's CLJS apparently, it can't work as a function - it has to be a macro.

p-himik10:08:43

*ns* is not a macro, but it doesn't exist in CLJS outside of macros or bootstrapped environments.

👍 1
stephenmhopper13:08:45

I have a map, m. I need to dissoc 6 fields from it and update 4 other fields on it. If I thread the map through these 10 forms, I’ll potentially end up with a bunch of garbage for GC. I have several different ideas of how to clean this up, but I’m looking for suggestions. How would you structure this code?

p-himik13:08:17

dissoc accepts multiple keys.

p-himik13:08:48

> I’ll potentially end up with a bunch of garbage for GC Is this truly a problem that you have observed and measured to be of a significant impact?

1
☝️ 3
stephenmhopper13:08:18

> dissoc accepts multiple keys. In my experience, criterium shows that route to be slower than just calling dissoc multiple times, but I know you and I have run into differing results from criterium in the past.

stephenmhopper13:08:10

>> I’ll potentially end up with a bunch of garbage for GC > Is this truly a problem that you have observed and measured to be of a significant impact? No, but this is just a side project where that part doesn’t really matter. I’m just using this as practice for when I build real applications where those implications are important. This piece of code that I’m optimizing is a “hot-spot” that is called many times in a short period of time. However, in the grand scheme of things, a few millis of performance difference won’t matter for this application.

CarnunMP13:08:52

Could select-keys instead of dissoc. Or even do the whole thing inside a (rather imperative) reduce-kv.

stephenmhopper13:08:29

I considered select-keys except this function doesn’t know all of the keys that might be in m. It just knows that there’s certain keys it wants to remove, and certain ones it wants to update.

stephenmhopper13:08:49

I gave reduce-kv a shot, but that was harder to read and slower than the alternatives

stephenmhopper13:08:26

Seems like turning the 6 calls to dissoc into a single call to dissoc is the best way to go

stephenmhopper13:08:34

Thanks, everyone!

👍 1
Ben Sless15:08:33

Hey, wanna cook with gas?

(alter-meta! #'clojure.core/dissoc assoc :inline
             (fn
               ([m] m)
               ([m k] `(. clojure.lang.RT (dissoc ~m ~k)))
               ([m k & ks] `(-> ~m ~@(map (fn [k] `(dissoc ~k)) (cons k ks))))))

👏 3
Ben Sless15:08:20

the 2+ keys arity of dissoc also generates garbage, and is somewhat slower

vlaaad16:08:25

You can use transients

Ben Sless17:08:08

that, too 🙂

didibus18:08:53

Yes transient is the answer here that's why they exist, for exactly these kind of use cases where you need to add or remove many elements at a time.

Ben Sless19:08:33

You still pay a pretty high cost for iteration, so dissoc should still be unrolled. It also generates garbage to allocate and iterate over the args

didibus19:08:32

Are you sure hotspot doesn't unroll this itself?

Ben Sless19:08:18

Oh yeah, I profiled that area to death. Maybe if it were array iteration it could have, but doing it with first/rest? Impossible

Ben Sless19:08:54

Up to and including JIT logs

didibus19:08:52

In this case its just threading through right? Do you know if threading and/or loop/recur can be optimized by the JIT

didibus19:08:33

Oh, are you talking about the dissoc that takes more than one arg? Does it use sequences to do so under the hood?

Ben Sless19:08:59

Yes. The rest args get packed in an array and it gets packed in a list facade. Iterating over it with first and rest returns a new *cursor" like object

👍 1
didibus19:08:19

Side note, I'd also be curious about transducers. Basically, what loop is best for JIT unrolling?

didibus19:08:43

Except for the inlining correct, so that's only if you use it in a HOF

Ben Sless19:08:01

Transducers are more like operator fusion

Ben Sless19:08:12

And they don't play as nice with primitives

didibus19:08:25

Wait, the inlining is your code, ok I got confused.

didibus20:08:50

Thought the core function had that inline code. Ok, ya I see why that arity is actually slower than just making repeated calls to disocc. I never used that arity to be honest, didn't even realize it was there.

stephenmhopper12:08:38

FWIW I profiled an option that uses reduce-kv with transient! but my results showed it to be slower than the other options

Ben Sless16:08:15

The most efficient solution is always unrolling a sequence of operations

didibus17:08:56

I've found that conversion to and back from transient adds a bit of overhead, so you want to use it when you have a lot of insert/remove. In your case, you have like 15 only, which isn't a lot of insert/remove, so maybe the transient conversion overhead makes it actually slower

didibus17:08:40

I also found loop/recur is always the fastest way to loop. And I mean unrolling like Ben is saying would be even faster.

didibus17:08:36

But to me these are two distinct problems. How fast can you iterate and execute a function repeatedly is one dimension. Unrolling will be fastest, followed by loop/recur, then iterator based loops like reduce or transducers, then sequence loops. And the other question is how do you insert a new key/value or remove one from a map most quickly. This will be faster on a transient map, but conversion from/back transient and persistent adds a fixed overhead. So for inserting a single thing it won't be faster probably, but if you're doing a lot of them inside a loop, it will pay off.

didibus17:08:02

My conclusion here is Ben is right (he's the expert at micro-opromizations afterall). If you know the set of keys, and it's under 100 (that's kind of a guess), unrolling and using normal assoc/dissoc is probably fastest. If you don't know the set of keys, I'd assume loop/recur would be best, and then if you'll have like 100+ (again just a guess) probably using transient will be fastest.

didibus17:08:34

Also, FYI, when using loop/recur you need to do so with iterator as well, if you use first/next/rest with it you're back at using sequences. loop/recur just avoids the function overhead for the body caused by reduce.

Ben Sless19:08:34

Loop/recur with first/rest can be slower than reducing over the sequence of keys, there's a good chance you'll be iterating over an array when you do so, which is hardware friendly

💯 1
didibus23:08:31

Ya that's why I clarified loop/recur with iterator