Fork me on GitHub
#clojure
<
2022-12-04
>
jaihindhreddy07:12:20

Can I get mutable primitive locals in Clojure? In long loop-`recur` forms, i'd like to sometimes mutate a loop-var (say a long). Is such a thing possible?

Ben Sless08:12:08

Not possible, but you can shadow with let

niwinz08:12:01

you have volatile! or java interop for internal local mutation

jaihindhreddy08:12:55

I don't want to incur the overhead of volatile!. Huh, I always thought there was some way to do it (for some reason). Context: I was trying to write a fast solution to day 4 of advent of code, and needed this there (https://clojurians.slack.com/archives/C0GLTDB2T/p1670139992100919?thread_ts=1670131095.411239&amp;cid=C0GLTDB2T).

lispyclouds09:12:12

not sure if volatile! incurs much overhead, but another way i can think of is having all of loop bindings in an (int-array ...) and mutate in the loop with aset-int maybe?

thanks3 1
jaihindhreddy09:12:17

The array with mutation would be the closest I guess. volatile! may be fast-enough in any particular situation, but I was wondering whether I can get the exact same bytecode as a mutable local in Java (out of curiosity). The advent of code solution for today was fast enough without mutation in the first place, so I didn't even try it 🙂.

lispyclouds10:12:20

loop-recur does seem to mutate the local bindings and with something like unchecked-add its pretty close!

(decompile (loop [x 0] (if (= x 10) x (recur (unchecked-add x 1)))))

// Decompiling class: user$fn__234
import clojure.lang.*;

public final class user$fn__234 extends AFunction
{
    public static Object invokeStatic() {
        long x;
        for (x = 0L; x != 10L; ++x) {}
        return Numbers.num(x);
    }

    @Override
    public Object invoke() {
        return invokeStatic();
    }
}

lispyclouds10:12:32

adding a type hinted fn,

(decompile (fn ^long [] (loop [x 0] (if (= x 10) x (recur (unchecked-add x 1))))))

// Decompiling class: user$fn_line_1__266
import clojure.lang.*;

public final class user$fn_line_1__266 extends AFunction implements L
{
    public static long invokeStatic() {
        long x;
        for (x = 0L; x != 10L; ++x) {}
        return x;
    }

    @Override
    public Object invoke() {
        return invokeStatic();
    }

    @Override
    public final long invokePrim() {
        return invokeStatic();
    }
}

1
Alex Miller (Clojure team)16:12:28

You can do it with java arrays

igrishaev18:12:16

Another options is to use with-local-vars :

(with-local-vars [a 0 b 0 limit 9]
  (loop [i 0]
    (if (= i @limit)
      (+ @a @b)
      (do
        (var-set a i)
        (var-set b i)
        (recur (inc i))))))

igrishaev18:12:02

The example is a bit dull yet works. Still, it's better to avoid mutable things in loop/recur

Samuel McHugh10:12:13

Are the two totally equivalent from a speed and memory perspective?

jaihindhreddy12:12:21

I don't think they are identical in-terms of the bytecode generated by the Clojure compiler as-is. But, with enough inlining, they should be equivalent. Whether-or-not the JVM inlines to that extent is something that I'm not aware-of (it probably depends on a bunch of factors).

Joshua Suskalo16:12:09

They are equivalent from a memory perspective. From a speed perspective I'd have to think hard about what goes on with comp and partial. But if this is at a stage where you care about the performance difference between these you should be using a named function which does the whole transformation without higher order functions and pass that to map.

jaihindhreddy16:12:46

Measuring these with criterium, they seem to neck-and-neck in speed. But yeah, as Joshua says, making the compiler and/or runtime do as little-work as possible is your best bet for performance. Which means (partial * 2) could be simplified to #(* % 2) (because this tells the system that we only care about arity-2 of *). Going further still, #(inc (* 2 %)) as the mapping-fn would probably be better.

Joshua Suskalo16:12:45

other order, but yes

✔️ 1
jaihindhreddy16:12:34

Oops, (* 2 (inc %)) yes.

p-himik12:12:35

The error does not seem sensible, does it?

=> (let [x 1]
     (clojure.core.match/match [1 1] [x x] 2))

Unexpected error (AssertionError) macroexpanding match/match at (...).
Pattern row 1: Pattern row reuses wildcards in [x x].  The following wildcards are ambiguous: x.  There's no guarantee that the matched values will be same.  Rename the occurrences uniquely.

Joshua Suskalo16:12:04

seems sensible to me. It's saying match doesn't support unification.

p-himik16:12:45

It's saying "wildcards are ambiguous", but x there is not a wildcard - it's a local binding.

Joshua Suskalo16:12:29

right, but does match ever support local bindings in patterns? I don't think so. They always shadow.

Joshua Suskalo16:12:03

So the problem here from match's perspective is the double occurance of x, the local is irrelevant.

p-himik16:12:06

It does. :)

Joshua Suskalo16:12:20

That surprises me

Joshua Suskalo16:12:36

Well then yes I agree this sounds like a bug

Joshua Suskalo16:12:42

premature verification

p-himik16:12:58

From https://github.com/clojure/core.match/wiki/Features: > It supports matching on local variables And from https://github.com/clojure/core.match/wiki/Basic-usage#locals: > Normally core.match will bind values to symbols, however it first tries to match against locals. From the implementation:

;; Local bindings in pattern matching are emulated by using named wildcards.
;; See clojure.lang.Symbol dispatch for `emit-pattern` 

Apple17:12:59

I'm reviewing my old AoC code. Any idea why first version is significantly slower, like 4sec vs 2sec? The difference is only in where the algorithm/md5 symbols are located.

(ns y2015
  (:import [java.security MessageDigest]))

(def algorithm (MessageDigest/getInstance "MD5"))

(defn md5 [^String s] (format "%032x" (BigInteger. 1 (.digest algorithm (.getBytes s)))))

;; 201504
(let [d (slurp "resources/201504")
      f (fn [regex] (->> (range) rest (some #(if (re-seq regex (md5 (str d %))) %))))]
  (map f [#"^00000" #"^000000"]))
;; ()
(ns y2015
  (:import [java.security MessageDigest]))

;; 201504
(let [d (slurp "resources/201504")
      algorithm (MessageDigest/getInstance "MD5")
      md5 (fn [^String s] (format "%032x" (BigInteger. 1 (.digest algorithm (.getBytes s)))))
      f (fn [regex] (->> (range) rest (some #(if (re-seq regex (md5 (str d %))) %))))]
  (map f [#"^00000" #"^000000"]))
;; ()

dpsutton17:12:14

no need to ping individuals like this

dpsutton17:12:47

if you use (set! **warn-on-reflection** true) then you'll probably see why

Alex Miller (Clojure team)17:12:35

algorithm will lose its type info on the def one

Apple17:12:57

but in the let block it doesn't?

Apple20:12:52

Any doc I can read about this behavior??

dpsutton20:12:20

https://clojure.org/reference/java_interop look at the type hint section. For these purposes, a “def” won’t keep the type hint and goes to object by default. but in the let bound form the type from the static call is propagated

Apple20:12:36

Type hints are metadata tags placed on symbols or expressions that are consumed by the compiler. They can be placed on function parameters, let-bound names, *var names (when defined)*, and expressions: got you. what does this part mean `var names (when defined)`

👍 1
zakkor22:12:52

Anyone know of a library that implements these two threading macros I found in a common lisp library? They seem very useful, and a potentially nicer solution compared to as->

seancorfield22:12:02

Maybe this https://github.com/rplevy/swiss-arrows ? (But I would say this does not make for readable/idiomatic code)

1
phill23:12:05

<> having to be a top-level element imposes some fragility relative to Clojure's as->, which (according to the docstring) "Binds name to expr..." so its depth doesn't matter. Also - while I can imagine that the cited -<> might seem attractive for use with multiple expressions in FORMS, Clojure's as-> is splendidly idiomatic as a modifier, so to speak, of a single expression somewhere in a ->. So to get the benefit of as-> in one expression in a thread you don't have to wrap any of the other expressions with it. All in all - perhaps as-> could be seen as an improvement over this -<>.

1
zakkor23:12:46

> -<< , -<<:p The Trystero Furcula, Parallel Trystero Furcula > -<>< , -<><:p The Diamond Fishing Rod, Parallel Diamond Fishing Rod Man and I thought Diamond Wand was a funky name 😆

seancorfield23:12:52

And I agree with Phill regarding these arrows. as-> is designed to provide a binding in the middle of a -> pipeline so you can easily use a function (or expression) where the threaded expression is not the first argument.

seancorfield23:12:13

(-> expr (f1 2 3) (as-> x (f2 1 x 3)) (f3 2 3))

skylize02:12:39

If you ignore the question of "idiomatic" and the fact that nobody reading your code will have any clue what these symbols mean... ...(realizing, of course, that these are actually pretty important considerations)... This does strike me as cleaner and more readable in the situations where you would want as->. Taking @U04V70XH6’s example:

(-> expr
  (f1 2 3)
  (as-> x (f2 1 x 3))
  (f3 2 3))
vs
(-<> expr
  (f1 2 3)
  (f2 1 <> 3)
  (f3 2 3))

cjohansen05:12:15

I would avoid both of those and find alternatives. Either make all the functions thread first or thread last friendly, or break it up.

1
xificurC09:12:05

anyone remembers a clojure arrow library that reimagined the meaning of let, when etc inside the thread macro? That was an interesting one but I didn't bookmark/star it

xificurC10:12:02

no, but this is gold too

ilmo11:12:34

I agree on the points on readability and idiomatism. That said, I’ve found the fx threading macros to be useful even in a simple threading pipeline when logging intermediary outputs.

Joe Reinhart12:12:03

If I find myself in a position to use as-> , I typically rewrite the entire expression to make use of the new binding. This provides the flexibility of as-> while maintaining readability.

(as-> expr $
  (f1 $ 2 3)
  (f2 1 $ 3)
  (f3 $ 2 3))

seancorfield17:12:35

@U04DJHRML3X That looks and feels so wrong to me since it reverses the normal "symbol expression" binding order we see everywhere else (and as-> is deliberately "the wrong way round" because it is intended to be used inside -> and not as a top-level form).

1
xificurC17:12:15

do you have a source to > it is intended to be used inside -> and not as a top-level form

seancorfield18:12:25

When it was introduced, Rich talked about it on the Google Groups mailing list, but it's pretty much impossible to search the archives for let-> or as-> (`let->` was one of the early names considered for it).

skylize18:12:06

I think @U04DJHRML3X’s suggestion is much easier to read than even the simplest example of an inlined as->. It gets the main benefits of -<>, from the original question, without being totally cryptic to anyone new to a codebase. I have no idea what Rich's intention was, but this usage seems much more natural to me: • Pick a name for positional threading and thread that name through a series of functions vs • Thread through a series of functions, and then use another threading macro to inject altered positionality in a single place. The primary mental model of a threading macro is to start at the top and push values through multiple functions. So why would we expect ->as be primarily limited to situations where it will usually only wrap a single function?

seancorfield18:12:31

Not "a single function" -- it could be any expression that uses the bound symbol.

seancorfield18:12:15

(-> expr
  (f1 2 3)
  (as-> x (let [y (f2 x)] (f3 x y)))
  (f4 2 3))
For example.

skylize18:12:07

Could be any expression, yes. But more often than not it is patching a single expression that just doesn't fit the mold for -> or ->>. I did say "usually".

seancorfield18:12:15

(that speaks to the design and use of as->)

xificurC18:12:08

that's a highly fabricated example 🙂

(let [x (f1 2 3), y (f2 x)]
  (f4 (f3 x y) 2 3)))

seancorfield18:12:10

Most examples are 🙂

xificurC18:12:41

how about

2 3 f1 dup f2 f3 2 3 f4
🙂

skylize18:12:25

> (that speaks to the design and use of as->) Well, it speaks to the use of. The wording of that article does not give me the impression he provided any meaningful input to the design of that particular feature.

skylize18:12:46

> To me, this clearly indicates that it is meant to be used in combination with ->. This strongly implies he is looking backward and guessing what Rich was thinking.

seancorfield18:12:19

If I can ever find Rich's original posts about it, I'll drop the link(s) here for you...

skylize18:12:38

I agree with the point in the article that (as-> expr x ...) is backward from the (as-> x expr ...) one might expect for use at the top level. But I disagree with the conclusion. I do not think the syntactical incongruity is inherently more important than the mental complexity of working out what an as-> thread is doing in the middle of an otherwise straightforward pipeline.

Joe Reinhart18:12:00

@U04V70XH6 I appreciate the feedback and @U90R0EPHA and @U09FL65DK for the discussion. Understanding the original intention for as-> explains the backward bindings which have definitely tripped me up before.