Fork me on GitHub
#clojure
<
2023-12-17
>
jeaye01:12:25

Hey folks, I've a runtime question regarding RestFn and arg matching. Let's assume I have a fn like:

(defn foo
  ([a]
   :fixed)
  ([a & args]
   :variadic))
Since this is variadic, it's an instance of RestFn and its getRequiredArity method will return 1. So, when I invoke it with one argument, I should end up here: https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/RestFn.java#L413 But the switch there, for the case of 1, will call doInvoke with two arguments, the first being the actual argument and the second being null. When I look at the decompiled Java for that function (shown in 🧵), I see that doInvoke with two arguments calls the variadic overload, but of course that's not what happens. So I'm wondering how this all ends up in the right place. I've implemented this in jank and I needed extra data to disambiguate this case; I'm just not yet seeing where Clojure is doing the same.

jeaye01:12:43

user=> (decompile (fn ([a] :fixed) ([a & args] :variadic)))

// Decompiling class: user$fn_line_1__2194
import clojure.lang.*;

public final class user$fn_line_1__2194 extends RestFn
{
    public static final Keyword const__0;
    public static final Keyword const__1;

    public static Object invokeStatic(final Object a, final ISeq args) {
        return const__1;
    }

    public Object doInvoke(final Object a, final Object o) {
        return invokeStatic(a, (ISeq)o);
    }

    public static Object invokeStatic(final Object a) {
        return const__0;
    }

    @Override
    public Object invoke(final Object a) {
        return invokeStatic(a);
    }

    @Override
    public int getRequiredArity() {
        return 1;
    }

    static {
        const__0 = RT.keyword(null, "fixed");
        const__1 = RT.keyword(null, "variadic");
    }
}

Bob B03:12:47

I might be missing something or just misreading, but the decompiled class overrides the one-arg invoke method mentioned with the switch, right? So in the case of a one-arg call, does it just call one-arg invokeStatic? Sorry if this isn't helpful

jeaye04:12:02

Ah, it's likely as simple as that. The single arg method is overridden and the switch is replaced simply by the vtable. The two-arg method is not overridden, so it goes into the switch for its base implementation.

jeaye04:12:15

Thanks for pointing out the obvious for me. :) I avoid dynamic dispatch for jank so much, I forget sometimes that Clojure uses it for everything.

👍 1
stephenmhopper13:12:05

If I have a reference to a Class that is a defrecord type (i.e. MyDefrecord), how do I lookup the corresponding map->MyDefrecord function? Right now, I have these:

(defn lookup-defrecord-map-fn
  "For a given defrecord type MyRecord, returns the resolved map->MyRecord function.
  Works with fully qualified classname symbols, but it noticeably slower than build-defrecord-map-fn"
  [^Class clz]
  (let [class-name (last (string/split (.getTypeName clz) #"\."))]
    (resolve (symbol (str "map->" class-name)))))

(defmacro build-defrecord-map-fn
  "For a given defrecord type MyRecord, returns the resolved map->MyRecord function.
  Does not work for fully qualified classname symbols. In those situations, use lookup-defrecord-map-fn"
  [defrecord-type]
  (resolve (symbol (str "map->" defrecord-type))))
The macro version is significantly faster (1.5ns vs 500ns) but I don't think the macro version (as it's currently written) will work in all cases

Alex Miller (Clojure team)16:12:15

Those vars are just wrapping a constructor - maybe easier to just invoke the constructor via interop?

stephenmhopper03:12:25

Yeah, thanks for the idea. That works too and is pretty straightforward. It's slower than the other two options, but it just pushes us from nanoseconds into microseconds, so it's probably fine for my use case

Alex Miller (Clojure team)04:12:26

If you’re doing the constructor lookup every time you could memoize

Alex Miller (Clojure team)05:12:29

Invoking the constructor is going to happen regardless but should be fast, its presumably the lookup that’s slow

stephenmhopper12:12:56

Are you referring to the lookup-defrecord-map-fn function above? I agree that memoization would be a good fit for that one. The interop code I came up with looks like this: (clojure.lang.Reflector/invokeStaticMethod MyDefrecord "create" (into-array Object [{:field-name :thing}])) That's the one that's currently the slowest and I imagine it's because of the reflection that's happening on each call. So I pulled out the Method lookup so I could cache it. Now I just call .invoke on the Method reference, but that's still slower than the code I posted above. But caching the reflection lookup calls dropped the runtime for this code from 1.8 microseconds down to 1 microsecond

stephenmhopper13:12:58

Oh, I was making this way harder than it needed to be. I can just call clojure.lang.Reflector/invokeConstructor and that yields performance around 155ns which puts this option somewhere between the first and second pieces of code I posted originally. I'll just go with that. Thanks, Alex!

Alex Miller (Clojure team)13:12:07

Reflector is an internal implementation class, so you really should call the Java reflection stuff directly (what Reflector calls)

👍 1
Alex Miller (Clojure team)13:12:04

The other newer approach to this is to use MethodHandles.Lookup, not sure how that compares perf wise

stephenmhopper13:12:40

That's not working for me. I'm not sure why though.

(def ^MethodType method-type (MethodType/methodType java.lang.Void/TYPE clojure.lang.IPersistentMap))
(def method-lookup (MethodHandles/lookup))
(def ^MethodHandle method-handle (.findConstructor method-lookup MyDefrecord method-type))
Is IPersistentMap not the right input parameter type for the constructor?

stephenmhopper13:12:53

This doesn't go directly through the constructor, but uses create instead and works:

(def ^MethodType method-type (MethodType/methodType MyDefrecord clojure.lang.IPersistentMap))
(def method-lookup (MethodHandles/lookup))
(def ^MethodHandle method-handle (.findStatic method-lookup MyDefrecord "create" method-type))
(.invokeWithArguments method-handle (into-array Object [{:field-name :thing}]))
Benchmarking just the invokeWithArguments call shows it to be slower than clojure.lang.Reflector/invokeConstructor though

Alex Miller (Clojure team)14:12:14

I would expect that to be getting faster in newer jvms. Using the positional constructor will also be faster if you know the field structure (don’t have to make or tear apart the map)