Clojurians Log v2

> I presume pure Clojure approaches like nested vectors and map from position to number would be one or more orders of magnitude slower than a plain Java array. this is correct, but I don't see how it in any way entails deftype

the thing that is going to provide performance is using an array, that is something that you can put in a var like any other value

You'd rather have a map of a column count, a row count and the array? The flat array alone doesn't tell you array dimensions.

Looks like I can use Class/forName with a try catch,

(Class/forName "question.Matrix")
;; => question.Matrix

Just to avoid the XY problem - why would you want to check for something like that in the first place?

My goal was to avoid redefining my type unnecessarily — each time I eval my namespace, I re-run the deftype expression. This gets instances in my test namespace out of sync with the class in my source namespace. I was trying to conditionally run deftype, somthing like

(when-not (class-exists? 'Matrix)
  (deftype Matrix [rows cols vals]))

Hope this answers your question, @p-himik!

I'd like to know if my conclusions make sense, or if I'm missing something.

If you eval whole namespaces, I'd suggest looking into the "reloaded" workflow. Just like the RDD, it has some quirks but nothing major. And after starting using it, together with occasional pin-point RDD, I don't even have to think about what uses what - at all. I never use #' either, and I pretty much have no problems, unless I do something rather complex that even with accurate RDD would be problematic, like some crazy mix of Integrant, memoization, and defonce.

But if that doesn't quite excite you, I usually rely on defonce for such things:

Clojure 1.12.0
user=> (defonce _X (deftype X []))
#'user/_X
user=> (def x1 _X)
#'user/x1
user=> (defonce _X (deftype X []))
nil
user=> (= x1 _X)
true

Oh, hold up - I used _X instead of X there.

Clojure 1.12.0
user=> (defonce _X (deftype X []))
#'user/_X
user=> (def x1 X)
#'user/x1
user=> (defonce _X (deftype X []))
nil
user=> (= x1 X)
false

yes — that reflects what I was seeing 😬

So it seems that the class that gets produced by deftype gets loaded during deftype form parsing. I would 100% use the "reloaded" workflow and forget about things like that entirely. :)

So … this abomination got me "define the class only if it hasn't been defined":

(defn class-defined? [class-sym]
  (let [classname (str (ns-name *ns*) "." (name class-sym))]
    (try
      (Class/forName classname)
      true
      (catch java.lang.ClassNotFoundException _ false))))

(defmacro deftypeonce
  "Define a type only when it hasn't already been defined (to limit the different
  types of objects in memory)"
  {:clj-kondo/lint-as 'clojure.core/deftype}
  [classname & more]
  (when-not (class-defined? classname)
    `(deftype ~classname ~@more)))

(deftypeonce Matrix [rows cols vals])

An abomination indeed. :D

Potemkin has a really nice solution for this

You could lift it easily or depend on it. But it’s quite nice for this

Do you mean deftype+? https://github.com/clj-commons/potemkin/blob/15d0118fa1c4581b93f46768395d30367ecee371/src/potemkin/types.clj#L291

Yes. There are a few that keep their previous forms around and don’t reevaluate themselves if they don’t change

Thank you - that's definitely an improvement on my hack above!

Taking the XY problem one step further - why do you need deftype? I've been using Clojure since ~2012 and I have never used it, so I'm kinda curious to see questions about it in #beginners.

(As in, I would definitely not point a beginner in the direction of deftype in the first place.)

Hi Gary 🙂 I guess I could have asked in #clojure. My goal is to experiment with https://en.wikipedia.org/wiki/Finite_element_method in Clojure. For that, I'd like to get my hands dirty with matrix algebra. Matrix operations need to be fast enough, and I presume pure Clojure approaches like nested vectors and map from position to number would be one or more orders of magnitude slower than a plain Java array. I suspect I'll move to a more established library like https://generateme.github.io/fastmath/clay/vector_matrix.html or https://neanderthal.uncomplicate.org/ — but I want to get my hands dirty too so that I have a sort of reference implementation — so I can see whether my own implementation gives the same results, and how much slower my implementation is than Fastmath / Neanderthal. specifically — I suspect swapping a deftype-Matrix with rows, cols and a Java array for nested Clojure vectors would result in a less interactive experience, as operations that should (could) have been instant suddenly require noticable waiting. ——— But please let me know if you think this doesn't make sense, and that I should leave deftype for something else! Attaching some code in case you want to have a look.

Note that Java arrays, unlike C pointers, know their own length.

So if you know you're working with matrices, you can set things up so you know you're always going to have an array of arrays of floats, and you can ask the first (outer) array for its length to know the number of rows, and the first array of the nested array for the number of columns.

Here's a quick example:

(defn to-matrix
  [v]
  (assert (->> v (map count) set count (contains? #{0 1}))
          "All rows must be of the same length.")
  (if (empty? v)
    (make-array Float/TYPE 0 0)
    (let [m (make-array Float/TYPE (count v) (count (first v)))]
      (doseq [y (range (count v))
              x (range (count (first v)))]
        (aset m y x (get-in v [y x])))
      m)))

(defn from-matrix
  [arr]
  (->> (seq arr)
       (map seq)))

(defn matrix-plus
  [^"[[F" m1 ^"[[F" m2]
  ;; check dimensions
  (assert (and (= (alength m1) (alength m2))
               (= (alength ^"[F" (aget m1 0)) (alength ^"[F" (aget m2 0))))
          "Invalid dimensions.")
  (if (and (zero? (alength m1))
           (zero? (alength m2)))
    (make-array Float/TYPE 0 0)
    (let [rows (alength m1)
          cols (alength ^"[F" (aget m1 0))
          result ^"[[F" (make-array Float/TYPE rows cols)]
      (loop [col 0
             row 0]
        (aset ^"[F" (aget result row) col (+ ^float (aget m1 row col)
                                             ^float (aget m2 row col)))
        (cond (and (= row (dec rows)) (= col (dec cols))) result
              (= col (dec cols)) (recur 0 (inc row))
              :else (recur (inc col) row))))))

(from-matrix
  (matrix-plus
    (to-matrix [[1 2 3] [4 5 6]])
    (to-matrix [[-1 -2 -3] [1 1 1]])))

Thanks for explaining! I've assumed that one single array would be faster than one array for each row — fewer allocations in total, and "pointer following" for access after creation. I haven't done any benchmarks for that, though.

That's a fair point. It may indeed be the case, depending on access pattern and how you organize things within that one array. My understanding is that generally speaking "walking down" an array is fast, jumping around in an array is only fast if the entire array fits in an L1 line, and jumping from one array to another, when they've been allocated at the same time, is roughly equivalent to jumping around within a single array.

You could still have a matrix be a three-element array where the first two elements are the dimensions, and the third element is a single array with all of the data in it. Or, if you're going to calculate indices anyway, just use a single array where the first two elements are the dimensions and the rest is data.

If you're at that level of performance, though, it's definitely worth benchmarking.