Fork me on GitHub
Ben Sless10:07:47

Curious why change from ArrayList to LinkedList in the transformers' implementation. Aren't arrays better for iteration and cache?


LinkedList has much simpler iterator impl. Didn't see any visible perf difference thou.

Ben Sless11:07:04

This is slightly theoretical, but ArrayLists have a smaller memory footprint and their backing array is contiguous in memory which ought to give better performance. Also, although the iterator implementation is simpler, ArrayLists would probably benefit from branch predictions, while pointer-chasing with linked lists gives you no guarantees

Ben Sless11:07:24

Modern CPUs like arrays 😞

👍 2

I guess you talk about I, same as @UK0810AQ2, would choose ArrayList, especially that we know the size of data. Theoretically it should be faster in allocation of memory and further iteration, but well, always good to see relevant data. Personally I don't care about performance that much so I don't mind either of them. Still, I'm curious, @U055NJ5CC why the iterator's impl matters to you since its interface is the same?


Collection transformers will be rewritten by @UK0810AQ2 to remove all iteration. But, perf comes from impl, not perf. Was really iteresting read, I still think the iteration is faster on LL, but, irrelevant here. As a generic data structure AL is much better.

Ben Sless17:07:36

I don't know if the composition based solution will be faster. Will try to poke at it with jmh

Ben Sless11:07:42

Right, jmh is running, results in a few hours

Ben Sless13:07:08

([:map-transformer/linked-list {:count 1} [6.156812627775496E7 "ops/s"]]
 [:map-transformer/linked-list {:count 2} [3.2994990990360465E7 "ops/s"]]
 [:map-transformer/linked-list {:count 4} [1.7037915234624855E7 "ops/s"]]
 [:map-transformer/linked-list {:count 8} [7557295.433684901 "ops/s"]]
 [:map-transformer/linked-list {:count 16} [1566793.27656629 "ops/s"]]
 [:map-transformer/linked-list {:count 32} [704492.6211734915 "ops/s"]]

 [:map-transformer/array-list {:count 1} [6.3440679176932156E7 "ops/s"]]
 [:map-transformer/array-list {:count 2} [3.439043251403162E7 "ops/s"]]
 [:map-transformer/array-list {:count 4} [1.689684037729955E7 "ops/s"]]
 [:map-transformer/array-list {:count 8} [7481296.308118063 "ops/s"]]
 [:map-transformer/array-list {:count 16} [1550320.2392137954 "ops/s"]]
 [:map-transformer/array-list {:count 32} [713659.2962506406 "ops/s"]]

 [:map-transformer/comp {:count 1} [8.510126517455931E7 "ops/s"]]
 [:map-transformer/comp {:count 2} [4.2607291886022285E7 "ops/s"]]
 [:map-transformer/comp {:count 4} [1.9971178945141207E7 "ops/s"]]
 [:map-transformer/comp {:count 8} [8451028.653528532 "ops/s"]]
 [:map-transformer/comp {:count 16} [1599471.9346010373 "ops/s"]]
 [:map-transformer/comp {:count 32} [737048.961960025 "ops/s"]])

Ben Sless14:07:26

Another avenue I want to explore is creating from an array. Can't avoid an array copy anyway, so why not copy just once?

Ben Sless19:07:49

more plausible to achieve with a tuple transformer, don't know if can be done with map

Ben Sless05:08:04

I think I found a faster way to transform a tuple:

(defn -tuple-transformer [ts]
  (let [ts (map (fn [[k t]] (let [i (int k)] (fn [^objects arr] (aset arr i (t (aget arr i))) arr))) ts)
        t (apply -comp ts)]
    (fn [^clojure.lang.APersistentVector x]
      (clojure.lang.LazilyPersistentVector/createOwning (t (.toArray x))))))
Will check some arities combinations, initial tests look promising


If you want to have different error messages for different predicates in a malli schema, would it be wrong to do this


[:and [:fn {:error/message "error 1"} some-constraint]
[:fn {:error/message "error 2"} some-other-constraint]]


what’s the schema for a malli schema? e.g. for a key in a map there’s a key in a map that can be a schema [:map [:a int?] [:spec :???schema???]]


The term I've used to describe this is alternately "meta-schema" or "self-describing schema" - there isn't one yet for malli schemas.


ah ok, thanks


I’ll just use any? I guess


couldn’t figure out what to search for


the word schema appears a lot 😉


is it just malli/schema??


hm no that doesn’t seem to work

Noah Bogart14:07:40

does malli care about stuff like volatile!? I have a schema for a map and i’m thinking of changing it to a volatile, so how do I put that in my schema? I changed the schema to [:volatile [:map …]] but that didn’t work


The term I've used to describe this is alternately "meta-schema" or "self-describing schema" - there isn't one yet for malli schemas.