Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2021-07-30

Channels

#malli
<
2021-07-30
>
Ben Sless10:07:47

Curious why change from ArrayList to LinkedList in the transformers' implementation. Aren't arrays better for iteration and cache?

ikitommi10:07:35

LinkedList has much simpler iterator impl. Didn't see any visible perf difference thou.

Ben Sless11:07:04

This is slightly theoretical, but ArrayLists have a smaller memory footprint and their backing array is contiguous in memory which ought to give better performance. Also, although the iterator implementation is simpler, ArrayLists would probably benefit from branch predictions, while pointer-chasing with linked lists gives you no guarantees

Ben Sless11:07:24

Modern CPUs like arrays 😞

👍 2
greg15:07:15

I guess you talk about https://github.com/metosin/malli/commit/dd19b9e. I, same as @UK0810AQ2, would choose ArrayList, especially that we know the size of data. Theoretically it should be faster in allocation of memory and further iteration, but well, always good to see relevant data. Personally I don't care about performance that much so I don't mind either of them. Still, I'm curious, @U055NJ5CC why the iterator's impl matters to you since its interface is the same?

ikitommi16:07:23

Collection transformers will be rewritten by @UK0810AQ2 to remove all iteration. But, perf comes from impl, not perf. Was really iteresting read, I still think the iteration is faster on LL, but, irrelevant here. As a generic data structure AL is much better.

Ben Sless17:07:36

I don't know if the composition based solution will be faster. Will try to poke at it with jmh

Ben Sless11:07:42

Right, jmh is running, results in a few hours

Ben Sless13:07:08

([:map-transformer/linked-list {:count 1} [6.156812627775496E7 "ops/s"]]
 [:map-transformer/linked-list {:count 2} [3.2994990990360465E7 "ops/s"]]
 [:map-transformer/linked-list {:count 4} [1.7037915234624855E7 "ops/s"]]
 [:map-transformer/linked-list {:count 8} [7557295.433684901 "ops/s"]]
 [:map-transformer/linked-list {:count 16} [1566793.27656629 "ops/s"]]
 [:map-transformer/linked-list {:count 32} [704492.6211734915 "ops/s"]]

 [:map-transformer/array-list {:count 1} [6.3440679176932156E7 "ops/s"]]
 [:map-transformer/array-list {:count 2} [3.439043251403162E7 "ops/s"]]
 [:map-transformer/array-list {:count 4} [1.689684037729955E7 "ops/s"]]
 [:map-transformer/array-list {:count 8} [7481296.308118063 "ops/s"]]
 [:map-transformer/array-list {:count 16} [1550320.2392137954 "ops/s"]]
 [:map-transformer/array-list {:count 32} [713659.2962506406 "ops/s"]]

 [:map-transformer/comp {:count 1} [8.510126517455931E7 "ops/s"]]
 [:map-transformer/comp {:count 2} [4.2607291886022285E7 "ops/s"]]
 [:map-transformer/comp {:count 4} [1.9971178945141207E7 "ops/s"]]
 [:map-transformer/comp {:count 8} [8451028.653528532 "ops/s"]]
 [:map-transformer/comp {:count 16} [1599471.9346010373 "ops/s"]]
 [:map-transformer/comp {:count 32} [737048.961960025 "ops/s"]])

Ben Sless14:07:26

Another avenue I want to explore is creating from an array. Can't avoid an array copy anyway, so why not copy just once?

Ben Sless19:07:49

more plausible to achieve with a tuple transformer, don't know if can be done with map

Ben Sless05:08:04

I think I found a faster way to transform a tuple: 

(defn -tuple-transformer [ts]
  (let [ts (map (fn [[k t]] (let [i (int k)] (fn [^objects arr] (aset arr i (t (aget arr i))) arr))) ts)
        t (apply -comp ts)]
    (fn [^clojure.lang.APersistentVector x]
      (clojure.lang.LazilyPersistentVector/createOwning (t (.toArray x))))))
Will check some arities combinations, initial tests look promising

danielneal11:07:21

If you want to have different error messages for different predicates in a malli schema, would it be wrong to do this

✅ 2
danielneal11:07:42

[:and [:fn {:error/message "error 1"} some-constraint]
[:fn {:error/message "error 2"} some-other-constraint]]

danielneal13:07:30

what’s the schema for a malli schema? e.g. for a key in a map there’s a key in a map that can be a schema [:map [:a int?] [:spec :???schema???]]

respatialized14:07:02

The term I've used to describe this is alternately "meta-schema" or "self-describing schema" - there isn't one yet for malli schemas.

danielneal14:07:57

ah ok, thanks

danielneal14:07:01

I’ll just use any? I guess

danielneal13:07:54

couldn’t figure out what to search for

danielneal13:07:27

the word schema appears a lot 😉

danielneal14:07:48

is it just malli/schema??

danielneal14:07:28

hm no that doesn’t seem to work

Noah Bogart14:07:40

does malli care about stuff like volatile!? I have a schema for a map and i’m thinking of changing it to a volatile, so how do I put that in my schema? I changed the schema to [:volatile [:map …]] but that didn’t work

respatialized14:07:02

The term I've used to describe this is alternately "meta-schema" or "self-describing schema" - there isn't one yet for malli schemas.