https://github.com/datalevin/datalevin/blob/b3d3f88629007671655bce490a9b0926465a0cfe/src/datalevin/txlog/codec.clj#L550
I'm getting ArithmeticException: integer overflow 🧵
(defn- ensure-room!
^ByteBuffer [^ByteBuffer bf ^ThreadLocal tl ^long needed]
(if (>= (.remaining bf) needed)
bf
(let [required (+ ^long (long (.position bf)) ^long needed)
new-cap (loop [cap (long (max 1 (.capacity bf)))]
(if (>= cap required)
cap
(recur (* 2 cap))))
^ByteBuffer grown (ByteBuffer/allocate (int new-cap))]
(.flip bf)
(.put grown bf)
(when tl (.set tl grown))
grown)))
- bf: a ByteBuffer at position 536,871,101, with capacity/limit 1,073,741,824 and 536,870,723 bytes remaining.
- needed: 536,870,913 bytes.So it doubles it to 2,147,483,648.
Clojure 1.12.3
user=> (int 2147483648)
Execution error (ArithmeticException) at java.lang.Math/toIntExact (Math.java:1371).
integer overflow
user=> (int 2147483647)
2147483647I know that's not much info. What am I doing wrong? I generated a bunch of embeddings and I want it to generate the hnsw index.