Clojurians Log v2

What is serialization without i/o?

The io parts are interfaces with many implementations so what are you planning to do?

Kafka will do the I/O. Kafka gives you hooks for how you want to transform (serialize/deserialize) before/after it does I/O. So, here is example of how I create a CBOR serdes for Kafka, note that I use encode and decode functions, that is what I am thinking I want/need from Transit...

(ns cktc.cbor
  (:require
   [clj-cbor.core :as cbor]
   [jackdaw.serdes.fn :as jsfn]
   )
  (:import org.apache.kafka.common.serialization.Serde
           org.apache.kafka.common.serialization.Serdes)
  (:gen-class
   :implements [org.apache.kafka.common.serialization.Serde]
   :prefix "EdnCborSerde-"
   :name cktc.cbor.EdnCborSerde))

(set! *warn-on-reflection* true)

(defn cbor-serializer []
  (jsfn/new-serializer
   {:serialize (fn [_ _ data]
                 (cbor/encode data))}))

(defn cbor-deserializer
  []
  (jsfn/new-deserializer
   {:deserialize
    (fn [_ _ data]
      (clojure.walk/keywordize-keys (cbor/decode data)))}))

(defn cbor-serde
  "Implements an EDN SerDes (Serializer/Deserializer)
  using cbor.
  (RFC 8949 Concise Binary Object Representation)
  
  "
  []
  (Serdes/serdeFrom (cbor-serializer) (cbor-deserializer)))

(def EdnCborSerde-configure
  (constantly nil))

(defn EdnCborSerde-serializer
  [& _]
  (cbor-serializer))

(defn EdnCborSerde-deserializer
  [& _ ]
  (cbor-deserializer))


;; gen-class with AOT for the cbor_serdes.clj:

;; clj -e "(compile 'slc.cbor-serdes)"

;; make sure classes (default *compile-path*) is mkdir-ed and on classpath (e.g. deps.edn :paths ["src" "resources" "classes"]

;; restart repl with (:require ... [slc.cbor-serdes]) and (:import (slc.cbor_serdes EdnCborSerde)) (if compilation went well the import should work)

The arities of the clj-cbor encode and decode fns are interesting/instructive.... https://github.com/greglook/clj-cbor#clj-cbor

I suspect there is a way to do it because the two are separated conceptually in transits very design but you will probably need to rely on private interfaces

At least in transit-js I know it’s possible because I did this many years ago to send/receive transit to/from web workers without serializing to string or reading from string (ie using structured cloning)

I don’t remember the details but it definitely wasn’t via public interfaces