data-science

pieterbreed 2025-05-15T15:58:40.977319Z

Hi everyone 👋🏽 , I've got a little bit of code here I've been working on for a few years, and it might be interesting to somebody (maybe??); I'm trying to gauge interest. Does this snippet look interesting to you? Do you struggle to work with kafka and data in the repl? Does this look simpler than what you are used to?

user> 
(def msgs (-kafka-utils/load-messages-from-confluent-topic
           :bootstrap-server (:kafka-bootstrap-server cfg)
           :topics           ["v2-update"]
           :offset-reset     "earliest"
           :collect-messages? true
           :nr-msgs           100
           :msg-filter        #(= 0 (:partition %))
           :extra-strategies [[:strategy/SeekToTimestampOffset "2025-05-14T08:00:00.00Z"]
                              [:strategy/StringSerializer :consumer :key]
                              [:strategy/EdnSerializer :consumer :value]
                              [:strategy/ConfluentCloud
                                  :api-key    (:confluent-api-key cfg)
                                  :api-secret (:confluent-api-secret cfg)]]))

user> (first msgs)
{:topic "v2-update",
 :partition 0,
 :offset 83000261,
 :value
 {:movement.acceleration/x -0.003,
  ,,,
  :movement.acceleration/y 0.028},
 :key "name/864275075792047",
 :timestamp #inst "2025-05-14T08:00:00.279Z"}

user> (count msgs)
100

pieterbreed 2025-05-15T15:59:01.308229Z

If this is off-topic, I apologise in advance.

Harold 2025-05-15T16:04:40.595709Z

certainly seems on-topic to me 🆒