Fork me on GitHub
#data-science
<
2021-05-02
>
awb9904:05:32

can someone tell me how I can convert http://tech.ml dataset back to clojure? dataset->data gives me vectors, but I want a row by row representation.

Daniel Slutsky05:05:08

@UCSJVFV35 tablecloth.api/rows has various output formats. On a lower level, tech.v3.dataset/mapseq-reader returns a Reader (in the dtype-next sense) of rows as maps.

Santiago21:05:32

I have a structure that looks like

{:UC48G7530085 [{:price 693.4965 :unixtimestamp 16893454}
                {:price 700.3344 :unixtimestamp 16893455}
                {:price 702.4444 :unixtimestamp 16893456}]}
And I need to create data for a candlestick chart which shows • min price • max price • open price • close price • open and close timestamp for slices of 1 min in time (I should ahve around 1 price per second). I know how I would do this in R in a data.frame, but I must use this structure for an experiment i’m running. Any data people here have a suggestion for how to do this in clojure? I’m not sure how to create a sequence of timestamps at precise/even 1min increments e.g. 12:00:00, 12:01:00 and not 12:00:53, 12:01:53 from unix timestamps

raspasov23:05:18

You can use a library like juxt/tick

(require '[tick.alpha.api :as t])
(let [now (t/now)
      now' (t/truncate now :minutes)]
 (t/range
  (t/beginning now')
  (t/end (t/instant (t/at (t/tomorrow) (t/midnight))))
  (t/new-duration 1 :minutes)))

raspasov23:05:53

=> (#time/instant”2021-05-02T23:17:00Z” #time/instant”2021-05-02T23:18:00Z” #time/instant”2021-05-02T23:19:00Z” #time/instant”2021-05-02T23:20:00Z” #time/instant”2021-05-02T23:21:00Z” #time/instant”2021-05-02T23:22:00Z” …. )

Santiago11:05:04

Nice thanks!

👍 2
Daniel Slutsky18:05:54

@U3RGL6XNF might have something to say about this time-related thread.

Santiago08:05:32

(defn candlestick-data
  "Returns a vector of maps with the data to build a candlestick chart"
  [price-quotes]
  (->> price-quotes
       (mapv #(assoc-in % [:truncated-dt] (t/truncate (:datetime %) :minutes)))
       (group-by :truncated-dt)
       (filter (fn [[_ v]] (> (count v) 1)))
       (into {})
       (reduce-kv
         (fn [m k v]
           (let [high-price (max-key-val :price v)
                 low-price (min-key-val :price v)
                 candle-start-unix (min-key-val :unixtimestamp v)
                 candle-end-unix (max-key-val :unixtimestamp v)
                 open-price (-> v
                                first
                                :price)
                 close-price (-> v
                                 peek
                                 :price)
                 n-quotes (count v)
                 res {:high-price high-price
                      :low-price low-price
                      :open-price open-price
                      :close-price close-price
                      :candle-start k
                      :candle-end (t/>> k (t/new-duration 1 :minutes))
                      :candle-start-unix candle-start-unix
                      :candle-end-unix candle-end-unix
                      :n-quotes n-quotes}]
             (conj m res)))
         [])))
worked for me 😄 not sure if it’s the “best” solution but the whole API is responding in 3ms so I’m happy and again impressed with Clj juxt/tick is dope! It’s exactly what I was looking for 🙂 let’s me work with unix timestamps as much as possible until the last minute

clojure-spin 3
lightsaber 3
raspasov01:05:25

@UFPEDL1LY Very glad to hear!