Fork me on GitHub
#datascript
<
2017-11-14
>
jeremys14:11:59

Hey everyone

jeremys14:11:27

I was wondering if somebody has already experimented with persisting datascript DBs on the file system or on something like sqllite.

thedavidmeister06:11:42

@U0FR867U1 i just send datoms to my db, indexed on t, grouped by e+a

thedavidmeister06:11:00

i pull out the "latest" v's for each e+a and send them all back to datascript when needed

jeremys14:11:39

Hey man thanks for the reply. I am sure there are more subtleties to your method though. I’ll keep it in mind.

thedavidmeister05:11:25

it gets trickier if you want to share your datoms across different users/devices

thedavidmeister05:11:31

but if it's 1:1 there's not much to it

jjttjj17:11:25

@U0FR867U1 I have a very "dumb" version as well by just shoving a datascript db in a duratom from here https://github.com/jimpil/duratom

jjttjj17:11:57

(ns durascript
  (:require [datascript.core :as d]
            [datascript.db :as ddb]
            [duratom.core :as duratom]
            [taoensso.nippy :as nippy]
            [ :as io]
            [clojure.edn :as edn]
            [duratom.utils :as duratom-utils])
  (:import ( PushbackReader)))

(nippy/extend-freeze clojure.lang.Atom
                     ::atom ; A unique (namespaced) type identifier
                     [x data-output]
                     (.writeUTF data-output (pr-str @x)))

(nippy/extend-thaw ::atom ; Same type id
                   [data-input]
                   (atom (edn/read-string  (.readUTF data-input))))

(nippy/extend-freeze datascript.db.DB
                     ::DB ; A unique (namespaced) type identifier
                     [x data-output]
                     ;;(println "type:" (type data-output))
                     ;;(.writeUTF data-output (pr-str x))
                     (#'nippy/write-utf8-lg data-output (pr-str x))
                     )

(nippy/extend-thaw
 ::DB ; Same type id
 [data-input]
 (edn/read-string {:readers d/data-readers}
                  (#'nippy/read-utf8-lg data-input)))

(defn read-db-edn-from-file!
  "Efficiently read large data structures from a stream."
  [source]
  (with-open [r (PushbackReader. (io/reader source))]
    (edn/read {:readers d/data-readers} r)))

(defn edn-db [file-path]
  (with-meta (duratom/duratom :local-file
                              :file-path file-path
                              :init (ddb/empty-db)
                              :rw {:read  read-db-edn-from-file!
                                   :write duratom-utils/write-edn-to-file!} )
    {:listeners (atom {})}))


(defn nippy-db [file-path & [schema]]
  (with-meta (duratom/duratom :local-file
                              :file-path file-path
                              :rw {:read  nippy/thaw-from-file
                                   :write nippy/freeze-to-file}
                              :init (if schema
                                      (ddb/empty-db schema)
                                      (ddb/empty-db)))
    {:listeners (atom {})}))

jeremys19:11:43

@U0D4G0Q4U I am starting to see how you do it now that my brain had time to process the info... @U064UGEUQ Thx, th aproach is an interesting one also I will think about it too.

thedavidmeister01:11:12

yeah, there's always pr-str, lol

thedavidmeister01:11:23

depends if you want to somehow preserve historical info or not

jeremys19:11:48

True and pr-str isn’t far from the transit solution... The thing is, it would be nice to be able to embed datomic in an app without having to start a transactor in another process and having it running in JS instead of Java for electron apps... Short of that I am seeing what can be done with datascript.

thedavidmeister02:11:08

@U0FR867U1 you can run datascript in clj and cljs, so you have a lot of freedom there

thedavidmeister02:11:30

as long as you don't want to try to do distributed state, or historical queries

thedavidmeister02:11:56

then persistence is very simple with datascript and you should keep it that way 🙂

jeremys20:11:47

@souenzzo Thx mate it might be enough to make save files in a desktop app and keeping the state in a db.