Fork me on GitHub

Hey everyone


I was wondering if somebody has already experimented with persisting datascript DBs on the file system or on something like sqllite.


@U0FR867U1 i just send datoms to my db, indexed on t, grouped by e+a


i pull out the "latest" v's for each e+a and send them all back to datascript when needed


Hey man thanks for the reply. I am sure there are more subtleties to your method though. I’ll keep it in mind.


it gets trickier if you want to share your datoms across different users/devices


but if it's 1:1 there's not much to it


@U0FR867U1 I have a very "dumb" version as well by just shoving a datascript db in a duratom from here


(ns durascript
  (:require [datascript.core :as d]
            [datascript.db :as ddb]
            [duratom.core :as duratom]
            [taoensso.nippy :as nippy]
            [ :as io]
            [clojure.edn :as edn]
            [duratom.utils :as duratom-utils])
  (:import ( PushbackReader)))

(nippy/extend-freeze clojure.lang.Atom
                     ::atom ; A unique (namespaced) type identifier
                     [x data-output]
                     (.writeUTF data-output (pr-str @x)))

(nippy/extend-thaw ::atom ; Same type id
                   (atom (edn/read-string  (.readUTF data-input))))

(nippy/extend-freeze datascript.db.DB
                     ::DB ; A unique (namespaced) type identifier
                     [x data-output]
                     ;;(println "type:" (type data-output))
                     ;;(.writeUTF data-output (pr-str x))
                     (#'nippy/write-utf8-lg data-output (pr-str x))

 ::DB ; Same type id
 (edn/read-string {:readers d/data-readers}
                  (#'nippy/read-utf8-lg data-input)))

(defn read-db-edn-from-file!
  "Efficiently read large data structures from a stream."
  (with-open [r (PushbackReader. (io/reader source))]
    (edn/read {:readers d/data-readers} r)))

(defn edn-db [file-path]
  (with-meta (duratom/duratom :local-file
                              :file-path file-path
                              :init (ddb/empty-db)
                              :rw {:read  read-db-edn-from-file!
                                   :write duratom-utils/write-edn-to-file!} )
    {:listeners (atom {})}))

(defn nippy-db [file-path & [schema]]
  (with-meta (duratom/duratom :local-file
                              :file-path file-path
                              :rw {:read  nippy/thaw-from-file
                                   :write nippy/freeze-to-file}
                              :init (if schema
                                      (ddb/empty-db schema)
    {:listeners (atom {})}))


@U0D4G0Q4U I am starting to see how you do it now that my brain had time to process the info... @U064UGEUQ Thx, th aproach is an interesting one also I will think about it too.


yeah, there's always pr-str, lol


depends if you want to somehow preserve historical info or not


True and pr-str isn’t far from the transit solution... The thing is, it would be nice to be able to embed datomic in an app without having to start a transactor in another process and having it running in JS instead of Java for electron apps... Short of that I am seeing what can be done with datascript.


@U0FR867U1 you can run datascript in clj and cljs, so you have a lot of freedom there


as long as you don't want to try to do distributed state, or historical queries


then persistence is very simple with datascript and you should keep it that way 🙂


@souenzzo Thx mate it might be enough to make save files in a desktop app and keeping the state in a db.