Datascript is normally used in ClojureScript web-apps as an in-memory DB where you maintain the app state (like the UI/UX state). Datahike is used for very small applications, it be a good candidate for a desktop app written in Clojure. Or a client/server app, but with very small number of concurrent users. So maybe internal tooling and such. It doesn't scale to large dataset and it runs on a single machine, so it is made for client/server apps where your server is expected to run in a single node, thus non-distributed. Don't expect it to be resilient either, it won't replicate itself or anything, so have backups. In neither case is PostgressSQL involved. With Datascript, sometimes people have POstgressSQL as their backend DB to their webapp, and DataScript as the client-side in-memory DB used by the client.

Otherwise, if you need high scale, resiliency, fault tolerance, you are looking at using Datomic on the backend or Crux. Those would replace your use of PostgressSQL.

So, best to think of it this way: For server backend: - Where you'd use SQLite -> use Datahike or Datalevin - Where you'd use PostgressSQL, MongoDB, etc. -> use Datomic or Crux For client-side apps: - Where you'd use SQLite -> use Datahike or Datalevin or DataScript (where you'd save/load to/from EDN on disk) - Where you'd just store data in-memory in atoms or variables -> use DataScript or Asami

Or you can always choose to also just use SQLite, PostgressSQL, H2, and other relational databases. The above is if you'd prefer using a Datalog based more graph oriented DB instead.

@U0K064KQV this is exactly the kind of answer i was looking for! Very helpful in wrapping my head around where these tools fit into the stack.

I’m looking to try a graph oriented db for a project I’m starting. It’s a relatively standard web app with a user interface in the browser and needs auth and data store. Crux sounds like the best free option?

Well, I'd say it depends what this project is. Like if its a hobby, you can start with Datalevin or Datahike. That will give you a chance to learn the whole Datalog thing. And then if you need to scale, you could afterwards consider switching to Crux.

And if your browser user interface is a SPA, you can try using DataScript in it.

Yes it’s a hobby project with no users yet so the datahike idea might be good to try. And yes it’s an SPA in react so I think I’d like to try datascript as well.

One question about hooking the client up to the server - do you just use standard JSON over http to communicate, and then stick the parsed data into datahike/datascript?

Its more typical to use Transit when your use case is Clojure backend and ClojureScript frontend, and you don't intend other languages to be using the APIs. You should look into: https://github.com/metosin/muuntaja/ [preferred most commonly used] https://github.com/ngrunwald/ring-middleware-format [simpler, you could just use EDN directly with this one, only downside is slower than muuntaja]

The advantage of Transit over raw JSON is that Transit can encode all the Clojure standard types like keywords, where-as JSON can't.

Using EDN directly is nice, but the disadvantage is the compression and performance of serializing/deserializing JSON isn't as optimized as Transit.

A good place to start on the server would be with: ring/ring-core ring/ring-jetty-adapter metosin/reitit metosin/muuntaja

you’re a saint @U0K064KQV this is a treasure trove of info for a beginner and gives me a solid foundation and direction to work in. Many thanks! 🙂

Asami also has disk storage now btw

Cool!

I know that I will have many false negatives, but it is worth it to ensure the files from my workflows are up to date.

yes certainly, for this simple case i didn’t want to introduce any hypothetical scenario just for the sake of a solution, but a construct like

(defn xif [f if-xform else-xform])

i wonder if i am hitting the limitation of transducers here? or is this possible?

tried looking at xforms library. it didn’t have anything either

actually i wonder if it even makes sense

e.g. if it was then what would the following even do if some-branching-condition kept alternating for each element

(comp
 (map parse-string)
 (xif some-branching-condition
      (comp
       (take 10)
       (filter extra-work)
       (map some-work))
      (map some-work))
 (map process))

my bad, some-branching-condition is a function of the input

oh - I misunderstood here, deleting my irrelevant suggestions

i don’t think it makes much of a sense except for simple branches as explained here https://clojurians.slack.com/archives/C03S1KBA2/p1623687952379900?thread_ts=1623687432.378800&cid=C03S1KBA2

i was probably trying to squeeze too much out of transducers 😅

from a higher level vantage point, it's much easier to use comp / fn first then optimize with transducers later

doing the transducing part first puts more demands on your code reading ability, is much more error prone, and can't do anything functions can't, except for doing it faster

(defn if-tx [test-fn true-xf false-xf]
  (fn [rf]
    (let [t (true-xf rf)
          f (false-xf rf)]
      (fn
        ([] (rf))
        ([result] (f (t result)))
        ([result input]
         (if (test-fn input)
           (t result input)
           (f result input)))))))

(comment

  (sequence (comp (map inc)
                  (if-tx even? (map str) (map double)))
            (range 10))

  )

i certainly won’t use that 😛 but i learnt something about the inner working of transducers today (that let binding)

I do keep thinking that it would be nice to have more expressive ways of combining transducers together ... but I can't say for certain that I've really got that problem ... I keep having doubts that I'm just creating a big mess and just writing the straight code would be easier to read and work with ... I guess it depends on how much you end up sharing transducers across modules (or whatever)

i am actually using core async which is why I'm leveraging transducers more often. without them i end up something like https://blog.golang.org/pipelines

which is fine

anything that gets difficult via transducers can be easily plucked out into its own function that accepts from an in chan and outputs to an out chan

The transducer loops one element at a time and applies all transforms. So you can just do:

(sequence (comp (map inc)
                (map #(if (even? %) (str %) (double %))))
          (range 10))

that's a good call - there's no reason the function inside map can't contain a conditional

hey look, I wrote stuff down... https://puredanger.github.io/tech.puredanger.com/2009/06/10/clojure-rich-hickey/