Fork me on GitHub
#datascript
<
2017-07-27
>
thedavidmeister11:07:57

@tonsky i just used datascript in clj for the first time today

thedavidmeister11:07:02

glad it exists! šŸ™‚

thedavidmeister11:07:24

i used it with joplin to migrate some eavt data stored in another db

Niki11:07:29

Awesome!

kurt-o-sys12:07:04

Yeah, it's cool... I'm busy writing a datascript based 'database' to use server-side. (https://gitlab.com/kurtosys/lib/factoidic)

hmaurer12:07:41

What will be its advantage over Datomic in-memory?

kurt-o-sys12:07:11

Well... the use case is: small applications that can't afford Datomic - the license is only 1 year, unless the free version, but that one requires a peer and transactor. For many applications, it's not necessary.

kurt-o-sys12:07:26

It'll not be in-memory, btw.

kurt-o-sys12:07:32

It'll have durable storage.

kurt-o-sys12:07:46

I'd prefer to use Datomic, but their licensing and pricing model is, in the use case I describe, not sustainable.

kurt-o-sys12:07:15

or: much easier r to set-up, free and embedded.

hmaurer12:07:16

@U0E0XL064 I can relate to this. Iā€™m currently on the way to use Datomic in a small, non-profit application. Will your API be compatible with Datomicā€™s API? (e.g. to allow smooth transition)

kurt-o-sys12:07:35

yes... it'll use the Datascript engine

kurt-o-sys12:07:44

Which is pretty much the same as Datomic.

kurt-o-sys12:07:32

I'm checking performance now - it's not really bad.

kurt-o-sys12:07:28

and yes, small non-profits is where I'm aiming at, they don't need millions of writes per minute etc šŸ™‚.

kurt-o-sys13:07:32

First trial: writing about 15000-20000 datoms/s right now, which is not bad at all. Will need more tests, though.

hmaurer14:07:54

@U0E0XL064 good enough for small-medium apps

kurt-o-sys15:07:00

@U5ZAJ15P0 yeah, well, writes shouldn't be the problem. Not that many systems do 1000s of writes/s. I have to check on the reads - but a lot will depend on the query (and how the user optimizes them). Read performance should be the same as datascript read (query, pull, ...) performance. I will try on a not-so-small synthetic dataset.

kurt-o-sys18:07:13

adding more datoms/transaction increases throughput a lot (roughly: # datoms x4 -> throughput x2). Batching could be a good idea in very high throughput systems.

hmaurer18:07:00

Out of curiosity, is Datomicā€™s technology patented in any way? e.g. could one get into trouble for building a ā€œcompetingā€ system that uses the exact same concepts and API?

kurt-o-sys18:07:42

don't think so... I mean, I don't copy the same thing: it's a datascript engine + durability of all transactions. There are no peers and many other features that are there in Datomic.

kurt-o-sys18:07:16

Moreover, it may make the step to datalog much easier... meaning, people my start with a more simple system, and switch to Datomic when necessary.

kurt-o-sys18:07:51

I just can't switch back to something else... I'm addicted to datalog šŸ™‚.

kurt-o-sys18:07:18

and I miss the small free embedded thing.

hmaurer18:07:50

I 100% understand the addiction to Datalog

kurt-o-sys18:07:00

But I'm not copying Datomic's architecture or so.

kurt-o-sys18:07:26

Anyway... they didn't want to adapt pricing for non-profits, so well, I had to do something šŸ™‚

kurt-o-sys18:07:40

> ... In terms of pricing, unfortunately we do cannot offer general discounts for Datomic Pro past the initial 12-month trial period. ...

myguidingstar18:08:47

@U0E0XL064 I've read factoidic README. It says factoidic will allow time-travel. Does that mean you are going to save retract datoms to durable storage and modify query engine so that it will take retract datoms into account?

kurt-o-sys18:08:59

Right, almost: I will store all the transactions. I don't have to change the query engine, all I have to change is to change the database (or database value) the query acts upon. Meaning: if you say: give me the state at time x, it will - not implemented yet - give you the database at time x and you can query that one.

kurt-o-sys18:08:39

it pretty similar to the way datomic works.

myguidingstar18:08:31

I'm also curious about your approach for durable storage. Will factoidic just copy from durable storage to memory and query against in-memory data, or will it use durable storage's filter (ie SQL where statements) instead?

kurt-o-sys18:08:29

it will construct an in-memory database (datascript). No SQL-queries.

kurt-o-sys18:08:58

It's really no rocket-science: it's datomic + append-only durable storage.

kurt-o-sys18:08:18

I hope I'll have time to finish a first version soon...

myguidingstar18:08:27

@U0E0XL064 btw, I think react native developers can make use of factoidic for mobile apps, so cljc may be the way to go. Of course no pressure, we can add it later šŸ™‚

kurt-o-sys18:08:42

lol, right šŸ™‚

kurt-o-sys18:08:51

well, the only thing that has to change is durability.

kurt-o-sys18:08:24

My first thought was to right to a file, but I replaced that to an embedded H2 db, to avoid opening/closing files etc.

myguidingstar18:08:12

thanks for your work, I'm sure factoidic will be useful for many not-so-big projects

kurt-o-sys18:08:21

Making durability more pluggable would make it pretty straight forward to implement for browsers, mobile, ...

kurt-o-sys18:08:15

If you have any ideas or requests, don't hesitate to add them... The pluggable durability is in my head somewhere, but I may have to add it somewhere in the README or wiki or ...