This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-01-30
Channels
- # announcements (15)
- # asami (26)
- # babashka (10)
- # babashka-sci-dev (18)
- # beginners (81)
- # biff (6)
- # calva (6)
- # cider (1)
- # clerk (1)
- # clj-kondo (34)
- # clojure (50)
- # clojure-belgium (1)
- # clojure-berlin (1)
- # clojure-europe (20)
- # clojure-nl (1)
- # clojure-norway (22)
- # clojure-uk (2)
- # clojurescript (1)
- # clr (4)
- # community-development (3)
- # data-science (8)
- # datomic (3)
- # gratitude (1)
- # honeysql (6)
- # instaparse (2)
- # jobs (1)
- # jobs-discuss (13)
- # kaocha (7)
- # london-clojurians (1)
- # lsp (6)
- # malli (8)
- # matcher-combinators (9)
- # missionary (3)
- # nbb (8)
- # off-topic (20)
- # pathom (16)
- # polylith (2)
- # practicalli (3)
- # rdf (1)
- # re-frame (7)
- # reagent (3)
- # releases (2)
- # reveal (6)
- # rewrite-clj (22)
- # shadow-cljs (64)
- # tools-build (7)
- # xtdb (13)
Hi! Asami doesn’t expose a way to access the metadata of its transactions, right? I can get the current version number w/ as-of-t
or timestamp w/ as-of-time
of the db and I can get a historical snapshot of the db based either on the version number or a date time, And I can get the list of versions (i.e. 0 … the current one). But I cannot find out what transaction/version/timestamp a datom was transacted/retracted at. So e.g. if I want to find out when a datom was transacted I’d need to search through the past versions of the db to find the first one that has it. Or am I overlooking something?
Every statement is given an ID when it is inserted. This is a monotonically increasing long.
Transactions occur at particular ID points. So you know which transaction a statement was inserted in by its ID
That is great to hear. Thank you
The plan is for queries to allow patterns of:
[?e ?a ?v ?t]
where ?t
will be bound to the transaction ID, and:
[?e ?a ?v :as ?s]
where ?s
will be bound to the statement ID
Patterns can be truncated, which is to say, the ?t
is optional. Also, it can still be included in a full pattern;
[?e ?a ?v ?t :as ?s]
To explain why Asami progress has been slow lately:
• I had used clojure.core/format
in error messages, but this is not supported in ClojureScript. So I wrote https://github.com/quoll/clormat
• I wanted to load work data into Asami, so I wrote https://github.com/quoll/raphael
• I want to write that back out (to turtle format), so I wrote a separate piece of code (not yet complete) which I’m calling https://github.com/quoll/raphael
• I am still writing my SPARQL parser, which I’ve named Twylyte (not much pushed yet)
I need to integrate all of these, but I’m trying to make them as modular as I can, and putting them in separate packages helps enforce that.
I wonder, are you fan of ninja turtles or Italian artists. Sounds like the former... :-)
One of my previous projects had already been called Yertle :rolling_on_the_floor_laughing:
Meanwhile, I’ve been asked to write a Clojure workshop on data structures, which I’ve spent my last few weeks on. That’s nearly done.
Anyway… except for the workshop, I’ve continued work on this. I just haven’t released it yet.
Oh, and I’m partway through integrating core to implement with
. It works differently with in-memory and on-disk, and I need to ensure the behavior is the same, so it’s taking time
Wow, that is lot of work you have done (and still have ahead of you)! Good luck!
In terms of schedule, I have to finish this workshop, but I’m wrapping up on it now. Then I’ll be looking to finish the with
implementation, and the integration of Raphael and Donatello (i.e. importing and emitting TTL). Then I’ll be working on finishing SPARQL. I don’t need everything there, but I want the basics in place. Then I’ll be expanding the query patterns to include transaction and statement IDs.
The next items on the list are: • Adding metadata to results to indicate if they are “local” or “global”. Local results are groups of numbers. Global results are the same data converted to Strings/keywords/URIs/etc. Right now, everything is global, but keeping everything local until final projection will make for much faster querying. • Writing a B-Tree for the Data Pool. I still like the AVL-tree for the triple indices, but the data pool has different access characteristics. • Building an in-block Patricia trie implementation. This is needed for full-text indexing. Yes, I know that everyone uses Lucene (including AWS with ElasticSearch/OpenSearch), but this doesn’t work for ClojureScript, nor does it give me storage flexibility. • Implementing block storage in Redis, and not just in memory-mapped file. • Implementing block storage in Postgres and MySQL
:rolling_on_the_floor_laughing:
@quoll if you’d also be open to use an existing lib for SPARQL https://github.com/yetanalytics/flint might be worth a look
oh, I now see it mentions Asami in the README: Flint borrows certain syntactic conventions from the Datomic and Asami query and update languages.
Is there a way to update a value atomicaly in Asami? In PostgreSQL I could do set column = column + 1
but for Asami I either have to hope nothing else changed the DB between I read the current value and compute the next one or use something like transact-update instead of transact-data, right?