Fork me on GitHub
#asami
<
2021-02-19
>
quoll03:02:36

BTW, I’m not at work tomorrow

pithyless17:02:32

In https://github.com/threatgrid/asami/wiki/Entity-Structure:

(#datom [:tg/node-10499 :db/ident :tg/node-10498 1 true]
 #datom [:tg/node-10499 :tg/entity true 1 true]
 #datom [:tg/node-10499 :name "Fitzwilliam" 1 true]
 #datom [:tg/node-10499 :home "Pemberley" 1 true])
Am I correctly assuming that's a typo and should be:
#datom [:tg/node-10499 :db/ident :tg/node-10499 1 true]

quoll16:02:13

Fixed this. Thank you for the feedback!

pithyless16:02:03

No problem, I just was trying to grok what the :db/ident was (and how it differs from Datomic's :db/ident); and the example made me do a double-take. Two questions that I had unanswered after reading the docs: 1. Why have both :db/id and :db/ident, if I can explicitly set :db/id myself and :db/ident is also treated as a global identifier of a single node? Is it an indexing issue? Or are there certain api functions that expect ident, but not id? For example, I checked MemoryDatabase d/entity but it actually considers both as valid inputs. 2. Is there any interest in supporting a lookup ref syntax ala Datomic (e.g. [:email ""] ) in the future? Or is that considered out of scope? This also came up as I tried to understand Asami's identities. All I could find was a closed issue without a followup: https://github.com/threatgrid/asami/issues/97

quoll17:02:25

I’ll address the #1 to start with: It’s a little different to Datomic. :db/ident is an explicit attribute added to entities. It can be any value. If you don’t supply one, then Asami allocates it, defaulting to using a loopback on the node. For the in-memory value, that node is represented by a keyword with a prefix of :tg/node-. In Datomic, you might explicitly state that you want a node using datomic.Peer/tempid, and after it’s inserted then it looks like a number (distinguished from actual long values be appearing in the “entity” position of a statement, or if it’s in the “value” position, then it gets determined by the attribute datatype). Asami’s in-memory store just uses these magic keywords. (the on-disk store… which is Real Soon Now… uses long values internally, not keywords). Anyway, the :db/ident will either be what you specify, or it will refer to itself. :db/id is different. It is an implicit attribute that does not appear in the database. Instead, it’s used to refer to the entity that is represented by the node. To explain this, I want to explicitly describe the entity structures in the graph (I realize that you’ll know a lot of it, but I want to make sure we’re in the same place). Consider a simple entity:

{:db/ident "simple"
 :foo "bar"}
This has 2 attributes: :db/ident and :foo. To represent this in a graph, I need to have a node that will represent them. Let’s call that node my-entity. The graph for this entity can then be specified with the edges:
[my-entity :db/ident "simple"]
[my-entity :foo "bar"]
Allocating a node to represent structures like this means that we can also build nested structures:
{:db/ident "nested"
 :foo "hello"
 :bar {:foo "world"}}
The top level structure will be allocated a node (call it outer) and the nested structure will be allocated its node (call it inner):
[outer :db/ident "nested"]
[outer :foo "hello"]
[outer :bar inner]
[inner :foo "world"]
Of course, in an in-memory database in Asami, then outer may be :tg/node-1 and inner might be :tg/node-2 The question is, how can I refer to the node that represents an entity if I am just using the entity map style of structure? The entity has various attributes (like :db/ident and :foo, but no direct way to refer to the node itself. This is what :db/id does. In fact, Datomic uses :db/id to do exactly the same thing when inserting. So if I say that I want to insert an entity of:
{:db/id :my-marvelous-entity
 :db/ident "mine"
 :foo "hello"
 :bar {:foo "world"}}
Then the statements that this will be turned into are:
[:my-marvelous-entity :db/ident "mine"]
[:my-marvelous-entity :foo "hello"]
[:my-marvelous-entity :bar :tg/node-3]
[:tg/node-3 :foo "world"]
I can even add a :db/id to the nested entity.

quoll17:02:10

I hadn’t really thought about the ref syntax, but it should be doable. It’s similar to using :db/ident in that it has to do a lookup. I just need to make sure I don’t forget any codepaths that could be affected by it

quoll17:02:05

Actually… :db/ident is already a kind of lookup ref, so the machinery is basically there

quoll17:02:00

The main difference is that I used the {:db/ident value} syntax for that, instead of [:db/ident value] (I don’t recall when Lookup Refs were introduced into Datomic, but they weren’t there early on. Asami reflects an older set of APIs in Datomic)

pithyless17:02:05

Yeah, thanks for the explanation. The way I see it, the primary difference is :db/ident in Asami is a "global" lookup ref and in Datomic is a "namespaced" lookup ref.

quoll17:02:56

I’ll probably just do the lookup ref, look up the data, and if there’s more than one value, throw an ex-info

quoll17:02:13

That’ll offer the best compatibility, I think

pithyless17:02:21

I was wondering if perhaps it's more difficult, because of the open-world assumption and no schema that explicitly states that some attribute will be used as a reference lookup-ref

pithyless17:02:32

But offering a way to do a lookup + throwing error if more than one exists may be a nice way for compatibility.

pithyless17:02:33

This is my understanding of the difference:

;; Transactions - Datomic - need to make sure :person/name is unique for people

 [{:db/id "parent"
   :person/name "Jill"}
  {:person/name "Susie"
   :person/parent "parent"}]

 ;; or
 [{:person/name "Susie"
   :person/parent {:person/name "Jill"}}]

 ;; or, assuming Jill already exists...
 [{:person/name "Susie"
   :person/parent [:person/name "Jill"]}]

pithyless17:02:44

;; Transactions - Asami - need to make sure :db/ident is unique for all datoms

 [{:db/ident "jill"
   :person/name "Jill"}
  {:person/name "Susie"
   :person/parent {:db/ident "jill"}}]

 ;; or
 [{:person/name "Susie"
   :db/ident "susie"
   :person/parent {:db/ident "jill"
                   :person/name "Jill"}}]

 ;; or, assuming Jill already exists...
 [{:person/name "Susie"
   :db/ident "susie"
   :person/parent {:db/ident "jill"}}]

pithyless17:02:01

;; Datomic - queries
 [?e :person/parent [:person/name "Jill"]
  ?e :person/name ?name]

 ;; Asami - queries
 [?p :db/ident "jill"
  ?e :person/parent ?p
  ?e :person/name ?name]

quoll17:02:21

Are you sure about queries allowing that?

quoll17:02:06

From the Datomic docs: > Lookup refs have the following restrictions: > - The specified attribute must be defined as either :db.unique/value or :db.unique/identity. > - When used in a transaction, the lookup ref is evaluated against the specified attribute’s index as it exists before the transaction is processed, so you cannot use a lookup ref to lookup an entity being defined in the same transaction. > - Lookup refs cannot be used in the body of a query though they can be used as https://docs.datomic.com/on-prem/query/query.html#multiple-inputs.

quoll17:02:21

That last thing suggests that you can’t using them in a query

pithyless17:02:24

I'm pretty sure queries allow it, because we write a lot of code that depends on that sugar syntax :]

pithyless17:02:37

(sorry, need to go afk ~1h)

quoll17:02:40

It’s a reasonably easy transformation on the query, but a surprising one, given that there is explicit documentation that says that you can’t do it 🙂

pithyless18:02:14

@U051N6TTC from the Datomic docs: > Resolving Entity Identifiers in V Position > Datomic performs automatic resolution of https://docs.datomic.com/on-prem/schema/identity.html#entity-identifiers, so that you can generally use entity ids, idents, and lookup refs interchangeably. https://docs.datomic.com/on-prem/query/query.html#limitations

pithyless18:02:28

ah, I see now: > Lookup refs cannot be used in the body of a query though they can be used as https://docs.datomic.com/on-prem/query/query.html#multiple-inputs.

quoll18:02:41

I think it might actually be easier to just let them through in a query. So instead of their example of:

(q '[:find ?artist-name
     :in $ ?country
     :where [?artist :artist/name ?artist-name]
            [?artist :artist/country ?country]]
   db [:country/name "Belgium"])
it would actually be easier for me to just accept this instead:
(q '[:find ?artist-name
     :in $
     :where [?artist :artist/name ?artist-name]
            [?artist :artist/country [:country/name "Belgium"]]]
   db)
If I accept that, then it would automatically support the parameter query as well

pithyless18:02:57

^ I just verified in a REPL both versions of that kind of query work in Datomic (assuming :artist/country is a unique identity)

pithyless18:02:11

so apparently the docs are a little misleading (or not up-to-date)

quoll18:02:19

probably not up to date

quoll18:02:26

Datomic’s API keeps evolving

quoll18:02:52

Which is one reason for certain missing features. I haven’t kept up with Datomic over time

pithyless18:02:13

But maybe it changed b/c of this? > Allow lookup refs for V position in users of VAET index, including :db.fn/retractEntity. https://docs.datomic.com/on-prem/changes.html#0.9.4766

pithyless18:02:22

I've been using them like that in queries since I can remember; although your links to the docs had started to make me doubt my sanity. 😅

quoll18:02:02

That’s OK

quoll18:02:51

I was using Datomic several years before Lookup refs came out. I’ve learned a few new things since, but I haven’t put the time in to learn everything

quoll18:02:00

(they came out in February 2014)

quoll18:02:38

Anyway, it’s not going to be top of my queue, but I’ve added this: https://github.com/threatgrid/asami/issues/112

👍 3
pithyless18:02:42

Thanks for taking the time and sorry for the long tangents! 🙂

quoll19:02:24

No problem. I want to interact with people more, even if it’s just to convince everyone that the project is alive 🙂

quoll19:02:31

… and responsive

quoll18:02:43

Huh... yes. I must have copy pasted from something else and then tried to hand tweak for consistency

quoll18:02:02

I’m in a car right now. I’ll try to fix when I get home