This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-02-19
Channels
- # announcements (13)
- # asami (43)
- # babashka (35)
- # beginners (175)
- # calva (23)
- # cider (5)
- # clj-kondo (68)
- # cljsrn (4)
- # clojure (61)
- # clojure-australia (7)
- # clojure-europe (20)
- # clojure-gamedev (59)
- # clojure-israel (11)
- # clojure-italy (4)
- # clojure-nl (2)
- # clojure-norway (21)
- # clojure-spec (12)
- # clojure-uk (43)
- # clojurescript (9)
- # cursive (56)
- # data-oriented-programming (5)
- # datascript (1)
- # events (1)
- # fulcro (16)
- # honeysql (46)
- # leiningen (1)
- # malli (4)
- # off-topic (12)
- # pathom (46)
- # re-frame (24)
- # reagent (14)
- # reitit (1)
- # reveal (8)
- # rewrite-clj (16)
- # ring (13)
- # sci (9)
- # spacemacs (14)
- # specter (2)
- # sql (2)
- # tools-deps (1)
- # vim (2)
In https://github.com/threatgrid/asami/wiki/Entity-Structure:
(#datom [:tg/node-10499 :db/ident :tg/node-10498 1 true]
#datom [:tg/node-10499 :tg/entity true 1 true]
#datom [:tg/node-10499 :name "Fitzwilliam" 1 true]
#datom [:tg/node-10499 :home "Pemberley" 1 true])
Am I correctly assuming that's a typo and should be:
#datom [:tg/node-10499 :db/ident :tg/node-10499 1 true]
No problem, I just was trying to grok what the :db/ident
was (and how it differs from Datomic's :db/ident
); and the example made me do a double-take. Two questions that I had unanswered after reading the docs:
1. Why have both :db/id
and :db/ident
, if I can explicitly set :db/id
myself and :db/ident
is also treated as a global identifier of a single node? Is it an indexing issue? Or are there certain api functions that expect ident, but not id? For example, I checked MemoryDatabase d/entity
but it actually considers both as valid inputs.
2. Is there any interest in supporting a lookup ref syntax ala Datomic (e.g. [:email "
) in the future? Or is that considered out of scope? This also came up as I tried to understand Asami's identities. All I could find was a closed issue without a followup: https://github.com/threatgrid/asami/issues/97
I’ll address the #1 to start with:
It’s a little different to Datomic. :db/ident
is an explicit attribute added to entities. It can be any value.
If you don’t supply one, then Asami allocates it, defaulting to using a loopback on the node. For the in-memory value, that node is represented by a keyword with a prefix of :tg/node-
. In Datomic, you might explicitly state that you want a node using datomic.Peer/tempid
, and after it’s inserted then it looks like a number (distinguished from actual long
values be appearing in the “entity” position of a statement, or if it’s in the “value” position, then it gets determined by the attribute datatype). Asami’s in-memory store just uses these magic keywords. (the on-disk store… which is Real Soon Now… uses long values internally, not keywords). Anyway, the :db/ident
will either be what you specify, or it will refer to itself.
:db/id
is different. It is an implicit attribute that does not appear in the database. Instead, it’s used to refer to the entity that is represented by the node.
To explain this, I want to explicitly describe the entity structures in the graph (I realize that you’ll know a lot of it, but I want to make sure we’re in the same place). Consider a simple entity:
{:db/ident "simple"
:foo "bar"}
This has 2 attributes: :db/ident
and :foo
. To represent this in a graph, I need to have a node that will represent them. Let’s call that node my-entity
. The graph for this entity can then be specified with the edges:
[my-entity :db/ident "simple"]
[my-entity :foo "bar"]
Allocating a node to represent structures like this means that we can also build nested structures:
{:db/ident "nested"
:foo "hello"
:bar {:foo "world"}}
The top level structure will be allocated a node (call it outer
) and the nested structure will be allocated its node (call it inner
):
[outer :db/ident "nested"]
[outer :foo "hello"]
[outer :bar inner]
[inner :foo "world"]
Of course, in an in-memory database in Asami, then outer
may be :tg/node-1
and inner
might be :tg/node-2
The question is, how can I refer to the node that represents an entity if I am just using the entity map style of structure? The entity has various attributes (like :db/ident
and :foo
, but no direct way to refer to the node itself. This is what :db/id
does. In fact, Datomic uses :db/id
to do exactly the same thing when inserting.
So if I say that I want to insert an entity of:
{:db/id :my-marvelous-entity
:db/ident "mine"
:foo "hello"
:bar {:foo "world"}}
Then the statements that this will be turned into are:
[:my-marvelous-entity :db/ident "mine"]
[:my-marvelous-entity :foo "hello"]
[:my-marvelous-entity :bar :tg/node-3]
[:tg/node-3 :foo "world"]
I can even add a :db/id
to the nested entity.I hadn’t really thought about the ref syntax, but it should be doable. It’s similar to using :db/ident
in that it has to do a lookup. I just need to make sure I don’t forget any codepaths that could be affected by it
Actually… :db/ident
is already a kind of lookup ref, so the machinery is basically there
The main difference is that I used the {:db/ident value}
syntax for that, instead of [:db/ident value]
(I don’t recall when Lookup Refs were introduced into Datomic, but they weren’t there early on. Asami reflects an older set of APIs in Datomic)
Yeah, thanks for the explanation. The way I see it, the primary difference is :db/ident
in Asami is a "global" lookup ref and in Datomic is a "namespaced" lookup ref.
I’ll probably just do the lookup ref, look up the data, and if there’s more than one value, throw an ex-info
I was wondering if perhaps it's more difficult, because of the open-world assumption and no schema that explicitly states that some attribute will be used as a reference lookup-ref
But offering a way to do a lookup + throwing error if more than one exists may be a nice way for compatibility.
This is my understanding of the difference:
;; Transactions - Datomic - need to make sure :person/name is unique for people
[{:db/id "parent"
:person/name "Jill"}
{:person/name "Susie"
:person/parent "parent"}]
;; or
[{:person/name "Susie"
:person/parent {:person/name "Jill"}}]
;; or, assuming Jill already exists...
[{:person/name "Susie"
:person/parent [:person/name "Jill"]}]
;; Transactions - Asami - need to make sure :db/ident is unique for all datoms
[{:db/ident "jill"
:person/name "Jill"}
{:person/name "Susie"
:person/parent {:db/ident "jill"}}]
;; or
[{:person/name "Susie"
:db/ident "susie"
:person/parent {:db/ident "jill"
:person/name "Jill"}}]
;; or, assuming Jill already exists...
[{:person/name "Susie"
:db/ident "susie"
:person/parent {:db/ident "jill"}}]
;; Datomic - queries
[?e :person/parent [:person/name "Jill"]
?e :person/name ?name]
;; Asami - queries
[?p :db/ident "jill"
?e :person/parent ?p
?e :person/name ?name]
From the Datomic docs: > Lookup refs have the following restrictions: > - The specified attribute must be defined as either :db.unique/value or :db.unique/identity. > - When used in a transaction, the lookup ref is evaluated against the specified attribute’s index as it exists before the transaction is processed, so you cannot use a lookup ref to lookup an entity being defined in the same transaction. > - Lookup refs cannot be used in the body of a query though they can be used as https://docs.datomic.com/on-prem/query/query.html#multiple-inputs.
https://docs.datomic.com/cloud/transactions/transaction-data-reference.html#entity-identifiers
I'm pretty sure queries allow it, because we write a lot of code that depends on that sugar syntax :]
It’s a reasonably easy transformation on the query, but a surprising one, given that there is explicit documentation that says that you can’t do it 🙂
@U051N6TTC from the Datomic docs: > Resolving Entity Identifiers in V Position > Datomic performs automatic resolution of https://docs.datomic.com/on-prem/schema/identity.html#entity-identifiers, so that you can generally use entity ids, idents, and lookup refs interchangeably. https://docs.datomic.com/on-prem/query/query.html#limitations
ah, I see now: > Lookup refs cannot be used in the body of a query though they can be used as https://docs.datomic.com/on-prem/query/query.html#multiple-inputs.
I think it might actually be easier to just let them through in a query. So instead of their example of:
(q '[:find ?artist-name
:in $ ?country
:where [?artist :artist/name ?artist-name]
[?artist :artist/country ?country]]
db [:country/name "Belgium"])
it would actually be easier for me to just accept this instead:
(q '[:find ?artist-name
:in $
:where [?artist :artist/name ?artist-name]
[?artist :artist/country [:country/name "Belgium"]]]
db)
If I accept that, then it would automatically support the parameter query as well^ I just verified in a REPL both versions of that kind of query work in Datomic (assuming :artist/country
is a unique identity)
lookup-refs were introduced here: https://docs.datomic.com/on-prem/changes.html#0.9.4556
Which is one reason for certain missing features. I haven’t kept up with Datomic over time
But maybe it changed b/c of this?
> Allow lookup refs for V position in users of VAET index, including :db.fn/retractEntity
.
https://docs.datomic.com/on-prem/changes.html#0.9.4766
I've been using them like that in queries since I can remember; although your links to the docs had started to make me doubt my sanity. 😅
I was using Datomic several years before Lookup refs came out. I’ve learned a few new things since, but I haven’t put the time in to learn everything
Anyway, it’s not going to be top of my queue, but I’ve added this: https://github.com/threatgrid/asami/issues/112
No problem. I want to interact with people more, even if it’s just to convince everyone that the project is alive 🙂