datomic 2020-03-05 | Slack Archive

lilactown00:03:06

this is sort of related to datomic, so I thought I might find people who knew here: is it possible to statically analyze a query and always know all of the attributes that a query depends on?

favila05:03:22

#Also sent to the channel

No, as attrs to match can be input, dynamically built, or you can call an arbitrary function

lilactown15:03:54

> dynamically built can you show me what you mean by that? the other two make sense

favila15:03:08

[?e :use-attr ?attr] [?e ?attr ?v]

😲 4

lilactown15:03:32

got it. thank you!

favila15:03:38

[(keyword ?foo ?bar) ?attr] [?e ?attr ?v]

favila15:03:21

[(rand-int 1 1000) ?attr] [?e ?attr ?v] 😏

lilactown15:03:30

hahaha

favila05:03:22

replied to a thread:this is sort of related to datomic, so I thought I might find people who knew here: is it possible to statically analyze a query and always know all of the attributes that a query depends on?

No, as attrs to match can be input, dynamically built, or you can call an arbitrary function

dmarjenburgh09:03:52

@lvbarbosa @favila I’m struggling with the same problem, but I want to match on multiple attributes and the matches to exclude is stored in datomic as well (not as separate input). E.g. a list of items:

[
 {:item/department "D1" :item/type "A"}
 {:item/department "D1" :item/type "B"}
 {:item/department "D1" :item/type "C"}
 {:item/department "D2" :item/type "B"}
 {:item/department "D3" :item/type "A"}
 {:item/department "D3" :item/type "C"}
 ]

And a list of tuples with [dep type]s to hide:

{:item/hidden [["D1" "A"] ["D3" "C"]]}

I got it working with the following query:

(d/q {:query '[:find (pull ?item [...])
               :where
               [?item :item/department ?dep]
               [?item :item/type ?type]
               [(tuple ?dep ?type) ?dep+type]
               [(q '[:find (set ?hidden)
                     :where [_ :item/hidden ?hidden]] $) [[?results]]]
               (not [(contains? ?hidden ?dep+type)])]
      :args [db]})

But I’m not sure about using the set function in the find clause of the subquery (it’s not documented). And I’m not sure if there is an easier/more performant way to do it

favila14:03:43

use distinct instead of set (honestly it might just be an alias for set) https://docs.datomic.com/cloud/query/query-data-reference.html#built-in-aggregates

favila14:03:44

Your query works? I don’t see how the ?hidden in your last clause is bound

favila14:03:34

I would move building the hidden set up higher. you can also issue two queries

favila14:03:50

supplying the output of one as the input to the next

favila14:03:26

this isn’t great because it can’t make use of indexes

dmarjenburgh15:03:02

I adjusted my actual case and typed it over, so it has an error. The ?hidden should be ?results.

dmarjenburgh15:03:48

Does distinct always yield a set? I assumed it would be a seq, like clojure.core/distinct.

dmarjenburgh15:03:54

Thanks for the feedback, I’ll try different approaches to see if there is a performance difference

favila15:03:41

consider an initial filter using the most selective part of the tuple

favila15:03:57

so that you can make use of any value indexes on item-department or item-type

favila15:03:18

alternatively, make a composite index and match against that instead

favila15:03:30

(probably a better option anyway)

hawkey16:03:28

hi, does someone know how to get statistics of Datomic database (total size, size by entity, index size …)?

Joe Lane16:03:15

@hawkey Using the Client API: https://docs.datomic.com/client-api/datomic.client.api.html#var-db-stats

marshall21:03:55

that sounds like a uniqueness violation

marshall21:03:28

does the schema of the one that didnt “repurpose” include a uniqueness attribute?

joshkh07:03:00

the failed repurposed ident was originally claimed by a ref attribute which was included in a unique composite tuple. i removed the unique constraint on that tuple, but i still get the same exception.

joshkh07:03:53

looking at the alias, and the tuple attr which refers to that alias, neither have a unique constraint

joshkh08:03:52

so the whole thing looks more like this: 1. :player/details, claimed by a ref attribute, aliased to :new-player/details 2. :player/details+region tuple attribute aliased to :new-player/details+region 3. unique by identity constraint removed from :new-player/details+region tuple attribute 4. transact {:db/ident :player/details :db/valueType :db.type/ref :db/cardinality :db.cardinality/one}

Ident :player/details cannot be used for entity 666, already used for :new-player/details

2020-03-05

Channels