This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-21
Channels
- # announcements (3)
- # architecture (5)
- # babashka (19)
- # beginners (145)
- # calva (1)
- # chlorine-clover (18)
- # cider (10)
- # clj-kondo (5)
- # cljsrn (3)
- # clojure (56)
- # clojure-berlin (19)
- # clojure-czech (2)
- # clojure-europe (32)
- # clojure-finland (2)
- # clojure-nl (3)
- # clojure-portugal (2)
- # clojure-spec (8)
- # clojure-uk (48)
- # clojurescript (48)
- # conjure (50)
- # cryogen (1)
- # cursive (28)
- # datomic (54)
- # depstar (12)
- # emacs (1)
- # events (8)
- # figwheel-main (6)
- # fulcro (4)
- # helix (4)
- # java (2)
- # jobs (2)
- # leiningen (1)
- # off-topic (29)
- # parinfer (4)
- # pathom (6)
- # portkey (3)
- # reagent (1)
- # remote-jobs (3)
- # reveal (16)
- # shadow-cljs (42)
- # sql (20)
- # tools-deps (11)
- # vim (4)
- # vrac (2)
does the not=
predicate work for datalog queries? e.g.
(d/q '[:find ?uid ?order
:in $ ?parent-eid [?source-uids ...]
:where
[?parent-eid :block/children ?ch]
[?ch :block/uid ?uid]
[?ch :block/order ?order]
[(= ?order ?source-uids)]]
@db/dsdb 48 #{0 1 2})
works but
(d/q '[:find ?uid ?order
:in $ ?parent-eid [?source-uids ...]
:where
[?parent-eid :block/children ?ch]
[?ch :block/uid ?uid]
[?ch :block/order ?order]
[(not= ?order ?source-uids)]]
@db/dsdb 48 #{0 1 2})
does not work
to elaborate, =
works for both value and collection comparisons, whereas not=
only seems to work for value comparisonsthere's a datalog specific not impl here https://docs.datomic.com/on-prem/query.html#not-caluses
This still isn’t what I expect, but note that in datalog it’s more idiomatic to use =
and !=
(d/q '[:find ?uid ?order
:in $ ?parent-eid [?source-uids ...]
:where
[?parent-eid :block/children ?ch]
[?ch :block/uid ?uid]
[?ch :block/order ?source-uids]]
@db/dsdb 48 #{0 1 2})
(d/q '[:find ?uid ?order
:in $ ?parent-eid [?source-uids ...]
:where
[?parent-eid :block/children ?ch]
[?ch :block/uid ?uid]
(not [?ch :block/order ?source-uids])]
@db/dsdb 48 #{0 1 2})
(d/q '[:find ?uid ?order
:in $ ?parent-eid ?source-uids
:where
[?parent-eid :block/children ?ch]
[?ch :block/uid ?uid]
[?ch :block/order ?order]
[(contains? ?source-uids ?order)]]
@db/dsdb 48 #{0 1 2})
@U09R86PA4 your first two codeblocks make sense to me! I tried your third codeblock earlier (`contains?`) but datascript didn't recognize my custom predicate for negation. It was fully qualified but idk
Is there a way to get a ‘projection’ of a database? For authZ purposes, I would like to run queries on a db that only contains the set of datoms that were returned from a query
you could do what I have done for cloud. proxy the db/conn values and inject middleware at that layer. then you can implement your own filtering before or after the reads occur
@U0510KXTU interesting. With what data does your middleware stack work with? Are the filters queries themselves, the results of which are then used as input to the next query, and so on? It seems with the client api you could end up performing large scans of the database if your filter was relatively wide...
I went all out and added Pedestal interceptors in the proxy. The enter fns decorate the queries before execution with extra where clauses. In that way you can 1/ limit access 2/ maintain good performance
for writes, the enter fns check that all references can be read using the same filters as the reads
Ah clever, that’s a great use of queries as data. I can see how you could have a toolbox of interceptors for common things like [?e :user/id ?user-id-from-cookie]
Thinking more, you could even build :accessible/to
into the schema, and assert it onto entities to authorize access by the referenced entity (ie a user). That might be generalized into an interceptor more gracefully.
exactly. almost anything can be generalised with this design. It’s non-trivial but worth it imho
There aren’t any docs. This technique relies upon undocumented (i.e. unsupported) use of the api client protocols. you can see an example of this here https://github.com/ComputeSoftware/datomic-client-memdb/blob/master/src/compute/datomic_client_memdb/core.clj
in the (unlikely) event that Cognitect changes these protocols, you can always refactor using this technique (which is where I first tried the middleware idea) https://github.com/stevebuik/ns-clone
is there a more efficient way to find all entities Y with any tuple attribute that references X?
(d/q '{:find [?tuple-entity]
:in [$ ?target-entity]
:where [[?tuple-attr :db/valueType :db.type/tuple]
[?tuple-entity ?tuple-attr ?refs]
[(untuple ?refs) [?target-entity ...]]]}
db entity-id)
this runs in around ~500ms given a few hundred thousand ?tuple-entity
s which isn't too slow for its purpose, but i am worried that it won't scale with my datai'm working with one database that has been modelled in such a way that entities with tuple attributes that are unique are no longer "valid" when any one of their tuple reference values is retracted. one drawback to having unique tuples is that you can end up with {:enrollment/[player+server+board [p1 s1 nil]}
after retracting a board, and then any subsequent retraction to another course will fail due to a uniqueness constraint so long as there is another enrollment
for [p1 s1 b2]
.
i have implemented a business layer API for retracting different "kinds" of entities that cleanup any tuples known to be "about" them. but in my real data i have many, many different kinds of entities, and many tuples that could be about any one+ of them. so when adding a new tuple to the schema, or transacting an existing tuple that includes a new kind of entity, there is a feeling of technical debt when the developer must know which retraction API functions to update.
since the schema was designed in such a way that tuples should not exist with nil values, i was hoping for a "catch all" transactor function that can clean up related tuples without making complicated decisions about which ones to look for.
(another option i explored was having component references from all entities back to tuples so that they are automatically retracted, but this proves to be just as tedious on the other end when transacting new entities)
This is possible if an enrollment becomes invalid (i.e. should be completely retracted) if any of player, server, or board are not asserted
I think that’s what you mean by this:
entities with tuple attributes that are unique are no longer "valid" when any one of their tuple reference values is retracted
then you could query for [?referring ?attr ?e] (vaet index), see if the attr is one of your special ones, and if so, emit [:db/retractEntity ?referring]
> your own retractentities function as in an API layer function or a transactor level function?
You could look at tupletype membership, but I think it’s going to be less surprising to have either a hardcoded list or your own annotation on the attribute, e.g. :required-for-entity-validity?
agreed, and that's where i'm at. but if i'm understanding you correctly, the problem still stands of knowing which tuple attributes reference which entities if i want to shorten the list of possible matches. in my case, nearly any tuple can reference nearly any entity.
since you know you are retracting, if you retract an attribute which is a member of a tuple, you know the tuple is going to get a null in it, so you can retract the entire referring entity
I still think it’s probably safer to annotate/enumerate attributes which you want this cascading behavior on