there's a datalog specific not impl here https://docs.datomic.com/on-prem/query.html#not-caluses

This still isn’t what I expect, but note that in datalog it’s more idiomatic to use = and !=

not= is clojure.core/not=, but those two are not necessarily clojure’s

Also, why not this?

(d/q '[:find ?uid ?order
            :in $ ?parent-eid [?source-uids ...]
            :where
            [?parent-eid :block/children ?ch]
            [?ch :block/uid ?uid]
            [?ch :block/order ?source-uids]]
          @db/dsdb 48 #{0 1 2})

or this for the negation?

(d/q '[:find ?uid ?order
            :in $ ?parent-eid [?source-uids ...]
            :where
            [?parent-eid :block/children ?ch]
            [?ch :block/uid ?uid]
            (not [?ch :block/order ?source-uids])]
          @db/dsdb 48 #{0 1 2})

or, if you want to keep a set:

(d/q '[:find ?uid ?order
            :in $ ?parent-eid ?source-uids
            :where
            [?parent-eid :block/children ?ch]
            [?ch :block/uid ?uid]
            [?ch :block/order ?order]
            [(contains? ?source-uids ?order)]]
          @db/dsdb 48 #{0 1 2})

(which is faster in some cases)

@U09R86PA4 your first two codeblocks make sense to me! I tried your third codeblock earlier (`contains?`) but datascript didn't recognize my custom predicate for negation. It was fully qualified but idk

oh this is datascript?

yeah, not sure if custom predicates are different in that case

I know on-prem has something like this via d/filter but Cloud does not afaik

thanks

you could do what I have done for cloud. proxy the db/conn values and inject middleware at that layer. then you can implement your own filtering before or after the reads occur

it’s complex but works well

@U0510KXTU interesting. With what data does your middleware stack work with? Are the filters queries themselves, the results of which are then used as input to the next query, and so on? It seems with the client api you could end up performing large scans of the database if your filter was relatively wide...

(I think this may be part of the reason why the client api doesn’t support filter)

I went all out and added Pedestal interceptors in the proxy. The enter fns decorate the queries before execution with extra where clauses. In that way you can 1/ limit access 2/ maintain good performance

doesn’t work for d/pull so, in that case, I filter the data in the leave fn instead

for writes, the enter fns check that all references can be read using the same filters as the reads

Ah clever, that’s a great use of queries as data. I can see how you could have a toolbox of interceptors for common things like [?e :user/id ?user-id-from-cookie]

Thinking more, you could even build :accessible/to into the schema, and assert it onto entities to authorize access by the referenced entity (ie a user). That might be generalized into an interceptor more gracefully.

exactly. almost anything can be generalised with this design. It’s non-trivial but worth it imho

Where are the API docs for the proxy? I’m not finding anything

There aren’t any docs. This technique relies upon undocumented (i.e. unsupported) use of the api client protocols. you can see an example of this here https://github.com/ComputeSoftware/datomic-client-memdb/blob/master/src/compute/datomic_client_memdb/core.clj

in the (unlikely) event that Cognitect changes these protocols, you can always refactor using this technique (which is where I first tried the middleware idea) https://github.com/stevebuik/ns-clone

i knew someone would ask that 😉

i'm working with one database that has been modelled in such a way that entities with tuple attributes that are unique are no longer "valid" when any one of their tuple reference values is retracted. one drawback to having unique tuples is that you can end up with {:enrollment/[player+server+board [p1 s1 nil]} after retracting a board, and then any subsequent retraction to another course will fail due to a uniqueness constraint so long as there is another enrollment for [p1 s1 b2]. i have implemented a business layer API for retracting different "kinds" of entities that cleanup any tuples known to be "about" them. but in my real data i have many, many different kinds of entities, and many tuples that could be about any one+ of them. so when adding a new tuple to the schema, or transacting an existing tuple that includes a new kind of entity, there is a feeling of technical debt when the developer must know which retraction API functions to update. since the schema was designed in such a way that tuples should not exist with nil values, i was hoping for a "catch all" transactor function that can clean up related tuples without making complicated decisions about which ones to look for.

(another option i explored was having component references from all entities back to tuples so that they are automatically retracted, but this proves to be just as tedious on the other end when transacting new entities)

what if instead you wrote your own retractentities function which does what you want?

This is possible if an enrollment becomes invalid (i.e. should be completely retracted) if any of player, server, or board are not asserted

is that true?

I think that’s what you mean by this:

entities with tuple attributes that are unique are no longer "valid" when any one of their tuple reference values is retracted

then you could query for [?referring ?attr ?e] (vaet index), see if the attr is one of your special ones, and if so, emit [:db/retractEntity ?referring]

> your own retractentities function as in an API layer function or a transactor level function?

You could look at tupletype membership, but I think it’s going to be less surprising to have either a hardcoded list or your own annotation on the attribute, e.g. :required-for-entity-validity?

transactor level function would be safest

agreed, and that's where i'm at. but if i'm understanding you correctly, the problem still stands of knowing which tuple attributes reference which entities if i want to shorten the list of possible matches. in my case, nearly any tuple can reference nearly any entity.

you don’t need to know about the tuples, but the attributes that compose the tuple

since you know you are retracting, if you retract an attribute which is a member of a tuple, you know the tuple is going to get a null in it, so you can retract the entire referring entity

oh hey, that just might work...

thank you for clarifying 🙂

I still think it’s probably safer to annotate/enumerate attributes which you want this cascading behavior on

yes, i'm with you on that. ideally it's something i can update via annotations on the schema rather than in the codebase.

correct

This is just wanting one piece of isComponent’s behavior

i've always thought of it as a "reverse component reference" :man-shrugging:

which isn't 100% accurate, but for some reason it's stuck in my head