Clojurians Log v2

Why not add [?out-eid :const-attr “const-Val”]?

Assuming I understand your example... that will not work when eid is a lookup ref. AFAIK, this is invalid query syntax: [[:lrefattr "lrefvalue"] :something-but-what? "something else, but what?"] so even if I could find a constant attr and value on every entity, the syntax is unacceptable. The goal of generalizing the scoping to an entity identified by any arbitrary eid (lref, ident, dbid) seems too ambitious without a little helper function like resolve-eid above.

You are altering the :where no matter what, correct? By appending something?

What I’m suggesting is the equivalent of a macro vs a fn: given some static input (lookup ref or long) and knowledge of the name of the binding you wish to restrict, add the appropriate where clause.

Yes, with backtick's template.

Is the issue that you don’t know the type of the original ?eid in the original query? (Not the type of how you want to restrict it)

Correct: the input eid could be a lookup ref, a db id or an ident. With some branching based on type detection, I could write a macro that generates differentiated where clauses ... that is an option.

The utility of this function is coercing any entity reference to a true eid; this is something you should do high in any query, and I think we made an impl on this very slack using rules. It seems strange to frame this as useful in this specific scenario where you want to add a clause to filter, because I would want any query to definitely have ?eid be an eid by that point

FWIW I would expect query transformations to have a shape like (update-in qmap [:query :where] conj my-filter) not backtick

As noted in my earlier comments, this is not about performance, it's rather a REPL convenience for debugging. In normal usage, the concatenated where clauses are not so simple, but for debugging it's useful to use the same dynamism to inject a where clause that scopes down to a single entity. I kinda remember a discussion using rules as well for something very similar... I will try to find it and post back here.

I'm not thinking of performance either, just your original question "[...] particularly one that does not depend on a custom query function?"

rules don't depend on a custom query function

mostly I'm struggling to imagine a case where I know the binding I want to restrict (`?eid` in our example) and want to restrict it, but don't know if its an entity id or lookup ref (or ident I guess). I think I just don't write queries like that, even at the repl

I think https://clojurians.slack.com/archives/C03RZMDSH/p1688916479328169 is the thread where a rule-based solution was proposed.

It's clunky to limit a general purpose REPL query to only working with lookup refs (or idents, or db ids) only because the where clauses are dynamically concatenated (an implementation detail). Outside the REPL, such a constraint isn't burdensome.

I am still trying to wrap my mind around the need for such resolve entity id function, a simple resolution around pull might be slightly faster than above, (btw, there is still the uncertainty I read here once about slightly different behavior from the ion client vs the cloud client, have not verified that) eg of a case of close proximity I’ve:

(defn lookup [db attr entity-id]
  (letfn [(pull [entity-id] (d/pull db [attr] entity-id))
          (get-attr [map] (get map attr))
          (resolve-many [result] (if (vector? result) (first result) result))
          (resolve-ref [result] (if (map? result) (:db/id result) result))]
    (some-> entity-id pull get-attr resolve-many resolve-ref)))

(defn lookup* [db path entity-id]
  (reduce (fn [eid next-attr] (lookup db next-attr eid))
          entity-id
          path))

(let [field-id '<some-long>] ;TODO
  (lookup* db [:college-list-entry/_custom-fields
               :student/_college-list-entries
               :student/account]
           field-id))

(defn get-level3-attr-details [db tx attr-eid]
  (-> (d/q '[:find ?student-eid ?student-name ?col-name ?col-category-type ?grad-year
             :in $ ?attr-eid
             :where
             [?attr-eid :college-list-entry/college ?col-eid]
             [?col-eid :college/name ?col-name]
             [?attr-eid :college-list-entry/category ?col-category]
             [?col-category :db/ident ?col-category-type]
             [?student-eid :student/college-list-entries ?attr-eid]
             [?student-eid :student/graduation-year ?grad-year]
             [?student-eid :student/account ?account-eid]
             [?account-eid :account/full-name ?student-name]
             [?account-eid :account/active? true]]
           (d/as-of db tx) attr-eid)
      last))

I'm not sure the solutions here will help you much with pagination. I'll restate the problem a bit: Problem: In Datomic Cloud, is it possible to resolve an arbitrary entity identifier ("eid") into a db id within a query (IOW, not as an argument to the query)? Motivation: dynamic construction of where clauses where input to the dynamic clause is an arbitrary eid. Solutions: no built-in solutions, but it's not hard to write a query fn (shown above) that solves the problem. Francis has also solved this using rules (linked above).

Pagination in Datomic Cloud is a whole other animal that I don't feel qualified to rigorously address. That should probably be in a separate thread, @simon, and I would be happy to chime in.

I guess the real issue isn't the Problem or Solutions but the Why(...the motivation), yet to see a use case for it at least in Datomic cloud.

I find myself occasionally needing to manually resolve an eid when I have a query that returns a lot of results. For example, imagine I have a query that returns eligible vehicles from a population of 100000 vehicles. Each vehicle represents some work to be done, say make a weekly call to a DMV API to check the insurance coverage. The query applies some conditions (vehicle must not be declared lost, must be newer than 1980 model year, etc) but even with those conditions, there are a lot of results. In production, I use strategies like sharding on the VIN, subdividing by registration state, etc. Each strategy is a dynamically applied set of where clauses concatenated onto the "standard" where clause conditions. When debugging, it can be convenient to scope the results to just one instance -in this example, a single VIN. I would like a strategy (=> set of dynamically applied where clauses) that scope the results to this single entity expressed as an eid (typically a lookup ref). But unfortunately there is no simple way to inject a where clause to constrain the results with an arbitrary eid. IOW, this is surprisingly hard to implement:

:where [....complex query conditions
              ~@(some-scoping-where-clause eid)