Hi @U052A8RUT your rule head looks like it needs tweaking as I think the ?proj needs to be an output, not an input:

(project-name-like? [?proj ?search-string])
;;->
(project-name-like? [?search-string] ?proj)

That said, I'm not sure how that could have been working previously :thinking_face:

I tried you suggestion, however the result remains the same.

"* XTDB circular dependency problem"
        (db/q (db/db node) '{:find  [?proj]
                             :in    [?search-string]
                             :where [[?proj :nexus.project/id]

                                     ;; WORKS
                                     #_(project-name-like? ?search-string ?proj)

                                     ;; WORKS
                                     #_(or-join [?proj ?search-string]
                                         [(text-search :nexus.project/name ?search-string) [[?proj]]])

                                     ;; WORKS
                                     #_(or
                                         [(text-search :nexus.project/name ?search-string) [[?proj]]])

                                     ;; THROWS: Circular dependency between ?proj and ?search-string
                                     ;; using XTDB version 1.21.0
                                     ;; HOWEVER: this works with Crux 21.02-1.15.0-beta
                                     (or-join [?search-string ?proj]
                                       (project-name-like? ?search-string ?proj))

                                     ;; THROWS: Circular dependency between ?proj and ?search-string
                                     ;; using XTDB version 1.21.0
                                     ;; HOWEVER: this works with Crux 21.02-1.15.0-beta
                                     #_(or
                                       (project-name-like? ?search-string ?proj))

                                     ]
                             :rules [[(project-name-like? [?search-string] ?out)
                                      [(text-search :nexus.project/name ?search-string) [[?out]]]]]}
          "ATR") => #{[(db-ref :nexus.project/id 1)] [(db-ref :nexus.project/id 3)]}

Actual: clojure.lang.ExceptionInfo: Circular dependency between ?proj and ?search-string
{:reason :juxt.clojars-mirrors.dependency.v1v0v0.com.stuartsierra.dependency/circular-dependency, :node ?proj, :dependency ?search-string}
 at juxt.clojars_mirrors.dependency.v1v0v0.com.stuartsierra.dependency.MapDependencyGraph.depend (dependency.cljc:87)
    xtdb.query$calculate_join_order$fn__94131$fn__94135.invoke (query.clj:1302)
    clojure.core.protocols$fn__8249.invokeStatic (protocols.clj:168)
    clojure.core.protocols/fn (protocols.clj:124)
    clojure.core.protocols$fn__8204$G__8199__8213.invoke (protocols.clj:19)
    clojure.core.protocols$seq_reduce.invokeStatic (protocols.clj:31)
    clojure.core.protocols$fn__8236.invokeStatic (protocols.clj:75)
    clojure.core.protocols/fn (protocols.clj:75)
    clojure.core.protocols$fn__8178$G__8173__8191.invoke (protocols.clj:13)
    xtdb.query$calculate_join_order$fn__94131.invoke (query.clj:1300)
    clojure.lang.PersistentVector.reduce (PersistentVector.java:343)
    xtdb.query$calculate_join_order.invokeStatic (query.clj:1293)
    xtdb.query$calculate_join_order.invoke (query.clj:1283)
    xtdb.query$compile_sub_query.invokeStatic (query.clj:1493)
    xtdb.query$compile_sub_query.invoke (query.clj:1468)
    xtdb.query$compile_sub_query$fn__94457.invoke (query.clj:1537)
    xtdb.query$compile_sub_query.invokeStatic (query.clj:1536)
    xtdb.query$compile_sub_query.invoke (query.clj:1468)
    xtdb.query$build_sub_query$fn__94579.invoke (query.clj:1609)
    xtdb.cache.second_chance.SecondChanceCache.computeIfAbsent (second_chance.clj:68)
    xtdb.cache$compute_if_absent.invokeStatic (cache.clj:10)
    xtdb.cache$compute_if_absent.invoke (cache.clj:9)
    xtdb.query$build_sub_query.invokeStatic (query.clj:1603)
    xtdb.query$build_sub_query.invoke (query.clj:1593)
    xtdb.query$build_or_constraints$iter__93492__93496$fn__93497$fn__93498$or_constraint__93505$iter__93519__93523$fn__93524$fn__93525$fn__93530.invoke (query.clj:1107)
    xtdb.query$build_or_constraints$iter__93492__93496$fn__93497$fn__93498$or_constraint__93505$iter__93519__93523$fn__93524$fn__93525.invoke (query.clj:1090)
    xtdb.query$build_or_constraints$iter__93492__93496$fn__93497$fn__93498$or_constraint__93505$iter__93519__93523$fn__93524.invoke (query.clj:1084)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    clojure.lang.LazySeq.sval (LazySeq.java:42)
    clojure.lang.LazySeq.seq (LazySeq.java:51)
    clojure.lang.RT.seq (RT.java:535)
    xtdb.query$build_or_constraints$iter__93492__93496$fn__93497$fn__93498$or_constraint__93505.invoke (query.clj:1115)
    xtdb.query$build_sub_query$constrain_result_fn__94630$fn__94631.invoke (query.clj:1625)
    xtdb.query$build_sub_query$constrain_result_fn__94630.invokePrim (query.clj:1624)
    xtdb.query$build_sub_query$constrain_result_fn__94630.invoke (query.clj:-1)
    xtdb.index.NAryJoinLayeredVirtualIndex.seek_values (index.clj:265)
    xtdb.index$layered_idx__GT_seq$step__91864.invokePrim (index.clj:317)
    xtdb.index$layered_idx__GT_seq$step__91864.invoke (index.clj:-1)
    xtdb.index$layered_idx__GT_seq.invokeStatic (index.clj:325)
    xtdb.index$layered_idx__GT_seq.invoke (index.clj:302)
    xtdb.query$build_sub_query$fn__94634.invoke (query.clj:1635)

AFAIU the problem occurs when combining an or or or-join with a Lucene query embedded in a rule.

The problem does not occur with Xtdb 1.20.0

Hey @U052A8RUT apologies for the late reply. Having reproduced this locally and reflected a little, I think this is a more general regression than just Lucene (it affects all such predicate functions), and warrants raising as a bug which I'll do next week. However as a workaround for now, you should be able to add an extra lvar + explicit unifying clause, e.g.:

(or-join [?search-string ?proj2]
         (project-name-like? ?search-string ?proj2))
[(== ?proj ?proj2)]

Thanks!

I explored the problem a little further and just opened this: https://github.com/xtdb/xtdb/issues/1783 (but have to park it for now)

If it's a non-trivial problem to have to adjust your existing queries with the workaround then it might be a good idea for us to sync this week to figure out a timeline - I'm always happy to chat in any case 🙂 Otherwise I'm going to have to park this for the time being (sorry!)

It's not an urgent matter for us at the moment. For the time being I can work with the previous xtdb version or try out your workaround. Thanks for offering help! :thumbsup:

FYI: the workaround solves it for us using the latest version :thumbsup:

Fabulous! I'll keep you posted on a fix 🙂

This should do it:

(q db
   '{:find [doc]
     :in [[category ...]]
     :where [[doc :category category]]}
   ["category1" "category2"])

thank you 🙂

(q db
   '{:find [(pull ?doc [*])]
     :in [[category ...]]
     :where [[?doc :category category]]}
   ["c1" "2"])

#{[{:category "c1",
    :xt/id #uuid"2b0ca3ed-d392-44b7-9f9e-a5bddcd35695"}]
  [{:category "c2",
    :xt/id #uuid"d413c4e4-4532-44f4-8e50-738bfcc2b497"}]}

At the moment you just have to do it in Clojure; I think this is maybe something the XT team will add in the future though? Hopefully at least 🙂. For myself I made a helper fn that I always use instead of xtdb.api/q: https://github.com/jacobobryant/biff/blob/master/src/com/biffweb/impl/xtdb.clj#L85 With that fn, if you omit the vector around the :find value, e.g. (q db '{:find (pull ?doc [*]) ..., then it'll wrap the results in a (map first ...) for you.