Hi @ouvasam I can't see anything obviously wrong. Are you able to paste the response of running crux/entity such that I can see the entire document(s)?

Alternatively, I have some free time now if you wanted to pair for the next 30m 🙂 just DM me to arrange

Yes that is this

{:translation/value qsdqsdqsqs kjh qsdkjh qsd, :translation/id :translation-en-728-1-1-DRAFT, :code-system/id 1, :translation/language en, :translation/status :DRAFT, :concept/id 728, :app/type :app/Translation, :jurisdiction/id 1, :crux.db/id :translation-en-728-1-1-DRAFT, :app/modification-date Wed Jun 24 15:27:27 CEST 2020}

the problematic query return [] and the last i sent return the correct [id? status?] I also tried to add status in :find but nothing more., I must do something wrong ...

Ah okay. Can you show what the :PUBLISHED version looks like also please?

Dear all, i have a problem with one query I have a record like this in the db

{:translation/value qsdqsdqsqs kjh qsdkjh, :translation/id :translation-en-728-1-1-PUBLISHED, :code-system/id 1, :translation/language en, :translation/status :PUBLISHED, :concept/id 728, :app/type :app/Translation, :jurisdiction/id 1, :crux.db/id :translation-en-728-1-1-PUBLISHED, :app/modification-date Wed Jun 24 15:27:27 CEST 2020}

Thanks, unfortunately I still can't see anything that might explain your situation. It could be useful to compare the two join orders, if you are able to set the logback.xml entry for crux.query=DEBUG If you are able to reproduce a minimal example in a gist or issue I can dig into it further. I can also see if I can reproduce it myself with some similar looking data

Many thanks @U899JBRPF i'll try to find first what it can be cause i am sure it between my chair and my screen that the problem should be 🙂

For now, i see that the problem appears after an "evict"

(defn evict
  [data]
  (println "evicting" data)
  (crux/submit-tx
   node
   (mapv (fn [d]
           (println "evict entity " d)
           [:crux.tx/evict d])
         data)))

so an item (i.e. document), with a completely unrelated :crux.db/id is somehow causing it, you think? hmm

arff ... i switch back to version 20.05-1.8.5-alpha-SNAPSHOT and everything seems to work fine I did try to do a gist with the principles applied but could not find any errors there ... It seems that i have multiple call to crux in parallel the problem appear The same queries and transactions works perfectly and suddenly i can't get the results. If i recreate a new one, the items that should have appears before reappears ... I continue to find what i made wrong

here is the gist but this code works perfectly ...

https://gist.github.com/ouvanous/f046da540656ed17beb509d579ebb27c#file-clj

Here is an example of the problem in the log i've made

# GET THE TRANSLATIONS FOR A CONCEPT PASSING params 728 and 1
java.lang.Long 728 java.lang.Long 1 TRANSLATIONS BY CONCEPT  #{[:translation-en-728-1-1-published] [:translation-fr-728-1-1-published] [:translation-en-728-1-1-draft] [:translation-fr-728-1-1-draft]}

# INSERT AN ITEM BASED ON A DRAFT ONE 
INSERT ITEM  {:translation/value qsdqsqsdqsqsdddq, :translation/id :translation-en-728-1-1-published, :code-system/id 1, :translation/language en, :translation/status :published, :concept/id 728, :app/type :app/Translation, :jurisdiction/id 1, :crux.db/id :translation-en-728-1-1-published, :app/modification-date Thu Jun 25 14:20:07 CEST 2020}

# EVICT THE DRAFT ONE
evict entity  :translation-en-728-1-1-draft

# GET THE TRANSLATIONS FOR A CONCEPT PASSING params 728 and 1
java.lang.Long 728 java.lang.Long 1 TRANSLATIONS BY CONCEPT  #{}

As i said if create a new one, all the entities are retrieved after

no problem anymore when i switched back 20.05-1.8.5-alpha-SNAPSHOT

so I just ran the code in the gist and left a comment, but I'm not sure if my output is what you expect or not though, it looks right to me, please take a look. https://gist.github.com/ouvanous/f046da540656ed17beb509d579ebb27c#gistcomment-3354175

out of interest, how come you want to use evict and not delete ? Do you have a regulatory requirement?

I thought delete was only for an instance time not the complete record

and yes i get the same result with the gist which is correct . Really don't know what's going on with my full code ...

i try to add the logs with logback to have a better understanding

Here are the logs

15:58:01.171 [qtp1933800627-24] DEBUG crux.memory - :pool-allocation-stats {:allocated 4063232, :deallocated 262144, :in-use 3801088}
15:58:01.262 [crux-tx-consumer] DEBUG crux.tx - Indexing tx-id: 8 tx-events: 1
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.memory - :pool-allocation-stats {:allocated 4194304, :deallocated 262144, :in-use 3932160}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :query {:find [id? status?], :where [[e? :crux.db/id id?] [e? :app/type :app/Translation] [e? :translation/status status?] [e? :jurisdiction/id jid?]], :arg-keys [#{jid?}]}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :join-order :aev e? status? {:e e?, :a :translation/status, :v status?}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :join-order :ave jid? e? {:e e?, :a :jurisdiction/id, :v jid?}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :join-order :ave :app/Translation e? {:e e?, :a :app/type, :v :app/Translation}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :where [[:triple {:e e?, :a :crux.db/id, :v id?}] [:triple {:e e?, :a :app/type, :v :app/Translation}] [:triple {:e e?, :a :translation/status, :v status?}] [:triple {:e e?, :a :jurisdiction/id, :v jid?}]]
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :vars-in-join-order [:app/Translation jid? e? status? id?]
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :attr-stats {:crux.db/id 113, :translation/status 1, :jurisdiction/id 6}
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :var->bindings {id? #crux.query.VarBinding{:e-var nil, :var id?, :attr nil, :result-index 4, :result-name crux.query.value/id?, :type :pred, :value? true}, jid? #crux.query.VarBinding{:e-var e?, :var jid?, :attr :jurisdiction/id, :result-index 1, :result-name e?, :type :entity, :value? false}, e? #crux.query.VarBinding{:e-var e?, :var e?, :attr :crux.db/id, :result-index 2, :result-name e?, :type :entity, :value? false}, status? #crux.query.VarBinding{:e-var e?, :var status?, :attr :translation/status, :result-index 3, :result-name e?, :type :entity, :value? false}}
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :query-time-ms 2
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :query-result-size 0

delete is simpler to use, as keeping the history around helps reason about how data changes. I would always use it instead of evict unless you really have a hard requirement to fully remove old data. Also you can delete from now until max-time

thanks for the debug log, the interesting line is:

15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :vars-in-join-order [:app/Translation jid? e? status? id?]

how does it compare with your other query? the one that fails?

15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.memory - :pool-allocation-stats {:allocated 3801088, :deallocated 262144, :in-use 3538944}
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :query {:find [id? status?], :where [[e? :crux.db/id id?] [e? :app/type :app/Translation] [e? :translation/status status?] [e? :jurisdiction/id jid?]], :arg-keys [#{jid?}]}
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :join-order :aev e? status? {:e e?, :a :translation/status, :v status?}
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :join-order :ave jid? e? {:e e?, :a :jurisdiction/id, :v jid?}
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :join-order :ave :app/Translation e? {:e e?, :a :app/type, :v :app/Translation}
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :where [[:triple {:e e?, :a :crux.db/id, :v id?}] [:triple {:e e?, :a :app/type, :v :app/Translation}] [:triple {:e e?, :a :translation/status, :v status?}] [:triple {:e e?, :a :jurisdiction/id, :v jid?}]]
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :vars-in-join-order [:app/Translation jid? e? status? id?]
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :attr-stats {:crux.db/id 113, :translation/status 1, :jurisdiction/id 6}
15:58:00.701 [crux.query.query-pool-thread-12] DEBUG crux.query - :var->bindings {id? #crux.query.VarBinding{:e-var nil, :var id?, :attr nil, :result-index 4, :result-name crux.query.value/id?, :type :pred, :value? true}, jid? #crux.query.VarBinding{:e-var e?, :var jid?, :attr :jurisdiction/id, :result-index 1, :result-name e?, :type :entity, :value? false}, e? #crux.query.VarBinding{:e-var e?, :var e?, :attr :crux.db/id, :result-index 2, :result-name e?, :type :entity, :value? false}, status? #crux.query.VarBinding{:e-var e?, :var status?, :attr :translation/status, :result-index 3, :result-name e?, :type :entity, :value? false}}
15:58:00.702 [crux.query.query-pool-thread-12] DEBUG crux.query - :query-time-ms 2
15:58:00.702 [crux.query.query-pool-thread-12] DEBUG crux.query - :query-result-size 2
PATHOM QUERY by jid {:jurisdiction/id 1, :concept/id 728}
15:58:00.858 [crux.query.query-pool-thread-13] DEBUG crux.memory - :pool-allocation-stats {:allocated 3932160, :deallocated 262144, :in-use 3670016}
15:58:00.859 [crux.query.query-pool-thread-13] DEBUG crux.query - :query {:find [?id], :where [[?e :crux.db/id ?id] [?e :app/type :app/Translation] [?e :jurisdiction/id jurisdictionid] [?e :concept/id conceptid]], :arg-keys [#{conceptid jurisdictionid}]}
15:58:00.859 [crux.query.query-pool-thread-13] DEBUG crux.query - :join-order :ave conceptid ?e {:e ?e, :a :concept/id, :v conceptid}
15:58:00.859 [crux.query.query-pool-thread-13] DEBUG crux.query - :join-order :ave jurisdictionid ?e {:e ?e, :a :jurisdiction/id, :v jurisdictionid}
15:58:00.859 [crux.query.query-pool-thread-13] DEBUG crux.query - :join-order :aev ?e ?id {:e ?e, :a :crux.db/id, :v ?id}
15:58:00.859 [crux.query.query-pool-thread-13] DEBUG crux.query - :join-order :ave :app/Translation ?e {:e ?e, :a :app/type, :v :app/Translation}
15:58:00.859 [crux.query.query-pool-thread-13] DEBUG crux.query - :where [[:triple {:e ?e, :a :crux.db/id, :v ?id}] [:triple {:e ?e, :a :app/type, :v :app/Translation}] [:triple {:e ?e, :a :jurisdiction/id, :v jurisdictionid}] [:triple {:e ?e, :a :concept/id, :v conceptid}]]
15:58:00.860 [crux.query.query-pool-thread-13] DEBUG crux.query - :vars-in-join-order [:app/Translation conceptid jurisdictionid ?e ?id]
15:58:00.860 [crux.query.query-pool-thread-13] DEBUG crux.query - :attr-stats {:crux.db/id 112, :jurisdiction/id 5}
15:58:00.860 [crux.query.query-pool-thread-13] DEBUG crux.query - :var->bindings {conceptid #crux.query.VarBinding{:e-var ?e, :var conceptid, :attr :concept/id, :result-index 1, :result-name ?e, :type :entity, :value? false}, jurisdictionid #crux.query.VarBinding{:e-var ?e, :var jurisdictionid, :attr :jurisdiction/id, :result-index 2, :result-name ?e, :type :entity, :value? false}, ?e #crux.query.VarBinding{:e-var ?e, :var ?e, :attr :crux.db/id, :result-index 3, :result-name ?e, :type :entity, :value? false}, ?id #crux.query.VarBinding{:e-var ?e, :var ?id, :attr :crux.db/id, :result-index 4, :result-name ?e, :type :entity, :value? false}}
15:58:00.861 [crux.query.query-pool-thread-13] DEBUG crux.query - :query-time-ms 3
15:58:00.861 [crux.query.query-pool-thread-13] DEBUG crux.query - :query-result-size 2
java.lang.Long 728 java.lang.Long 1 TRANSLATIONS BY CONCEPT  #{[:translation-fr-728-1-1-published] [:translation-fr-728-1-1-draft]}
LOOKUPVECTOR #{[:translation-fr-728-1-1-published] [:translation-fr-728-1-1-draft]}
evicting data [:translation-fr-728-1-1-draft]
evict entity  :translation-fr-728-1-1-draft
15:58:01.171 [qtp1933800627-24] DEBUG crux.memory - :pool-allocation-stats {:allocated 4063232, :deallocated 262144, :in-use 3801088}
15:58:01.262 [crux-tx-consumer] DEBUG crux.tx - Indexing tx-id: 8 tx-events: 1
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.memory - :pool-allocation-stats {:allocated 4194304, :deallocated 262144, :in-use 3932160}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :query {:find [id? status?], :where [[e? :crux.db/id id?] [e? :app/type :app/Translation] [e? :translation/status status?] [e? :jurisdiction/id jid?]], :arg-keys [#{jid?}]}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :join-order :aev e? status? {:e e?, :a :translation/status, :v status?}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :join-order :ave jid? e? {:e e?, :a :jurisdiction/id, :v jid?}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :join-order :ave :app/Translation e? {:e e?, :a :app/type, :v :app/Translation}
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :where [[:triple {:e e?, :a :crux.db/id, :v id?}] [:triple {:e e?, :a :app/type, :v :app/Translation}] [:triple {:e e?, :a :translation/status, :v status?}] [:triple {:e e?, :a :jurisdiction/id, :v jid?}]]
15:58:02.465 [crux.query.query-pool-thread-14] DEBUG crux.query - :vars-in-join-order [:app/Translation jid? e? status? id?]
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :attr-stats {:crux.db/id 113, :translation/status 1, :jurisdiction/id 6}
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :var->bindings {id? #crux.query.VarBinding{:e-var nil, :var id?, :attr nil, :result-index 4, :result-name crux.query.value/id?, :type :pred, :value? true}, jid? #crux.query.VarBinding{:e-var e?, :var jid?, :attr :jurisdiction/id, :result-index 1, :result-name e?, :type :entity, :value? false}, e? #crux.query.VarBinding{:e-var e?, :var e?, :attr :crux.db/id, :result-index 2, :result-name e?, :type :entity, :value? false}, status? #crux.query.VarBinding{:e-var e?, :var status?, :attr :translation/status, :result-index 3, :result-name e?, :type :entity, :value? false}}
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :query-time-ms 2
15:58:02.466 [crux.query.query-pool-thread-14] DEBUG crux.query - :query-result-size 0

The starnge part is that this can go good but suddenly it does stop working. Sames queries, same evict but may be a litttle bit faster ?

oh, so it's intermittent even if everything stays the same (code, data, crux version)? are you using await-tx between the queries?

what's interesting in that log is 1) the join order is the same, which is good, and 2) seeing the attribute-stats changing (which aren't completely consistent in how they get updated, but if you add a 50ms sleep between all ops they should be consistent enough):

:attr-stats {:crux.db/id 113, :translation/status 1, :jurisdiction/id 6}
:attr-stats {:crux.db/id 112, :jurisdiction/id 5}
:attr-stats {:crux.db/id 113, :translation/status 1, :jurisdiction/id 6}

Hi Jeremy, i did try all the above, delete, await-tx, ... but at some times, i have the same trouble. I'll redo all the process to catch the problem. Many thanks for your help

It's no problem. The offer of pairing for 30m is still open, but it would have to be Monday :)

Many thanks for this proposition ! I did restart from scratch and remove unnecessary calls and everything seems to work fine now. I also use delete instead as evict. And put some little delay between calls. Many many thanks for your help and sorry if all my questions were finally not related to crux but more about my code Have a nice day and a nice week-end

Wonderful! I'm glad you persevered and fingers crossed that's the end of it!

Have a lovely weekend also :)

is the :abc/def here correct?

Yes I did try to change the name thinking stays was a reserved keyword

Yeah both related issues with args got fixed in the last release: https://github.com/juxt/crux/pull/888 https://github.com/juxt/crux/pull/885