Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2022-04-16

Channels

#xtdb
<
2022-04-16
>
sheluchin16:04:53

Something happened to my dev db and now I can get document details using explicit field names in :find, but :find [(pull foo [*])] produces nil, as does xt/entity. I can repopulate my data from scratch, but I'd like to understand what's happening. Can anyone shed some light on why using pull or requesting an entity by ID might be failing?

refset16:04:55

Hey, it sounds like your doc-store might be in a bad state 😕 Please can you share your start-node config map and confirm which XT version you're using? > I can get document details using explicit field names This will be because values can often be served directly from indexes without needing access to the (cached) doc-store

refset16:04:02

Did you only notice this happen after you restarted the JVM or the node?

sheluchin16:04:46

Yes, but let me be clear, this is almost certainly my fault and not some bug I came across. I'm wrapping up some major refactoring and ran one of my ETL routines when I shouldn't have, and then went to restore an old backup. > This will be because values can often be served directly from indexes without needing access to the (cached) doc-store Is there some way to specifically check if this is happening?

👍 1
refset17:04:26

You could try creating a new node (e.g. different index-store dir) and see if it is able to build a new index or whether latest-completed-tx gets stuck at 0 or some point earlier than latest-submitted-tx

sheluchin12:04:28

Thanks for the tip @U899JBRPF. If I have my config as such: 

:xtdb/tx-log               {:xtdb/module "xtdb.jdbc/->tx-log"              
                            :connection-pool :xtdb.jdbc/connection-pool}   
                            ; :block-cache :xtdb.rocksdb/block-cache}}     
:xtdb/document-store       {:xtdb/module "xtdb.jdbc/->document-store"      
                            :connection-pool :xtdb.jdbc/connection-pool}}}
Using a Postgres backend, is the old pg_dump sufficient to properly backup the golden stores which can be used to rebuild the index? There are no special flags required to make a proper backup?

refset12:04:25

I haven't used pg_dump extensively before so I can't answer with certainty... but I don't see why XT should need extra flags. Nothing looks obviously wrong with that config :thinking_face:

sheluchin12:04:23

I think maybe part of what happened was starting to do too many operations shortly after a backup restore, and before the index was fully rebuilt.

👍 1