(let [node (crux/start-node
              {:crux.node/topology '[crux.standalone/topology
                                     crux.kv.lmdb/kv-store]
               :crux.kv/db-dir (str (io/file "db"))})]
  (crux/submit-tx node [[:crux.tx/put
                         {:crux.db/id :img,
                          :description "",
                          :tags
                          ["female"
                           "grass"],
                          :source "",
                          :format "PNG",
                          :mime-type "image/png"}]])
  (crux/q (crux/db node)
          '{:find [id]
            :where [[e :crux.db/id id]]
            :full-results? true}))

It seems to work with rocksdb, but rocksdb has other issues for me -- it never seems to let go of its lock.

It works with RocksDB but not with LMDB

it works for me

How do I get RocksDB to let go of a lock?

no idea about rocksdb

LMDB is giving me issues now too, it's defaulting to memKv for no discernable reason

20-07-11 02:57:48 canterlot WARN [crux.kv.memdb:1] - Using sync? on MemKv has no effect. Persistence is disabled.
20-07-11 02:57:48 canterlot INFO [crux.tx:1] - Started tx-consumer
20-07-11 02:57:48 canterlot INFO [crux.hash.jnr:1] - unknown
20-07-11 02:57:48 canterlot INFO [crux.hash:1] - Using libgcrypt for ID hashing.

(defn get-crux-node
  [database]
  (crux/start-node {:crux.node/topology '[crux.standalone/topology
                                          crux.kv.lmdb/kv-store]
                            :crux.kv/db-dir (str (io/file database))}))

Why is it trying to use MemKv?

But then my db/data.mdb file has grown...

So confusing

Can someone explain what is going on here? Is this a case of the index not being updated before the query is run? How can this be prevented?

pengine.core> (defn start-standalone-node ^crux.api.ICruxAPI [storage-dir]
    (crux/start-node {:crux.node/topology '[crux.standalone/topology]
                      :crux.kv/db-dir (str (io/file storage-dir "db"))}))

  (def node (start-standalone-node "crux-store"))
=> #'pengine.core/start-standalone-node=> #'pengine.core/node
pengine.core> (defn q
    [query]
    (crux/q (crux/db node) query))
=> #'pengine.core/q
pengine.core> (defn add-test-doc []
                (crux/submit-tx
                 node     
                 [[:crux.tx/put
                   {:crux.db/id (uuid)
                    :type :test}]]))
=> #'pengine.core/add-test-doc
pengine.core> (defn find-test-doc-id []
                 (let [query {:find '[?e]
                              :where [['?e :type :test]]}] (q query)))
=> #'pengine.core/find-test-doc-id
pengine.core> (let [doc (add-test-doc)]
                (find-test-doc-id))
=> #{}
pengine.core> (let [doc (add-test-doc)]
                (find-test-doc-id))
=> #{[#uuid "72f065f8-522b-45fc-825e-360e6799cccc"]}

If I want to re-mount the db say in repl to do some queries, what do?

@frosku you can certainly do that. I usually have my Crux node defined as a component in an Integrant or component set up, and access it via the 'system'. Or you can just def it somewhere.

I'm choosing the directory via a variable to a CLI script

https://github.com/Frosku/mirrorpool/blob/master/src/mirrorpool/core.clj ^ The variable is parsed here

https://github.com/Frosku/mirrorpool/blob/master/src/mirrorpool/derpi.clj ^ Then fed into get-crux-node here.

What I'd like to do is ideally while the script is running, query the db to make sure stuff is going into it.

-rw-r--r-- 1 frosku users 26820608 Jul 11 12:56 data.mdb

It definitely seems to be

But then when I try to re-open and query node I get nothing.

(defn h
  []
  (let [node (get-crux-node "/derp/db")]
    (crux/q (crux/db node) '{:find [id] :where [[e :crux.db/id id]] :full-results? true})))

(h)
=> #{[nil]}

Which Crux version @frosku

[juxt/crux-core "20.06-1.9.1-beta"]
                 [juxt/crux-lmdb "20.06-1.9.1-alpha"]

Try with 20.07-1.9.2-beta

There's a particular issue you're hitting with 1.9.1 that's fixed in 1.9.2

Same version bump for lmdb?

Yes, but I think that's irrelevant.

But better to align

Still -alpha?

Yes

I believe lmdb is still alpha, you'll get a maven download error if you get it wrong

What's the issue btw?

Still #{[nil]}

When the script closes, the query returns data, but when I try to fire up a repl with the same node, it returns #{[nil]}

Is there something I'm not understanding?

And then if I run it for a different set of data it returns nil at the end of script execution

OK, so turns out I needed to do a bit more to persist the event log and then I could query back again

(defn get-crux-node
  [database]
  (crux/start-node {:crux.node/topology '[crux.standalone/topology
                                          crux.kv.lmdb/kv-store]
                    :crux.kv/db-dir (str (io/file database "db"))
                    :crux.standalone/event-log-kv-store 'crux.kv.lmdb/kv
                    :crux.standalone/event-log-dir (str (io/file database "evt"))
                    :crux.standalone/event-log-sync? true
                    :crux.kv/sync? true}))

Ok

I think this is a common "gotcha" when starting with crux - it seems like a good thing to make more prominent in the documentation under standalone node configuration

100%, I definitely found it very confusing