This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-07-26
Channels
- # announcements (6)
- # beginners (69)
- # calva (10)
- # clj-kondo (9)
- # clojure (32)
- # clojure-uk (9)
- # clojuredesign-podcast (13)
- # clojurescript (14)
- # clojureverse-ops (2)
- # conjure (24)
- # cursive (12)
- # data-science (1)
- # datomic (13)
- # graalvm (5)
- # jobs-discuss (4)
- # malli (6)
- # meander (33)
- # off-topic (9)
- # pedestal (2)
- # re-frame (38)
- # reitit (6)
- # rum (4)
- # shadow-cljs (6)
- # tools-deps (8)
- # xtdb (26)
I still have trouble understanding everything 😄
So, I have some code that adds entites and retrieves them. Between my tests I reset the database and everything seems to work. But then, all of a sudden, crux/entity
stops working.
This query returns a uuid as expected:
(ffirst (crux/q (crux/db n) {:find '[e]
:where '[[e :apigen/type t]
[e :email ?email]]
:args [{'t :apigen/user '?email ""}]}))
=> #uuid"645b8639-1589-41ef-b0c1-933471ba8c59"
But when I call entity on it:
(crux/entity (crux/db n) (ffirst (crux/q (crux/db n) {:find '[e]
:where '[[e :apigen/type t]
[e :email ?email]]
:args [{'t :apigen/user '?email ""}]})))
=> nil
it returns nil.Hi @U0677JTQX I think I know what this might be....can you post your topology map please?
{:crux.node/topology '[crux.standalone/topology crux.http-server/module crux.kv.rocksdb/kv-store]
:crux.http-server/port 3988
:crux.kv/db-dir (str (io/file storage-dir "db"))}
Ah, okay so it looks like the problem is that your indexes are persisted but your event log is not. You should just need to add a :crux.standalone/event-log-dir
key so that the db can return the full documents (which are stored in the event log, not the indexes). Running a standalone node with persisted indexes but without a persisted event log shouldn't really be supported, sorry about that.
We're currently working on simplifying this whole area: https://github.com/juxt/crux/issues/818
In the meantime I will add more notes about needing the event-log-dir in the various docs/tutorials to help others
Hi @UA2JG2Y11 would you be using Crux in-process? Due to how nippy works, Crux can currently handle arbitrary Java objects as values...but we might limit that a bit in future. The full list of primitive datatypes in the indexes, that support efficient seeks, can be seen here: https://github.com/juxt/crux/blob/master/crux-core/src/crux/codec.clj#L133
I'm following the tutorial using the rocks db storage implementation.
(crux/submit-tx
node
[[:crux.tx/put
{:crux.db/id :dbpedia.resource/Pablo-Picasso ; id
:name "Pablo"
:last-name "Picasso"}
#inst "2018-05-18T09:20:27.966-00:00"]])
was run and then i restart my repl. However, I'm getting results that seem strange to me. I've restarted the repl and i can query this but not get the entity.
working.cli> (crux/q (crux/db node)
'{:find [e]
:where [[e :name "Pablo"]]})
#{[:dbpedia.resource/Pablo-Picasso]}
working.cli> (crux/entity (crux/db node) :dbpedia.resource/Pablo-Picasso)
nil
The id seems to be there but crux/entity
returns nil for that idand the rocks db started as per the configuration example
(defn start-rocks-node [storage-dir]
(crux/start-node {:crux.node/topology '[crux.standalone/topology
crux.kv.rocksdb/kv-store]
:crux.kv/db-dir (str (io/file storage-dir "db"))}))
(def node (start-rocks-node "testing"))
Hi @U11BV7MTK this same problem came up in a similar thread earlier today. You just need to add an :crux.standalone/event-log-dir
configuration to make sure everything gets persisted properly (and I will attempt to add a note to the tutorial and docs right now 🙂 ): https://clojurians.slack.com/archives/CG3AM2F7V/p1595759489376600
Sorry for the confusion...
Funny, I just saw this post and thought, hm, I have seen this today in my environment 😄
Again, sorry to have not nipped this in the bud sooner. Hopefully these changes will makes things clearer in the docs: https://github.com/juxt/crux/commit/e322a00a1c1c8fb81b9928ee20f6201505605b92 (I can't modify the current version docs before we cut 1.10.0 later in the week)
will this commit update the published information at https://opencrux.com/docs#config-rocksdb ?
@U899JBRPF It seems like for me the setting still does not work. I thought it did, but did not come back to it until today. So this is my topology map:
{:crux.node/topology '[crux.standalone/topology crux.http-server/module crux.kv.rocksdb/kv-store]
:crux.http-server/port crux-http-port
:crux.standalone/event-log-dir (io/file storage-dir "event-log")
:crux.kv/db-dir (io/file storage-dir "indexes")}
But when I restart my server I have the same error happening, it does find my users via a query, but /entity returns nil.
I see, the storage/indexes has some files, but no "event-log" folder ist created.
If I create that manually no files are added there either.Hi again, I'm not working today but @U050V1N74 should be able to help you debug
Hey @U0677JTQX - apologies, you'll also need :crux.standalone/event-log-kv-store 'crux.kv.rocksdb/kv
in that map - I'll ensure this is included in the docs. We're making changes to this setup as part of 1.11 - Crux was quite different when it was introduced 🙂
@U050V1N74 Awesome, I think I got it now, at least I have some files in the event-log-folder 🙂