xtdb 2022-10-25 | Slack Archive

Is that possible to do much nicer: eg. how to check is that closed?

(def closed? (atom "false"))

(defn kv-store [dir]
  {:kv-store {:xtdb/module 'xtdb.rocksdb/->kv-store
              :db-dir      (io/file dir)
              :block-cache :xtdb.rocksdb/block-cache
              :sync?       true}})

(defn start-xtdb! [conf]
  ^IXtdb (IXtdb/startNode
           (let [xtdb-path (-> conf :xtdb :dev :dir)
                 xtdb-port (-> conf :xtdb :dev :port)
                 ^Map m {:xtdb/tx-log              (kv-store (str xtdb-path "/tx-log"))
                         :xtdb/document-store      (kv-store (str xtdb-path "/doc-store"))
                         :xtdb/index-store         (kv-store (str xtdb-path "/index-store"))
                         ;; Configuring the Block Cache 
                         :xtdb.rocksdb/block-cache {:xtdb/module 'xtdb.rocksdb/->lru-block-cache
                                                    :cache-size  (* 512 1024 1024)}
                         ;; optional:
                         :xtdb.lucene/lucene-store {:db-dir (str xtdb-path "/lucene-dir")}
                         :xtdb.http-server/server  {:port xtdb-port}}]
             m)))

(mount/defstate ^:dynamic ^{:on-lazy-start :throw}
                *xtdb*
                :start (do (println (str "XTDB database start"))
                           (reset! closed? "false")
                           (start-xtdb! conf))
                :stop (when (not (keyword? *xtdb*))
                        (try
                          (do
                            (.close *xtdb*)
                            (.status *xtdb*))
                          (catch IllegalStateException e
                            (when (= "XTDB node is closed" (.getMessage e))
                              (reset! closed? "true"))))))

refset21:10:58

Hi @U2T2ZEVPC I'm not clear on the problem you're trying to solve here (/ the problem you have solved). Please can you describe it a little more? This previous thread on "how do I know if a node is closed" may be of use: https://clojurians.slack.com/archives/CG3AM2F7V/p1661950972927149

sb23:10:21

Ok, I try to find out. My code is working, but very dirty. So no different solution. What I would like to solve: get feedback Rocksdb really closed or not, because mount sometimes give back false info.

refset09:10:47

Hmm, so are you finding that .close is not blocking correctly / behaving synchronously?

refset09:10:25

> mount sometimes give back false info is this resulting in segfaults?

sb09:10:59

Yes, I use in this way the mount.. as I shared.. if not.. 1 blocked from 5-6 restart. Mount said stopped but really not fully.. like locked something

sb09:10:30

I dont know why. That is stable version now as you wrote the test, the dirty version

sb09:10:28

Maybe that is coming rocksdb installation on osx.. I will find out later. Ps. I read about that was a problem at Python too.. etc in the past

sb09:10:00

Im not a senior Clojure dev.. so therefore I can mainly just drop questions. I dont want to say anything as fact. If you understand me.

👍 1

refset09:10:21

Well I'm not quite a senior Clojure dev either 😅

refset09:10:26

I haven't used Mount before so I don't have a good understanding of how it could/should all work together. However, if you are keen to get to the bottom of this I should be able to help you if you could create a small, isolated project (i.e. a new repo on Github) that consistently reproduces the issue, alongside the solution. I'm wondering whether perhaps Mount isn't even required to demonstrate the real underlying issue :thinking_face:

👍 1

sb09:10:45

I drop to mount channel too 😜

sb09:10:54

Ok I create the issue’s repo on github and everybody can check it.

🙏 1

sb09:10:10

That is a good idea!

tolitius14:10:50

> I’m wondering whether perhaps Mount isn’t even required to demonstrate the real underlying issue I think it is a true statement > mount said stopped but really not fully.. like locked something mount does not really say anything what you see is a result of a stop function (a block of code after the :stop 👆)

sb14:10:21

Mount have a fn to get back info about statuses. Not just .. print every step.

sb14:10:13

. I saw: A write as in docs B write as above, like test. So, at A sometimes I get error, at B case never

tolitius14:10:01

to @U899JBRPF’s point I would recommend to try to reproduce it without mount the error comes from the code block in your :stop function, mount just calls it on (mount/stop) it is best to isolate the problem vs. wrap it in tools

👍 1

👀 1

sb16:10:46

sb16:10:58

On the weekend I do it

sb18:11:53

@U899JBRPF ~~I tested out the full code and the solution:~~ ~~(defn start-xtdb! [conf]~~ ~~^IXtdb (xt/start-node (rocks-node-opts conf)))~~

sb18:11:56

Ok, the mistake was I used (declare ^:dynamic ^IXtdb *xtdb*) instead of def.. beginning of the namespace… I never checked 😕 😞🤯

refset19:11:57

Oh no! Although I'm really glad to hear you got to the bottom of it eventually 👏 What was the intended use-case for using (declare ^:dynamic ^IXtdb *xtdb*) ? Testing?

sb22:11:27

I used at user.clj > backend/db setup where was the lifecycle management, so effected to everything. Eg. at tests:

(defn test-rocksdb-fixture
      "(clojure.test/use-fixtures :once test-rocksdb-fixture)"
      [f]
      (start-with-args {:mode :test} [#'conf #'*xtdb*])
      (f)
      (stop [#'*xtdb* #'conf]))

👍 1

sb22:11:09

that is call *xtdb* > (mount/defstate *xtdb* .. which is depend on config and special args.

sb22:11:32

That was a typo.. instead of def… (maybe I copied an old code..)

😬 1

sb18:10:03

After this.. xt/submit-tx isn’t working.. on xtdb, I can use just Transaction/Builder.

sb18:10:17

Is that possible to do much easier? Nicer, better? 🙂

sb19:10:21

I know, that isn’t perfect.. but you can see fully (maybe easier in this way understand my problem) https://github.com/damesek/deps-fullstack I work on this ( long time ago.. was a plan .. renew my stack from lein).

sb19:10:50

ns project.clj.db.persons

2022-10-25

Channels