Fork me on GitHub
#asami
<
2021-10-10
>
william12:10:58

Here's a thing that I don't fully understand: if delete-database is a no-op for memory-databases, and in fact I can still query an in-memory db after deleting it, when is the data garbage collected?

quoll12:10:10

There’s a map that connects URLs to connections. So if you get rid of your last reference to a connection, you can get it back by asking for it again by URL. Calling delete-database will remove this entry, so when your last reference to the connection is gone then it can be GC’ed

william12:10:10

but what happens if I'm developing the sofware in shadow-cljs for example? I mean, I tried commenting out all the references to a db (I already deleted it from the connection list) but I still get the values back when I query

quoll12:10:42

What are you querying?

william12:10:46

something like:

(defonce workspace (db/connect "asami:"))

(comment
  "Cancel the current database-try"
  (db/delete-database "asami:"))

@(db/transact workspace {:tx-data [{:db/id -1 :name :do}]})

(db/q '[:find ?a ?b ?c :where [?a ?b ?c]] (db/db workspace))

quoll12:10:25

workspace is a reference

quoll12:10:02

You didn't let it go, so it's not eligible to be garbage collected

william12:10:42

but, when I comment out workspace and then do > workspace at the repl, I get undefined

william12:10:09

then I uncomment the things, and at the query I get a bunch of stuff

quoll13:10:43

So, you delete then reconnect to the same URL and there's data? That's unexpected. It should be removed from the connection map. Can you show this with some code?

william13:10:10

something like:

william13:10:14

(comment
  "the connections are emtpy"
  @db/connections
  ;; => {}
  "but still, there's data"
  (db/q '[:find ?a ?b ?c :where [?a ?b ?c]] (db/db workspace))
;; => ([:tg/node-25 :name :do]
;;     [:tg/node-25 :db/ident :tg/node-25]
;;     [:tg/node-25 :tg/entity true]
;;     [:tg/node-19 :name :do]
;;     [:tg/node-19 :db/ident :tg/node-19]
;;     [:tg/node-19 :tg/entity true]
;;     [:tg/node-23 :name :do]
;;     [:tg/node-23 :db/ident :tg/node-23]
;;     [:tg/node-23 :tg/entity true]
;;     [:tg/node-14 :name :do]
;;     [:tg/node-14 :db/ident :tg/node-14]
;;     [:tg/node-14 :tg/entity true]
;;     [:tg/node-27 :name :do]
;;     [:tg/node-27 :db/ident :tg/node-27]
;;     [:tg/node-27 :tg/entity true]
;;     [:tg/node-16 :name :do]
;;     [:tg/node-16 :db/ident :tg/node-16]
;;     [:tg/node-16 :tg/entity true]
;;     [:tg/node-17 :name :do]
;;     [:tg/node-17 :db/ident :tg/node-17]
;;     [:tg/node-17 :tg/entity true]
;;     [:tg/node-20 :name :do]
;;     [:tg/node-20 :db/ident :tg/node-20]
;;     [:tg/node-20 :tg/entity true]
;;     [:tg/node-22 :name :do]
;;     [:tg/node-22 :db/ident :tg/node-22]
;;     [:tg/node-22 :tg/entity true]
;;     [:tg/node-12 :name :do]
;;     [:tg/node-12 :db/ident :tg/node-12]
;;     [:tg/node-12 :tg/entity true]
;;     [:tg/node-18 :name :do]
;;     [:tg/node-18 :db/ident :tg/node-18]
;;     [:tg/node-18 :tg/entity true])
  )

quoll13:10:06

That shows you querying a connection that you got before removing it from the map

william13:10:31

if I do:

(db/q '[:find ?a ?b ?c :where [?a ?b ?c]] (db/db (db/connect "asami:")))
I don't get anything back as expected

quoll13:10:33

I could reinitialize the connection when you delete, but you called delete, so initializing it would seem to be the wrong thing?

william13:10:06

I guess my surprise comes from the fact that after I make sure that workspace is not defined anymore, when it's redefined it still shows up with the old data

william13:10:40

I'm thinking about just avoiding references and using always (db/db (db/connect "url")). Would that strategy have any countereffect?

quoll13:10:17

It's like you're asking

(let [a (get a-map :key)
      b-map (dissoc a-map :key)]
  (println a))
this will work. The only way to throw away the data in a is for that reference to go out of scope

quoll13:10:50

That strategy would be a tiny bit slower

quoll13:10:37

So you want to use delete-database as a shorthand to drop all the data and clear it out?

william13:10:17

yes, that's what delete-database would imply for me; but maybe I just need a quick way of clearing out the db - I wasn't able to find that in the code

william13:10:47

it's also a thing about hanging resources; I would have expected delete-database to free the resources used. But again, it's not really necessary that that function does it, as long as we have some convenient way to do it

quoll13:10:06

It does free the resources. But since most of that is just memory structures and you were hanging onto references, then the structures are still where your holding them

quoll13:10:35

Is more apparent with files, since files actually get removed

quoll13:10:04

I expect to do a release today. It's a small change to update a connection to an empty database when it gets deleted, so I could do that

🙌 1
william13:10:36

oh I see than, it's probably not an asami problem per se, but the way I'm using references when using shadow-cljs at the top level; yes that would fix my problem I think

quoll13:10:18

Give me time. Right now I’m lying in bed on a Sunday morning with a phone. 😴

william13:10:47

sure, there's absolutely no rush 🙌

william17:10:56

Just putting this here for later. In Clojurescript, this returns no watchers:

(def workspace-str "asami:")
(def workspace (db/connect workspace-str))
(add-watch (:state workspace) :simple-print
           (fn [a b c d] (println "On workspace: " [a b c d])))
(.-watchers (:state workspace))

quoll19:10:20

Change the final line to:

(.-watches (:state workspace))

quoll19:10:22

We all make typos. I was just lucky to spot it 🙂

🙌 1
william19:10:41

oh... :man-facepalming: 😂

quoll22:10:48

Asami 2.2.0 is now out. There are 3 main changes: • transact is now synchronous on the JVM, and will not return until the end of the transaction, or until timing out. The timeout can be set with a system property of asami.txTimeoutMsec, or datomic.txTimeoutMsec. Long timeouts could be needed for multi-gigabyte imports. transact-async returns a future that will be completed when the transaction is done. • Datoms returned in transaction results are now reflective of database changes, and not just what was requested. So if 20 statements are inserted, and 5 of them already existed, only 15 datoms will be returned. If 2 deletions are requested, and only 1 of those statements exists, then only one delete datom will be returned. • The delete-database function will now clear and reset any existing connections for that URL. This allows the connection to be reused. If data is transacted into the connection, then it will be picked up again by the registry

🎉 2
❤️ 4
🙌 1
quoll23:10:56

Strangeloop is behind me, so I have time to myself again.

quoll23:10:29

Unfortunately, it looks like I can’t spend work hours on Asami anymore, but that’s OK. I did a lot of Asami outside of work hours anyway 😊

❤️ 3
🤞 1