Fork me on GitHub
#datahike
<
2022-10-18
>
localshred20:10:20

Hey friends, I've been using datahike happily for about 6 months now, really loving it. Today I wanted to add a test db to my docker-provided postgres. All the config/store params are identical as my dev db, but when I run d/database-exists? or d/create-database I get an exception (see thread). I'm utterly baffled, as I can't find anywhere in the docs that says you're limited to one datahike db per postgres instance. See thread for deps versions of everything. Thanks!

❤️ 1
localshred20:10:50

1. Caused by clojure.lang.ExceptionInfo
   Failed to connect to store
   {:error "FATAL: database \"myapp_dev_test\" does not exist",
    :cause nil,
    :trace
    [[org.postgresql.core.v3.QueryExecutorImpl receiveErrorResponse "QueryExecutorImpl.java" 2533],
     [org.postgresql.core.v3.QueryExecutorImpl readStartupMessages "QueryExecutorImpl.java" 2645],
     [org.postgresql.core.v3.QueryExecutorImpl <init> "QueryExecutorImpl.java" 138],
     [org.postgresql.core.v3.ConnectionFactoryImpl openConnectionImpl "ConnectionFactoryImpl.java" 255],
     [org.postgresql.core.ConnectionFactory openConnection "ConnectionFactory.java" 49],
     [org.postgresql.jdbc.PgConnection <init> "PgConnection.java" 211],
     [org.postgresql.Driver makeConnection "Driver.java" 459],
     [org.postgresql.Driver connect "Driver.java" 261],
     [java.sql.DriverManager getConnection "DriverManager.java" 664],
     [java.sql.DriverManager getConnection "DriverManager.java" 208],
     [next.jdbc.connection$get_driver_connection invokeStatic "connection.clj" 141],
     [next.jdbc.connection$get_driver_connection invoke "connection.clj" 136],
     [next.jdbc.connection$url_PLUS_etc$reify__9282 getConnection "connection.clj" 359],
     [next.jdbc.connection$make_connection invokeStatic "connection.clj" 385],
     [next.jdbc.connection$make_connection invoke "connection.clj" 369],
     [next.jdbc.connection$fn__9295 invokeStatic "connection.clj" 408],
     [next.jdbc.connection$fn__9295 invoke "connection.clj" 406],
     [next.jdbc.protocols$fn__9098$G__9093__9105 invoke "protocols.clj" 24],
     [next.jdbc.result_set$fn__10007 invokeStatic "result_set.clj" 908],
     [next.jdbc.result_set$fn__10007 invoke "result_set.clj" 831],
     [next.jdbc.protocols$fn__9150$G__9115__9159 invoke "protocols.clj" 33],
     [next.jdbc$execute_BANG_ invokeStatic "jdbc.clj" 244],
     [next.jdbc$execute_BANG_ invoke "jdbc.clj" 231],
     [konserve_jdbc.core$new_jdbc_store$fn__10375 invoke "core.clj" 276],
     [clojure.core.async$thread_call$fn__13326 invoke "async.clj" 484],
     [clojure.lang.AFn run "AFn.java" 22],
     [java.util.concurrent.ThreadPoolExecutor runWorker "ThreadPoolExecutor.java" 1149],
     [java.util.concurrent.ThreadPoolExecutor$Worker run "ThreadPoolExecutor.java" 624],
     [java.lang.Thread run "Thread.java" 748]]}
                  core.clj:   32  konserve-jdbc.core/prep-ex
                  core.clj:   29  konserve-jdbc.core/prep-ex
                  core.clj:  292  konserve-jdbc.core/new-jdbc-store/fn
                 async.clj:  484  clojure.core.async/thread-call/fn
                  AFn.java:   22  clojure.lang.AFn/run
   ThreadPoolExecutor.java: 1149  java.util.concurrent.ThreadPoolExecutor/runWorker
   ThreadPoolExecutor.java:  624  java.util.concurrent.ThreadPoolExecutor$Worker/run
               Thread.java:  748  java.lang.Thread/run

localshred20:10:21

Deps are

io.replikativ/datahike      {:mvn/version "0.5.1516"}
io.replikativ/datahike-jdbc {:mvn/version "0.1.2-SNAPSHOT"}

localshred20:10:46

aero/integrant config being passed in is

:myapp.db/name #profile {:dev  #env DB_NAME
                          :test #join [#env DB_NAME "_test"]
                          :prod #env DB_NAME}
:myapp/db {:keep-history? true
           :name #ref [:myapp.db/name]
           :store  {:backend  :jdbc
                    :dbtype   "postgresql"
                    :host     #env DB_HOST
                    :port     #long #or [#env DB_PORT "5432"]
                    :dbname   #ref [:myapp.db/name]
                    :user     #env DB_USER
                    :password #env DB_PASSWORD}}

localshred21:10:31

tl;dr with one db already created, a second fails to create (or to be checked d/database-exists? without throwing a "database does not exist" fatal error from the underlying postgres lib org.postgresql.core)

localshred21:10:05

calling d/database-exists? on the :dev profile config for the db simply returns true (as expected). swapping the config profile to :test and all hell breaks loose

localshred21:10:54

running d/create-database on the :dev profile config throws a :db-already-exists error (again, as expected)

localshred21:10:46

Hmm. After digging some more, it appears that the top-level :name is not used in either of those two functions. The [:store :dbname] is used in both cases, and unfortunately the postgres DB with that name has to already exist. konserve creates a single table which apparently manages all the datoms transacted through datahike (?).

localshred21:10:59

So. d/create-database and d/database-exists? must already need the postgres db of that name to be setup, in order to create the datahike db

localshred21:10:14

Open to correction if anyone followed along and has some insight for me that contradicts my understanding

whilo07:10:36

I am not familiar with how the Postgres backend exactly works, @U4GEXTNGZ or @UQVCR784A might know it better, but I can imagine that the Postgres DB must exist already.

timo12:10:20

@U5RFD1733 You are correct. The Postgres-DB needs to exist.

👍 1
localshred15:10:18

Understood, thanks for clarifying 🙂

whilo07:10:15

@U4GEXTNGZ Can there be multiple konserve stores per database?

1
localshred10:10:00

I don’t think it supports that. I ended up adding another Postgres instance to my docker compose and swap to that connection before running tests

localshred11:10:00

Though now that I say that I could just as easily have used one instance and just created a separate db for tests in that same instance in my dockerfile

timo12:10:33

that's what I thought as well @U5RFD1733... I never tried it but it shouldn't be a problem to create another db on the same postgres-instance.

localshred12:10:52

Ya I think my brain was so fried at the time that it didn’t even present itself as an option :face_palm::skin-tone-2:

whilo20:10:30

Since we only need a table in effect it might be good to be able to create multiple konserve stores in one database. In cases where you pay by DB instances this would be beneficial, e.g. on Heroku that might be beneficial (not sure).

localshred20:10:15

but with most dbs you can create multiple dbs per instance, so you should be able to avoid that issue. it's 1:1 for datahike/konserve <-> postgres db, but you can add multiple pg dbs per instance