datahike

whilo 2024-09-11T08:34:56.283699Z

Unfortunately, our release bot is currently on strike for datahike. Thanks to @uppfinnarjonas a new release of datahike is out which doubles the query performance on the benchmark suite of the Swedish government. Thanks to @timok’s work we also get native builds on github now. The dthk binary and babashka pod are already available for Linux amd64 and aarch64, but the Mac OS build still fails at a unit test.

bdbrodie 2024-09-12T07:51:57.030629Z

That is an impressive performance improvement, looking forward to testing this release soon.

❤️ 1
whilo 2024-09-11T17:19:00.374229Z

The bot is alive again 😄. Thanks @timok!

🎉 1
silian 2024-09-11T17:52:57.959679Z

Heroku occasionally rotates database and provides new credentials. When trying to deploy with the new credentials, datahike is throwing the config-does-not-match-stored-db error.

silian 2024-09-11T17:54:58.123459Z

I don't believe I specified :scope in my app in the older deployments but specifying it now (with the new db credentials) doesn't seem to help or override the error. I can't even connect to the current deployment db to make a copy of it, because datahike will throw config-does-not-match-stored-db.

silian 2024-09-11T17:55:37.092349Z

Somewhat at a loss. I don't want to destroy anything without making a copy but I can't access the db!

whilo 2024-09-11T18:16:33.715069Z

Which datahike version are you using?

whilo 2024-09-11T18:17:05.775419Z

I relaxed the config check in a recent version, but I might have to expand that.

whilo 2024-09-11T18:17:17.756699Z

How are the credentials passed? I assume as part of the store config.

silian 2024-09-11T18:17:28.803869Z

just updated to most recent. Still failed

silian 2024-09-11T18:17:34.091359Z

Same error

whilo 2024-09-11T18:17:37.571479Z

Ok

silian 2024-09-11T18:17:51.711879Z

Yes, in the store config

silian 2024-09-11T18:19:59.355399Z

Like so:

(def cfg
  {:store {:backend :jdbc
           :dbtype "postgresql"
           :host ""
           :port 5432
           :user ""
           :password ""
           :dbname "dc6bbrav2u4lve"
           :scope "already-set-this-huh"}})

whilo 2024-09-11T18:40:22.605089Z

Is the dbname changing for you?

whilo 2024-09-11T18:40:59.062239Z

The store identity for JDBC should not depend on the password.

whilo 2024-09-11T18:43:48.113109Z

This is how the store is identified https://github.com/replikativ/datahike-jdbc/blob/main/src/datahike_jdbc/core.clj#L21

whilo 2024-09-11T18:44:20.675659Z

I assume that table stays the same, then the culprit should be dbname here.

whilo 2024-09-11T18:44:40.755079Z

@alekcz360 has worked with JDBC a lot, but I think not with Heroku yet.

whilo 2024-09-11T18:48:20.926919Z

We can relax that further, the config check is there to not accidentally corrupt a database.

whilo 2024-09-11T18:48:29.402479Z

By using the wrong configuration.

alekcz 2024-09-11T18:54:46.617949Z

@feedmyinbox02_clojuri in the error it tells you what has changed. Is the dbname the culprit?

silian 2024-09-11T19:16:53.746819Z

dbname does change, actually

silian 2024-09-11T19:17:57.748779Z

Let me check the error report

silian 2024-09-11T19:22:10.856029Z

:diff
({:store
  {:dbname "dc6bbrav2u4lve", 
   :password "", 
   :user "u10ue9c465kc7o", 
   :host "c4[...]."}}
 {:store 
  {:dbname "d3ldb5vhikjho2", 
   :password "", 
   :user "jhhyishypdwycq", 
   :host "ec2[...]."}}
 {:keep-history? true,
  :search-cache-size 10000,
  :index :datahike.index/persistent-set,
  :store {:port 5432, :dbtype "postgresql", :backend :jdbc}, :store-cache-size 1000, :attribute-refs? false, :crypto-hash? false, :schema-flexibility :write, :branch :db})

whilo 2024-09-11T19:22:31.701839Z

Ok, this is a bit odd, but I guess it is what it is.

whilo 2024-09-11T19:23:35.660209Z

Probably the best way to deal with oddities like this is to allow deactivation of the config check altogether at your own risk.

silian 2024-09-11T19:24:03.445629Z

How would I do that?

silian 2024-09-11T19:24:38.307349Z

That is a boolean I can set somewhere in the store?

whilo 2024-09-11T19:25:11.010989Z

I need to add an option for that.

whilo 2024-09-11T19:25:26.870839Z

I think I would probably add a field to the config itself to avoid dynamic bindings.

alekcz 2024-09-11T19:25:34.135719Z

We'd have to update the datahike with a property like allow-unsafe-configor something along those lines

whilo 2024-09-11T19:25:52.568849Z

Alternatively it could be an environment variable, but then it is global.

silian 2024-09-11T19:27:10.309039Z

Is there anything I can do at the moment less nuclear than reconstituting the entire production db?

silian 2024-09-11T19:27:54.770479Z

... oh wait, I can't do a migration or copy. I don't know how to get a new conn. This is blocking me from making a new conn even

silian 2024-09-11T19:30:16.979039Z

Wait, this is bad. How can I make a copy of the production db using Datahike tools if this error is blocking?

alekcz 2024-09-11T19:32:11.025909Z

@whilo you want to work on a new cut? or should I pick it up? I can do it this evening?

whilo 2024-09-11T19:33:41.123749Z

@feedmyinbox02_clojuri try this PR https://github.com/replikativ/datahike/pull/693

whilo 2024-09-11T19:34:13.579259Z

Lmk whether it works. You need to set :allow-unsafe-config true in the configuration.

✅ 1
whilo 2024-09-11T19:34:34.898179Z

I might have to add stuff the config specification, I have not checked this fix yet.

silian 2024-09-11T19:34:38.948689Z

Ok, trying now. Thank you

silian 2024-09-11T20:40:04.041419Z

Hi whilo, it didn't seem to work. I am DM'ing you direct copy-paste of error

silian 2024-09-11T20:50:31.843099Z

The PR (#693) provided by whio worked! 😊 Thank you

whilo 2024-09-11T20:58:29.216759Z

You are welcome! Lmk if there is anything else causing you issues.

1
whilo 2024-09-11T22:23:33.262119Z

This release contains a simple fix to deactivate our conservative configuration checks when connecting, as some people stumbled over those.

whilo 2024-09-11T22:23:57.282999Z

Thanks to @feedmyinbox02_clojuri for reporting this problem on Heroku.

🙏 1
silian 2024-09-11T23:18:14.533609Z

Thanks to the team for being excellent, speedy and responsive! (Just like Datahike! ;-)

❤️ 2