Clojurians Log v2

@feedmyinbox02_clojuri Yes, this is always possible.

We can relax this validation, it is mostly there to avoid accidentally overwriting existing DB's with the wrong configuration. @feedmyinbox02_clojuri What are your respective DB configs that clash?

I created a copy of production postgres db via heroku CLI pg:pull (which employs pg_dump). Local config I was attempting to connect to:

(def local-cfg
        {:store {:backend :jdbc
                 :dbtype "postgresql"
                 :host "localhost"
                 :port 5432
                 :user "postgres"
                 :password "whatever"
                 :dbname "from-heroku"}})

(def cfg
        {:store {:backend :jdbc
                 :dbtype "postgresql"
                 :host "ec2..."
                 :port 5432
                 :user "..."
                 :password "..."
                 :dbname "..."}})

Hmm, I see. I think the host is the problem here.

How :host is specified?

Sorry if I'm not understanding

I think this is where the consistency check failed. Could you use the "ec2...http://amazonaws.com" address in both cases (instead of localhost)?

Hmm, I think I am not being clear. The redacted values in cfg are my production credentials, which I know not to share and have replaced here with "..." The local db shows up in the Postgres GUI app I use to run a local Postgres server. When I try to connect to that local db, wouldn't the :host have to be "localhost"?

The cfg map that connects to production db is actually something like:

(def cfg 
  {:store 
    {:backend :jdbc
     :dbtype "postgresql" 
     :host "ec2..."  
     :port 5432  
     :user "ddxjb2hijdllx"  
     :password ; random string
     :dbname ; random string 
}})

IIUC, if I use the http://amazonaws.com EC2 address it will connect to the production db, which is not what I want. I want to connect to my local copy to experiment, etc.

Then they are different databases and should never clash, i.e. for some reason during your connection it sees the other db. I am not sure why this is happening.

Yes, when I (def conn (d/connect local-cfg)) I get the error ensure-stored-config-consistency. I will try again and include more of the error output perhaps.

@whilo isn't the stored-config stored in the konserve store which is a pg_dump from the prod-db? As I understand it we need a way to change the dump to accept the new configuration, isn't that the case here?

Yip that's correct. Doing a pg_dump would copy the config across. Hence the error. We'd need something like a create-new-from. Like a one time overide. Also, if I remember correctly the cache is part of the config check (accidentally I presume) which we need to fix.

I see, we haven't covered this case yet. Maybe we should allow overriding the consistency check and setting a new config.

This can be done by adding a guard for this check https://github.com/replikativ/datahike/blob/main/src/datahike/connector.cljc#L187.

@feedmyinbox02_clojuri Are you comfortable with open PRs?

Ideally we want people to be able to hack Datahike to their needs.

Where hack means adapt 😉 We are not hacking around in the codebase, but it should be able to explore modifications easily.

I've never done one but that shouldn't stop me!

Great! If you need help just ask here or put up the PR and we can discuss it there

I created the copy using the heroku CLI which provides pg:pull to copy a db. (I believe it runs pg_dump.)

hey @feedmyinbox02_clojuri. Hmm, I know this problem.... don't know exactly how to get around it. It is a check that validates the connection. I did not implement this validation and don't exactly know how to get around it. I guess the config you were using in prod is stored inside the dump. Maybe @whilo can chime in.

Thanks timo. I suppose I can try to find this row in local and just delete? (Afraid this might corrupt somehow.)

What is best practice to keep local and production instances of a Datahike db?

I would like to continue developing with Datahike but don't want to experiment on production db. What is best practice to create copy of prod to experiment with?

so the exception you are seeing above is expected. it's a way to globally identify databases that you can connect to. There should be a way to use a dump, not sure how though.

@whilo wants to improve documentation and there were a lot of changes as of recently so it is needed

@alekcz360 might have an idea as well on how to use a dump from prod locally

I suppose I could re-create the entire db by replaying all transactions. (I recall seeing this somewhere in docs, I think in migrate namespace.)