Clojurians Log v2

I’m trying to do this:

https://stackoverflow.com/questions/68665010/how-to-run-a-command-in-the-background-in-dockerfile?noredirect=1#comment121349858_68665010

tldr, I’m running these commands:

1 USER postgres
  2 RUN initdb
  3 RUN postgres -D /var/lib/postgresql/data &
  4 RUN psql -f bin/sql/postgres-db.sql -U postgres
  5 RUN psql -f bin/sql/postgres-table.sql -U postgres -d datomic
  6 RUN psql -f bin/sql/postgres-user.sql -U postgres -d datomic

=> ERROR [11/22] RUN psql -f bin/sql/postgres-db.sql -U postgres          0.3s
------
 > [11/22] RUN psql -f bin/sql/postgres-db.sql -U postgres:
#14 0.277 psql: error: could not connect to server: No such file or directory
#14 0.277 	Is the server running locally and accepting
#14 0.277 	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
------
executor failed running [/bin/sh -c psql -f bin/sql/postgres-db.sql -U postgres]: exit code: 2

@ps This channel is for discussions about docker as it relates to Clojure. I think this issue is best handled on stack overflow. That said, this isn't possible. Each statement in a Dockerfile, like RUN, is run independently and in isolation of every other statement in a Dockerfile. It's not possible to run a command in the background to start a service which can be connected to by other commands in other statements in the same Dockerfile.

@ps you should use the /docker-entrypoint-initdb.d/init-user-db.sh and put your commands there; as described here: https://github.com/docker-library/docs/blob/master/postgres/README.md#initialization-scripts

@tvaughan @rahul080327 @jaret so doing what @rahul080327 suggested works, but I have a similar problem further on in the dockerfile related to running #datomic . After doing all the postgres storage initialization, I have these three commands, to create a db and then to start the datomic server:

36 RUN bin/transactor sql-transactor-template.properties&
 37 RUN clj create_db.clj
 38 CMD bin/run -m datomic.peer-server -h 0.0.0.0 -p 8998 -a myaccesskey,mysecre    t -d humboi,"datomic:    atomic&password=datomic"

1 (ns create_db
  2   (:require [datomic.api :as d]))
  3
  4 (prn "creating db"
  5   (d/create-database "datomic:"))
  6 (System/exit 0)

Gives the error:

#25 22.41 15:31:06.919 [main] WARN datomic.kv-sql-ext - {:event :sql/validation-query-failed, :query "select 1", :pid 7, :tid 1}
#25 22.41 org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

@ps Please see my statement above: > Each statement in a Dockerfile, like RUN, is run independently and in isolation of every other statement in a Dockerfile. It's not possible to run a command in the background to start a service which can be connected to by other commands in other statements in the same Dockerfile. which means don't do this: > 36 RUN bin/transactor sql-transactor-template.properties& Create one container image, and run the transactor and peer separately

@tvaughan what is this?: https://docs.docker.com/config/containers/multi-service_container/

that's a very simple process manager 😉

you might as well just include runit or something

people used to do that and imho it's a Bad Idea ™️

that works only in dev context, not in production

> @tvaughan what is this? As @lukaszkorecki said, a bad idea

@tvaughan okay, so I created another container that solely runs the peer and the first one is left at running the transactor, but how would the two containers talk to each other? Specifically, if the transactor is running then this:

1  (ns create_db
  1   (:require [datomic.api :as d]))
  2
  3 (prn "creating db"
  4   (d/create-database "datomic:    datomic?user=datomic&password=datomic"))
  5 (System/exit 0)

you can look into something like https://docs.docker.com/compose/

I mean they will eventually talk with k8s, but what’s going to be the create-database string?

one way to do this is read connections from environment variables. set something like DATOMIC_URL for that container and use the k8s or some other orchestration to set it

Create a bridge network, e.g. docker network create --driver bridge datomic-network Then run the transactor and peer on this network, e.g. docker run --name datomic-transactor --network datomic-network ... and docker run --name datomic-peer --network datomic-network ... Use the name of the running containers to connect, e.g. in this example the peer would connect to datomic-transactor

@tvaughan a strange thing is happening on running the datomic transactor container. It runs for a few seconds but then gives:

Starting datomic:sql://<DB-NAME>?jdbc:, you may need to change the user and password parameters to work with your jdbc driver ...
System started datomic:sql://<DB-NAME>?jdbc:, you may need to change the user and password parameters to work with your jdbc driver
Terminating process - Lifecycle thread failed
java.util.concurrent.ExecutionException: org.postgresql.util.PSQLException: Connection refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.

localhost:5432 is most likely not correct

you need to point it to the postgres container. most likely its name. like @tvaughan mentioned in the network setup

here is an example of a docker compose using multiple services and a postgres container: https://github.com/bob-cd/bob/blob/main/docker-compose.yml#L53

@tvaughan does that mean that in addition to the transactor, a peer, I need a separate postgres container too?

Yes (though strictly speaking it just needs to be accessible from inside the running transactor container)

but as you said running two services in the same container is a bad idea

I mean a process, in this case the datomic transactor, needs to be able to connect to it. Postgresql should not be running inside the datomic transactor container. Postgresql could be another running container, or something like RDS