This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-04-18
Channels
- # ai (2)
- # announcements (11)
- # beginners (34)
- # biff (14)
- # clerk (14)
- # clj-kondo (25)
- # clojure (27)
- # clojure-austin (1)
- # clojure-conj (6)
- # clojure-denmark (1)
- # clojure-europe (20)
- # clojure-hamburg (1)
- # clojure-nl (1)
- # clojure-norway (28)
- # clojure-uk (2)
- # clojuredesign-podcast (6)
- # clojurescript (43)
- # cursive (4)
- # data-science (2)
- # emacs (9)
- # hyperfiddle (9)
- # introduce-yourself (2)
- # jobs (3)
- # lsp (32)
- # missionary (31)
- # nbb (8)
- # off-topic (23)
- # rdf (23)
- # re-frame (10)
- # reitit (11)
- # releases (3)
- # rewrite-clj (4)
- # shadow-cljs (7)
- # specter (6)
- # sql (7)
- # xtdb (7)
Hello everyone, I’m asking myself some questions regarding SQL connections in some API request handler scenario. One SQL connection per request or several, sequentially? During the lifecycle of a request, I acquire “just in time” a connection from the pool, when I execute a SQL query. I often end up acquiring/releasing several connection, one for each SQL query, in sequence. However I remember from Rails the “connection per request”, acquired by a middleware and reused during the whole request lifecycle. My naive opinion of this former approach is the SQL connection could be held but inactive, during doing some slow computation outside of SQL, let’s say an HTTP call to an external service. Since this a common pattern, it’s probably not so simple. Does anyone have experience to share? Several connection pool or one? When the the connection pool is used with a mix of high throughput, very short transactions, and low throughput, long running transactions, I find that the slow queries are creating short bursts of pool saturation, which in turn create response time spikes on all other requests. I’m wondering if anyone has experience with having 2 connection pools, one for short transactions and another one for long running transaction. I’ve not found much resources online about this, only in HikariCP’s https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-Sizing#caveat-lector. Thanks!
Usually you don't start with separate pools, but separate databases, a read/write database for fast online processing, and a largely read only (possibly a replica) database for analytics and reporting
Hey team, has anyone played around with streaming the Postgres WAL in clojure? I am playing around with this, and following https://access.crunchydata.com/documentation/pgjdbc/42.0.0/replication.html Here's my current progress:
(def pg-conn (-> pool
(.getConnection)
(.unwrap org.postgresql.PGConnection)))
;; create slot
(sql/execute! pool ["SELECT pg_create_logical_replication_slot('test', 'wal2json');"])
;; log sequence number
(def lsn (org.postgresql.replication.LogSequenceNumber/valueOf
(:pg_current_wal_lsn (sql/select-one pg-conn
["SELECT pg_current_wal_lsn()"]))))
(def s (-> pg-conn
(.getReplicationAPI)
(.replicationStream)
(.logical)
(.withSlotName "test")
(.withStartPosition lsn)
(.start)
)
)
But I get the error
; (err) ERROR: syntax error at or near "START_REPLICATION"
; (err) Position: 1
Logs in postgres show that this statement is sent:
START_REPLICATION SLOT instant_server_stopa LOGICAL 0/40DCC70
and indeed something like this is generated here:
https://github.com/pgjdbc/pgjdbc/blob/f61fbfe7b72ccf2ca0ac2e2c366230fdb93260e5/pgjdbc/src/main/java/org/postgresql/core/v3/replication/V3ReplicationProtocol.java#L91
I am not 100% sure what the issue is. I am using Postgres-flavored Aurora. I am pretty sure logical replication is supported in Aurora, but maybe this command isn't? If anyone has ideas I'm all ears!