This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-10-02
Channels
- # announcements (15)
- # aws-lambda (1)
- # babashka (2)
- # beginners (25)
- # calva (2)
- # cider (6)
- # cljfx (6)
- # clojure (24)
- # clojure-boston (1)
- # clojure-dev (2)
- # clojure-europe (31)
- # clojure-uk (2)
- # clojurescript (48)
- # community-development (4)
- # conjure (1)
- # datahike (4)
- # datascript (10)
- # datomic (1)
- # deps-new (1)
- # fulcro (10)
- # juxt (1)
- # malli (7)
- # missionary (1)
- # nrepl (2)
- # off-topic (2)
- # other-languages (5)
- # polylith (4)
- # random (1)
- # react (2)
- # shadow-cljs (9)
- # tools-deps (21)
Good morning All. I am trying to learn how to test the interactions with the database. I have been trying to use https://github.com/Bigsy/pg-embedded-clj, For migrations, I use https://github.com/weavejester/ragtime. How can I copy schema from development db, to the pg-embedded db. I can use pg_dump
to export the schema and I simply tried to jdbc/execute! the schema like so:
(log/info (ojdbc/execute! ds (str/split
(slurp "resources/test_db.sql")
#"(?m)\n\s*--;;\s*\n")))
But I get this result:
Execution error (PSQLException) at org.postgresql.jdbc.PgStatement/checkNoResultUpdate (PgStatement.java:269)
Yes it does
You probably have something like
SELECT pg_catalog.set_config('search_path', '', false);
in the result of pg_dump
.
So you need to turn it and any other potential SELECT
statements into statements that don't return anything.
Like this:
DO
$$
BEGIN
PERFORM pg_catalog.set_config('search_path', '', FALSE);
END
$$;
OK, I commented it out, it seems to work.
@U2FRKM4TW do you build schema fixture this way or is there a cleaner way of doin it, is the best approach or should I do this by migrating schema?
I'm not doing any tests that involve DB right now. But I think it's a reasonable approach when the test DB is created in exactly the same way as you would create a production DB.
We do all our production updates via migrations, the same as we do in dev/test and in QA -- with the exception that we have some dev/test-only migrations that do not run on QA or production (these are often setting up fake local data for testing new features). Mostly these are straight-up SQL migrations: we have five migrations that completely teardown databases and rebuild the bare-bones stuff (that only runs on dev/test!) and then we have close to 900 migrations that run to build the dev/test DB up from that bare-bones baseline. The migration process runs all migrations that need to be applied, but skips the dev/test ones on QA and production -- but it's the same process run on all tiers.
That migration process is an uberjar we build in CI and auto-deploy to QA (and, later, to production) so the SQL migrations are fully-automated.
Thank you @U04V70XH6, this makes sense to me. I just had no idea how others are doing it. I will attempt this method.
OK both methods work fine. The challenge with the pg_dump
is the need to edit the exported schema. So I am going with something like:
; ------------------------------------------------------------------------------
(def ds {:dbtype "postgresql"
:classname "org.postgresql.Driver"
:subprotocol "postgresql"
:subname (str "//localhost:" 54321 "/postgres")
:user "postgres"})
; ------------------------------------------------------------------------------
(def ragtime-config
{:datastore (rjdbc/sql-database ds)
:migrations (rjdbc/load-resources "migrations")})
; ------------------------------------------------------------------------------
(defn with-pg-fn
"Startup with the specified configuration;
executes the function then shuts down."
[f]
{:pre [(fn? f)]}
(try
(pg/init-pg {:port 54321 :log-redirect "wibble.log"})
(log/info (repl/migrate ragtime-config))
(f)
(finally
(pg/halt-pg!))))
OK, amazing day, I had no idea how to test against db this morning, now I able to run tests against my migrated schema, thank you @U2FRKM4TW and @U04V70XH6!
The ordering of keys
and vals
isn't always consistent right? So this could be potentially buggy:
(defn fuse [kvs]
(post-complete (apply juxt (vals kvs))
(fn [acc]
(zipmap (keys kvs) acc))))
(assuming kvs is a map)for a given map, keys and vals always share the same order
(in the same clojure version at least)
if you need to pair them it's best to call (seq m)
The order is guaranteed to be consistent as long as the map is the same object. The docstrings mention it.
This seems a bit strange on first glance :face_with_raised_eyebrow: ... Am I missing something?
(= [:kw] '(:kw))
=> true
(= ['sym] '('sym))
=> false
I think this should help illuminate the issue:
> (pr-str '('sym))
"((quote sym))"
also:
> (= (list (list 'quote 'sym)) '('sym))
true
from https://clojure.org/reference/reader#_quote
> Quote (')
> 'form
⇒ (quote form)
or
> (= ['sym] '(sym))
true
> (= '[sym] '(sym))
true
> (= [(quote sym)] (quote (sym)))
true