This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-11-14
Channels
- # announcements (8)
- # babashka (37)
- # beginners (4)
- # biff (19)
- # cider (18)
- # clj-kondo (52)
- # clojure (26)
- # clojure-brasil (5)
- # clojure-dev (12)
- # clojure-europe (34)
- # clojure-nl (1)
- # clojure-norway (32)
- # clojure-uk (7)
- # core-logic (1)
- # data-science (5)
- # emacs (14)
- # honeysql (11)
- # hyperfiddle (37)
- # jobs (1)
- # malli (4)
- # off-topic (38)
- # pedestal (7)
- # portal (30)
- # releases (1)
- # remote-jobs (1)
- # tools-build (8)
- # vim (12)
Found a bug in production. Used the prod-repl to find it, soft-deploy to deploy the fix. took 20 minutes. Felt a surge of power course through me.

Hello all. I'm exploring Clojure web development (hobby project to start) and have started to look into Biff. I'm having some difficulties with deployment (I'm following the official tutorial), so I was hoping I could get some help here. The tutorial assumes Digital Ocean and Nginx, which is fine—but that's not my setup, and so I'm unsure what I need to do a deploy. Currently, my remote setup, which is my target for deploying projects, is a fairly vanilla Arch server that runs Lighttpd. It currently serves up some static sites as well as a private wiki; nothing too serious. My question is, what are the tools that I need to do it? I'm pretty sure with a bit of fumbling I can figure it out; but after a bunch of searching I've not found a lot of help. Any suggestions as to where to start would be greatly appreciated. My level of experience: none with Babashka other the Biff tutorial. A medium amount with Clojure as a language, but not very much with Clojure tooling. A medium amount with Arch and Lighttpd.
Hey! You'll want to take a look at https://github.com/jacobobryant/biff/blob/master/example/server-setup.sh (to understand what kind of stuff is normally added to a fresh ubuntu server) and https://github.com/jacobobryant/biff/blob/master/tasks/src/com/biffweb/tasks.clj (to see how the server is interacted with). You shouldn't need to change tasks.clj at all, but you'll need to make a custom version of server-setup.sh.
At a high level, server-setup.sh does the following:
• installs a few programs like Clojure and https://github.com/athos/trenchman
• creates an app
user for the biff app
• sets up a git repo for push-to-deploy (you won't need this if your local machine is mac/linux/WSL--it's just a fallback for plain windows users/people who don't have rsync)
• creates a systemd service for the biff app so that it'll run on server start (and so that it'll be auto-restarted if it crashes)
• gives the app
user sudo permissions for restarting the systemd service
• sets up a firewall with ufw
• sets up nginx as a reverse proxy for your app (i.e. incoming http requests go to nginx first, then nginx passes them to your clojure app)
• generates and installs an SSL certificate with certbot
to understand what happens when you deploy, you can read the source for the bb deploy
task: https://github.com/jacobobryant/biff/blob/5295ec19947eae0a1c71a8d9365c32ed081e3344/tasks/src/com/biffweb/tasks.clj#L362
basically it just uses rsync
to copy the files over to the /home/app
directory on your server, then it tells the server to restart the app.
anyway--I'm guessing the main different thing you'll need to do is just configure lighttpd for the biff app; the rest of it will hopefully be just translating apt-get
commands to pacman
and of course, if you run into any snags and you don't mind paying an extra $7/month, you can always just set up a separate server for the biff app 😉
Running into an issue with :biff/ensure-unique (see thread)
Trying to run this transaction:
(defn create-cohort [ctx name]
(biff/submit-tx ctx
[{:db/doc-type :cohort
:db/op :create
:cohort/name name}
[::xt/fn :biff/ensure-unique {:cohort/name name}]]))
Like so (create-cohort (biff/assoc-db @main/system) "mom and dad")
Seeing the error Transaction violated a constraint
{:tx
([:xtdb.api/match #uuid "6c964091-9868-4e5b-a05e-e911e17fbd95" nil]
[:xtdb.api/put
{:cohort/name "mom and dad",
:xt/id #uuid "6c964091-9868-4e5b-a05e-e911e17fbd95"}]
[:xtdb.api/fn :biff/ensure-unique #:cohort{:name "mom and dad"}])}
xtdb.clj: 386 com.biffweb.impl.xtdb/submit-with-retries
xtdb.clj: 374 com.biffweb.impl.xtdb/submit-with-retries
xtdb.clj: 407 com.biffweb.impl.xtdb/submit-tx
xtdb.clj: 403 com.biffweb.impl.xtdb/submit-tx
biffweb.clj: 639 com.biffweb/submit-tx
biffweb.clj: 631 com.biffweb/submit-tx
cohort.clj: 7 org.stinkless.rekonstruction.cohort/create-cohort
Here is the entire contents of the db as reflected by (biff/q (:biff/db (biff/assoc-db @main/system)) '{:find [(pull entity [*])]
:where [[entity :xt/id]]})
(Note the weird [nil])
#{[#:xt{:fn
(fn
[ctx kvs]
(let
[kvs
(for
[[i [k v]]
(map-indexed vector kvs)
:let
[sym (symbol (str "v" i))]]
{:k k, :v v, :sym sym})
query
{:find '[doc],
:limit 2,
:in (mapv :sym kvs),
:where (vec (for [{:keys [k sym]} kvs] ['doc k sym]))}]
(when
(<
1
(count
(apply
xtdb.api/q
(xtdb.api/db ctx)
query
(map :v kvs))))
false))),
:id :biff/ensure-unique}]
[nil]}
that is really weird... I just tried running a similar transaction on a new biff project and it worked. Not sure what's up with that [nil]
either. Maybe try stopping the app, deleting the storage
directory, then starting it up again and see if the transaction works?
Sorry for the confusion. I am using postgres now 🙂 This explains the [nil]
(biff/q (:biff/db (biff/assoc-db @main/system)) '{:find [entity]
:where [[entity :xt/id]]})
#{[#uuid "3f1df8fa-dbe4-4718-9e8e-a8e0097efd0d"]
[#uuid "d3f99f8e-1cf6-485e-9c23-7b634df707dd"]
[#uuid "00a35bd5-ec7c-4a57-9263-58a99751b16c"]
[:biff/ensure-unique]
[#uuid "de7faee5-6452-4ae8-add4-e143baa913ff"]
[#uuid "74a7cddf-5b7a-455d-81cb-c975ff92d692"]
[#uuid "428b94ee-2f0c-4e12-a240-050905f5e7b5"]
[#uuid "8ada6c68-6881-4207-bd6a-82b6ab8ca98b"]
[#uuid "9b786f18-c3d7-4339-973a-6af657babc50"]
[#uuid "2adbc49d-8646-46bc-8165-74a1df438c31"]
[#uuid "37fe7131-edce-47ad-b002-fce2aa4fba6f"]
[#uuid "b7882f39-0284-46f3-b916-f04c5c82f0b8"]
[#uuid "774f1b28-1aa7-4cdd-963a-8ad6601dbd1f"]
[#uuid "cbf57103-247b-4cc3-82bc-5c3087cb1766"]
[#uuid "a7d19e93-79f9-4d7a-9afe-e23f965bdf62"]}
I thought I had wiped and recreated my db as follows, but apparently that isn't working
root@cheapstories:~# systemctl stop app
root@cheapstories:~# sudo -i -u postgres dropdb cheapstories
root@cheapstories:~# sudo -i -u postgres createdb cheapstories
root@cheapstories:~# systemctl start app
ah got it. to wipe the DB you'll need to both drop the postgres DB and delete the storage
directory, since the latter is still used for indexes. maybe that'll fix it?
Is this the storage directory?
grep data_directory /etc/postgresql/12/main/postgresql.conf
data_directory = '/var/lib/postgresql/12/main' # use data in another directory
root@cheapstories:~# grep storage /etc/postgresql/12/main/postgresql.conf
root@cheapstories:~# ls -la /var/lib/postgresql/12/main/
total 92
drwx------ 19 postgres postgres 4096 Nov 14 19:28 .
drwxr-xr-x 3 postgres postgres 4096 Nov 10 16:54 ..
drwx------ 5 postgres postgres 4096 Nov 14 21:44 base
drwx------ 2 postgres postgres 4096 Nov 14 19:29 global
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_commit_ts
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_dynshmem
drwx------ 4 postgres postgres 4096 Nov 14 21:44 pg_logical
drwx------ 4 postgres postgres 4096 Nov 10 16:54 pg_multixact
drwx------ 2 postgres postgres 4096 Nov 14 19:28 pg_notify
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_replslot
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_serial
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_snapshots
drwx------ 2 postgres postgres 4096 Nov 14 19:28 pg_stat
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_stat_tmp
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_subtrans
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_tblspc
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_twophase
-rw------- 1 postgres postgres 3 Nov 10 16:54 PG_VERSION
drwx------ 3 postgres postgres 4096 Nov 10 16:54 pg_wal
drwx------ 2 postgres postgres 4096 Nov 10 16:54 pg_xact
-rw------- 1 postgres postgres 88 Nov 10 16:54 postgresql.auto.conf
-rw------- 1 postgres postgres 130 Nov 14 19:28 postmaster.opts
-rw------- 1 postgres postgres 110 Nov 14 19:28 postmaster.pid
Oh, I get it. /home/app/storage
yes 🙂
that totally worked!!
perfect!
if you had previously dropped the postgres table without deleting storage
, that possibly would've explained the weirdness; XT would've been in a bad state