This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-12-30
Channels
- # announcements (5)
- # beginners (3)
- # biff (11)
- # cider (5)
- # clj-kondo (5)
- # cljdoc (4)
- # clojure (18)
- # clojure-dev (7)
- # clojure-europe (5)
- # clojure-gamedev (8)
- # clojure-norway (223)
- # clojurescript (4)
- # data-science (22)
- # emacs (5)
- # exercism (2)
- # fulcro (2)
- # graalvm (6)
- # jobs-discuss (4)
- # lsp (16)
- # matrix (1)
- # off-topic (41)
- # scittle (16)
- # xtdb (3)
Some notes about stuff I've been working on recently: https://forum.tfos.co/t/in-progress-changes-to-config-and-tasks/114
@U028ART884X I'm interested in your impressions of this since we've discussed config before and I believe you were in favor of doing it all through env variables.
I liked the previous approach because the variable name is 100% the same in the config.
And then you see that for the production setup you will use a database while for local setup you'll use basic file-based persistence.
And you don't need to look at the config file to see what setup you have for which env -- maybe you still need to look there but not for understanding the env var structure.
What I see in the refactored config is that the variable name has to be converted into kebab-case with /
sign somewhere in the middle.
Also the "deploy structure" is gone from that file and instead you expect the file to be "correct".
And for me the new version is very similar to simply running the env var retrieval through a helper function with a bunch of or
statements to apply defaults.
At this point I'd consider if I even want to use this new config format and I'd think about simply loading the env variables from the CLI myself (or maybe using a library) -- especially since I'd know that I can find that variable where it's used in the code by the variable name rather than guessing where the /
should be or what regex I should write to match files to find uses of BIFF_MIDDLEWARE_SECURE
by using grep -r "biff.middleware.secure" .
.
i.e. I think there is less value in the new config format even though you provide some defaults.
I'd rather want to crash the app startup when a variable is not provided as it will be a signal for me to look into the garbage that I inputted my the .env
file.
I understand the parsing aspect because parse-boolean
is a good way to specify how to parse things. And it's probably good to specify it there.
tl;dr;
I don't like boilerplate but I see value in seeing your partial config for each environment.
So I like the maps that you introduced because it's the necessary evil but I don't like the removal of the current structure and introduction of multiple .env
files.
Currently I have only one .env
file with both configs and most of it is shared.
Thanks for asking. I hope you'll choose the best design!
Thanks for the detailed feedback. Currently thinking through this.
At a high level, I think there are basically three situations that I'm thinking about/trying to serve:
1. Private apps deployed with bb deploy
(the most common situation for Biff apps I'm sure)
2. Private apps deployed with some other method, e.g. uberjar + docker / some PaaS like http://fly.io
3. Public/open-source apps (including https://github.com/jacobobryant/biff-postgres like the ones I've written to accompany various how-to guides)
For #1, the current way is fine/probably best. Mainly I'm trying to come up with an approach that will make #2 and #3 smoother. e.g. to combine those two, imagine there's an open-source Biff app you wanted to deploy to http://fly.io or some other PaaS. I'm trying to figure out what that should look like--what files are checked into the git repo, what are the instructions for people to deploy the app with their own config...
A large part of my thinking was that I wanted to keep as much of the config "structure" as possible checked into the source code, and then have all non-checked-in config be in the form of simple env variables. PaaSes generally have good support for setting config via env vars, so the instructions are just "deploy this repo as is + set your own env vars in the PaaS console/whatever."
However, this setup is less convenient for #1, which again is the most common situation. Maybe I can structure things a bit differently so you still have a config.edn
file like there is now, but it has some reader tags like #env "FOO"
, #secret "BAR"
etc... similar to https://github.com/juxt/aero. And have config.edn
be checked into source by default. Then for #1, people can do what they do now--put normal config directly in config.edn
and put secrets in secrets.env
. But then it's easy to move any additional config values out of config.edn
and into environment variables if you want.
In fact maybe just using Aero to parse config.edn
would do the trick. Would be worth thinking about/experimenting with. I am wondering if it has too many features though and defining a couple of simple #env
and #secret
reader tags would be all Biff needs.
Anyway I'll do a second take on a proof-of-concept and report back!
got a simple thing set up with Aero. I think that's actually gonna be the way to go.
I really appreciate the time spent on deployment capabilities. It is such a thrill to get a project live so easily. It would be neat to be able to easily dockerize a Biff app. The impediment there for me has mostly to do with my clumsiness with docker in general (rather than anything to do with Biff). But +1 on anything that streamlines or assists containerizing.
I think dockerizing is not as basic as the shellscript deploy because what happens after you build it? You also have to understand where to put this docker image and also understand scaling part of things (what's the point of dockerizing if we don't do it?). And the script handles the deploy to the end. So probably the docker deploy script should include a terraform config that manages everything to the end. And this is way more work than just containerizing.
I totally hear you. And Jacob knows I'm attracted to unnecessary complexity.
I am in the process of adding an uberjar command + a Dockerfile to the template app. For most people I think sticking with bb deploy
/ not using docker will be best. But in some cases people will have a specific platform in mind they want to deploy to using Docker (kubernetes / http://fly.io / ...), and then if Biff can take care of the Biff-specific parts of building a Docker image, they can take that and deploy it.
(I have/had a working Dockerfile + bb uberjar
command, but in the process of cleaning that up I started thinking it'd be worthwhile to try switching from bb
to clj
for tasks first, and then since I was working on that I figured might as well revamp the config stuff at the same time...)
config take 2: https://forum.tfos.co/t/in-progress-changes-to-config-and-tasks/114/5?u=jacobobryant
Thanks for the detailed feedback. Currently thinking through this.
At a high level, I think there are basically three situations that I'm thinking about/trying to serve:
1. Private apps deployed with bb deploy
(the most common situation for Biff apps I'm sure)
2. Private apps deployed with some other method, e.g. uberjar + docker / some PaaS like http://fly.io
3. Public/open-source apps (including https://github.com/jacobobryant/biff-postgres like the ones I've written to accompany various how-to guides)
For #1, the current way is fine/probably best. Mainly I'm trying to come up with an approach that will make #2 and #3 smoother. e.g. to combine those two, imagine there's an open-source Biff app you wanted to deploy to http://fly.io or some other PaaS. I'm trying to figure out what that should look like--what files are checked into the git repo, what are the instructions for people to deploy the app with their own config...
A large part of my thinking was that I wanted to keep as much of the config "structure" as possible checked into the source code, and then have all non-checked-in config be in the form of simple env variables. PaaSes generally have good support for setting config via env vars, so the instructions are just "deploy this repo as is + set your own env vars in the PaaS console/whatever."
However, this setup is less convenient for #1, which again is the most common situation. Maybe I can structure things a bit differently so you still have a config.edn
file like there is now, but it has some reader tags like #env "FOO"
, #secret "BAR"
etc... similar to https://github.com/juxt/aero. And have config.edn
be checked into source by default. Then for #1, people can do what they do now--put normal config directly in config.edn
and put secrets in secrets.env
. But then it's easy to move any additional config values out of config.edn
and into environment variables if you want.
In fact maybe just using Aero to parse config.edn
would do the trick. Would be worth thinking about/experimenting with. I am wondering if it has too many features though and defining a couple of simple #env
and #secret
reader tags would be all Biff needs.
Anyway I'll do a second take on a proof-of-concept and report back!