Fork me on GitHub
#biff
<
2023-09-15
>
Martynas Maciulevičius09:09:22

What is a good way to run all tests? What I want is to not need to load all tests into REPL but instead run them via a command. Maybe it's a good idea to have a babashka task that executes all tests? The framework forces me to put my tests into src/ but then if I add a testrunner it expects to run from test/. Do you have this scenario in mind in some way?

Jacob O'Bryant15:09:22

you can put them in test. you probably want to add an alias to deps.edn that adds test to the classpath, and then have the test runner include that alias. a babashka task sounds like a good idea; it might just call something like (clojure "-M:test")

Jacob O'Bryant17:09:10

just took a look at what we have set up at work, and I think this should do the trick:

;; deps.edn
 :aliases {:extra-paths ["test"]
           :extra-deps {lambdaisland/kaocha {:mvn/version "1.0.629"}}}
# shell
clojure -M:test
maybe update the kaocha version

Martynas Maciulevičius05:09:27

I was thinking whether it would make sense to release this as part of the sample project of Biff. Because if I add my own configuration for myself then you may add this into the default and then I'd have to adjust it.

Jacob O'Bryant06:09:11

I probably will put something like this into the template project. The testing story has been on my todo list for a while, though it's still fairly far down there. I wouldn't worry about duplicating effort though. The example I pasted above is just two lines 🤷. plus updates to the template project are optional for existing projects; it's fine to just bump the biff version and leave the rest of your app alone. no need to stick to the defaults.

👍 1
Jacob O'Bryant23:09:21

I just pushed v0.7.13 which contains some bug fixes for various babashka tasks, including some regressions that were introduced in v0.7.11 and previous releases. Also there's a UX improvement around ssh; if you haven't set up ssh-agent or similar already and you're using an encrypted private key file, you'll only be asked for your decryption password once per bb command. I've tested on linux, mac and windows; hopefully all the kinks have been worked out. It'd be great if a couple people (cc @invertisment_clojuria) would try upgrading and running bb deploy as a smoke test. To upgrade, set your Biff dependency in tasks/deps.edn to com.biffweb/tasks {:git/url "" :deps/root "tasks" :tag "v0.7.13" :sha "6460590"}. Assuming everything works I'll write up some release notes for v0.7.13 sometime next week. changes: • if there isn't already an ssh-agent session, biff tries to start one for you, so you don't have to enter your ssh key password multiple times in a single command. • fix a regression in secrets.env from a little while ago that I believe made bb dev completely broken on plain windows (i.e. not WSL)... • fix a regression from v0.7.11 which made bb css (and several other commands by extension) broken if you don't have npm installed (even if you're using the standalone tailwind binary, as is the default). • fix another dumb bug from v0.7.11 in which bb deploy / bb soft-deploy is completely broken if you don't have rsync installed (i.e. on windows) • show a helpful error message if your tailwind binary is corrupted maybe I should start doing release candidates...

Martynas Maciulevičius05:09:05

I didn't update the project's biff version, only the version of the tasks. Project uses "v0.7.11" and tasks use "v0.7.13". This happens in the start. Seems to be ok for now.

Martynas Maciulevičius05:09:52

But after it compiles things it goes haywire (I entered my ssh password a couple of times):

Martynas Maciulevičius05:09:13

I didn't really mind to enter my password twice and I was happy with "v0.7.9". That was not a problem and I didn't care. I simply wanted it to work reliably. That's it. I don't care if it does it twice (as long as it's not more than twice). It's not about ergonomics of entering/not entering my key.

Jacob O'Bryant06:09:13

hmmmm Looks like the ssh agent isn't working. It's not causing any problems; it's just not preventing you from being asked for your password five times. I just counted up the number of times that bb deploy will ask for your password if you don't have an ssh agent session working, and it is in fact five 🙂 . If you entered your password that last time in the second screenshot, that should be the last time it asks. I can re-refactor bb deploy so that the max number of times it asks for your password is two (an rsync call and an ssh call). I would like to get the ssh agent thing working still, but limiting the number of ssh/rsync calls would be good as a fallback. What output do you get if you run:

eval $(ssh-agent)
ssh-add
echo $?
ssh-add -l
ssh  echo hello
(replacing with wherever your app is deployed) It should ask for your password after ssh-add and then not ask for it when you run ssh ..., but based on the screenshots I'm guessing the opposite will occur. hopefully ssh-add prints a helpful error message or something

Martynas Maciulevičius06:09:22

It failed on ssh-add:

Martynas Maciulevičius07:09:41

If I do this it works but then I'll not have the issue and this won't be solved. It will be fixing my config but it won't solve the 5-times password thing. And yes. Currently ssh-add -l prints one key that I added.

Martynas Maciulevičius07:09:54

If I try to deploy from a different terminal then it doesn't work. The first bb deploy call is with the version 9, the second call is with the version 13 (this white stripe is me removing title of the app):

Jacob O'Bryant18:09:14

> If I do this it works but then I'll not have the issue and this won't be solved. It will be fixing my config but it won't solve the 5-times password thing. I don't understand what you're saying here; can you rephrase? If you run ssh-add ~/.ssh/id_martin_digitalocean as in the first screenshot and then run bb deploy, does bb deploy still ask for your password? In any case I'm just trying to figure out where the root problem is so that if possible I can update biff to handle it. What output do you get from:

eval $(ssh-agent)
ssh-add
echo $?
i.e. a plain ssh-add without specifying your key file? I'm wondering if for some reason ssh-add isn't inferring where your keyfile is, even though ssh and rsync are able to find it.

Jacob O'Bryant18:09:26

Also: I've just pushed another commit. You can change tasks/deps.edn to have com.biffweb/tasks {:git/url "" :deps/root "tasks" :tag "v0.7.14" :sha "aad7173"} In this update: • The ssh/rsync commands are consolidated, so even if ssh-agent doesn't work, bb deploy will only ask for your password twice instead of five times. (Unless you're on Windows/don't have rsync...) • There'll be an error message if the ssh-add command fails • If you have any trouble with the ssh agent stuff, you can put :biff.tasks/skip-ssh-agent true in the :tasks section of config.edn

Martynas Maciulevičius18:09:30

> I'm wondering if for some reason ssh-add isn't inferring where your keyfile is I have 5 keys so I think that single ssh-add doesn't add the correct key. The digitalocean key is only used for this deployment and nothing else. I have separate keys for everything. What I did in the first screenshot was that I opened a second terminal and expected it all to work.

Jacob O'Bryant19:09:58

oh, yeah. I figured you probably had multiple ssh keys but didn't think through the implications, which I should have. ssh-add might be adding one of your other keys, or something. probably not much that's worth doing to try to handle that situation since I think even ssh sometimes has to just try each key one by one till it finds one that works. at least there's the message about keychain already. re: ssh-add and opening a new terminal: unless you're using keychain or you've set up ssh-agent in some other way so that it applies to your entire login session, the ssh-agent session is typically scoped to just the current terminal. so if you open a new one, you have to do eval $(ssh-agent); ssh-add <key file path> again. in any case, v0.7.14 should limit the password prompts to 2, so if that works then I think there's no other changes needed in the biff tasks for now. if the messages from bb deploy about keychain get annoying then you can set :biff.tasks/skip-ssh-agent true.

Jacob O'Bryant20:09:10

(I guess ssh/rsync does seem to be figuring out which key file it needs to decrypt, so there should be some way I can infer it the same way--but, eh. maybe I'll check if ssh-add can take a host name as a parameter or something)

Martynas Maciulevičius09:09:27

It seems to work well. I think this is good. There's an error in the beginning and the instruction should also be good.

Jacob O'Bryant19:09:49

Perfect, thanks for testing that.

Jacob O'Bryant18:09:26
replied to a thread:I just pushed `v0.7.13` which contains some bug fixes for various babashka tasks, including some regressions that were introduced in `v0.7.11` and previous releases. Also there's a UX improvement around ssh; if you haven't set up ssh-agent or similar already and you're using an encrypted private key file, you'll only be asked for your decryption password once per bb command. I've tested on linux, mac and windows; hopefully all the kinks have been worked out. It'd be great if a couple people (cc @invertisment_clojuria) would try upgrading and running `bb deploy` as a smoke test. To upgrade, set your Biff dependency in `tasks/deps.edn` to `com.biffweb/tasks {:git/url "<https://github.com/jacobobryant/biff>" :deps/root "tasks" :tag "v0.7.13" :sha "6460590"}`. Assuming everything works I'll write up some release notes for v0.7.13 sometime next week. changes: • if there isn't already an ssh-agent session, biff tries to start one for you, so you don't have to enter your ssh key password multiple times in a single command. • fix a regression in `secrets.env` from a little while ago that I believe made `bb dev` completely broken on plain windows (i.e. not WSL)... • fix a regression from v0.7.11 which made `bb css` (and several other commands by extension) broken if you don't have `npm` installed (even if you're using the standalone tailwind binary, as is the default). • fix another dumb bug from v0.7.11 in which `bb deploy` / `bb soft-deploy` is completely broken if you don't have rsync installed (i.e. on windows) • show a helpful error message if your tailwind binary is corrupted maybe I should start doing release candidates...

Also: I've just pushed another commit. You can change tasks/deps.edn to have com.biffweb/tasks {:git/url "" :deps/root "tasks" :tag "v0.7.14" :sha "aad7173"} In this update: • The ssh/rsync commands are consolidated, so even if ssh-agent doesn't work, bb deploy will only ask for your password twice instead of five times. (Unless you're on Windows/don't have rsync...) • There'll be an error message if the ssh-add command fails • If you have any trouble with the ssh agent stuff, you can put :biff.tasks/skip-ssh-agent true in the :tasks section of config.edn