Fork me on GitHub
#babashka
<
2023-05-18
>
anovick14:05:20

I notice that my babashka script takes longer than the intended (<1 second) runtime šŸ˜• on 8 PDF files it can take >5 seconds even. Can I use babashka.fs and babashka.process from a non-babashka CLJ program (on the JVM)? I'd like to compare the performance of the two, even though I'm pretty sure the other shell tool I'm using (`pdftk`) is the main performance obstacle. Is there anything else I need to change in my script? can I just run it with clj (assuming it is namespaced as shown here: https://github.com/amitnovick/bookmark-files-combiner/blob/main/src/bookmark_files_combiner/core.clj)

borkdude14:05:30

> Can I use babashka.fs and babashka.process from a non-babashka CLJ program (on the JVM)? Yes, of course

2
borkdude14:05:53

You just have to add those deps to a deps.edn file

šŸ‘ 2
anovick15:05:54

I'll try that šŸ™‚ thanks

joakimen15:05:03

In 95% of the cases where I shell out, I do the same thing ā€¢ run the command ā—¦ on exit 0, return :out as string ā—¦ on exit != 0, "pretty"-throw (no stack trace) with :err as message Hence I have this snippet floating all over my different scripts

(defn run [& args]
  (let [{:keys [out err exit]} (apply p/sh args)]
    (if (zero? exit)
      (str/trim out)
      (throw (ex-info (str/trim err) {:babashka/exit exit})))))
Is there a more standardised way to achieve this behaviour using babashka.process?

borkdude15:05:47

@U0422G22932 (:out (shell {:out :string} ...args...)) would be a shorter way of writing this. If you want the babashka/exit behavior you can use (babashka.tasks/shell ...) instead

šŸ‘€ 2
Peter Tonner18:05:51

is there a way to make an "extensible" CLI? basically I'm trying to build commands up from a base command and re-use CLI building blocks between them. Here's a basic example of what I'm trying:

(def my-cli {:org.babashka/cli {:alias {:n :dry-run?}}})
(defn base
  my-cli
  [{:keys [dry-run?]}]
  (if dry-run? (prn "here's what I'll do: ...") (actually-do!)))
(defn extend
  (assoc-in my-cli [:org.babashka/cli :alias :i] :interactive?)
  [args]
  (when interactive? (do-something!))
  (base args))
but this doesn't work, I think b/c the function metadata can't be a symbol

borkdude19:05:59

@U03G25L065N You can do this with:

(alter-meta! #'extend ...)
or
(defn ^{:org.babashka/cli (assoc-in (:org.babashka/cli my-cli) [:alias :i] :interactive?)} extend [])

borkdude19:05:13

So something like this:

(def my-cli {:alias {:n :dry-run?}})

(defn ^{:org.babashka.cli my-cli} base
  [{:keys [dry-run?]}]
  (when dry-run? (prn "here's what I'll do: ...") ))

(defn base+
  ^{:org.babashka.cli (assoc-in my-cli [:alias :i] :interactive?)}
  [args]
  (prn args))

Peter Tonner19:05:07

awesome, thank you!

Frank Henard19:05:43

I want to use babashka or even plain Clojure for server provisioning instead of Ansible. I found bbssh https://github.com/epiccastle/bbssh for shelling into my vps and running commands, but I'm having trouble connecting. CircleCI is failing on bbssh and it hasn't had a commit for 4 months. Is anyone else using bbssh or something like it, and if so any recommendations?

Frank Henard19:05:51

here's the debug text

Frank Henard19:05:56

The error I'm getting is Auth fail for methods 'publickey'

Frank Henard19:05:47

I can ssh into the vps outside of bbssh fine

Bob B21:05:17

I'm aware of (but haven't used) <https://github.com/epiccastle/spire>, which I guess might be one level up from bbssh, but I don't how much overlap there is

šŸ‘ 2
Kimo13:05:39

Crispin is rewriting spire, and bbssh is the first of the new modules. So spire might just work, since it's a different codebase. I've used it to provision a remote via pubkey, it's pretty cool what it can do. I looked over your log but I can't find any answers, sorry.

šŸ‘ 2
Crispin02:05:32

Looks like you are connecting but you cannot authenticate with your keypair. Have you given bbssh a key to use? Are you using an ssh agent to provide a key? Outside of bbssh, are you using any custom ssh config to tailor the connection parameters to that machine?

Frank Henard15:05:54

@U1QQJJK89, here's my connection details

(-> (bbssh/ssh "99.99.99.99"
                   {:username "ubuntu"
                    :identity "/Users/frankhenard/.ssh/id_rsa"})
        (bbssh/exec "echo 'I am running remotely'" {:out :string})
        deref
        :out
        )
I'm pretty sure I'm not using custom ssh config

Frank Henard15:05:52

I can ssh in fine from zsh with ssh [email protected]

Frank Henard15:05:26

this is an aws ec2 ubuntu instance

Crispin05:05:08

@U06BQ07JS in the bbssh script you are connecting with the private key stored in /Users/frankhenard/.ssh/id_rsa. Does the public key equivalent of this exist on the server for user ubuntu as an authorized key? When setting up an ec2 instance amazon will often create a new key for you to connect with. This new key may be being used in your ssh [email protected] invocation instead of your personal rsa key if you have added that aws key to your ssh-agent keychain. Probably the best way to debug this is turn verbose on when issuing your working ssh connection: ssh -v [email protected]. Look for a line like debug1: Authentications that can continue: publickey and then just after that it will show what key it is actually offering eg. debug1: Offering public key: RSA SHA256:hash_here /path/to/keyfile. Is this working ssh key offering the /Users/frankhenard/.ssh/id_rsa key, and does that key succeed?

Crispin05:05:10

One key difference between ssh proper and bbssh, is ssh command line will try many authentication methods one after another until one succeeds. In bbssh, when you specify :identity you are asking to only authenticate with that identity, not to try that identity first, and then if that fails, try offering keys in your running ssh-agent.

Crispin05:05:51

You can try to get bbssh to connect using the ssh agent by removing the identity line completely so it reads more like your ssh invocation. ie

(-> (bbssh/ssh "99.99.99.99"
                   {:username "ubuntu"})
        (bbssh/exec "echo 'I am running remotely'" {:out :string})
        deref
        :out
        )
If you are running an ssh-agent this will use that agent to find keys to connect.

Crispin05:05:20

You can find out if you are running an ssh-agent by issuing the following command at the command line ssh-add -l

Frank Henard19:05:31

@U1QQJJK89 Thanks for all your help with all this. I'm still having trouble connecting Here's the results of ssh -v [email protected]

Frank Henard19:05:57

> Does the public key equivalent of this exist on the server for user ubuntu as an authorized key? Yes, it's the first one which I added. The 2nd one is the original one that AWS puts on the server and gave me the private key file for results of ssh-add -l:

The agent has no identities.

Crispin01:05:55

@U06BQ07JS Something about your key that bbssh doesn't like. Try the methods mentioned here https://epiccastle.io/bbssh/03-how-to.html and see if they work. Particularly try "Allow connection to a legacy server that only supports RSA/SHA1 signatures".

Crispin01:05:10

Another thing to try is to add the key to the ssh agent with ssh-add /Users/frankhenard/.ssh/id_rsa first, and then tell bbssh to connect without any key identity passed to it (so it uses the ssh-agent).

Crispin02:05:42

If these don't work, in order to debug the problem I will need to recreate the situation, and for that I need to know exactly what type of RSA key this key of yours is. PKCS#1, PKCS#8 or OpenSSH. Have a read here https://superuser.com/questions/1515261/how-to-quickly-identify-ssh-private-key-file-formats. If you read the header line of the PEM file it can give you info (BEGIN RSA PRIVATE KEY vs BEGIN PRIVATE KEY vs BEGIN ENCRYPTED PRIVATE KEY vs BEGIN OPENSSH PRIVATE KEY) and ssh-keygen -l -f /Users/frankhenard/.ssh/id_rsa can give you the key length. I can then generate the same kind of key, and test against an Amazon EC2 ubuntu 22 instance.

Crispin12:05:05

@U06BQ07JS ok don't worry about any of that. I've managed to replicate the bug. It's an issue with using just :identity as a method of connecting. As a workaround, you should find it will work if you add your key to your ssh-agent and then connect without any :identity configuration at all. I am working on a fix now and will release a new version when that's done, hopefully later tonight.

nice 2
Crispin12:05:24

(add your key to your ssh agent from the command line with ssh-add /Users/frankhenard/.ssh/id_rsa)

Crispin02:05:19

@U06BQ07JS Give bbssh 0.4.0 a try with the plain :identity auth. It should be fixed.

Frank Henard12:05:38

will do, and I'll let you know, thanks!

Frank Henard22:05:40

@U1QQJJK89, that worked, nice fix!!

šŸ‘ 2