Fork me on GitHub
#babashka
<
2022-09-22
>
Crispin04:09:45

How can I programmatically get the file attributes for a file in bb? I don't want to shell out (I will be recursively querying file modes) but a lot of java.nio.file. seems to not be available.

lispyclouds05:09:00

this is using the java.nio.file.Files/readAttributes

Crispin05:09:46

ah this looks like it! What to I pass to attributes?

Crispin05:09:32

awesome got it

Crispin05:09:35

user=> (fs/read-attributes "deps.edn" "posix:permissions")
{:permissions #{#object[java.nio.file.attribute.PosixFilePermission 0x4faa298 "OWNER_READ"] #object[java.nio.file.attribute.PosixFilePermission 0xcf0e8b4 "GROUP_WRITE"] #object[java.nio.file.attribute.PosixFilePermission 0x7e168d90 "GROUP_READ"] #object[java.nio.file.attribute.PosixFilePermission 0x1e7245da "OTHERS_READ"] #object[java.nio.file.attribute.PosixFilePermission 0x1cd3b138 "OWNER_WRITE"]}}

1
mdallastella06:09:55

Morning. Does anybody ever used bb inside a systemd service? Any cons?

lispyclouds06:09:45

should be fine, its like any other binary. what are you trying to do?

mdallastella06:09:55

Starting a couple of Oracle dbs... unfortunately the Oracle documentation is a bit messy. I found a way with bash but I feel more confident with Clojure

lispyclouds06:09:09

yeah should work, you can have a

[Service]
...
ExecStart=bb your-script.clj ...
...

1
borkdude07:09:08

Also you can use babashka.process/exec to replace the bb parent process with another process, in case you don't want bb to keep running after starting the other process

mdallastella07:09:18

Thanks, I was already thinking about that! Appreciate the confirm.

Søren Sjørup09:09:30

With babashka.curl I’m using the {:throw false} option. But what is the idiomatic way of checking for error? When the request is successful I get a :err "" key in the response. I would’ve expected the err key to not be there when the request is successful.

borkdude09:09:52

:err can be anything including logging, don't attach too much meaning to that. similar to when shelling out

borkdude09:09:29

status code is probably the best thing to check

Søren Sjørup09:09:57

Thanks. I tried using :status but that is nil when the protocol is garbled.

❯ bb -e '(curl/head "" {:throw false})'
{:status nil, :headers {}, :body "", :err "curl: (1) Protocol \"httpsx\" not supported or disabled in libcurl\n", :process #object[java.lang.ProcessImpl 0x7803597c "Process[pid=18897, exitValue=1]"], :exit 1}

borkdude09:09:02

then check :exit too

borkdude09:09:33

whenever exit is positive, it means an error

Søren Sjørup09:09:46

ah, thank you!

👍 1
the-alchemist17:09:20

@borkdude: please keep up the great work 🙂

2
❤️ 1
the-alchemist17:09:42

got a question, tho: anyone try using the pod-babashka-aws with a custom AWS profile? I always get Access Denied because it’s using the default but aws s3 ls in the terminal works because it’s using the profile defined in the env variable AWS_PROFILE

borkdude17:09:44

@the-alchemist Thanks! There is more info on credentials here: https://github.com/babashka/pod-babashka-aws#credentials I also recommend checking out https://github.com/grzm/awyeah-api which is cognitect aws-api adapted for bb

the-alchemist17:09:35

yeah, i tried creating a profile-credentials-provider but that didnt’ work either :thinking_face: I don’t think the pod is using the same credentials process as the Java SDK

the-alchemist17:09:58

do I have a typo or something?

(def s3-client (aws/client {:api :s3
                            :credentials-provider (credentials/profile-credentials-provider "MY_CUSTOM_PROFILE")}))

borkdude17:09:40

Unfortunately I don't use AWS a lot, but perhaps @U7ERLH6JX and/or @U0EHU1800 can help you. I recommend trying @U0EHU1800’s library, awyeah-api too, which is probably easier to debug, since it's not a pod

the-alchemist17:09:32

ok, thanks 🙂 I’ll take a look what’s @U7ERLH6JX’s relationship with the pod / awyeah-api?

borkdude17:09:44

lispyclouds works a lot with AWS

borkdude17:09:08

and has also contributed to this pod

👍 1
the-alchemist18:09:06

DEBUG [com.grzm.awyeah.credentials:?] - Unable to fetch credentials from environment variables.

borkdude18:09:30

Also try with the normal cognitect aws api I'd say

the-alchemist18:09:39

yeah, that was my next step, thx

the-alchemist18:09:17

doesn’t work with cognitect-labs/aws-api, sign, so gotta dig into that

grzm18:09:36

Note that that's debug message. I don't recall if that's echoed when it's not picking up AWS_PROFILE or just the AWS_SECRET_KEY and the like. We use AWS_PROFILE with awyeah-api (with bb) and aws-api (Clojure) all the the time, so I suspect there's something else going on. Maybe dump your env vars via System/getenv to see what the runtime is seeing.

hoppy18:09:29

sorry, should have thread posted.

hoppy18:09:17

this works with wasabi, so pretty much everything is custom - no environment required

the-alchemist18:09:57

@U19EVCEBV: do you mind posting a redacted copy of your profile config from ~/.aws/config? mine is :

[profile Developer]
role_arn = arn:aws:iam::123465854:role/DeveloperRole
source_profile = default
output = json
region = us-east-1
I wonder if it’s the role_arn that’s messing things up

hoppy18:09:10

sure, sec

hoppy18:09:55

[default] aws_access_key_id=?? aws_secret_access_key=?? [wasabi] aws_access_key_id = ?? aws_secret_access_key = ??

borkdude18:09:12

Maybe [profile Developer] should be [Developer]?

the-alchemist18:09:41

nah, aws s3 ls --profile Developer works great

the-alchemist18:09:22

oh, interesting. my access key and secrets are in ~/.aws/credentials, not ~/.aws/config

hoppy18:09:02

as are mine ?

hoppy18:09:43

here's the whole of the script. Just a personal project to sync a bunch of still pictures up to s3 on wasabi - maybe of some use for ya.

the-alchemist18:09:15

yeah, i dunno. i’ll put this aside for now. everything works OOTB for aws cmdline and boto3. i guess i’ll have to manually write some Java code to see if it’s the Java SDK

the-alchemist18:09:51

but thank you everyone 🙂

👍 1
the-alchemist18:09:41

oh, haha, i was right. role_arn is not supported: https://github.com/cognitect-labs/aws-api/issues/172

borkdude18:09:18

Perhaps @U0EHU1800 can support it in awyeah-api 🙈

grzm19:09:44

I'd have to look and see what would be involved. My current self-imposed remit is to have feature parity with aws-api, not necessarily going beyond. (We also use awyeah-api/aws-api in common code paths via cljc in house, so explicitly supporting a awyeah-api feature that we couldn't also use with aws-api would be a bit counter-productive. Gotta have a think.

👍 2
borkdude19:09:16

Makes sense!

wilkerlucio21:09:04

hello, question, in babashka process, is there a way to get a callback when the process is done? or blocking is the only way? (wondering if I can have an async thing without having to make a new thread to block in)

Bob B01:09:52

I'm fairly confident there isn't a built-in callback - an option would be to throw it in a future (using a thread from a pool)

wilkerlucio01:09:13

yeah, I was trying to figure if there was a way, since it already runs async (considering we can fire multiple commands and wait for those later), I wonder if there isn't a way to get notified when it ends, instead of having to block for it, so we could avoid adding a new thread

borkdude06:09:30

I think the underlying process stuff supports this. You can get the process via the :proc key. We can expose this via a function or so. Issue welcome

borkdude07:09:38

$ bb -e '(let [prom (promise) proc (:proc (babashka.process/process ["ls"]))] (-> (.onExit proc) (.thenRun (fn [] (deliver prom (.exitValue proc))))) @prom)'
0

borkdude11:09:52

I now published process 0.2.9 with:

(deftest exit-fn-test
         (let [exit-code (promise)]
           (process ["ls"] {:exit-fn (fn [proc]
                                       (deliver exit-code (:exit @proc)))})
           (is (int? @exit-code)))))))

borkdude11:09:00

so the :exit-fn option is new

💯 1
wilkerlucio12:09:01

amazing, thanks man!

wilkerlucio12:09:36

@borkdude one thing I notice, the process object that gets in :exit-fn, the :exit value is nil there, and when using :out :string, I also have to deref from :out to get the string, do you think we can make the map behave there the same way as when we deref the process?

wilkerlucio12:09:53

I notice if I just deref the proc that comes on :exit-fn I get all I want, so I'm wondering what you think, not sure if there is any extra delay if that comes done from bb proc, anyway its easy enough now that I'm find with whatever option you prefer

wilkerlucio12:09:50

this is what I wanted to do from start, and now it works great:

(defn process [cmd opts]
  (p/create
    (fn [resolve reject]
      (letfn [(exit-fn [response]
                (let [{:keys [exit] :as r} @response]
                  (if (pos? exit)
                    (reject r)
                    (resolve r))))]
        (proc/process cmd (assoc opts :exit-fn exit-fn))))))
this wraps the thing in a promise, so now I can use it with Promesa to get async behavior without adding an extra thread 😄

borkdude12:09:06

@U066U8JQJ Ah, do you think I should have deref-ed the response perhaps? I think that makes more sense... I can do another release

👍 1
borkdude12:09:54

I think that's a very nice snippet. Maybe we should include it in the README

wilkerlucio12:09:14

sure, use it as you wish 🙂

borkdude15:09:01

@U066U8JQJ Now released 0.2.10 which returns the already deref-ed result to the exit-fn

🙌 1
🎉 1
wilkerlucio15:09:25

I suggest mentioning version 0.2.10 in the docs, if a user tries that snippet on 0.2.9 it will fail because :exit will be nil

borkdude15:09:54

fixed

👍 1