Fork me on GitHub
#babashka
<
2021-12-06
>
awb9914:12:58

How can I get the out value as a string from babashka.tasks/shell ? I get this: {:proc #object[java.lang.ProcessImpl 0x6942dbc4 Process[pid=27469, exitValue=0]], :exit 0, :in #object[java.lang.ProcessBuilder$NullOutputStream 0x628b0922 java.lang.ProcessBuilder$NullOutputStream@628b0922], :out #object[java.lang.ProcessBuilder$NullInputStream 0x2aa44cab java.lang.ProcessBuilder$NullInputStream@2aa44cab], :err #object[java.lang.ProcessBuilder$NullInputStream 0x2aa44cab java.lang.ProcessBuilder$NullInputStream@2aa44cab], :prev nil, :cmd [guix describe]}

borkdude14:12:26

@hoertlehner

(-> (shell {:out :string} "ls") :out)

awb9914:12:49

are the out options specified somewhere?

borkdude14:12:21

but I should add an example of this to the tasks as well

awb9915:12:11

when I specify :out :string, then I do get the output, which is good.

awb9915:12:24

However, I would like to see the shell output of the app I am running in the mean time.

awb9915:12:45

I have a shell task that runs for a few days (it is cross compiling a guix operating system),

awb9915:12:17

and I want to take only the last output of the script (which is the iso image file location), but show all the other output in the mean time.

borkdude15:12:30

Perhaps it's best to use tee and write to a file and to stdout. And then you can read from the file at the end

awb9915:12:38

I think thi sis a quite common useage pattern. Many apps log debug/info/warning messages during runtime that wa.

borkdude15:12:48

or write the iso name to a file at the end

borkdude15:12:51

and then read from that file

borkdude15:12:26

usually log messages go to stderr, not stdout

borkdude15:12:52

you can capture stdout with :out and stderr with :err , by default they are both inherited (you see them as they come out)

awb9915:12:12

Thats a good info that log messages go to stderr

awb9915:12:17

a very good convention.

👍 1
awb9915:12:36

inherited?

awb9915:12:46

I dont see any output during runtime now anymore.

borkdude15:12:51

you can also make your own output handler: a loop that reads the output line by line, and prints it and scans it for certain interesting information. This is described in the README of babashka.process.

awb9915:12:51

only the final result.

awb9915:12:56

since I added :out :string

awb9915:12:08

I will read that.

borkdude15:12:53

> I dont see any output during runtime now anymore. You can try :err :inherit to make sure stderr is still visible

borkdude15:12:13

but if they log to stdout, then it ends up in the string result

awb9915:12:41

Is there any demo that shows how to use babashka tasks with cli menu utils?

awb9915:12:59

I have now 30 tasks in my bb.edn file

awb9915:12:22

and for many tasks, it would be good to have a little menu, that lets one select say a profile out of a list of profiles.

awb9915:12:40

That would make my bb tasks shorter.

borkdude15:12:51

You can probably do this in your shell, using some shell specific code. Or some other program you can shell out to. https://askubuntu.com/questions/1705/how-can-i-create-a-select-menu-in-a-shell-script

borkdude15:12:05

There are also bash auto-completions in the book.

awb9915:12:20

the bash auto completions is a really cool thing.

borkdude15:12:21

And you can often group tasks by task-prefix:foo:bar and then use the auto-complete

awb9915:12:32

I cI didnt have time to play around wiht it,

awb9915:12:37

but really a good idea.

awb9915:12:03

the bash auto completions display a menu?

awb9915:12:16

or they only show after I enter something that can be completed?

awb9915:12:49

I am just a week or so into babashka tasks,

borkdude15:12:55

In my zsh I see this when I type bb up<TAB>

$ bb up
update-project-clj  upload-jar

awb9915:12:03

but I can tell you that I have already eliminated 100 or more shell scripts.

awb9915:12:14

They are all babashkarized now.

borkdude15:12:24

I'll be afk now for a bit.

awb9915:12:50

thanks again

cap10morgan20:12:29

Has anyone run into a ClassNotFoundException in data.xml's usage of codec.base64 when building babashka in a Docker container?

borkdude20:12:21

is that specific to docker? can you make a repro?

cap10morgan20:12:01

yeah it seems to be; looks like it fails w/ similar errors elsewhere in the code if I disable the xml feature

cap10morgan20:12:05

builds fine outside docker

cap10morgan20:12:23

even a lean build fails

cap10morgan20:12:51

I was trying to build a linux/arm64 binary

cap10morgan20:12:20

which worked in a fedora vm, but I wanted to get it working in docker since (IIRC) GitHub Actions don't have arm runners yet

cap10morgan21:12:40

not sure yet; I need to investigate a little more. all I changed was to make it download the aarch64 graalvm tarball

cap10morgan21:12:48

I can open a ticket w/ just that if you want

borkdude21:12:39

what are you doing to build. Just executing script/uberjar + script/compile?

cap10morgan21:12:55

just running your docker build, so yeah

borkdude21:12:16

but we are also running that Docker image in CI, so there must be something different?

cap10morgan21:12:53

different arch and thus different graalvm distribution are the only differences I'm aware of

cap10morgan21:12:06

trying it in docker in fedora now

borkdude21:12:52

but can you actually do this without Docker without failing?

borkdude21:12:01

The build failure looks similar to what I'm sometimes seeing if I have already AOT-compiled and then run using a different Clojure version.

cap10morgan21:12:45

it seems to be working in Docker on Fedora

cap10morgan21:12:01

might have to restart or reset my docker desktop on mac and see if that helps

borkdude21:12:32

Are you sharing local build stuff with your Docker env?

cap10morgan21:12:45

well, it's almost entirely your Dockerfile still (except for the patch in the issue I just opened). so it's still doing the COPY . .. but I'm using o/w clean master HEAD & v0.6.8 tag.

cap10morgan21:12:37

ok it worked in fedora docker so I'm betting this is a weird environment thing in my mac docker desktop. I'll keep playing around w/ it.

cap10morgan21:12:04

hoping to end up w/ a PR that can build a linux arm64 release in docker in your CI (I have a multi-platform docker-based GitHub Action I've been using already that uses qemu to build the arm64 binary on the x86_64 standard action runners)

cap10morgan21:12:38

and then we can build an arm64 docker image around that

cap10morgan21:12:03

adding .cpcache/ and target/ to .dockerignore fixed it

cap10morgan21:12:30

you were right that it was bringing in things from the host build that it shouldn't

borkdude21:12:31

yeah, you were probably coping target in there and this caused that bug

borkdude21:12:58

ok, github issue can be closed then

cap10morgan21:12:58

want me to put up a PR just for that change first?

cap10morgan21:12:10

or is it easier for you to just do it?

borkdude21:12:19

which change exactly?

cap10morgan21:12:05

adding those dirs to .dockerignore

cap10morgan21:12:18

target and .cpcache

borkdude21:12:28

sure, go ahead

cap10morgan21:12:56

oh you're already building linux arm64 binaries, just not a docker image?

borkdude22:12:43

I think @U7ERLH6JX suggested we make the images multi-architecture

cap10morgan22:12:50

ok, I'll put up a PR w/ just the Dockerfile changes to allow multi-platform builds

cap10morgan22:12:56

since that seems to work now

cap10morgan22:12:50

do you want me to make an issue first?

cap10morgan22:12:10

PR is up, but I'm happy to open an issue for it too if you'd like

borkdude22:12:12

Issue first is good with some background. Then @U7ERLH6JX can hopefully also comment on this. He knows all about this Docker stuff,

borkdude22:12:59

E.g. I don't know if this TARGETARCH is standard practice etc. Hopefully you can explain it to me and I will learn something.

cap10morgan22:12:54

and actually, the PR probably needs a bit more tweaking to continue working in non-buildx docker setups, which you probably want

cap10morgan22:12:03

I'll do that now

borkdude22:12:35

great. please put the link to the docs in the issue as well

👍 1
cap10morgan22:12:15

ok that falls back on an amd64 build now if you run it w/o BuildKit

cap10morgan22:12:21

should be good to go

borkdude22:12:35

is ARCH also a default? in our circleci builds we have BABASHKA_ARCH

borkdude22:12:20

and also BABASHKA_STATIC, BABASHKA_MUSL

cap10morgan22:12:21

Now, that's just one I made up. I need it to be separate from TARGETARCH b/c of how that behaves in all the different scenarios.

cap10morgan22:12:37

BABASHKA_ARCH would probably work fine there if you prefer

borkdude22:12:56

Maybe have a look where and how this is used in the compile script

👍 1
borkdude22:12:08

I'm going afk now, back tomorrow

👍 1
cap10morgan23:12:03

Yeah there's a little more to do around making this resilient with static builds (i.e. fail with a useful error message because that's only supported on amd64 so far). I'll push another commit or two tomorrow.

jkrasnay23:12:22

Does anyone have a quick tip on zipping up files from a Babashka script? I was hoping for something in babashka.fs but it only has unzip

borkdude06:12:48

You can look at the source in tools build and copy that. It works in bb as well. I’ll add a similar function to fs soon

jkrasnay12:12:02

Oh, good idea. Thanks!

borkdude13:12:35

but it probably needs some tweaks to cover some edge cases