babashka 2023-05-06 | Slack Archive

Casey07:05:14

Are there any shell helpers, for example to check if a program exists in PATH? I want my build script to detect if podman or docker is in the PATH and use the right one.

elken07:05:25

If you don't need it to be xplatform you can shell out to which

Casey07:05:11

ah yea that's a good idea.. we really only need linux support

🙌 2

Casey07:05:16

(defn container-runtime-cmd []
  (if (not (str/blank? (exec "which podman")))
    "podman"
    "docker"))

Casey07:05:25

simple but effective

elken07:05:40

Clojure in a nutshell 🙂

borkdude08:05:11

There is babashka.fs/which

borkdude08:05:15

I'm not sure what exec is here, but if this is from babashka.process then this won't have the desired effect

borkdude08:05:36

Just use:

(or (fs/which "podman") "docker")

🧡 6

pesterhazy09:05:03

I recommend the api docs for babashka.fs cover to cover, lots of goodies in there

💯 4

teodorlu11:05:06

I found a pattern I was quite happy with for supporting --dry-run with babashka.cli. It gives you behavior like this:

$ bb mycli.clj dostuff
total 12K
drwxr-xr-x   3 teodorlu teodorlu 4.0K 2023-05-06 13:16 .
drwxr-xr-x 163 teodorlu teodorlu 4.0K 2023-05-06 11:28 ..
drwxr-xr-x   2 teodorlu teodorlu 4.0K 2023-05-06 13:17 bbdispatch
$ bb mycli.clj dostuff --dry-run
(babashka.process/shell "ls -lah ..")

The trick is to dispatch the side effects, and let dispatch respect --dry-run. Lisp + shell + FP = ❤️!

🎉 2

borkdude12:05:15

Instead of eval I'd probably would have used resolve or requiring-resolve :)

teodorlu12:05:14

Right, agreed. Didn't think of that, thanks! Eval "smells like it's too wide" to me.

pesterhazy13:05:54

I guess this could be done in with-redefs as well?

pesterhazy13:05:25

Related to set -x maybe

borkdude13:05:13

I'd say with-redefs should be limited to testing. It's not thread-safe and kind of meh...

teodorlu16:05:27

Yeah, with-redefs could do the trick. But then, one would have to remember to override every side-effecting function? I'd be a little cautious about overriding functions like spit. I also kind of like that I have to call dispatch explicitly for side effects. Perhaps it should be called run-effect! instead. Here's the real code I wrote: https://github.com/iterate/olorm/blob/2f30149a976859567ebade62bd471be865a17e91/cli/src/olorm/cli.clj#L51-L92 It's an internal (but public) tech microblog I'm running with a couple of coworkers, in Norwegian. Babashka made it so nice to use. $ olorm create, type in $EDITOR, :wq (or equivalent), and the new thing is published. I tried to make it minimally intrusive in an otherwise busy day.

pesterhazy07:05:36

Thanks for sharing! The context I'm thinking of is unit testing shell scripts. Maybe a wrapper similar to your dispatch could help

teodorlu07:05:53

Oooh, right. I don't feel like I have a great solution for that yet. I tend to just write a dry-run option, and try it out.

👍 2

anovick12:05:47

Is this a proper way to "shell out" to another binary?

(shell
  (apply str ["pdftk ",
              (.getFileName pdf-file-path),
              " dump_data | grep NumberOfPages"]
   )
)

where pdf-file-path is a java Path object like this:

pdf-file-path
=>
#object[sun.nio.fs.UnixPath
        0xe7824bd
        "/path/to/dir/week01.pdf"]

I have to serialize the Path to a string first, right?

borkdude12:05:13

You can just call str on it

borkdude12:05:51

Also you can supply separate arguments to shell, no need to concatenate them as a single string

anovick12:05:44

Hmm not sure what my error is:

(shell
  (apply str ["pdftk ",
              (str pdf-file-path),
              " dump_data | grep NumberOfPages"]
   )
)
Error: expecting "output" keyword.  Instead, I got:
   |
Errors encountered.  No output created.
Done.  Input errors, so no output created.
clojure.lang.ExceptionInfo:  bookmark_files_combiner.core REPL:1:1

anovick12:05:25

When I ran the command in my terminal it gave me the output I wanted, without error

anovick12:05:33

$ pdftk /path/to/dir/week01.pdf dump_data | grep NumberOfPages
NumberOfPages: 6

anovick12:05:09

Seems like a similar issue to this: https://gitlab.com/pdftk-java/pdftk/-/issues/44

anovick12:05:00

though I don't think I'm using the Java port of this program? I thought I was just using my system's pdftk (C or C++)

anovick12:05:37

It also says in the issue: "This issue is present in the original C++ pdftk"

anovick12:05:04

So probably the problem is the tool, not anything else

borkdude12:05:40

Oh don’t use the pipe symbol, that is bash syntax

borkdude12:05:29

Just capture the output string with :out :string and then process that string

borkdude12:05:57

Also to re-iterate: https://clojurians.slack.com/archives/CLX41ASCS/p1683375531982329?thread_ts=1683375407.989049&channel=CLX41ASCS&message_ts=1683375531.982329

borkdude12:05:40

Can’t write any code in here on the phone but perhaps someone like @U06F82LES or @U3X7174KS can help you write it out

anovick12:05:40

So the issue is the ending | grep NumberOfPages

👍 2

anovick12:05:47

{:out :str} ?

anovick12:05:52

(shell {:out :str}
  (apply str ["pdftk ",
              (str pdf-file-path),
              " dump_data"]
         )
  )

Like this?

borkdude12:05:19

:string, it’s in the docs :)

anovick12:05:13

There's nothing in the book when I use the web search for ":str" or ":string"...

borkdude12:05:15

Also again you don’t need to do the apply str thing, just provide those directly :-)

anovick12:05:31

haha I'm sorry I just don't know how to do that

borkdude12:05:45

Go to the docs of the babashka.process library on GitHub

teodorlu12:05:01

I'm also on my phone, I can try to help out when I get back home!

borkdude13:05:07

😎 2

📖 2

anovick13:05:45

This gives me an error...

(shell {:out :string}
         ["pdftk ",
           (str pdf-file-path),
             " dump_data"]
         )

anovick13:05:21

after I removed the (apply str ...) part

anovick13:05:19

But more importantly, I'm not sure how to pipe this stuff to grep, this is what I tried...

(shell "grep "
       (shell {:out :string}
         (apply str ["pdftk ",
                     (str pdf-file-path),
                     " dump_data"]
                )
         )
 )

borkdude13:05:46

And now remove the vector

borkdude13:05:23

Just pass the individual “words”

borkdude13:05:07

I’ll wait with another reply until I’m home because typing code in the phone is frustrating me

anovick13:05:17

Oh that works! I had to also remove whitespace

anovick13:05:22

but I see what you mean now...

anovick13:05:30

it inserts whitespace between words

borkdude13:05:03

And about grep: just parse the string yourself is what I would do

anovick13:05:13

anovick13:05:31

How do I access the string?

anovick13:05:38

(shell {:out :string}
 "pdftk",
             (str pdf-file-path),
             "dump_data"

 )

borkdude13:05:00

You can however pass the string to grep if you need to using the :in argument, but it’s probably not hard to just process this manually in Clojure

anovick13:05:01

From what I inspected, this gives me a Process

borkdude13:05:38

The :out result of the shell call contains the string

anovick13:05:44

Alright, I'll try reading some to figure this out

anovick13:05:24

Yay filtering worked 🙂

(filter
  (fn [s] (str/starts-with? s "PageMediaNumber:"))
  (str/split
    (:out (shell {:out :string}
                 "pdftk",
                 (str pdf-file-path),
                 "dump_data"
                 )
      )
    #"\n"
    )
  )

🎉 4

anovick13:05:36

🚃 Choo Choo my script is on its way!

borkdude13:05:53

Nice! You can use str/split-lines instead of split + regex here :)

anovick18:05:27

If my BB script creates some files in the process of its runtime, which aren't part of the script's goal, should I create these files as temporary files? with babashka.fs/create-temp-file maybe? and if so, does this function return the path of the newly created temp file?

borkdude18:05:47

Yes, this returns the path of the temp file, but the temp file won't be automatically clean up, unless you reboot your computer (which cleans up the temp folder). If you want those temp files to be cleaned up on bb exit, you can do:

(let [temp-file (doto (fs/create-temp-file)
                        (fs/delete-on-exit))]
  temp-file)

🙏 5

anovick19:05:26

Wow that worked so smoothly on first try... so pleasant to use!! :hugging_face:

anovick19:05:13

How can I receive arguments to my BB script from the user? say a relative path or an absolute path?

elken19:05:42

https://book.babashka.org/#_parsing_command_line_arguments

❤️ 2

borkdude19:05:20

Hmm, that part of the book should actually point to: https://github.com/babashka/cli

elken19:05:02

Has it been recently updated? I'm sure I noticed something else that wasn't up-to-date

borkdude19:05:20

Please report those things so I can update them

borkdude19:05:33

It's not like I'm re-reading the book every day ;)

elken19:05:33

Makes good bedtime reading no? 😉

anovick19:05:11

Where should I look? suppose I want to receive arguments like this:

$ bb my-script.clj ./path/to/dir ./path/to/another/dir

anovick19:05:24

two relative paths in this case

anovick19:05:25

• org.babashka/cli • clojure.tools.cli • babashka/cli Which namespace?

borkdude19:05:29

@UDHL22ZFE https://github.com/babashka/cli

borkdude19:05:32

Check out the README there

borkdude19:05:24

btw, I've updated the docs here now: https://book.babashka.org/#_parsing_command_line_arguments

borkdude19:05:02

To parse two positional arguments, you don't need any CLI parsing library. You can simply do:

(let [[file1 file2] *command-line-args*]
  ...)

anovick19:05:15

Awesome!!

anovick19:05:25

😀

borkdude19:05:30

but if you need to parse additional options you can do this:

user=> (require '[babashka.cli :as cli])
nil
user=> (cli/parse-args ["file1" "file2" "--action" "delete"])
{:args ["file1" "file2"], :opts {:action "delete"}}

anovick19:05:38

@borkdude from your last code snippet: Is this something that you would pass as the argument? is this an EDN notation?

{:args ["file1" "file2"], :opts {:action "delete"}}

borkdude19:05:10

No, that is the return value of the call to cli/parse-args

borkdude19:05:22

What you saw there was a REPL session

anovick19:05:35

oh yea I see now

anovick19:05:41

I'm just gonna use the positional arguments option you showed earlier with command-line-args I have another question on how to deal with the relative path string:

(let [[pdfs-dir-path-string
       bookmarks-yaml-files-dir-path-strings
       output-pdf-dir-path-string
       ] *command-line-args*]
  (pprint  pdfs-dir-path-string)
  (pprint  bookmarks-yaml-files-dir-path-strings)
  (pprint  output-pdf-dir-path-string)
)

All of these are just strings like so:

$ bb ./src/bookmark_files_combiner/core.clj resources/pdf-files/ resources/bookmark-files/ resources/output-boomark-file/
"resources/pdf-files/"
"resources/bookmark-files/"
"resources/output-boomark-file/"

My question is, how can I tell that this is a relative path and not an absolute path?

anovick19:05:36

Is it just about checking if the first character is / and treating it as absolute path, otherwise treat it as relative path?

anovick19:05:07

I'm not sure if that's the best heuristic

anovick19:05:23

Oh I see some utility functions: • fs/absolute? • fs/relative?

anovick20:05:00

So if it's a relative path, I just convert it to an absolute path and go on about my business assuming it's all absolute paths after that?

borkdude20:05:10

yes, those are the utility functions you need

borkdude20:05:24

you can use fs/absolutize to convert relative to absolute

anovick20:05:27

Hey guys, so I just finished the script! Started working on it yesterday... Never used BB before, and only dabbled with Clojure a few times prior to this I had so much fun!! :star-struck: Shoutout to everyone in this amazing community that just keep providing their faithful assistance, without you this wouldn't have been possible! (I would've gotten stuck and quit) 🙏 Special shoutout to @borkdude 😇 for being extremely quick, attentive and just so awesome to a total stranger like me on the internet 🙂 I uploaded my code to GitHub because I want Clojure to succeed and I can share stuff that I did with friends 🙂 https://github.com/amitnovick/bookmark-files-combiner

❤️ 16

teodorlu09:05:48

Nice! If you want to be able to install a system script (so that you can run bookmark-files-combiner from any folder), bbin might be able to help you do that! More info on #babashka-bbin / https://github.com/babashka/bbin

teodorlu09:05:33

And I think it's quite amazing that you're able to get this done without much prior clojure experience! 💯

anovick11:05:01

I tried to install babashka/bbin on my Ubuntu 22.04 LTS with the Homebrew method but I got an error. It's probably something wrong with my disk space, though I'm not sure how to fix it.

$ brew install babashka/brew/bbin
==> Tapping babashka/brew
Cloning into '/home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/babashka/homebrew-brew'...
remote: Enumerating objects: 399, done.
remote: Counting objects: 100% (143/143), done.
remote: Compressing objects: 100% (78/78), done.
remote: Total 399 (delta 97), reused 103 (delta 65), pack-reused 256
Receiving objects: 100% (399/399), 59.46 KiB | 801.00 KiB/s, done.
Resolving deltas: 100% (209/209), done.
Tapped 64 formulae (79 files, 172.6KB).
==> Tapping borkdude/brew
Cloning into '/home/linuxbrew/.linuxbrew/Homebrew/Library/Taps/borkdude/homebrew-brew'...
remote: Enumerating objects: 1114, done.
remote: Counting objects: 100% (231/231), done.
remote: Compressing objects: 100% (94/94), done.
remote: Total 1114 (delta 157), reused 210 (delta 137), pack-reused 883
Receiving objects: 100% (1114/1114), 138.50 KiB | 984.00 KiB/s, done.
Resolving deltas: 100% (737/737), done.
Tapped 9 formulae (39 files, 251.4KB).
==> Fetching dependencies for babashka/brew/bbin: borkdude/brew/babashka
==> Fetching borkdude/brew/babashka
==> Downloading 
==> Downloading from 
############################################################ 100.0%
==> Fetching babashka/brew/bbin
==> Downloading 
==> Downloading from 
#-#O#- #                                                          
==> Installing bbin from babashka/brew
==> Installing dependencies for babashka/brew/bbin: borkdude/brew/babashka
==> Installing babashka/brew/bbin dependency: borkdude/brew/ba
Error: An exception occurred within a child process:
  Errno::ENOSPC: No space left on device - sendfile

anovick11:05:53

In general my machine has free 40GB of disk space available, but maybe for some partition I need to allocate more, dunno

teodorlu11:05:38

It's also possible to install bbin manually. It's just a babashka script! That's what I do on my Linux system, I don't usually install things with homebrew.

teodorlu11:05:18

Here's the manual installation instructions: https://github.com/babashka/bbin/blob/main/docs/installation.md#manual-linux-and-macos

🙏 2

anovick11:05:37

Alright! that worked

anovick11:05:44

Got it on my system now

anovick11:05:21

I got my script on a file path bookmark-files-combiner/core.clj Wonder how I can pass that on to bbin so that the global available name for it is bookmark-files-combiner` and not core.clj

borkdude11:05:50

It is a bit worrying that anovick got that obscure error message though when installing bbin via brew

➕ 2

borkdude11:05:19

@UDHL22ZFE you can use bbin install <script> --as bookmark-files-combiner

anovick11:05:35

Oh there's a flag for that! I'll try it :hugging_face:

anovick11:05:35

OK that was so easy that I've regained my faith in computing altogether 🙂

😄 2

anovick11:05:03

I just run

$ bookmark-files-combiner pdf-files/ bookmark-files/ output-boomark-file/

and it just works! no need to call bb no need to navigate to the script... it's so much easier ❤️

🎉 6

❤️ 2

borkdude12:05:40

@UDHL22ZFE If you want feedback on your clojure coding, you could ask in #C053PTJE6 too. E.g. I noticed you're using:

(defn foo [x]
  (def y 1) ;; inline def
  (+ x y))

which we usually write as:

(defn foo [x]
  (let [y 1]
    (+ x y))

Notice def vs let def always introduces a global variable, even if you create it in the middle of the function, vs let always creates local (immutable) bindings

😍 4

👍 2

borkdude12:05:18

If you set up a linter like #CHY97NXE2 it will also tell you this. The linter can be integrated with your editor

anovick12:05:25

Seems like a very useful tool to have... indeed I do not write idiomatic Clojure and that could help me out... But then I'd have to install some Cursive plugin and install the clj-kondo tool and make sure that it works... and it's just a bit of a headache for me... I hate spending time on environment setup

anovick12:05:40

If I were to really be doing Clojure all day I would've considered that but right now I'm a student and don't have time to spend on this unfortunately... maybe in the future 🙂

borkdude12:05:53

#C02UN1B0998 will give you clj-kondo in Cursive

anovick12:05:41

Is it built in though?

anovick12:05:49

or do I have to install clj-kondo separately?

borkdude12:05:25

you don't have to install it separately if you install #C02UN1B0998

anovick12:05:34

oh... ok then I'll try

anovick12:05:15

Alright so I installed this Clojure Extras pugin...

anovick12:05:31

and the clj-kondo was triggered on my classpath

anovick12:05:53

but there's no warnings that weren't there before by just the Cursive static analysis

anovick12:05:25

the def forms for example... no warning on that

borkdude12:05:16

hmm, then something isn't working. not sure why

anovick12:05:49

Yea... 😕 that sucks

borkdude12:05:13

maybe you just need to edit the file

borkdude12:05:24

to trigger the linting

anovick12:05:46

Hmm

anovick12:05:57

so I started writing, and the Clojure Extras plugin gave me an error

anovick12:05:21

that's what it gave me:

Error trying to annotate file

java.lang.Throwable: Control-flow exceptions (like ProcessCanceledException) should never be logged: ignore for explicitly started processes or rethrow to handle on the outer process level
	at com.intellij.openapi.diagnostic.Logger.ensureNotControlFlow(Logger.java:264)
	at com.intellij.idea.IdeaLogger.doLogError(IdeaLogger.java:151)
	at com.intellij.idea.IdeaLogger.error(IdeaLogger.java:142)
	at com.intellij.openapi.diagnostic.Logger.error(Logger.java:206)
	at com.github.brcosta.cljstuffplugin.extensions.CljKondoAnnotator.lintWithBuiltinLinter(CljKondoAnnotator.kt:116)
	at com.github.brcosta.cljstuffplugin.extensions.CljKondoAnnotator.doAnnotate(CljKondoAnnotator.kt:61)
	at com.github.brcosta.cljstuffplugin.extensions.CljKondoAnnotator.doAnnotate(CljKondoAnnotator.kt:31)
	at com.intellij.codeInsight.daemon.impl.ExternalToolPass.doAnnotate(ExternalToolPass.java:220)
	at com.intellij.codeInsight.daemon.impl.ExternalToolPass.doAnnotate(ExternalToolPass.java:214)
	at com.intellij.codeInsight.daemon.impl.ExternalToolPass$1.lambda$run$0(ExternalToolPass.java:192)
	at com.intellij.codeInsight.daemon.impl.ExternalToolPass.runChangeAware(ExternalToolPass.java:289)
	at com.intellij.codeInsight.daemon.impl.ExternalToolPass$1.lambda$run$2(ExternalToolPass.java:192)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$runProcess$2(CoreProgressManager.java:188)
	at com.intellij.openapi.progress.impl.CoreProgressManager.lambda$executeProcessUnderProgress$12(CoreProgressManager.java:608)
	at com.intellij.openapi.progress.impl.CoreProgressManager.registerIndicatorAndRun(CoreProgressManager.java:683)
	at com.intellij.openapi.progress.impl.CoreProgressManager.computeUnderProgress(CoreProgressManager.java:639)
	at com.intellij.openapi.progress.impl.CoreProgressManager.executeProcessUnderProgress(CoreProgressManager.java:607)
	at com.intellij.openapi.progress.impl.ProgressManagerImpl.executeProcessUnderProgress(ProgressManagerImpl.java:60)
	at com.intellij.openapi.progress.impl.CoreProgressManager.runProcess(CoreProgressManager.java:175)
	at com.intellij.openapi.progress.util.BackgroundTaskUtil.runUnderDisposeAwareIndicator(BackgroundTaskUtil.java:365)
	at com.intellij.openapi.progress.util.BackgroundTaskUtil.runUnderDisposeAwareIndicator(BackgroundTaskUtil.java:343)
	at com.intellij.codeInsight.daemon.impl.ExternalToolPass$1.run(ExternalToolPass.java:191)
	at com.intellij.util.ui.update.MergingUpdateQueue.execute(MergingUpdateQueue.java:332)
	at com.intellij.util.ui.update.MergingUpdateQueue.execute(MergingUpdateQueue.java:322)
	at com.intellij.util.ui.update.MergingUpdateQueue.lambda$flush$1(MergingUpdateQueue.java:271)
	at com.intellij.util.ui.update.MergingUpdateQueue.flush(MergingUpdateQueue.java:285)
	at com.intellij.util.ui.update.MergingUpdateQueue.run(MergingUpdateQueue.java:240)
	at com.intellij.util.concurrency.QueueProcessor.runSafely(QueueProcessor.java:241)
	at com.intellij.util.Alarm$Request.runSafely(Alarm.java:388)
	at com.intellij.util.Alarm$Request.run(Alarm.java:377)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at com.intellij.util.concurrency.SchedulingWrapper$MyScheduledFutureTask.run(SchedulingWrapper.java:223)
	at com.intellij.util.concurrency.BoundedTaskExecutor.doRun(BoundedTaskExecutor.java:241)
	at com.intellij.util.concurrency.BoundedTaskExecutor.access$200(BoundedTaskExecutor.java:31)
	at com.intellij.util.concurrency.BoundedTaskExecutor$1.execute(BoundedTaskExecutor.java:214)
	at com.intellij.util.ConcurrencyUtil.runUnderThreadName(ConcurrencyUtil.java:212)
	at com.intellij.util.concurrency.BoundedTaskExecutor$1.run(BoundedTaskExecutor.java:203)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:702)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:699)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:399)
	at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:699)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: com.intellij.openapi.progress.ProcessCanceledException
	at com.intellij.openapi.application.impl.ReadMostlyRWLock.startRead(ReadMostlyRWLock.java:108)
	at com.intellij.openapi.application.impl.ApplicationImpl.runReadAction(ApplicationImpl.java:939)
	at com.intellij.openapi.application.ReadAction.compute(ReadAction.java:68)
	at com.intellij.openapi.editor.impl.DocumentImpl.getText(DocumentImpl.java:934)
	at com.github.brcosta.cljstuffplugin.extensions.CljKondoAnnotator.getPsiFileContent(CljKondoAnnotator.kt:125)
	at com.github.brcosta.cljstuffplugin.extensions.CljKondoAnnotator.getTempLintFile(CljKondoAnnotator.kt:131)
	at com.github.brcosta.cljstuffplugin.extensions.CljKondoAnnotator.lintWithBuiltinLinter(CljKondoAnnotator.kt:102)
	... 40 more

borkdude12:05:56

not sure what this is, unfortunately

2023-05-06

Channels