Fork me on GitHub
#babashka
<
2020-09-10
>
goomba01:09:50

struggling a bit to get the output of a ProcessBuilder workflow (my Java is weak)

goomba01:09:13

(defn find-files! []
 (let [proc (-> (ProcessBuilder. ["find" "./some-dir"])
                 (.redirectOutput ProcessBuilder$Redirect/INHERIT)
                 (.redirectError ProcessBuilder$Redirect/INHERIT)
                 (.start))
        proc-input (.getOutputStream proc)]
    (into [] ????)))

borkdude06:09:13

@goomba Not sure where you're going with that example: you want to feed input to the process?

goomba13:09:50

It's probably safe to assume that I'm totally misunderstanding ProcessBuilder and the output streams. My goal here is to capture the output of a long running process, like a large "git clone", and capture the output into a string or a vector of strings.

goomba13:09:30

In some previous work I tried using shell/sh, but it doesn't seem suitable for long running tasks

goomba13:09:07

I tried looking into the Java API for how to work with the redirects of the ProcessBuilder but I got lost in the sauce

borkdude13:09:27

yeah, let me try to write that

borkdude13:09:23

What's not clear in your example: would like also to feed input to that process?

borkdude13:09:33

And why do you redirect output? This means it will be sent straight to stdout

borkdude13:09:29

@goomba

(require '[ :as io])

(defn find-files [path]
  (let [proc (-> (ProcessBuilder. ["find" path])
                 (.start))
        output (.getInputStream proc)]
    (future
      (run! prn (line-seq (io/reader output))))))

@(find-files (first *command-line-args*))
$ bb /tmp/foo.clj /tmp/version-clj
"/tmp/version-clj"
"/tmp/version-clj/test"
"/tmp/version-clj/test/cljx"
"/tmp/version-clj/test/cljx/version_clj"
...

borkdude13:09:06

Confusingly, you get the output of the process using .getInputStream because it returns an inputstream

goomba13:09:44

sweet! thank you

borkdude13:09:32

This is an in progress library which should make all of this easier: https://github.com/babashka/babashka.process

borkdude08:09:45

Any thoughts on this discussion? https://github.com/http-kit/http-kit/issues/442#issuecomment-690040016. If you have, consider posting your thoughts there.

borkdude10:09:18

I also made a #http-kit channel now

dharrigan11:09:38

I've added a simple example of using Babashka to rewrite an existing bash script into Clojure to perform spotifyd notifications on Linux: https://github.com/dharrigan/spotifyd-notification

borkdude11:09:06

Cool! Feel free to add to https://github.com/borkdude/babashka/blob/master/doc/libraries.md which also features projects

goomba13:09:20

Curious about techniques for Babashka applications-as-scripts. For instance, I created two files, config.clj and core.clj (which requries config.clj with a deps.edn and the traditional project structure. However from a non-root directory when I tried bb -f ../path/to/src/core.clj , I got the cannot find my-project.config.clj error message

goomba13:09:39

If I wanted to have a more complex setup but still use it as a script, would I be looking at making an Uberjar?

borkdude13:09:54

Either an uberscript or uberjar yes

borkdude13:09:37

@goomba or put the dir with the scripts on the classpath correctly using -cp

goomba13:09:12

that has the deps?

goomba13:09:41

I gotta tell you @borkdude this damn project is so much damn fun hahaha

borkdude13:09:00

Good to hear :) So if your sources are in <project>/src then classpath should be src, but you then go into .. then your classpath should become <project>/src but maybe using absolute paths for that is the least confusing

goomba13:09:40

understood. So regardless of what location I'm in, if I set the -cp to the src of the project (relatively or absolutely, as long as it is correct). Does that mean then that if I have src/project_name/*.clj files, and the namespaces are (ns project-name.something), it should still be src?

borkdude13:09:04

no, you can put it anywhere you want, and make the classpath anything you want

goomba13:09:04

to be more specific:

$ tree 
.
├── deps.edn
├── Makefile
└── src
    └── react_deployment
        ├── config.clj
        └── core.clj

goomba13:09:39

so still point to src with the classpath?

borkdude13:09:40

you can also make the classpath the current directory and have no src dir for example

borkdude13:09:14

foo/bar.clj becomes (ns foo.bar) yes.

goomba13:09:44

yep that did it

goomba13:09:53

I can't wait to try out some crazy socket/async stuff

thomas.ormezzano15:09:35

Hey, I have been writing a bit of babashka lately (it’s absolutely amazing btw) I was wondering what was the best way to interact with AWS with babashka. I have been doing some stuff with generating and running commands with clojure.java.shell but I was wondering if there was a way to either use the java sdk or if a library to integrate with the AWS api or something (couldn’t find something here https://github.com/borkdude/babashka/blob/master/doc/libraries.md)

thomas.ormezzano15:09:36

(or would that be a good candidate for a pod or something ?)

borkdude15:09:40

@thomas.ormezzano It's been asked a couple of times before. I think you could maybe wrap the aws using clojure.java.shell or maybe create a pod around the Python AWS SDK.

thomas.ormezzano15:09:28

ok amazing thank you will take a look at this 👍

nate18:09:23

very interesting idea to write an AWS pod in another language, maybe Go would be a good candidate too

borkdude18:09:52

Yes, Go would also be a good candidate, since it also compiles to native and should have very good startup

borkdude18:09:19

I'm not using AWS myself that much, so I'm leaving this one up to the babashka community :)

borkdude18:09:54

And where I can I'll try to help of course

lukaszkorecki18:09:13

This is v. relevant to something I'll be working on soon, no promises but I might tackle this

lukaszkorecki18:09:21

or something like that

viesti18:09:32

I did some hack with lambda, attaching babashka and awscli layers, shelling to aws cli worked nicely, although awscli is a python thing so not that stand-alone, compared to go

borkdude18:09:26

@lukaszkorecki The pod could live under the babashka org when it's more or less done, if you wanted to. Also ok to keep it under your company org, etc.

lukaszkorecki18:09:22

@borkdude I'm still at the gathering requirements stage, so it's not been fully decided if we're going to go with Clojure (jvm) or BB. But if we go with BB, it's def going to be open source

borkdude12:09:16

Could be both. If you're going with the JVM library and make it GraalVM compatible, it could also be wrapped in a pod

lukaszkorecki13:09:47

True! I'll keep you updated and bug you with questions ;-)

michael81923:09:40

Have you guys seen this? I am using it for some minor stuff but it's got the entire set of services for AWS. https://github.com/cognitect-labs/aws-api