Fork me on GitHub
#babashka
<
2021-04-22
>
Karol Wójcik07:04:27

@borkdude I would like to pack some babashka pods into a .zip file in some automatic manner. How would you do it?

borkdude08:04:44

@karol.wojcik Probably manually download them and zip them

Karol Wójcik08:04:24

1. Supposing user has the pod defined in that way: (load-pod 'org.babashka/aws "0.0.5") then if I provide pod in .zip via babashka classpath will babashka take it from local filesystem? 2. Can somehow inspect a script and check which pods it declared?

borkdude08:04:08

@karol.wojcik Is the user calling the pod or your framework?

Karol Wójcik08:04:11

@borkdude Actually babashka-runtime will call the user script like so:

bb -cp src:holy-lambda-babashka-0.1.27.jar:holy-lambda-default-retriever-0.0.4.jar:clojure-1.10.3.jar:core.specs.alpha-0.2.56.jar:spec.alpha-0.2.194.jar:PODS_PATH -m USER_DEFINED_ENTRYPOINT

Karol Wójcik08:04:33

@borkdude so framework 😄 but only on aws runtime side 🙂

borkdude08:04:24

what pods is your framework using? btw, pods aren't resolved from the classpath, you can use either a fully qualified symbol so it loads from the registry or a file system path

borkdude08:04:34

I'm afk for a bit

Karol Wójcik08:04:53

@borkdude Ok so no PODS_PATH then. From which path babashka reads babashka pods? Can the case for fully qualified symbol first check whether pods are available locally? That would make the integration with babashka pods easy on holy-lambda side.

borkdude08:04:26

@karol.wojcik you can write (load-pod "/tmp/foo/pod") to just load a pod from the file system, there is no special path for this

borkdude08:04:59

In the case of a fully qualified symbol it is trying to load from ~/.babashka/pods/repository/org.babashka/postgresql/0.0.4 for example when you write (load-pod 'org.babashka/postgresql "0.0.4")

borkdude08:04:24

and if it is not there, it will try to download it from the pod registry and install it there

borkdude08:04:19

Does that make sense?

borkdude08:04:27

It's similar to .m2 but for pods ;)

borkdude08:04:25

So if you can organize your environment in such a way that these pods are in the right place, this will work

Karol Wójcik09:04:18

Thank you so much for the explanation @borkdude

Karol Wójcik09:04:52

@borkdude Where pods are downloaded in Windows?

borkdude09:04:21

@karol.wojcik To wherever (System/getProperty "user.home") says your home dir is

borkdude09:04:38

it also respects XDG_CACHE_HOME

Karol Wójcik09:04:44

Ok got it. I will cp pods from .babashka to zip file and it will just work.

borkdude09:04:31

@karol.wojcik Except that pods are platform specific. So if you are on Windows, those are .exe files and these do not run on AWS

borkdude09:04:51

same for macOS

Karol Wójcik09:04:06

Ups. Right. I need only linux artifacts.

borkdude09:04:09

There should probably be a step in the docker image to download the pods or something

Karol Wójcik09:04:55

Is there a way that I can say to babashka "Hey bb just download the pods! Don't do anything else!!"?

borkdude09:04:30

Not right now. But we could make something like this in bb.edn: you declare the pods there and then you could say bb pods --download or something

Karol Wójcik09:04:51

Yep. That would be awesome.

borkdude09:04:16

Currently you can do this by making a script download_pods.clj and just call load-pod there with nothing else

Karol Wójcik09:04:58

Ok will do so. May I make an issue to support bb pods --download?

👍 2
jamescroft17:04:00

Hello. Using babashka.process is it possible to stream the stdout to the parent process whilst also capturing the output? I’m using bb as a task runner and launching the figwheel cljs compilation process. It takes a while so I want the output to display as it occurs, but I want also want to capture output so I can exit with a non zero code if the output contained any warnings.

borkdude17:04:04

@jamescroft you can display the output as it occurs by setting :out :inherit :err :inherit but I'm not sure why you want to capture the output?

borkdude17:04:18

and why are you not using shell in the task?

borkdude17:04:00

oh now I get it, you are not using bb's task runner

borkdude17:04:22

@jamescroft why don't you just exit with the exit code of the process?

borkdude17:04:02

Something like this:

(require '[babashka.process :refer [process]])

(-> (process '["bb" "-e" (loop [i 0]
                           (if (= i 3)
                             (do (prn :stop) (System/exit 1))
                             (do (Thread/sleep 1000)
                                 (prn :foo)
                                 (recur (inc i)))))]
             {:inherit true})
    deref :exit (System/exit))

borkdude17:04:41

or use check which only throws on a non-zero exit:

(require '[babashka.process :refer [process check]])

(-> (process '["bb" "-e" (loop [i 0]
                           (if (= i 3)
                             (do (prn :stop) (System/exit 1))
                             (do (Thread/sleep 1000)
                                 (prn :foo)
                                 (recur (inc i)))))]
             {:inherit true})
    check)

jamescroft17:04:24

@borkdude I’m using the bb task runner, but the task definition just calls a function (not using shell ). The reason I want to capture output is that I want to look for the string “Compile Warning” that may occur in the stdout. If I see a “Compile Warning” then I want the process to exit with a non zero code even though the child process would have been a zero code.

jamescroft17:04:24

So currently I have something like:

(defn throw-if-output-contains [{:keys [out err] :as process} s]
  (when (str/includes? out s)
    (println out)
    (println "Exiting, stdout contains:" s)
    (System/exit 1))
  (when (str/includes? err s)
    (println err)
    (println "Exiting, stderr contains:" s)
    (System/exit 1))
  process)

(defn compile-cljs [{:keys [env]}]
  (-> (sh ["clojure" "-A:frontend:build-min"])
      (throw-if-output-contains "Compile Warning")))
But with that, the output only gets printed after the child process finishes. Ideally i’d like the output to appear as it happens.

borkdude17:04:49

@jamescroft

(require '[babashka.process :refer [process destroy]]
         '[ :as io]
         '[clojure.string :as str])

(def proc (process '["bb" "-e" (loop [i 0]
                                 (if (= i 3)
                                   (do (prn :compile-warning)
                                       (recur (inc i)))
                                   (do (Thread/sleep 1000)
                                       (prn :foo)
                                       (recur (inc i)))))]
                   {:shutdown destroy}))

(def output (:out proc))

(with-open [rdr (io/reader output)]
  (loop [lines (line-seq rdr)]
    (when-first [l lines]
      (println l)
      (when (str/includes? l "compile-warning")
        (System/exit 1))
      (recur (next lines)))))

borkdude17:04:09

Perhaps your process writes to stderr, but the same trick applies to that

jamescroft17:04:11

Thank you! I see the approach now. I’ll try something like that.