Fork me on GitHub
#babashka
<
2021-06-07
>
borkdude10:06:43

Mindbending demo of building an AWS Lambda front-end with babashka and scittle that interacts with other AWS features like Polly: https://twitter.com/FieryCodDev/status/1401843357555511301

🤯 7
Endre Bakken Stovner15:06:17

In my REPL I get:

user> (require '[babashka.process :as p :refer [process]])
;; => nil
user> (process ["ls"])
Error printing return value (IllegalArgumentException) at clojure.lang.MultiFn/findAndCacheBestMethod (MultiFn.java:179).
Multiple methods in multimethod 'simple-dispatch' match dispatch value: class babashka.process.Process -> interface clojure.lang.IDeref and interface clojure.lang.IPersistentMap, and neither is preferred
How do I fix this?

Endre Bakken Stovner15:06:12

I guess you aren't meant to be able to print processes like that...

borkdude16:06:59

Is this a bb repl?

borkdude16:06:01

It seems cider is using pprint in the REPL then: https://github.com/babashka/process#clojurepprint

Endre Bakken Stovner16:06:31

Do you also know why CIDER might be eating my stderr? The below gives stderr output in a regular REPL, but not in CIDER:

(require '[babashka.process :as p :refer [process]]
         '[ :as io])
(def bwa-mem (process ["bash" "-c" "bwa mem -R '@RG\\tID:A\\tSM:A'  /Users/endrebakkenstovner/everclear/snakemake-flow/data/genome.fa /Users/endrebakkenstovner/everclear/snakemake-flow/data/samples/A.fastq | samtools view -Sb - > /Users/endrebakkenstovner/everclear/snakemake-flow/bwa-map/genome/hg19/sample/A/bwa-map.bam"] {:err :inherit
                           :shutdown p/destroy}))
(with-open [rdr (io/reader (:out bwa-mem))]
  (binding [*in* rdr]
    (loop []
      (let [line (read-line)]
        (println :line line)
      (when (not (nil? line))
        (recur))))))

Endre Bakken Stovner16:06:44

Just curious. Otherwise I'll ask in CIDER.

Endre Bakken Stovner16:06:07

The above command would only show stderr output.

borkdude16:06:30

the stderr of a process is written to System/err, if you use :inherit, not to *err*

borkdude16:06:46

Unless you configure it with {:err *err*}

borkdude16:06:24

that is probably the issue

Endre Bakken Stovner16:06:43

Yes, it was! I had been playing around with it for so long XD, thanks!

Endre Bakken Stovner16:06:46

Anyways, can I use :shutdown to issue a callback after process has finished? Or is some other way recommended? I'd love to do (process bwa-map {:shutdown #(do (notify-others %) (p/shutdown %))

borkdude16:06:49

currently there is no callback that you can add when the process exits, but it is possible to add this

Endre Bakken Stovner16:06:15

But can't I cheat in the way I showed above?

borkdude16:06:37

I think you can. But the :shutdown hook is only executed when the JVM or babashka shuts down, not directly when the process itself ends

borkdude16:06:51

but in Java 11 there is an onExit hook

borkdude16:06:01

which babashka.process could also support

borkdude16:06:21

it depends on your use case, so far I didn't need it

Endre Bakken Stovner16:06:45

That would be neat. My program dispatches plenty of jobs (think make) and I would love to be notified when they finish (ideally with all the info found in (process ...)) so that I could process the results easily.

borkdude16:06:21

you could use this as of today, but using Java interop

borkdude16:06:28

the process is available under :proc

Endre Bakken Stovner16:06:54

Mind if I ask you questions if I can't get it working? XD I'll start looking into it now 🙂

borkdude16:06:41

sure. is your target with this usage the JVM or bb?

borkdude16:06:55

I don't think bb allows interop with CompleteableFuture yet

borkdude16:06:32

user=> (def proc (:proc (babashka.process/process ["ls"])))
#'user/proc
user=> (def cf (.thenApply (.onExit proc) (reify java.util.function.Function (apply [this p] (.exitValue p)))))
#'user/cf
user=> (.get cf)
0
@UT770EY2K

🙏 2
Endre Bakken Stovner17:06:32

This worked beautifully, thanks.

mathpunk16:06:57

I expected this to work from a bb script, since I saw a usage of http://java.net classes in an example in the book:

(import [ DatagramSocket
         DatagramPacket
         InetSocketAddress])
yet, Unable to resolve classname: java.net.InetSocketAddress :thinking_face:

mathpunk16:06:27

i've done no setup other than the shebang so if I have to get into pods or a bb.edn, I haven't tried that

mathpunk16:06:42

A selection of Java classes are available, see babashka/impl/classes.clj in babashka's git repo.

mathpunk16:06:26

The example I saw was at the end of this section: https://book.babashka.org/#_nrepl

mathpunk16:06:48

do I need to do something classpath-y in order to invoke an import like that?

mathpunk16:06:38

gotcha, didn't consider that a subset of http://java.net might be present

borkdude16:06:55

we could consider adding it if there is a good use case for it and if the added binary size is still reasonable. could you explain why you need it?

mathpunk16:06:40

I am communicating with datadog via UDP datagram

mathpunk16:06:04

I might should just use curl instead

borkdude16:06:30

Are you the same person I recently gave the statsd example to? I think that is all you need

mathpunk16:06:48

I am not, I bet I could reuse it

mathpunk16:06:02

great, I'm sending events rather than incrementing a thing but, I think I can see enough to find my way toward that

borkdude18:06:34

I added InetSocketAddress for the next release so the next person won't run into this issue again

mathpunk22:06:27

It's been a pretty successful day getting babashka to help me do a task. One thing though: I created a second .clj file, and went to require the other one, and it's not found. I figured: 1. I'll probably have to learn more about classpaths to do that; but also, 2. Maybe if I split up namespaces that's a sign I'm using babashka for too large a problem?

borkdude07:06:25

@U0E9KE222 1. This works pretty similar to how normal Clojure projects work. Add a bb.edn instead of a deps.edn and add your source directories to :paths. An alternative to using the classpath is using load-file.

borkdude07:06:46

2. No, this is totally fine

mathpunk23:06:15

I'm adding functionality to existing shell scripts, which will call my program, so I thought it was the right thing to grab. I could be wrong and it's a good time in this project to figure out if I am 🙂