Clojurians Log v2

> node packages api has a large surface area Not sure what you mean here... I haven't actually tried this myself. GraalVM polyglot could be interesting, but I think the JS package there can't be very node-specific

As in there are several functions that we’d want to interleave with our other code. So it’s not trivial to write a single script to call, it would have to be multiple. If that makes sense?

Perhaps you could do it via nREPL?

For now we can get buy. I’ve just been blown away with how magical nbb is in terms of making node actually usable. 🤣

:)

Oh, that’s interesting, so start an nrepl server in nbb process and then call out to that, from babashka/clojure?

That’s exactly the mad science sort of answer I was looking for.

Where would be the best place to start?

what you could do is either: Start nbb first with an nREPL server and then connect to it from the JVM to execute the stuff you want, or vice versa

That’s actually brilliant.

I guess you could also spin up nbb's nrepl server from the JVM as a shell process

you don't even need the nREPL protocol, you can do it via stdin/stdout via this API: https://github.com/babashka/nbb/blob/main/doc/api.md#nbbrepl

Thats even better, the exact sort thing I do in emacs all the time. Don’t know how I missed it.

I guess elisp has always been “the silly place”

Thank you!

No problem, let me know if it works out or not

I’ll defo do a write up blog post if it does. 🎉

So here’s what I have so far, half working. A cljs nbb file.

(ns core
  (:require [nbb.repl :refer [repl]]))

(repl)

(ns app.nbb
  (:require [clojure.java.process :as p]
            [ :as io]))
            
(def nbb-process
  (p/start "nbb" "resources/nbb/core.cljs"))
  
(def stdin (-> nbb-process p/stdin))
(def ^java.io.Writer writer (io/writer stdin))

(def stdout (-> nbb-process p/stdout))
(def ^java.io.Reader reader (io/reader stdout))

(.write writer (str '(+ 3 4 5)))

if only nbb had a prepl, this would be easier :) but perhaps as a workaround you write a special value and read it back

as a delimiter

nrepl is also more structured

you could try to start the nbb nrepl server and then read values using this: https://github.com/babashka/nrepl-client

haha, yeah. Good shout with the special character. It’s more the pain of java low level read api. Can’t use spit/slurp as they close the connection, working out termination etc. maybe nrepl + nrepl client is the right level of abstraction, I’ll check it out.

Still amazed it kinda works.

you can do something like this:

(binding [*in* stdout] (read-line))

Somehow I don't see any input here:

(ns app.nbb
  (:require [babashka.process :as p]
            [ :as io]))

(def nbb-process
  (p/process {:err :inherit} "npx" "nbb" "repl"))

(def stdin (-> nbb-process :in))
(def ^java.io.Writer writer (io/writer stdin))

(def stdout (-> nbb-process :out))
(def ^java.io.Reader reader (io/reader stdout))

(.write writer (str '(+ 3 4 5) "\n"))
(.flush writer)

(future
  (binding [*in* stdout]
    (while
        (when-let [line (read-line)]
          (println :> )
          line))))

@(promise)

Yeah so I’m running into the same issue, where I can see the evaluation is happening with :out :inherit but I can’t seem to read normally as in it just blocks.

Assumed the blocking was the reader trying to read and not having anything to read (and not using a termination character).

This program works (at least in bb):

(ns app.nbb
  (:require [babashka.process :as p]
            [ :as io]))

(def nbb-process
  (p/process {:err :inherit} "npx" "nbb" "repl"))

(def stdin (-> nbb-process :in))
(def ^java.io.Writer writer (io/writer stdin))

(def stdout (-> nbb-process :out))
(def ^java.io.Reader reader (io/reader stdout))

(.write writer (str '(+ 3 4 5) "\n"))
(.flush writer)

(.close writer)

(while
    (when-let [line (.readLine reader)]
      (println :> line)
      line))

Works in clojure land too!

something like this will make it more easily readable:

(.write writer (str ::begin "," '(+ 3 4 5) "," ::end "\n"))

but runs into the issue I was having before in that it closes the stream

which seems to kill the process

This is because .close writer , I did that on purpose

once you're finished with nbb, closing the reader kills the REPL

but not closing it leaves the REPL open

right so now I just need to stop reading when :app.nbb/end is read so as not to block

(str ::begin "," '(+ 3 4 5) "," ::end "\n") <- this is a nice trick

I'm not 100% sure what happens

it seems node crashes with Error: write EPIPE sometimes

haven’t seen that crash yet 🤞

This seems to kinda work in my case:

(defn read-val []
  (when-let [line (.readLine reader)]
    (println :> line)
    line))

(read-val) ;; repl greeting

(.write writer (str ::begin "," '(+ 3 4 5) "," ::end "\n"))
(.flush writer)

(read-val) ;; app begin
(read-val) ;; 12
(read-val) ;; app end

(.write writer (str ::begin "," '(assoc {:a 1} :b 2) "," ::end "\n"))
(.flush writer)

(read-val) ;; app begin
(read-val) ;; {:a 1 :b 2}
(read-val) ;; app end

it seems kinda clunky. nrepl is probably more robust

Yeah, does feel like just re-inventing nrepl

yeah or prepl which is also suited for this, but not implemented yet in nbb

I’ll play around with an nrepl version and see how that goes. Thanks for all the help!

(def nbb-process
  (p/start "nbb" "resources/nbb/core.cljs"))

(def stdin (-> nbb-process p/stdin))
(def ^java.io.Writer writer (io/writer stdin))

(def stdout (-> nbb-process p/stdout))
(def ^java.io.Reader reader (io/reader stdout))
  
(defn execute [sexp]
  (.write writer (str ::begin "," sexp "," ::end "\n"))
  (.flush writer)
  (while
      (when-let [line (.readLine reader)]
        (when-not (re-find #":app.nbb/end" line)
          (println :> line)
          line))))

(execute
  '(ns core
     (:require
      ["@metaplex-foundation/umi-bundle-defaults" :refer [createUmi]]
      ["@metaplex-foundation/umi" :as umi]
      ["@metaplex-foundation/mpl-bubblegum" :as gum]
      [promesa.core :as p]
      [cljs-bean.core :refer [->clj]])))

(execute
  '(def umi-conn
     (-> (createUmi "")
       (.use (gum/mplBubblegum)))))

yup

Here’s the nrepl version works a treat package json:

{
  "dependencies": {
    "@metaplex-foundation/mpl-bubblegum": "^4.2.0",
    "@metaplex-foundation/umi": "^0.9.2",
    "@metaplex-foundation/umi-bundle-defaults": "^0.9.2",
    "@solana/web3.js": "^1.95.2"
  }
}

(ns core
  (:require [nbb.nrepl-server :as nrepl]))

(nrepl/start-server! {:port 1337})

(ns app.nbb
  (:require [clojure.java.process :as p]
            [nrepl.core :as nrepl]))

(def nbb-nrepl
  (p/start "nbb" "resources/nbb/core.cljs"))

(defn execute [sexp]
  (with-open [conn (nrepl/connect :port 1337)]
    (->> (-> (nrepl/client conn 1000)
           (nrepl/message {:op   "eval"
                           :code (str sexp)}))
      (map nrepl/read-response-value)
      nrepl/combine-responses)))

(comment
  (try
    ;; Works but throws an exception as it can't read the response:
    ;; "Could not read response value: #object[Is core]"
    (execute
      '(ns core
         (:require
          ["@metaplex-foundation/umi-bundle-defaults" :refer [createUmi]]
          ["@metaplex-foundation/umi" :as umi]
          ["@metaplex-foundation/mpl-bubblegum" :as gum])))
    (catch Throwable _))

  (execute
    '(def umi-conn
       (-> (createUmi "")
         (.use (gum/mplBubblegum)))))

  ;; {:id "a73c3698-8d00-416d-9d21-42be56c0be7c",
  ;; :ns "core",
  ;; :value [#'core/umi-conn],
  ;; :status #{"done"}}
  )
  

Thanks again for all the help!

awesome

why not just start nrepl from the command line without the script? not that it matters

So I was having issues picking up the package json

ah

it seemed to run in the wrong directory, and even passing :dir option in didn’t seem to fix it

weird

yeah, probably just me not using the process api properly.

looking forward to the blog post when you have time some day ;)

you could probably re-use the nrepl client over multiple calls, but cool that you have it working

Yeah, got a backlog already, need to put one out on back pressure for virtual threads. Turns out infinite threads can be a bit of a pain when you have a slow consumer, who knew? Thank you for nbb! The library I never thought i’d need. The space I’m currently working in is just very auto generated JS SDK heavy and nbb means I can stay in my clojure happy place.

Ahh yes that’s a good point.