How are you shelling out exactly?

And what does attach do?

with sh

attach(dataframe) installs the dataframe columns as individual variables.

In the failing case those variables are all NULL

Can you show how you are shelling out?

Yep - but hold on - I have tried so many options….

You could also try babashka.process

I’m requiring:

[clojure.java.shell :refer [sh]]

Then…

(sh “Rscript” “resources/r_model_tests/kidney/waiting/adjcox.R”)

There are variants where I pass the current directory to Rscript, but in this variant I just set it to the correct dir in adjcox.R.

And why would this work with bash but not with sh? Maybe you could try:

(babashka.process/sh ["Rscript" ...])

I wondered about babashka.process, but I could not see the benefit since the R code is self-contained. It reads its own files and writes it’s own results to file.

Maybe you can log the current directory, etc from the Rscript, to debug what's different.

Yeah - there isn’t a problem with the current directory.

OK - I’ll have a go with babashka.process…

Have you checked the :err output?

And the :exit code?

I would be able to take a closer look if you could make a repro in a github repo I could check out

and tell me how to install Rscript

For now you could maybe shell out to bash to call Rscript, if that works

Yes - the err: output is exactly what you would expect from R when it sees NULL instead of the correct value for one f the header variables.

(sh "bash" "-c" ...)

Another layer… Interesting idea….

OK You’ve given me some things to try. Thanks!

I'm pretty sure this can be fixed, given the right repro. :thumbsup:

I’ll come back to this shortly.

(sh "bash" "-c" ...) has the same attach issue

(babashka.process/sh ["Rscript" "resources/r_model_tests/kidney/waiting/adjcox.R"])

{:proc #object[java.lang.ProcessImpl 0x5eabdbcd "Process[pid=2775, exitValue=1]"], :exit 1, :in #object[java.lang.ProcessImpl$ProcessPipeOutputStream 0x3d98c7c6 "[email protected]"], :out "", :err "Error in adjcox(cent = tests$cent[i], sex = tests$sex[i], eth = tests$eth[i],  : \n  object 'tx_sex_m' not found\nExecution halted\n", :prev nil, :cmd ["Rscript" "resources/r_model_tests/kidney/waiting/adjcox.R"]}

tx_sex_m is the first column header that should be attached.

If you want to make a ready to go repro I can clone and execute locally, including a csv file, and instructions how to install Rscript, I can take a look. No ideas at this point.

Perhaps it helps searching for "Java shell out Rscript attach" or so

OK - I’ll see if I can isolate it. It should be simple to do.

My wife is also using R, so it would be interesting for me to figure this out :)

Well that was interesting. The issue only appears in the Calva babashka repl. In other contexts it’s fine. Gist is here: https://gist.github.com/gmp26/b859872eeb53c753ece08c7e632f7b02

Nice! So it's a Calva, maybe general GUI + shelling out problem or so

Congrats on discovering this

Maybe also try with a JVM REPL

and with a babashka nREPL connected with lein repl :connect in a terminal maybe

As @U02N27RK69K hinted, it may also have to do something with tty-ish

Lemme try :)

there's a profile named bb

example https://twitter.com/tatut/status/1511025200766332936

The first trouble I have with the README and get going quickly is that I can't see where connect is coming from

(require '[clj-chrome-devtools.core :as core]
         '[clj-chrome-devtools.automation :as auto])

(def a (auto/create-automation (core/connect)))
(auto/to a "")
(auto/print-pdf a "clojure.pdf"
                {:print-background true
                 :paper-width 8.3
                 :paper-height 11.7})

yeah, should probably add my sample script

the connect actually needs chrome to be already up, there's a launcher ns as well which will launch a chrome instance

How do I launch a chrome with a debug port?

might also be nice to add to the README

I already have chrome running, I can't figure it out without restarting it

Now just relaunched with the port, it worked!

22:       (throw (ex-info (str "Error in command " method ": " (:message error))
          ^--- Error in command Page.printToPDF: Printing is not available
23:                       {:request payload

you must be running --headless for PDF printing to work, don't know why that is

check README again, I added a quickstart help that links to a script

For the bb script example, you can do this without the classpath hassle:

(ns devtools
  (:require [babashka.deps :as deps]))

(deps/add-deps
 '{:deps {tatut/devtools {:git/url ""
                          :git/sha "62a8aba789bbda6d954d17f2dc6f92dd74ec8619"}
          org.babashka/spec.alpha {:git/url ""
                                   :git/sha "1a841c4cc1d4f6dab7505a98ed2d532dd9d56b78"}}})

(require '[clj-chrome-devtools.automation :as auto]
         '[clj-chrome-devtools.core :as core])

(def a (auto/create-automation (core/connect)))
(auto/to a "")

that's neat

Do you think this or a similar approach could also work for firefox?

This is already very cool btw!

Thanks a lot for making it bb compatible

I don't know what kind of protocol firefox has... chrome has the CDP protocol that is described as json and you can easily autogenerate the functions

I started this bb support because I had an idea to use AWS Lambda custom runtime to have a lambda that takes in HTML and returns back PDF

but could also be used for all sorts of browser automation things now

for sure! For AWS Lambda #nbb could also be an interesting option: https://github.com/babashka/nbb/blob/main/doc/aws_lambda.md There are many pdf things for Node.js as well. But I don't want to distract you too much from this project, since I could see it being very useful for browser testing in bb + JVM

As for lambda: if you could move clojure.spec aside too, then this code would load even faster.

I might bring clojure.spec.alpha as a built-in in the future, then that becomes irrelevant for startup time, but right now it has to load a bunch of spec code at startup

this relies heavily on clojure.spec

well, it might not be absolutely necessary, but all the autogenerated commands have fdef specs

I could imagine moving the specs to auxiliary namespaces and having some switch to skip loading them...

you could even detect bb and skip them by default

btw, regarding firefox, it does support a subset of the CDP, so you can use it as-is

just tried, it supports the same --remote-debugging-port=<port> parameter

If your program relies on spec, then that's good, keep it! But often spec is loaded not for any good reason. Not in this case.

@U11SJ6Q0K Just spitballing here, but would it help in any way if a browser were controllable through nREPL: https://twitter.com/borkdude/status/1511456422667735042/photo/1 I feel there could be some kind of synergy between the debug protocol and the thing I'm currently working on

I noticed when trying firefox on the debug protocol, that after:

(def a (auto/create-automation (core/connect)))
(auto/to a "")

hmm, I was able to print a PDF with firefox

it was very much slower than chrome but worked

about sci, you could use CDP to just do js eval on the browser

The only arg parsing changes came from the bb.edn pods stuff ;)

Repro welcome, we can do a follow up release if it's critical

what makes you think it's related to arg parsing, out of curiosity?

I thought you hinted at that, but maybe I've not read good enough

ah ok

darn, thought you had noticed something I hadn't 😉

haven't been able to repro in a more minimal example yet... but I'll keep working on it

probably best to git bisect

you can just run with jvm using a git sha

hmm... don't think I've done that before. is that documented somewhere?

to clarify, I've done git bisect, but haven't run bb w/ jvm (is that bb clojure? not sure how to do that w/ an arbitrary git sha)

@U06FS3DLH I mean, just the process of choosing a git sha, checking if the error happens there, etc.

not literally git bisect, but just splitting the commit range in half, trying, etc.

ok

you can just invoke babashka as a git dep + -M -m babashka.main

ah, I see. I'll give that a try. thanks!

how do you deal with the git submodules when you do that? according to jira they're not supported in deps.edn git deps

oh right. well, maybe use a local root + git reset then

I must say that I always only use babashka JVM as a local root locally

:thumbsup: makes sense

ok false alarm. it was a problem in my code after all. just strange that it behaved so differently in 0.7.8. it's possible it was a combination of things I hadn't tested before. anyway, I don't think babashka is to blame either way.

🎉 great to hear we didn't forget to add a test for something :)