Fork me on GitHub
#babashka
<
2020-01-08
>
borkdude10:01:00

Now removed the reliance on any private sun or Unix-specific class by applying a trick: https://github.com/borkdude/babashka/blob/2b0cb6fb1cf2f86a9bdf22dd6b954a35ec1be21f/src/babashka/impl/classes.clj#L178 Reflection happens on the public class and that seems to work

borkdude10:01:38

So that should also work for JDK11

borkdude13:01:01

@UG1C3AD5Z I upgraded the jdk 11 branch. It fails on CI with an out of memory error. https://circleci.com/gh/borkdude/babashka/3364

borkdude13:01:45

On linux, given enough memory, I feel that it should now be possible to compile.

sogaiu14:01:26

this is the jdk11 branch, right? seems to have worked here πŸ‘ got to hear my fans spin up

sogaiu14:01:20

all tests seemed to have passed too

sogaiu14:01:55

may be you saw that https://github.com/oracle/graal/issues/2015#event-2928406659 was labeled as a bug within the last day?

rickmoynihan11:01:36

Curious about this line in the rationale of deps.clj: > This offers an arguably easier way to get going with deps.edn based projects in CI. Just download an installer script, execute it with bash or Powershell and you’re set. Installer scripts are provided for linux, macOS and Windows. Isn’t that essentially the same installation process as tools.deps?

borkdude12:01:39

Not for macOS and Linux

borkdude12:01:48

Sorry Windows

borkdude12:01:09

So not for macOS and Windows

borkdude13:01:23

First successful linux build of babashka on JDK11. Binary for the grabbing in #babashka_circleci_builds !

jeroenvandijk14:01:38

Is there a way to retrieve the path from the code of the current file that is being executed? E.g. in bb src/path/foo.bb I would like to get the value src/path/foo.bb somehow

jeroenvandijk14:01:25

Is this on purpose?

user=> (.getName *ns*)
Method getName on class sci.impl.vars.SciNamespace not allowed! [at line , column ]

jeroenvandijk14:01:45

This gives me what I want as well (str *ns*)

jeroenvandijk14:01:41

yes! thanks πŸ™‚

borkdude14:01:17

@jeroenvandijk You can use *file* for that

borkdude14:01:26

and the second example: (ns-name *ns*)

borkdude14:01:47

(oh sorry, slack was lagging, I see Nate answered it already)

jeroenvandijk17:01:59

In case you are interested, I'm trying to build a script in babashka that can delegate to clojure when needed:

(defn handle-in-clj [msg cli-args]
  (println msg)
  ;; TODO stream output with ProcessBuilder instead
  #_(def ws (-> (ProcessBuilder. (into ["clj"] args)) (.start)))

  (let [cmd (into ["clj" *file*] (map str) cli-args)
        ; _ (prn "CMD" cmd)
        p (apply clojure.java.shell/sh cmd)]
    (exit (:exit p)
          (if (zero? (:exit p))
            (:out p)
            (if (clojure.string/blank? (:err p))
              (:out p)
              (:err p))))))

(defmacro delegate-to-clj [& body]
  #?(:bb `(handle-in-clj "Delegating to clj" *args*)
     :clj `(do
             [email protected])))

jeroenvandijk17:01:22

For simple scripts you have the benefit of minimal boot time. For things that require clojure you can already do some input validation before starting the heavy process and saving you time in case of errors. All without having to think about it 😎

jeroenvandijk17:01:29

That's the idea anyway :)

borkdude18:01:28

@jeroenvandijk Take a look at the deps.clj script, it shows how to stream out and err

borkdude18:01:28

@jeroenvandijk Take a look at the deps.clj script, it shows how to stream out and err

jeroenvandijk08:01:15

Thank you! I'll have a look

jeroenvandijk11:01:32

Works in Babashka! Funilly enough the JVM doesn't recognize ProcessBuilder$Redirect/INHERIT It is able to run (ProcessBuilder$Redirect/INHERIT), but that doesn't work in Babashka

borkdude11:01:20

I'm not sure what you mean. deps.clj is tested with both babashka and JVM clojure

borkdude11:01:16

$ clojure $(which deps.clj) -Spath

borkdude11:01:32

$ bb $(which deps.clj) -Spath

borkdude12:01:37

Can you make a repro of an isolated piece of code that runs with bb but not with clojure?

borkdude12:01:08

^ @jeroenvandijk πŸ™‚

jeroenvandijk15:01:17

@U04V15CAJ ah sorry, I made a mistake. Everything does work. Sorry to bother

borkdude20:01:03

spartan.test, a test lib that can be used with bb, now has testing, thrown? and thrown-with-msg?: https://github.com/borkdude/spartan.test

borkdude21:01:23

Some progress made on adding clojure.data.xml. Feel free to read+test: https://github.com/borkdude/babashka/issues/153

borkdude21:01:22

Seems it also works with 11.0.5:

[email protected]:/tmp$ ./bb
Babashka v0.0.61-SNAPSHOT REPL.
user=> (require '[clojure.data.xml :as xml])
nil
user=> (let [xml (xml/parse-str "<items><item>1</item><item>2</item></items>")] (xml/emit-str xml))
"<?xml version=\"1.0\" encoding=\"UTF-8\"?><items><item>1</item><item>2</item></items>"
user=> (System/getProperty "java.version")
"11.0.5"

borkdude21:01:48

I'm always a bit wary of making choices of what should go into babashka. E.g. should I have added clojure.data.json instead of cheshire.core since I already have the other clojure.* stuff? Should I remove core.async because nobody is using it? What will happen to the binary size in the long run?

Nate Sutton21:01:17

I would love it if you kept core.async

Nate Sutton21:01:29

I don't really have thoughts on the json stuff, though

Nate Sutton21:01:17

one of my selling points to coworkers is "you can do/fetch things in parallel in a way that's not completely ridiculous like in bash"

borkdude21:01:42

yep, but you can already do that with pmap? πŸ™‚

borkdude21:01:32

Maybe I should do a yearly survey πŸ˜‰

Nate Sutton21:01:55

I suppose that's true but there's definitely more power in core.async

Nate Sutton21:01:17

but I get the concern about binary size

Nate Sutton21:01:20

and what's good enough

borkdude21:01:47

I'm not sure how much binary size matters to people. Does only the download size matter, or also the extracted size? What's still acceptable?

Nate Sutton22:01:44

what matters most to me is startup time

borkdude22:01:16

For me too.

borkdude22:01:29

Startup time doesn't seem to be affected by the number of libs I add to it

Nate Sutton22:01:35

could core.async be optional? an external library you could pull in?

Nate Sutton22:01:55

babashka.async πŸ˜‰

Nate Sutton22:01:32

I'm a single data point but download size doesn't matter all that much to me

borkdude22:01:06

Maybe I should put together a questionnaire.

Nate Sutton22:01:16

it's not a bad idea

borkdude22:01:08

Anyway. Adding clojure.data.xml seems like a good idea, but I'd like to hear if it's not πŸ™‚

Nate Sutton22:01:51

supporting the common data formats seems smart

Nate Sutton22:01:03

json, xml, edn, yaml

borkdude22:01:15

and also probably hiccup for html

borkdude22:01:21

generating html at least

Nate Sutton22:01:31

you could make a static site generator then

Nate Sutton22:01:16

now we just need something like this for babashka, pick your own libraries troll https://jqueryui.com/download/

borkdude22:01:48

the only way to make that work currently is to let users enable the libraries via some UI and circleci will build it

borkdude22:01:23

and if someone already has chosen the exact combination of libs before, you will just get the already compiled version immediately

Nate Sutton22:01:56

yeah, like that

Nate Sutton22:01:07

it's kind of wild but it also gives a lot of flexibility

borkdude22:01:09

cool idea for the future. I made an issue for it.

Nate Sutton22:01:20

awesome πŸ™‚

Nate Sutton22:01:38

in theory they could supply the deps in general and it could try to bundle them?

Nate Sutton22:01:46

that might be getting into a little extreme territory

Nate Sutton22:01:01

probably only want blessed deps

borkdude22:01:20

yeah, I don't think that's going to work. I think it has to be curated manually somehow

borkdude22:01:37

adding macros is a bit of a manual effort

borkdude22:01:43

there could be a branch where people could PR their new favorite library bindings and some config so this UI knows how to enable it

borkdude22:01:55

but if people really want their personal libs in bb, they might as well just fork it

borkdude22:01:02

and build it themselves on circle

Nate Sutton22:01:50

making that easy is a neat idea, at least

Nate Sutton22:01:04

so many ideas

Nate Sutton22:01:06

so little time

Nate Sutton22:01:22

I've barely had time to work on my filesystem stuff

Nate Sutton22:01:40

and I see you figured out ways to not pull whole classes per platform

Nate Sutton22:01:09

well, as long as it obeys and interface that's more generic

Nate Sutton22:01:13

like Path vs UnixPath

borkdude22:01:28

yes, happy that worked out nicely

Nate Sutton22:01:30

I'm wondering if that will work with FileSystem

mkvlr22:01:56

if we finally release an api for http://nextjournal.com notebooks we could build it there

mkvlr22:01:37

our default environments now already ship with babashka, jet & friends

mkvlr22:01:46

and graal 19.3 is our default JVM

borkdude22:01:06

that would be awesome... you could just select the libs as a notebook

borkdude22:01:19

instead of a custom web ui

borkdude22:01:18

@mkvlr what memory / CPU / resource restrictions do you have in nextjournal for free / OSS use?

mkvlr22:01:17

currently 15GB of RAM + 1 GPU is free but happy to bump that if you need more

borkdude22:01:37

wow, thats already lots more than circleci.

mkvlr22:01:05

we also have a lot less users 😼

borkdude22:01:26

I'll make a note of this in the issue

mkvlr22:01:51

I’ll see if I can come up with a proof of concept

mkvlr22:01:48

so how do I build babashka with custom libs?

borkdude22:01:37

@mkvlr in babashka.main.clj there is a map which determines which namespaces are included. dissoc-ing that map would already have an effect of which libs are left out

borkdude22:01:18

I think graalvm is clever enough to see that it should leave out the libs from the compiled artifact if you dissoc them

borkdude22:01:42

but in the end I would do this via some edn config file probably that is read at build time

borkdude22:01:56

to construct that map

mkvlr22:01:07

ok, so we want to allow to pass in the whole map via edn or how do you envision the api?

borkdude22:01:59

Not sure yet.

borkdude22:01:17

Maybe just getting the build running in the notebook is a good first step?

mkvlr22:01:17

Syntax error (ClassNotFoundException) compiling at (babashka/impl/classes.clj:188:16).
java.lang.UNIXProcess$ProcessPipeOutputStream

borkdude22:01:38

hmm, that might not be the most recent master

borkdude22:01:52

also you need jdk8

mkvlr22:01:09

no, I checkout out the latest release

borkdude22:01:08

there is no mention anymore of UnixProcess classes, removed that today, so I can't explain that error. Can you show the git sha?

mkvlr22:01:36

changed it to master now

mkvlr22:01:55

but it’s still JDK 11

mkvlr22:01:06

Execution error (FileNotFoundException) at babashka.impl.clojure.core.server/loading (server.clj:12).
Could not locate sci/core__init.class, sci/core.clj or sci/core.cljc on classpath.

borkdude22:01:20

also, there is a circleci config that shows how / where to download graalvm etc

borkdude22:01:42

I'm afk now, happy to continue tomorrow

mkvlr22:01:13

same, have a good night!

mkvlr22:01:49

cool, it compiles now with JDK 11

mkvlr22:01:15

almost, but it got further…