Fork me on GitHub
#babashka
<
2020-03-21
>
sogaiu00:03:34

all tests passed here :thumbsup:

sogaiu09:03:24

anyone investigated cross-platform cryptographic hash approaches?

sogaiu09:03:39

just looking to use sha1 or similar from bb

sogaiu09:03:35

anyone had luck getting http://java.io.File's deleteOnExit to work in babashka?

sogaiu10:03:58

regarding sha1-ing of strings, here's something that i've tested on linux and windows: https://gist.github.com/sogaiu/fa930abb815daade41dd24e0c76f4d86

sogaiu10:03:25

it may also work on macos

borkdude10:03:53

@sogaiu

(def hashed (.digest (.getInstance java.security.MessageDigest "SHA-1")
                     (.getBytes "babashka")))

(doseq [b hashed]
  (print (format "%02X" b)))
prints:
0AB318BE3A646EEB1E592781CBFE4AE59701EDDF

borkdude10:03:20

@sogaiu doesn't that work cross platform?

sogaiu10:03:42

@borkdude ah thanks -- that works on linux. i don't see why it shouldn't on windows. will check.

sogaiu10:03:51

yes, it works on windows too 🙂 thanks again!

borkdude10:03:25

cool 🙂

sogaiu10:03:26

here's a defn version:

(defn sha1
  [a-string]
  (let [hashed (.digest (.getInstance java.security.MessageDigest "SHA-1")
                 (.getBytes a-string))]
    (apply str (map #(format "%02X" %) hashed))))

borkdude10:03:17

now improved docs with defn version

sogaiu10:03:39

so efficient with the StringWriter 🙂

borkdude10:03:49

oh this is actually very handy for updating my homebrew stuff

borkdude10:03:59

right now I was doing this in bash still, you know, legacy scripts 😉

babashka 1
borkdude11:03:44

@hobosarefriends The above example is interesting, maybe also inspiring for your problem to package up a bb script as a standalone thing. Maybe packaging it as a Docker image could work. 🙂

borkdude11:03:16

Just push it to docker hub mno/thing and then let the consumer run it as docker run --rm mno/thing

Mno11:03:43

That seems like something that could be abused fairly quickly

Mno11:03:17

Also the dependency removal is what I'm trying to achieve from the binary packaging

Mno11:03:22

I probably don't understand what it means exactly

Mno11:03:33

So I'll look into it later

Mno11:03:20

But if they switch out their dependency from the bb binary to docker its still dependent

borkdude11:03:36

I mean, you can make a Dockerfile:

FROM borkdude/babashka
COPY yourscript
ENTRYPOINT bb yourscript
and then push it to dockerhub.

Mno11:03:36

Even if it is more likely that docker is on someone's computer 😅

borkdude11:03:57

making a tiny bash script that downloads bb on their system and then runs your script also could work 😉

Mno11:03:59

Haha yeah I suppose

Mno11:03:12

But then it's just hiding the dependency

Mno11:03:47

I guess I want an immutable independent micro application

Mno11:03:24

I should work on this while we're isolating :rolling_on_the_floor_laughing:

Mno13:03:56

That's graat already

Mno13:03:27

And the brightside is its basically universal

borkdude14:03:17

and you can package any dependencies along with the image

borkdude14:03:41

I wonder what the startup time on linux is for docker. On macOS it's not too great, like 400ms 😉

borkdude14:03:58

but this is probably due to some virtual machine indirection thing

Mno14:03:11

The downside is startup time goes up dramatically 😅

Mno14:03:23

That kinda defeats the purpose

Mno14:03:08

Though that's probably a fixed cost, and half a second isn't too much unless you're running it a bunch

renewdoit14:03:37

Any thought on bring a simple template function into babashka?

renewdoit14:03:09

I think comb is small enough to include. I already tested it is compatible.

borkdude15:03:51

@renewdoit responded in the issue

borkdude16:03:39

@renewdoit to recap: the library comb can be interpreted with babashka, but cannot be compiled with graalvm native, because graalvm native cannot handle clojure.core/eval.

borkdude16:03:28

therefore it cannot be included, that's the technical argument. the other argument is that we should consider carefully all alternatives before including something in bb, because once it's in, it's likely that we'll have to maintain it forever

borkdude16:03:19

But it's nice that you pointed at this lib, because now we can list it with the compatible libraries: https://github.com/borkdude/babashka/#comb

nate19:03:13

running into a bit of an inconsistency and wondering if it's a bug in my code or something in babashka (story in thread)

nate19:03:08

mostly a proof of concept, but I'm working on a bb script that uses comb to template edn data

nate19:03:27

$ cat data/names.edn
{:name "Bob"}
{:name "Marcia"}

nate19:03:48

if I run my code with deps.clj:

$ cat data/names.edn | bb -I --stream -cp $(deps.clj -Spath) "(ns user (:require [comb])) (apply comb/-main *command-line-args*)" -t "<%= name %>"
data: {:name "Bob"}
Bob
data: {:name "Bob"}
Bob

nate19:03:36

but if I uberscript it and run it:

$ cat data/names.edn | bb -I --stream -f uberscripts/comb.clj  -t "<%= name %>"
data: {:name "Bob"}
Bob
data: {:name "Marcia"}
Marcia

nate19:03:34

I can't figure out why the classpath version doesn't properly re-bind user/*input* to each input map

nate19:03:00

$ bb --version
babashka v0.0.75

nate19:03:15

trying with latest to see if that might affect things

borkdude19:03:26

classpath version = --stream -cp one?

borkdude19:03:23

I'm not sure what I'm looking at. The output of both programs seems identical?

nate19:03:46

oh, in the first one, Bob is printed for both lines of input data

borkdude19:03:48

oh you get Bob twice in the first one

nate19:03:39

bb 0.77 same results

borkdude19:03:04

not sure, this requires debugging. maybe you can repro it using a simpler example

nate19:03:06

I don't really mind to terribly much as I'm planning on using the uberscript versions after development

nate19:03:26

sure thing, I'll see about setting up a minimal repro

nate19:03:40

@borkdude Here's a repo with minimal code for reproduction. I also put the commands in the readme. https://github.com/justone/bb-input-repro

nate20:03:19

You're most welcome. Thanks again for babashka. It's supremely wonderful.

borkdude20:03:31

Thanks, that's a big compliment. I found something:

"stream?" true
dude
read from in {:name Bob}
args nil
input {:name Bob}
dude
read from in {:name Marcia}
args nil
input {:name Bob}

nate20:03:19

Oh interesting.

borkdude20:03:57

yeah, so $ cat names.edn | clj -A:bb-local -I --stream '*input*' works as expected... hmm

borkdude20:03:36

this also works as expected:

$ cat names.edn | clj -A:bb-local -I --stream -cp src "(ns user (:require [testinput])) (prn \"X\" user/*input*) "

nate20:03:17

Indeed.

nate20:03:11

When I first ran the one that prints two Bobs, I figured it was something with my code. And then I tried the uberscript version and was surprised that worked.

borkdude21:03:15

I'm nearing a solution.. getting there.

nate21:03:10

Ooh, cool!

borkdude21:03:45

Note that:

$ cat names.edn | bb -I --stream -cp src "(ns user2 (:require [testinput] :reload)) (apply testinput/-main *command-line-args*)"
Bob
Marcia
does work 😉

borkdude21:03:24

Pushed the fix to master.

nate21:03:20

Ah, forcing it to reload the namespace does the trick.

nate21:03:23

wow, reading through the code change

nate22:03:05

@borkdude tried out the master build and it works perfectly!

borkdude22:03:39

re-using the existing var did the trick. the code that re-inserted a new item into the state every iteration was still from the time that we didn't have vars.

nate22:03:38

Oh, very cool. Thanks!

penryu23:03:47

First, thank you so much for babashka, @borkdude! It scratches an itch that I'd only just begun to realize I had. 😄

penryu23:03:36

I'm running into an issue where clojure's reader function fails to read files off linux's /proc fs, apparently due to the way procfs reports virtual file metadata.

penryu23:03:25

user=> (-> "/proc/loadavg" io/reader line-seq)
Execution error (IOException) at .FileInputStream/available0 (FileInputStream.java:-2).
Invalid argument

penryu23:03:16

Apparently there's a workaround using java's FileReader instead of the BufferedReader that Clojure's reader uses, but I'm unable to import it under babashka:

user=> (import .FileReader)
Unable to resolve classname: .FileReader [at line 1, column 1]

penryu23:03:15

Is there another way for bb to read procfs file entries?

penryu23:03:42

If not, (shell/sh "cat" "/proc/loadavg") is always an option, but hoping there's a way that doesn't require a new process.

borkdude08:03:55

I’ll have to add support for FileReader. Can you post these details in an issue?

penryu08:03:48

Ah, I wasn't sure if it was intentional. I can even try my hand at a PR if you like.

borkdude09:03:32

creating a new process with babashka isn't too bad though, it's what you would have done in a bash script as well

borkdude09:03:19

I'm not sure in what other cases FileReader would be useful. Note that it will add size to the binary if we include it, so if the use case is fairly niche, shelling out might be preferable

penryu09:03:23

In bash I would have done something like: cut -d' ' -f1,2,3 < /proc/loadavg, which afaict would have been one external process cut with /proc/loadavg redirected as its stdin. In babashka I can (and do) just use str/split and keep-indexed for this, so it can be all in process. I kind of feel like my case is niche and easily worked around using cat, but since procfs misreports inode data for all its virtual file entries, which are numerous. io/reader and slurp would be equally unusable for them as well. But it would be just as easily worked around using cat. As a result of procfs being my primary driver, the test would be OS-specific.

borkdude10:03:54

The size with filereader is 42025276. Without filereader it's 42008876. That's 16kb, almost nothing if you ask me, so I'm ok with adding it.

borkdude10:03:14

A conditional which tests if the OS is linux inside the unit test would be good.

borkdude10:03:26

I'm already working on a branch now 🙂

borkdude10:03:15

lein test babashka.main-test
"output:" "17.90 14.68 11.61 4/5580 2101\n"

borkdude10:03:16

ok, it's on master now

penryu11:03:34

My native-image install seems to be broken, but the jar seems to work just fine:

[git:master] kanin:…/code/github/babashka>% rlwrap java -jar target/babashka-0.0.79-SNAPSHOT-standalone.jar
Babashka v0.0.79-SNAPSHOT REPL.
Use :repl/quit or :repl/exit to quit the REPL.
Clojure rocks, Bash reaches.
user=> (import '.FileReader)
user=> (require '[ :as io])
user=> (-> "/proc/loadavg" FileReader. io/reader slurp)
"0.57 0.28 0.11 3/239 2989\n"

borkdude11:03:03

I recommend just downloading GraalVM in your Downloads folder and then setting GRAALVM_HOME to it. It's as simple as that.

borkdude11:03:12

babashka currently uses version 19 java 8.

borkdude11:03:28

There are pre-built binaries in #babashka_circleci_builds

penryu11:03:28

Oh, they moved native-image to a separate download!

penryu11:03:46

Building now.

👍 1
penryu11:03:09

Thanks, @borkdude! 🙂