Fork me on GitHub
#babashka
<
2021-01-22
>
Marco Pas07:01:53

I am a bit lost on how one should write and execute unit tests for bb scripts. Checked the bb book but could not find any pointers on how to start doing this. Quick remark that this question is coming from a clojure newbie 🙂 Is there something like $ bb run-tests ?

Marco Pas07:01:04

Would be more then happy to write some howto that maybe can be added to the book if this is usefull by the way 🙂

borkdude07:01:57

@marco.pasopas go to the book and Ctrl-f on clojure.test. It is in there somewhere

Marco Pas08:01:55

Found the recipe indeed trying to understand what it is doing 🙂

Marco Pas08:01:00

Fighting name-spaces and folders 🙂

Marco Pas08:01:03

But i will get there 🙂

borkdude08:01:06

@marco.pasopas I now updated that part of the book, hopefully it's clearer

borkdude10:01:05

pod-babashka-aws 0.0.5 released: fixes an issue with nil in the aws-api response which prevented using it with the lambda API. Now it works.

Adrian Smith10:01:38

Will espresso for Graalvm allow Babashka to run more Clojure programs?

borkdude10:01:16

Maybe, possibly, I don't know yet :)

👍 3
Marco Pas12:01:37

@borkdude Thanks this surely helps.. But i am not there yet 🙂 Fighting the concepts of namespaces and filename is guess.. Having the following simple test project..

.
├── run-test.sh
└── test
    └── util_test.clj
# util_test.clj
(ns util-test)

(deftest testme
  (is (= true)))

(deftest testme2
  (is (= 8 (+ 1 2))))
# run-test.sh
#!/usr/bin/env bash
bb -cp "src:test:resources" \§
   -e "(require '[clojure.test :as t] '[util])
       (let [{:keys [:fail :error]} (t/run-tests 'util)]
         (System/exit (+ fail error)))
Still it shows up with running 0 tests running

borkdude12:01:30

@marco.pasopas '[util] should be 'util-test

borkdude12:01:05

also in run-tests

borkdude12:01:47

that invocation should throw with "Could not find namespace util" or something similar

borkdude12:01:05

The name of your namespace is not util, it is util-test

👍 3
Marco Pas12:01:05

Guess i need to include something more..

borkdude12:01:34

:refer [deftest]

borkdude12:01:42

The example has (t/deftest ...)

borkdude12:01:42

I mean, in util you need to require clojure.test as well and :refer [deftest] if you want to use it unqualified

👍 3
borkdude12:01:59

(ns util (:require [clojure.test :as t :refer [deftest is]]))

Marco Pas06:01:14

@borkdude You are really helping me out in understanding BB and also Clojure txs!!

Jakub HolĂ˝ (HolyJak)13:01:55

Hello! I have read https://github.com/babashka/pods but did not find an explanation of how bb finds the pod. Does it use https://github.com/babashka/pod-registry to find the source gh repo and then downloads whatever is needed? And what if I want to package the pod with my app (aws lambda, actually)? Thanks!

borkdude13:01:31

@holyjak You can load pods using a qualified symbol, like org.babashka/aws and a version "0.0.5". In that case it will download it from the pod-registry by looking up a manifest. When calling load-pod with a string or vector of strings, the pod is looked up on the local file system (either using the PATH, or using an absolute path)

borkdude13:01:02

If you want to take pods along into a docker image, probably the best is to download them manually inside the docker image and use an absolute or relative path

borkdude13:01:40

But you can also just copy the ~/.babashka directory where the downloaded pods are living, that should also work

borkdude13:01:36

@holyjak I noticed you were using clj-http-lite from source, this isn't necessary anymore, now that there is babashka.curl and org.httpkit as built-in clients

👍 3
borkdude13:01:50

But it still works

Jakub HolĂ˝ (HolyJak)14:01:15

@borkdude I guess it would be nice to update the "About" of https://github.com/babashka/babashka to point to https://book.babashka.org/ 🙏

borkdude14:01:05

done

❤️ 3
borkdude14:01:24

We will perhaps have a nice landing page on http://babashka.org sometime soon.

Jakub HolĂ˝ (HolyJak)15:01:17

FYI I will be looking into adding Oracle support to the sql pod

Jakub HolĂ˝ (HolyJak)15:01:18

@borkdude I have just noticed https://github.com/babashka/babashka/pull/642 is still open. What would you like me to do with it?

borkdude15:01:13

Up to you, you can close it or finish it. Beware of Oracle Licensing when distributing pods

Jakub HolĂ˝ (HolyJak)15:01:03

What is missing for it to be finished? I believe the licensing is nowadays ok, i.e. you can re-distribute, but I will check it again and link to some official docs.

borkdude15:01:25

I would prefer to add these vars to the pod first, so we can write real tests to verify if they work correctly and to see what impact they have on the size. See other message in channel

👍 3
borkdude15:01:45

If they are verified to work in the pod, they can be added to the feature flag as well

Jakub HolĂ˝ (HolyJak)15:01:21

I will first work on adding oracle to the pod then come back to extend the next-jdbc support

borkdude15:01:42

@holyjak When adding stuff, I would recommend to take very small steps, so we can monitor the growth of the binary

👍 3
borkdude15:01:13

Some things trigger excessive growth (usually runtime resolve or require), so taking it one var and test at a time is probably the best

Jakub HolĂ˝ (HolyJak)16:01:04

@borkdude here is my first attempt https://github.com/babashka/babashka-sql-pods/pull/15 However it fails to compile, with > Error: Incompatible change of initialization policy for java.lang.Math$RandomNumberGeneratorHolder: trying to change RUN_TIME from the command line to RERUN for random number generator > com.oracle.svm.core.util.UserError$UserException: Incompatible change of initialization policy for java.lang.Math$RandomNumberGeneratorHolder: trying to change RUN_TIME from the command line to RERUN for random number generator Any tips how to trouubleshoot & fix that? 🙏

borkdude16:01:40

@holyjak Yes, with GraalVM 20.3.0 this line should be removed

borkdude16:01:53

Feel free to also update GraalVM in circleci / appveyor to 20.3.0

Jakub HolĂ˝ (HolyJak)16:01:04

I have 20.3:

🐟  $GRAALVM_HOME/bin/native-image --version
GraalVM Version 20.3.0 (Java Version 11.0.9+10-jvmci-20.3-b06)
yet I still am getting this error 😭

borkdude16:01:26

> Yes, with GraalVM 20.3.0 this line should be removed I mean, you should actually delete that line

Jakub HolĂ˝ (HolyJak)16:01:50

ah, this one

--initialize-at-run-time=java.lang.Math\$RandomNumberGeneratorHolder \
in compile ?

borkdude16:01:21

and compile.bat

Jakub HolĂ˝ (HolyJak)16:01:11

I don't see this file

borkdude16:01:31

oh sorry, this is not part of this repo. there is no Windows version yet, ok

Jakub HolĂ˝ (HolyJak)16:01:43

I guess I should also update generate_circleci.clj to use 20.3.0?

borkdude16:01:53

and appveyor.yaml

borkdude16:01:08

you can do it as a separate PR

👍 3
borkdude16:01:06

@holyjak let's take this to the dev channel, I just invited you there

👍 3
borkdude19:01:24

Welcome @tgg

😄 3
👋 3
seancorfield22:01:47

Does babashka work with next.jdbc? I thought it did but I just got an issue opened that next.jdbc doesn't compile with GraalVM: https://github.com/seancorfield/next-jdbc/issues/156

borkdude22:01:27

@seancorfield We have a feature flag for next.jdbc and also some sql "pods" which are optional binaries which bb can download to get more features. These are all compiled with GraalVM native-image, but probably not with the most recent versions of next.jdbc.

borkdude22:01:07

@seancorfield The issue isn't very descriptive of what goes wrong, but we can update our next.jdbc to see if it still works, in the sql pods

borkdude22:01:11

we were using "1.1.610", now trying upgrade to 613

borkdude22:01:47

@seancorfield Just upgraded to 613 and everything works correctly: https://github.com/babashka/babashka-sql-pods

seancorfield23:01:51

I really appreciate you providing all that information in the GH issue!

Jakub HolĂ˝ (HolyJak)12:01:06

Also notice that the sql pod exposes only a subset of next-jdbc. My experience was that when I tried to add the .prepare ns, the compile time and CPU use exploded until my PC could not take it anymore. So it is possible that not all parts of next-jdbc can work in GraalVM (yet)

seancorfield22:01:54

I'm happy to review PRs that folks send in about GraalVM compatibility but it's very low priority for me since I have zero interest in native-compiled Clojure right now.

borkdude22:01:29

@seancorfield The most important thing for GraalVM users is avoiding reflection warnings, so as long as you have (set! *warn-on-reflection* true) everywhere, and you can avoid using require, resolve and eval (or variations) during runtime (top-level is ok), this will cover 99% of the problems

seancorfield22:01:05

I think I have (set! *warn-on-reflection* true) everywhere... Ah, I don't have it in next.jdbc.default-options or next.jdbc.plan. Will add it there (and fix any reflection warnings).

seancorfield22:01:50

Added. No reflection warnings. A couple of places have requiring-resolve (for whether camel-snake-kebab is on the classpath or not). There's one place in next.jdbc.prepare that refers to a symbol from next.jdbc.result-set (which already requires next.jdbc.prepare) so that's probably what @holyjak ran into...

seancorfield22:01:23

The CSK stuff is all handled in macros so if you do not have CSK on your classpath, there's no dynamic require in the runtime (but it does mean you can't create a GraalVM native image if you try to use CSK).

seancorfield22:01:47

That prepare one is nasty since it's trying to avoid a circular dependency. I'll open an issue for that.

borkdude23:01:34

Ah that makes a lot of sense, yes, these things can make native-images 30-50 mb larger and also analysis takes a lot longer. I sometimes work around circular dependencies by creating a volatile in another namespace and vreset-ing the defined function to that volatile, so I can use it from other places. A bit hacky, but it works

seancorfield23:01:12

Interesting workaround. In my case, prepare/execute-batch! is really in the wrong place. It should have been in result-set instead. But I could use your volatile approach to fix it -- although there might still be paths through people's code to next.jdbc.prepare that don't go through next.jdbc or next.jdbc.result-set which would be the two obvious candidates to include that vreset...