Fork me on GitHub
#babashka
<
2020-03-27
>
deep-symmetry00:03:44

I just noticed that I forgot to define a zero-arity version of one of the functions I’m calling from sci, but never discovered that because it worked the way I intended, which is as if I had passed a nil. Of course, if I call it that way from a Clojurescript REPL, it throws an arity error. Is this just a feature of the interpreter as compared to a compiler?

deep-symmetry00:03:51

(As an aside, the byte field generator I used sci for has worked wonderfully, I have ported my big, diagram-heavy LaTeX documents to a much more maintainable and attractive Antora site, here’s a sample section: https://djl-analysis.deepsymmetry.org/djl-analysis/track_metadata.html#_connecting_to_the_database And I am currently finishing off the documentation for the DSL the diagram generator offers.)

borkdude07:03:19

@deep-symmetry that’s probably a bug. Feel free to post an issue with a repro. And good to hear!

borkdude11:03:52

Can't repro this:

$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.597"}}}' -m cljs.main -re node
ClojureScript 1.10.597
cljs.user=> (require '[sci.core :as sci])
nil
cljs.user=> (sci/eval-string "(defn foo ([x] x)) (foo)")
Execution error (Error) at (<cljs repl>:1).
Cannot call foo with 0 arguments

deep-symmetry12:03:19

I’ll try to reproduce it for you. One difference in my case is that the function was defined in Clojurescript and is being invoked from an sci script.

borkdude12:03:58

that's on CLJS I think

borkdude12:03:04

and maybe more in general: JS

deep-symmetry12:03:27

Ok! I suspected as much and it wasn’t a problem so much as “hmm” moment.

deep-symmetry12:03:42

I asked only to try to better understand the interaction. In any case I added the missing arity to avoid confusion and the risk of cljs changing behavior in the future.

borkdude13:03:48

what you're seeing is basically this:

$ plk
ClojureScript 1.10.520
cljs.user=> (defn f [x] x)
#'cljs.user/f
cljs.user=> (f)
            ^
WARNING: Wrong number of args (0) passed to cljs.user/f at line 1
nil

borkdude13:03:03

but without the compiler warning

deep-symmetry14:03:17

Thanks, that makes sense! This is my first foray into Clojurescript, so a lot of learning is expected.

borkdude16:03:06

you've made something quite impressive, especially for someone who is new to CLJS

deep-symmetry04:03:26

Thank you! 😄 I’ve done plenty of work in JVM-hosted Clojure, which certainly gave me a nice starting point.

borkdude08:03:13

talk at ClojureD: https://youtu.be/Nw8aN-nrdEk

🙏 4
🎉 28
⚛️ 4
👏 8
coolj17:03:18

Is anyone using babashka in a Jenkins pipeline? It might be possible to invoke bb from Groovy but I am guessing that's a world of trouble. Running the bb via an sh command isn't possible from what I can tell.

borkdude17:03:57

> Running the bb via an sh command isn't possible from what I can tell. What's the problem with this?

coolj17:03:41

I think it's some sort of a chroot jail created for the Groovy process running the pipeline. It refuses execute the binary from /tmp or the local dir. Installing to /usr/local/bin is not an option for me. The bb installer works but that's just a bash script and that's allowed. Don't worry, I can run bb in a container that's created by the pipeline.

borkdude17:03:19

is that Docker containers?

borkdude17:03:06

bb also has a docker image you could use maybe

coolj17:03:14

Thanks, that is my next move.

coolj17:03:16

I could run an uberjar in the pipeline too. Java is installed as Jenkins is written in Java.

borkdude17:03:54

yeah, bb is available as an uberjar. is that still helpful since you could just be using clj?

coolj17:03:05

It's an option. It's heavyweight but it would work in this particular context. I like trying bb first and it's good to know there is the fallback uberjar option if all else fails. The bb docker image is probably a better choice here.

borkdude17:03:48

ah for prototyping that makes total sense

coolj18:03:28

this works in the pipeline

steps {
	    sh 'curl -o babashka-standalone.jar -sL '
	    sh 'java -jar  babashka-standalone.jar test.clj'
      }
That's pretty amazing for a 12M download.

coolj18:03:57

Who needs Groovy!

Darin Douglass20:03:06

i am, though from a dockerfile 😉

Darin Douglass20:03:01

i actually just hooked up the code to convert our helm-based kube manifests to edn-based which are parsed into yaml with bb

borkdude20:03:22

that's interesting, bb doesn't support yaml yet 🙂

Darin Douglass20:03:02

ya,

;; TODO: when bb supports a yaml library, use that.
(defn ->manifest

Darin Douglass20:03:48

edn -> yaml isn't super difficult if you don't care about allowing the different types of yaml formatting and special cases

borkdude20:03:55

ah, like manual

Darin Douglass20:03:04

heh ya, totally manual

Darin Douglass20:03:29

with some special munging for env configs/image tags and whatnot

borkdude20:03:06

maybe https://github.com/weavejester/comb could also be used to create some simple yaml files based on templating

borkdude20:03:31

https://github.com/tolitius/cprop/ is a lib for dealing with env variables.

borkdude20:03:38

both are available as libs, not built-in

Darin Douglass20:03:46

yep that's the one i bugged you about 😉

borkdude20:03:55

ah, sorry, forgot 🙂

Darin Douglass20:03:28

no worries! the code works by deep merging lists of edn maps which, IMO, feels much better than templates

nate20:03:53

I'm very interested in your approach here

nate20:03:06

I've been using kustomize for my kubernetes templating/merging

nate20:03:13

would love to do something with data

Darin Douglass20:03:00

awesome! i wouldn't call it revolutionary, b/c it's just doing some deep-merging and converting to yaml. here's a quick example:

# base/deployment.edn
{:api-version "apps/v1beta2"
 :kind "Deployment"
 :spec {:replicas 2
        :strategy {:rolling-update {:max-surge 0
                                    :max-unavailable 1}
                   :type "RollingUpdate"}
        :template {:metadata {}
                   :spec {:containers [{
                                        :liveness-probe {:http-get {:path "/healthcheck"
                                                                    :port 9001}
                                                         :initial-delay-seconds 30
                                                         :period-seconds 30}}]}}}}

nate20:03:11

awesome, thanks

Darin Douglass20:03:23

# app.yaml
{:from ["base/deployment"]
  :metadata {:name "some-app"}
  :spec {:replicas 1
         :selector {:match-labels {:app "some-app"}}
         :template {:metadata {:labels {:app "some-app"}}
                    :spec {:containers [{:name "app-name"
                                         :env {:some-value 1
                                               :aws {:access-key :
                                                     :secret-key :

Darin Douglass20:03:06

the env converts to [{:name "SOME_VALUE" :value 1} ...] pre-yaml so the app can bind with cprop

nate20:03:28

interesting

nate20:03:41

today's the first time I've heard of cprop, will have to check it out

nate20:03:49

I'm definitely going to pursue this soon

Darin Douglass20:03:59

: is referential, i.e. valueFrom the secret app against the access-key key

nate20:03:03

kustomize is way better than helm, but it's still super verbose

Darin Douglass20:03:21

i found kustomize's docs to be severely lacking and confusing

Darin Douglass20:03:28

so i basically ignored it

nate20:03:49

I definitely can understand that

Darin Douglass20:03:55

the problem with this approach is standard CD apps (a la flux) may not have a way to update tags automatically

Darin Douglass20:03:06

so i had to write my own jenkins pipeline (with bb ) to do it

Darin Douglass20:03:13

much like the cprop changes, once the yaml issue is resolved, it'll be nice to rm some code

borkdude20:03:00

I'm not sure which yaml library to pick. if you have anything to add there: https://github.com/borkdude/babashka/issues/263

Darin Douglass20:03:52

yeah i've been pondering it a little bit. for my use-case all i need are simple vectors and maps; nothing fancy

Darin Douglass20:03:33

hard to say if that's the general use-case though b/c the only reason i have (currently) to deal with yaml in clojure is to create kube manifests

borkdude22:03:33

so I finally found a usage for adding java.nio.file.FileSystem(s):

(require '[ :as io])
(import '[ URL HttpURLConnection])

(let [source (URL. "")
      dest (io/file "bb.zip")
      conn ^HttpURLConnection (.openConnection ^URL source)]
  (.connect conn)
  (with-open [is (.getInputStream conn)]
    (io/copy is dest)))

(set! *warn-on-reflection* true)

(let [bb (io/file "bb")
      fs (java.nio.file.FileSystems/newFileSystem (.toPath (io/file "bb.zip")) nil)
      to-extract (.getPath fs "bb" (into-array String []))]
  (java.nio.file.Files/copy to-extract (.toPath bb)
                            ^"[Ljava.nio.file.CopyOption;"
                            (into-array java.nio.file.CopyOption []))
  (.setExecutable bb true))
This script downloads a bb.zip file from Github (redirects work) and then unzips it (only the relevant file) and sets the bb file to executable.

👍 8
nate22:03:13

cool, saving this as an example in my notes

nate22:03:44

curious, what was the reason to write this script?

borkdude22:03:30

I want to get rid of depending on platform specific ways of dealing with zip files in deps.clj

borkdude22:03:22

I already replaced platform specific ways of downloading a file (shelling out to Powershell or bash) with the first part

nate22:03:33

heck of a type hint there (`^"[Ljava.nio.file.CopyOption;"`)

borkdude22:03:47

yeah, it ain't pretty... I asked in clojure if there was another way

borkdude22:03:56

bb can do without it though

borkdude22:03:08

clojure can also do without it, but in general I want to avoid reflection

borkdude23:03:37

this file is now part of a unit test: https://github.com/borkdude/babashka/blob/master/test/babashka/scripts/download_and_extract_zip.bb it downloads v0.0.78 of babashka as a zip file, extracts it to a tmp dir, runs it and then cleans up