Fork me on GitHub
#babashka
<
2020-07-29
>
onetom16:07:57

has anyone tried to manage a datomic on-prem installation with babashka? eg, generating cloudformation template, uploading it to s3 and deploy it with the desired parameters. i see there are some nice examples of how to "remote control" the aws utility: https://tech.toyokumo.co.jp/entry/aws_logs_babashka

jeroenvandijk16:07:16

@ Not specifically with babashka but with Clojure, yes

jeroenvandijk16:07:04

the aws part is still a bit painful in babashka due to lack of a proper aws library

onetom17:07:30

is there an obstacle integrating the https://github.com/cognitect-labs/aws-api ? i've used that from clojure itself too with great effect

jeroenvandijk17:07:08

But I think it’s good to try it and see how far you get. Maybe others, like me, missed something

jeroenvandijk17:07:09

And I think when I tried it with babashka there was a lot less functionality in babashka (no multimethods and many missing classes). Maybe it is different now

jeroenvandijk17:07:13

maybe the use of reify could still be a deal breaker

borkdude17:07:00

If that lib works with GraalVM then making a pod out of it could be interesting

borkdude17:07:12

assuming that the API is data-driven and no weird state is going on

borkdude17:07:51

Possible forking that lib, swapping the http client out for clj-http-lite or babashka.curl and making it more static could work

stijn07:07:09

we are using the aws-cli from babashka which is maybe a little bit more work than aws-api, but works well enough

onetom14:07:45

it would be interesting to see the aws cli utility wrapped into the same api as the cognitect client. if either of them doesn't work, you can quickly swap in the other one and see if it's any better πŸ™‚

borkdude14:07:48

a pod based on the python SDK would probably work pretty well. you pay only once the python startup (which is already milliseconds) and get to interact with probably a mature ecosystem. The only thing you would have to do is map all the commands of course, which could maybe be done using a clever trick

onetom14:07:21

thx. i haven't read up about pods yet and we are just starting to look into adopting babashka, but i will definitely remember this possiblity. for me currently the biggest obstacle of adopting babashka is that i can't get it via the nix package manager on macOS and even on linux i only have access to some 0.0.97 version or what. then we would need to have it available on circle ci too efficiently somehow, but i don't think there is any recent versions of it are included in any of the official circle ci docker images...

borkdude14:07:13

I don't know anything about nix, but running babashka in CI is as easy as curl the binary and run it.

onetom14:07:23

without having it in some official image, it would mean that it's not cached on the circle ci build machines, hence downloading it on every build would be noticably slower

borkdude14:07:43

the binary is pretty small, 20mb zipped or so

borkdude14:07:00

there's also a babashka docker image

borkdude14:07:38

you can also add caching for the downloaded binary. I do this with lots of things I install during builds on CircleCI

onetom14:07:45

sure, but that cache is a different cache than the docker layer cache. we are using that cache of course, for node_modules & .m2, but it's not exactly the fastest... and u r right, 20MB is not that terrible; for some reason i was calculating with the 67MB unzipped size.

onetom14:07:44

the point of using nix though is to pin down all dependencies of a project with laser precision, so it can be extremely agnostic to the environment it runs on.

borkdude14:07:21

the binary is self-contained

onetom14:07:49

here is a real-world example:

# To upgrade pinned versions, get the latest git commit SHAs:
#     git ls-remote  nixos-20.03 nixpkgs-unstable

with import (builtins.fetchGit {
    name = "nixos-20.03";
    ref = "refs/heads/nixos-20.03";
    rev = "bb8f0cc2279934cc2274afb6d0941de30b6187ae";
    url = ;
}) {};

let
    # project-jdk = adoptopenjdk-hotspot-bin-11;
    project-jdk = jdk11;
    clojure = callPackage ../deps/clojure { jdk11 = project-jdk; };
    proxychains = callPackage ../deps/proxychains {};
in

mkShell rec {
    buildInputs = [
    	coreutils cacert miller devd modd direnv proxychains
    	project-jdk clojure awscli mysql-client mysql
    ];

    shellHook = ''
        export PATH=$PATH:$PWD/bin
        export LC_CTYPE="UTF-8"
    '';
}

borkdude14:07:39

It's up to the maintainers of the nix packages to bring them up to date. I would like to help, but they haven't been asking questions

onetom14:07:13

this produces the very same sw environment on a linux or on a macos and developers don't need to be concerned with the OS this project is being deployed to or the CI servers config. all they need from the target environment is having the nix package manager installed and a /usr/bin/env; even bash is not expected to be under /bin/bash, because the version of bash is also controlled by nix

onetom14:07:32

what i was thinking, given the neck-breaking pace of babashka development, is that maybe we should setup a babashka channel overlay, similarly to the brew tap and let nix users get the latest binaries from there.

onetom14:07:08

unfortunately i don't have that much nix experience yet to pull this off myself alone, but if i would make it happen, would you include it into the installation instructions?

borkdude14:07:21

fwiw I also publish the uberjars. for clj-kondo there was one person who makes nix builds based on that

borkdude14:07:51

you could try to search the issues in clj-kondo for nix, and read those discussions

borkdude14:07:05

I do like the principles behind nix

onetom14:07:17

thx, i will investigate it then. im using nix for about ~3years now, without learning it deeply and it has already helped me immensely to carry around development environments to colleagues machines, without relying on them following readmes, which are typically not updated, because it would be a pain to update them for multiple platforms...

onetom14:07:02

i can recommend nix highly enough! on my thinkpad for example i just installed pop!_os, for the sake of a convenient desktop environment out of the box with good driver support, tiling window manager, etc then i just installed the nix package manager on it and all my projects work the same as they do on my macs. just git clone, cd, nix-shell, wait a bit for the downloads, run the project. i don't even run nix-shell manually, i let direnv take care of that every time i do a cd.

onetom14:07:22

no virtualization overhead and no filesystem and network isolation headaches, like in case of docker. pure joy πŸ™‚

borkdude14:07:35

yeah, very cool

onetom14:07:10

i don't know who else is a nix expert in the clojure community here, but in any case, if u happen to experiment with nix and have any issues, questions, don't hesitate to DM me! im happy to jump on a screen.so screen sharing session and help out.

borkdude14:07:46

Maybe send that to the channel instead of only in this thread

onetom14:07:57

that would be a bit out of context though, no? im offering this to u primarily, because helping you getting into nix would have the biggest ripple effect for the whole babashka community.

borkdude14:07:25

ah, I thought you meant "u" in the general sense πŸ™‚

borkdude14:07:43

I don't have any plans to put time in this, at least not soon. Already enough to do.