Fork me on GitHub
#announcements
<
2021-09-28
>
borkdude10:09:04

Announcing: neil https://github.com/babashka/neil A CLI which adds common features to your #clojure deps.edn projects. It's installable with Homebrew. If this is useful to you, I'd be happy to receive more features. Neil is made with #babashka :)

clojure-spin 20
gratitude 9
🔥 8
👍 4
leiningen 3
🎉 3
Karol Wójcik10:09:33

Would it be possibile to manage configs from graal-config with neil? Or it's not a scope of this tool?

borkdude10:09:15

Perhaps, haven't thought about it yet.

borkdude10:09:42

The first thing I'd like to fix is the formatting of the deps.edn which is a bit off.

eskos12:09:47

So the name is a donk at Lein(ingen)? In that context, is the envisioned scope to gather all the new low level things such as new build tools and deps.edn and combine them into an actual build integration tool through neil or is it just a way to have shorthands for aliases?

borkdude12:09:04

@U8SFC8HLP I plan to add the ability to inject tasks into a bb.edn and then you can use bb test to run tests in your project for example. This gives you some of the UX that lein has but still with the possibility to tweak things.

👍 2
borkdude12:09:40

but I want to make this optional so this tool can be used in a non-opinionated way

eskos12:09:27

Righto 🙂 This matters to me only in the sense that I want to understand where this whole uneaseness with Lein getting disfavored is going to go in the long run… I’m not so sure on the non-opinionated part, but that’s a separate discussion. Your goals for this sounds good 🙂

borkdude12:09:53

we'll see how it evolves :)

borkdude12:09:26

I find myself adding build.clj files to deps projects because depstar is deprecated so that fulfills the goal of this tool for me right now

borkdude12:09:51

for fun we could add neil test :only foo.bar which then invokes clojure -M:test -n foo.bar or so :P

borkdude12:09:13

I'm willing to receive a PR for that :P

kwrooijen12:09:24

I'd like to see neil being able to run tasks. Being able to manage (most) thinks within an easy to understand tool would also be good for beginners

kwrooijen12:09:59

clj cli just doesn't seem friendly enough (compared to lein)

metal 1
borkdude12:09:47

like neil test :only foo.bar ?

kwrooijen12:09:55

Yeah exactly

borkdude12:09:06

yeah I agree

borkdude12:09:11

PR welcome :)

kwrooijen12:09:29

I'll see if I have the time budget 😬

borkdude12:09:45

hehe no worries, no hurry

borkdude12:09:55

just collecting ideas is already valuable

kwrooijen12:09:01

Might want to think this through a bit more

kwrooijen12:09:25

Would be great to have configurable tasks (where the CLI tool shows you all options and such)

borkdude12:09:41

This is already what bb tasks is, have you used that?

kwrooijen12:09:53

hmm no I haven't

kwrooijen12:09:56

I'll look into that

lread13:09:00

Cool! You have a real skill in naming your projects!

Johan Thorén14:09:23

Naming is actually one of the hardest things in programming.

3
Steven Deobald17:09:22

Adds calendar invite for 2031. "See who remembers why this thing is called neil."

😄 1
jumar06:09:53

@U04V15CAJ I tried to install neil with

brew install babashka/brew/neil
but it failed:
==> Tapping babashka/brew
Cloning into '/usr/local/Homebrew/Library/Taps/babashka/homebrew-brew'...
remote: Enumerating objects: 44, done.
remote: Counting objects: 100% (44/44), done.
remote: Compressing objects: 100% (30/30), done.
remote: Total 44 (delta 7), reused 43 (delta 6), pack-reused 0
Receiving objects: 100% (44/44), 5.42 KiB | 1.81 MiB/s, done.
Resolving deltas: 100% (7/7), done.
Error: Invalid formula: /usr/local/Homebrew/Library/Taps/babashka/homebrew-brew/Formula/[email protected]
No available formula with the name "[email protected]". Did you mean [email protected] or [email protected]?
In formula file: /usr/local/Homebrew/Library/Taps/babashka/homebrew-brew/Formula/[email protected]
Expected to find class NeilAT008, but only found: Neil.
Error: Invalid formula: /usr/local/Homebrew/Library/Taps/babashka/homebrew-brew/Formula/[email protected]
No available formula with the name "[email protected]". Did you mean [email protected] or [email protected]?
In formula file: /usr/local/Homebrew/Library/Taps/babashka/homebrew-brew/Formula/[email protected]
Expected to find class NeilAT009, but only found: Neil.
Error: Invalid formula: /usr/local/Homebrew/Library/Taps/babashka/homebrew-brew/Formula/neil.template.rb
No available formula with the name "neil.template".
In formula file: /usr/local/Homebrew/Library/Taps/babashka/homebrew-brew/Formula/neil.template.rb
Expected to find class NeilTemplate, but only found: Neil.
Error: Cannot tap babashka/brew: invalid syntax in tap!

jumar06:09:12

I had an ancient version of babashka so tried to upgrade it just in case but neil install is still failing.

borkdude06:09:53

Weird. Can you make an issue? Perhaps you need to update brew as well?

jumar07:09:35

brew is typically auto-updated and an explicit update didn't help. I created https://github.com/babashka/neil/issues/5

jumar07:09:48

Btw. manual installation works fine.

borkdude07:09:41

Thanks. I'll look into if I can reproduce it. Can you also mention the brew version, macos or linux version?

borkdude09:09:43

@U06BE1L6T Something else was wrong, now also fixed

borkdude10:09:00

And added another feature:

$ neil add dep borkdude/sci :latest-sha true
$ cat deps.edn
{:deps {borkdude/sci {:git/url ""
                      :git/sha "a0dd7fa2a087ed1f524fd22cb73139f571ff469d"}}
 :aliases {}}

Karol Wójcik10:09:04

Does this feature works somehow with our graal-config? 😄

Karol Wójcik10:09:15

If yes we can put the usage in the description : )

borkdude10:09:35

@UJ1339K2B Hmm, we could add neil add dep clj-easy/graal-config :deps/root taoensso.timbre

Karol Wójcik10:09:19

Oh, yes pleeease ❤️

Karol Wójcik10:09:19

But probably we have do differentiate the graal-config from the regular dependencies, since it's multi artifact deps.edn monorepo

borkdude10:09:00

we have to make a separate command for it :)

borkdude10:09:28

neil add graal-config taoensso.timbre

borkdude10:09:44

but we also support :as com.github.clj-easy/graal-config-cheshire

borkdude10:09:18

or wait no you can just use that name

borkdude10:09:40

no that won't work

borkdude10:09:48

so :as should be good

Karol Wójcik13:09:37

I have tried:

neil add dep com.github.clj-easy/graal-config :deps/root config/com.taoensso/nippy :as com.github.clj-easy/graal-config-nippy :latest-sha
But this add a dependency with {:mvn/version nil}

borkdude13:09:30

Well this option isn’t implemented yet. Also you should provide true to latest sha

borkdude14:09:37

I'll work on this now

borkdude14:09:28

@UJ1339K2B Should work now:

$ neil add dep com.github.clj-easy/graal-config :deps/root config/com.taoensso/nippy :as com.github.clj-easy/graal-config-nippy :latest-sha true
$ cat deps.edn
{:deps {com.github.clj-easy/graal-config-nippy {:git/url ""
                                                :git/sha "7a2cc5b1b1652580e26d990cf5bac2ed48247b7e"
                                                :deps/root "config/com.taoensso/nippy"}}
 :aliases {}}
in neil v0.0.11

borkdude14:09:34

oops one bug

borkdude14:09:53

the git/url isn't correct

Karol Wójcik14:09:19

But, we almost got it 😄

borkdude14:09:21

Should be fixed in v0.0.12

eskos15:09:06

This is so minor irrelevance but wouldn't :sha/latest read better than :latest-sha :)

eskos15:09:27

Dunno which part would even be responsible of that in this stack...

borkdude15:09:25

I'm not sure if all options should be namespaced keywords ;)

borkdude15:09:36

git/sha, sha/git, ...

vncz16:10:41

Question: did you think about integrating this as a Clojure tool?

borkdude16:10:20

@U015VUATAVC I have considered this, but one of the goals of neil was to see if it's feasible to package a bb script as a standalone brew/scoop/nixos package

borkdude16:10:46

I might make a tool version of this later on. But one of the benefits of having it as a bb script is that it's fast. I don't want to wait a few seconds to add something to my deps.edn

Kelvin21:09:37

At Yet Analytics, we've recently open-sourced a core product written in Clojure - lrsql (aka SQL LRS). This project uses HugSql to interface with three different SQL backends (H2, SQLite, and Postgres) to store learning data. As part of the lrsql development process, we released colossal-squuid, an open-source Clojure(script) library that generates sequential UUIDs (SQUUIDs) which follow the https://datatracker.ietf.org/doc/html/draft-peabody-dispatch-new-uuid-format. We use SQUUIDs in lrsql as primary key IDs, where their strict monotonicity makes them useful as query cursors. lrsql repo: https://github.com/yetanalytics/lrsql colossal-squuid repo: https://github.com/yetanalytics/colossal-squuid

catjam 14
🚀 10
😻 1
🎉 3
Ivar Refsdal09:09:12

Great idea and project! It will surely be useful for the company where I wok. I submitted an issue now that proves and proposes a fix for a race condition.

pithyless07:09:33

@U02FU7RMG8M are you familiar with mulog's flake [0] and if so, can you share some insight on pros/cons of choosing one library over the other? Asking as someone who has not studied the code for colossal-squuid yet, so feel free to just respond "RTFM". :) [0] https://github.com/BrunoBonacci/mulog/blob/master/mulog-core/java/com/brunobonacci/mulog/core/Flake.java

Kelvin15:09:48

Never heard of it before so I'm reading through it, and two three things stood out: • Flake.java is part of a larger library, which in turn has a bunch of dependencies. Not only is colossal-squuid one library dedicated to generating SQUUIDs, I designed it to not have any dependencies at all (Clojure(script) and test deps excepted). • The IDs Flake.java generates are not UUIDs, which may not be optimal for applications that prefer UUIDs (e.g. certain DBMSs like Postgres have special optimized UUID types). • colossal-squuid has Clojurescript/JS support, while Flake.java is for, well, Java only.

👍 1
Kelvin17:09:42

One con of colossal-squuid vs Flake.java is performance; these are times measured on Java 11.0.10 on my machine (MacOS High Sierra with an i7 processor):

(criterium.core/bench (squuid/generate-squuid))

Evaluation count : 245862600 in 60 samples of 4097710 calls.
             Execution time mean : 247.688222 ns
    Execution time std-deviation : 16.444680 ns
   Execution time lower quantile : 235.489134 ns ( 2.5%)
   Execution time upper quantile : 284.262079 ns (97.5%)
                   Overhead used : 6.380512 ns

Found 3 outliers in 60 samples (5.0000 %)
	low-severe	 2 (3.3333 %)
	low-mild	 1 (1.6667 %)
 Variance from outliers : 50.0896 % Variance is severely inflated by outliers

(criterium.core/bench (str (squuid/generate-squuid)))

Evaluation count : 214467540 in 60 samples of 3574459 calls.
             Execution time mean : 289.453769 ns
    Execution time std-deviation : 21.778366 ns
   Execution time lower quantile : 273.336325 ns ( 2.5%)
   Execution time upper quantile : 340.385667 ns (97.5%)
                   Overhead used : 6.380512 ns

Found 8 outliers in 60 samples (13.3333 %)
	low-severe	 3 (5.0000 %)
	low-mild	 5 (8.3333 %)
 Variance from outliers : 56.7680 % Variance is severely inflated by outliers
So generating a SQUUID is about 7 times slower than generating a Flake, and with string conversion it's about 3.5 4 times slower.

🙏 1
Kelvin17:09:57

For purposes of fair comparison, I rebenched Flake generation on my machine. Generating flakes w/o string conversion was similar to the listed time (35 ns), but w/ string conversion it was better (69 ns).

Ivar Refsdal06:10:31

I will chip in with a small observation: Flake provides monotonicity per thread, whereas colossal provides monotonicity per JVM. I doubt that makes a big difference in practice for an application, but I find the latter neater. I'm no performance expert, but I suppose that (synchronization guarantees) might also explain some of the performance differences. Side note: μ/log looks like a great and interesting project. Another option for squuids can be the datomic api. Downsides: closed source. No easy way to get from millis to squuid as far as I know. https://docs.datomic.com/on-prem/clojure/index.html#datomic.api/squuid

👍 1
pithyless16:10:59

Thank you all for the observations, this is exactly what I was hoping for. I was considering porting flake to CLJS, so colossal library arrives on the scene at a great time. :) (Datomic - and DataScript - both have a squuid function, but unlike flake and colossal, they make no attempt to be monotonic).

❤️ 2
pithyless16:10:39

Thanks @U02FU7RMG8M and @UGJE0MM0W for following up! gratitude