Fork me on GitHub
#babashka-sci-dev
<
2022-05-16
>
borkdude17:05:29

@cldwalker Do you think we could disable the build test on nbb? It's taking an extra minute of build time but if the build doesn't change much, maybe we can do it more cleverly. I liked that the previous build was pretty fast https://app.circleci.com/pipelines/github/babashka/nbb/673/workflows/bb4a8488-7fe4-4597-9c36-462d58fa7dae/jobs/604

cldwalker17:05:52

Sure. I'll be updating nbb-logseq enough that I should be able to report any custom build issues pretty quick. If I ever make build/ changes, I could also see if there's a clever way to run these tests

cldwalker18:05:04

Also, thanks for the nbb release and cljs upgrade! Bumping to a release makes it easy for me to bump to the newest nbb

borkdude18:05:04

cool. we could also detect if some build code has changed or not, for example

cldwalker18:05:50

Right. I'm ok with disabling it for now if you'd like to speed up builds

borkdude18:05:42

ok, done

👍 1
lread20:05:13

Ah just the monorepo task listing/invoking part yeah? I was also glancing at the default task idea (which maybe feels like a separate issue? Dunno).

borkdude20:05:47

Yes, that's a separate issue

lread20:05:55

Is this just about task discovery and invoking for monorepos? Nothing more than that?

lread20:05:48

How deep would bb dig to find bb.edn files? Would it search all subdirs?

borkdude21:05:07

That's a good question. Something like this, takes 25 ms in the bb repo:

$ bb -e '(time (map str (fs/glob "." "**/bb.edn")))'
"Elapsed time: 24.973688 msecs"
("sci/bb.edn" "fs/bb.edn" "process/bb.edn")

lread21:05:35

Curious because I’ve not worked on a monorepo but do have one project with a https://github.com/cljdoc/cljdoc-analyzer/tree/master/modules/metagetta. I wonder if bb would report on this one too when I did a bb tasks

lread21:05:26

Perhaps some options around subdir searching in a root bb.edn ?

borkdude21:05:37

Yes, that would be the idea, to report it as

modules/metagetta/test run all tests

borkdude21:05:28

I would say:

bb tasks --subdirs --depth 5
or whatever

borkdude21:05:30

and not do it by default

borkdude21:05:04

The idea would be that you can always stay in the root dir of the mono repo and execute all tasks from there

lread21:05:53

Right… and if you cd downward bb would look from cur dir downward in your monorepo, I would assume.

lread21:05:02

Seems like a good idea. Especially the discovery part. What tasks does my monorepo have?

lread21:05:40

Would it also be interesting to ask in #polylith?

lread21:05:15

Or do they already have their own solution for this type of thing?

borkdude21:05:58

Not sure. I have a similar use case in clerk. I have a directory ui_tests: https://github.com/nextjournal/clerk/tree/main/ui_tests So from the root I could execute my ui_tests using:

bb ui_tests/test
but right now I'm doing that with: https://github.com/nextjournal/clerk/blob/9a04cf38fd4d7c771071ddc14538da673cd32f4f/bb.edn#L50

borkdude21:05:58

I think people will be confused when they try to re-use tasks from a subdirectory in the upper bb.edn, which imo isn't possible due to the "current working directory" problem

borkdude21:05:54

So bb ui_tests/test should work but not (run 'ui_tests/test)

borkdude21:05:32

unless that would invoke another bb process as well

borkdude21:05:05

yeah, I'm afraid I would introduce something that is easy on the command line but leads to confusion

lread21:05:08

Would dependencies still be scoped to a single bb.edn?

borkdude21:05:34

there would be no merging of tasks and deps going on, just a way to list them and invoke them

lread21:05:36

Yeah, good point, because I can do this, why can’t I do that?

lread21:05:22

I suppose just reporting all tasks and not supporting the running part could still be useful?

borkdude21:05:54

What if (run 'ui_tests/test:foobar) was synonymous to (shell {:dir "ui_tests"} "bb run test:foobar")

lread21:05:36

Yeah that would work.

lispyclouds22:05:34

> What if (run 'ui_tests/test:foobar) was synonymous to `(shell {:dir "ui_tests"} "bb run test:foobar") > ` This is pretty much what I do in my monorepos. This is the https://github.com/bob-cd/bob where i use it. Just that I have long names without a - delimiter. Would / as a delimiter be allowed as a normal task name and later on there's a folder with the same name and this has a different behavior? / is quite idiomatic namespacing in clj?

lispyclouds22:05:40

My personal preference would be to denote the subdir explicitly rather than a naming convention

lispyclouds22:05:32

Like bb subtask-run ui_tests/test:foobar or something. Similarly (subtask-run 'ui_tests/test:foobar)

lispyclouds22:05:03

If thats the case, I would expect the paths in my sub bb.edn to be relative to it. If its implemented as forking another bb or not could be abstracted?

borkdude06:05:15

Task names with slashes are normally not allowed so using a slash would already indicate a subtask

borkdude06:05:12

I do think we'd maybe have to use strings if those symbols contain more than one slash as the reader does not accept those

lispyclouds06:05:45

how about a new key in the bb.edn called :subprojects or something? That is a vector of strings showing the dirs where bb.edns are there and can be used when listing and/or running tasks.

borkdude06:05:21

Makes sense yes, I like that

lispyclouds06:05:06

I had many a nightmares when using gradle where the nesting is delimited by a : and a bit wary of name based behaviours

lispyclouds06:05:49

unless theres good editor nav support, sometimes quite hard to figure out where this task is

borkdude06:05:52

We could also just start with only supporting the command line option and for running within bb you'd still have to use shell, maybe that's less confusing

borkdude06:05:25

But (shell bb foo/task) would then set foo as the dir since that is the command line behavior

lispyclouds06:05:34

yeah more i think about it, the decoupling the name from behaviour seems like a nice way to future/confusion proof?

borkdude06:05:20

What do you mean with decoupling name from behavior?

lispyclouds06:05:01

so having a / in the name could have some sideeffects when shelling or not and some other delimiter like : would make it less semantic i think. so we deciding whether a task is a sub task or not based on its name is the coupling bit im thinking

lispyclouds06:05:18

with the :subprojects key thing, the names are not special

borkdude06:05:12

We could also give subprojects an alias even and those aliases can be used before the slash

borkdude06:05:38

In subprojects

borkdude06:05:18

So then it would not be coupled with the dir structure

borkdude06:05:30

Like a group name. Normal tasks could also get an optional group name

lispyclouds06:05:35

what is an example of the alias? cant really imagine it

borkdude06:05:35

Usually the name of the subproject. bb foo/test

borkdude06:05:44

Foo would be the alias here for sub dir foobarfoouglylongname

lispyclouds06:05:52

and if we have 2 or more levels of subdir nesting that should work here too i guess? im imagining a key called :aliases which is a map of keyword -> str? val is the potentially nested path.

lispyclouds06:05:38

parallel thought: also maybe we could limit it to one level nesting in the paths. the nested bb.edn could have a sub nested task which we can recursively discover. that way i think we can be in the reader's rules?

borkdude07:05:54

that would require you to add a bb.edn in each subdirectory which may not be what you want

lispyclouds07:05:15

well not each one but you can put where its needed, denoting a sub project and the recursive walk would ignore dirs with no bb.edn

lispyclouds07:05:02

ignore that dir and its children

borkdude07:05:58

sounds complicated

lispyclouds07:05:48

yeah for me too, its just another thought i had when i read about the reader limitation 😄

lispyclouds05:05:59

Similar to the :subprojects idea I was mentioning and more polished and established approach taken by Rust's Cargo: https://doc.rust-lang.org/book/ch14-03-cargo-workspaces.html

borkdude08:05:09

Can you tell me how this would apply to babashka? Preferably here: https://github.com/babashka/babashka/discussions/1044 As much detail as possible, better than handwaving :)