Fork me on GitHub
#tools-deps
<
2019-08-10
>
seancorfield00:08:07

I should probably see if I can synthesize a generic version of our build script and release it as open source...

kenny00:08:53

That would be super awesome 🙂 I just realized mine doesn't support aliases very nicely...

seancorfield00:08:55

We have a whole bunch of WS-specific alias expansions in it right now. build run thing will expand into whatever thing means in our project. So build run database-setup expands to build [ migrate migration photos dev ] which runs the :migrate alias in the migration subproject and then runs the :photos alias in the dev subproject.

seancorfield00:08:08

We also have an expand-deps.sh script that looks for :local/root deps in a subproject's deps.edn file and recursively expands the list of subprojects to include all transitive subprojects that are depended on -- so we can say build test-all api and it will expand to build tests environment datamapper worldsingles ... api and that in turn expands to running build test <subproject> for each subproject in turn 🙂

seancorfield00:08:17

prev=""
curr="[email protected]"
while test "$prev" != "$curr"
do
  deps="$curr"
  for d in $curr
  do
    if test -f "$clojure/$d/deps.edn"
    then
      roots=`fgrep local/root "$clojure/$d/deps.edn" |
        sed 's;.*local/root "../\(.*\)".*;\1;'`
      deps="$deps $roots"
    fi
  done
  prev="$curr"
  curr=`( for d in $deps; do echo $d; done ) | sort -u`
done
echo $curr
where $clojure is the monorepo root.

seancorfield00:08:21

All told, including the WS-specific task alias expansion and the 70 lines of help output, we're talking 350 lines of bash.

kenny00:08:44

Hmm, that's a lot of bash haha. I just about hit my tolerance with the wee 16 lines above 😛

seancorfield00:08:52

I've created some truly monstrous bash-based systems in my career... 1,000's of lines long. I probably wrote my first *nix shell script back in '79 tho'... neckbeard

kenny00:08:54

Writing a small script to do the necessary parsing in plain ol' Clojure only adds 1s overhead 🙂

seancorfield00:08:30

True, I could rewrite all this in Clojure (and shell out to clojure as needed for the repeated invocations). I could make all the WS-specific stuff data driven and external too. Maybe one day :thinking_face:

✔️ 1
alexmiller00:08:01

@kenny I made some progress on your perf issue today

alexmiller00:08:50

There’s some repository cache stuff built into Maven that we weren’t using so I’ve changed that and it makes a big difference.

alexmiller01:08:01

There’s still something I don’t understand about why you’re getting that s3 maven download every time, but it’s now only doing it once at least.

kenny01:08:27

Awesome! Is there a way for me to test it out before a release?

seancorfield01:08:07

@kenny Looking at the clojure script, I think the answer is "not easily".

seancorfield01:08:05

You'd have to build tools.deps JAR yourself and overwrite the version in the clojure install or hack the clojure script to point to that JAR elsewhere

kenny01:08:44

That doesn’t sound that bad. Perhaps writing bash makes tasks like that sound easy haha

alexmiller01:08:50

You don’t even need to build it

alexmiller01:08:03

Just clone the repo

alexmiller01:08:05

Then modify your clojure script to put the tools.deps.alpha/src/main/clojure dir at the front of the tools_cp var

alexmiller01:08:27

Or you can just wait a day or two for me to test and release it :)

dominicm06:08:32

There's one downside to not building it since the removal of the system deps.edn, you end up with templated version strings! So you need a hard clojure dependency.

alexmiller12:08:29

Ah, true. Well you can build and use the target output, that should work.