Hi gents! Can you please help me to understand how I can reload duct server when I change required library I use deps.edn together with lein

{:paths ["src"]
 :deps
 {...
  shared                         {:local/root "../shared"}}

I might be mistaken, but I think you should restart the repl

so deps.edn can rebuild classpath

there was something which supports dynamic load add-lib I think, but dunno much about it

Hmm, it’s not good idea to restart repl every time when I changed shared dependency

There's nothing different about using deps.edn vs lein in terms of reloading your app while you're working. I do it all the time with our Component-based apps: stop the system, reload any namespaces you need to, start the system again.

(although, to be honest, I almost never need to do that as I eval code into the running image all the time while I'm working, and I use Var references for a few key things around routes and middleware so I don't need to restart the app)

@y.khmelevskii in case of local dependencies, there’s no need to restart the jvm, since the dep folders are on your classpath you can just use cider-ns-reload-all or your editor’s equivalent to get the latest state

Oh, I misunderstand you If it is already on the class path - you just need reload tell, not restart

A bit of a tangent... it'd be useful to be able to switch from a git/maven dependency to a local one at runtime. That would remove my primary reason for needing to restart the REPL.

not possible

For all dep type cases or just maven?

I strongly recommend developing a tight REPL-Driven Development workflow where you eval code as you change it -- small, fast feedback loop -- rather than making a bunch of changes and then trying to restart/reload your app. Then these sorts of questions just don't even come up. I start my REPL/REBL combo up and tend to leave it running for days, sometimes even a week or more, and I might only need to restart my app once or twice a week.

@y.khmelevskii maybe this can be useful https://lambdaisland.com/blog/2018-02-09-reloading-woes

@seancorfield Not sure how that would solve this problem. The dependencies are immutable so you cannot make changes to them.

How often do you find yourself changing dependencies? (you can add new dependencies on the fly via the add-lib branch of tools.deps)

We have an 80K+ line code base at work and dozens and dozens of external lib dependencies -- and we update them in a controlled manner every few weeks or so.

Often. Currently we have ~15 internal libraries that a particular service may depend on. They are all git deps. If you find a bug or need to add something to a particular library, you need to stop your repl and then decide to either change the library dep to a local/root or push a new version of the lib.

It's not usually adding a dependency - that's rare. It's almost always updating to a newer version or switching to local/root.

Interesting. We don't run into that because we have a monorepo -- so all code is already a local dep. We can edit any of it or pull new versions down and, at worst, just reload all on the main namespace of the subproject we're working on.

(one of the main drivers for our switch from Boot to the CLI tools was that it would work better with our monorepo -- because we were having problems with Boot's pods and the shadow fileset thing it does)

Yeah I have been seriously considering switching us to a monorepo to remove this pain point. I hate having to restart the REPL to go and fix a small bug.

I've also thought about some sort of hackery that would temporarily update the .gitlibs file to a local/root file so you could dynamically switch between a git dep and a local one. Haven't written anything around that concept yet though. It feels a bit icky but so is restarting the REPL a lot.

@kenny we used to have multiple repos and switched to a monorepo - so much better

@mccraigmccraig @seancorfield How do you guys deal with ensuring tests/build steps for a particular project only run when that project changes?

@kenny we don't . we run CI across all modules in the monorepo on every commit

Ah. That seems tricky. That could easily result in extremely long build times for simple changes.

sure, a CI run is 10 mins for us, but it doesn't matter - your local dev loop is generally to run tests in one namespace from the editor and is lightning fast

10 mins would be ok but I don't think we'd be close to that. Many services need to build docker images, run integration tests, build cljs, and run hundreds of thousands of gen tests. I could see the build time easily going over an hour.

you could look at the paths changed by the last commit and only run tests in changed modules

although presumably you would also want a full run sometimes

Something like that may work. Though there doesn't appear to be any tooling around this sort of stuff yet.

I've been very close to digging in on that front because of how painful the REPL restarts are. A monorepo seems like quite a useful tool for development but it also feels like it may open a different can of worms. Blog posts around this topic would be very useful 🙂

@kenny (sorry, stepped away) I mostly run tests directly from the editor while I'm developing but we can run tests selectively for any combination of subprojects locally from the command line or a full suite end-to-end for a CI-style sweep. Again, tho', working in a tight feedback loop -- eval'ing code as you write it and either running test expressions from Rich Comment Forms or actual tests via the editor -- means that your risk of breaking anything is minimal.

@seancorfield Right - the dev story is great. I was talking about CI time for a monorepo and how a single push could easily result in very long build times.

@kenny while developing locally on 15 dependent libs, I'd definitely use local/root

Exclusively

Because of the bug fix scenario you described

You'd then need to guarantee that all the devs on your team have the same directory structure, I'd think.

No, just generate the deps file

It's data for a reason

Not sure what you mean. Also, I think that'd lose the valuable comments that we have in our deps.edn.

I mean: programmatically output the local deps file with some source file

Can comment the source file

Oh, interesting

We auto-generate an everything/deps.edn file from all the other deps.edn files (each subproject has its own) and then we generally run a REPL/REBL based on that "everything" deps file.

^^

If you use git deps locally, that conflicts with manual iteration on the dependency

A git SHA is a value, local dir is a mutable zone

So you have a deps-base.edn (or something like that) and then when you spin up a clj process, you generate a deps.edn based on the "environment" you want?

Yeah

Or generate it one time for a developer onboarding

Or whenever the source file changes

Exactly

That seems doable and would remove the need for a monorepo. This seems like very valuable info that should be a part of the deps.edn doc if it's not already.

monorepo is a big hammer

just a little read => update => spit is enough to avoid that hammer

> This seems like very valuable info that should be a part of the deps.edn doc if it's not already. I think different people have different approaches -- I don't think you can generalize this sort of thing for the base set of documentation.

True. An outline of various approaches you could use to solve these problems in the real world would be fitting.