Fork me on GitHub
#tools-deps
<
2020-03-06
>
devn03:03:01

Is there a good guide or rationale on http://tools.deps vs other options? Several beginners have expressed confusion about lein v http://tools.deps, and lamented the fact indirectly that they’ve now needed to learn both tools while leveling up, but that almost all of their professional work has involved lein.

devn03:03:53

I’m actually confused myself as a long time clojure user. Why did we need another half-finished build tool again? I think I personally understand some of the why, but there’s no doubt it’s contributed confusion to would-be clojure users.

devn03:03:15

There are numerous answers here that will be acceptable to me, a long-time clojure user, but the fragmentation of beginner tutorials sucks.

devn03:03:30

Sorry to be a downer. This ship has sailed, but I have to let y’all know that new users are frequently wondering why they have to learn two build tools.

andy.fingerhut03:03:58

One answer is that tools.deps is not a build tool. Some more discussion comparing Leiningen vs. tools.deps capabilities here: https://clojureverse.org/t/is-there-a-sales-pitch-for-switching-to-deps-edn-from-lein-in-2020/5367

devn03:03:42

So, that’s kind of pointing directly at the problem.

andy.fingerhut03:03:04

I can point you at resources that you may or may not find useful to point beginners at. I've got no horse in this race.

Bobbi Towers03:03:13

I think David Nolan made a pretty good case here https://www.youtube.com/watch?v=F0Lv53lop2g

devn03:03:32

I would appreciate any handy resources, and I’ll cop to soapboxing a bit here because it pains me to see beginners who are serious learners being legitimately confused by the ecosystem.

devn03:03:54

I remember when I started, and people were using ivy and gradle and maven. Lein came into fashion, then cake was absorbed into lein.

seancorfield03:03:43

> Why did we need another half-finished build tool again? CLI/`deps.edn` is very specifically not a build tool.

devn03:03:03

you can explain this all day to new users

seancorfield03:03:32

New users do not need a build tool. They need a REPL. They need to run programs.

devn03:03:43

that’s nonsense imho

devn03:03:32

like you can be right, but it doesn’t help new user confusion

seancorfield03:03:20

The "problem" is that most books and lots of online tutorials that have been around for ages all show Leiningen as part of the first steps in using Clojure. We're in a transition period right now because Leiningen has been around for years and new tooling is appearing. This happens any time a tech community has new tooling come up to replace older, existing tooling.

devn03:03:50

stop right there

devn03:03:56

You said it’s not a build tool

devn03:03:02

What transition?

seancorfield03:03:03

There was a time when Boot was gaining ground and folks complained about that too.

seancorfield03:03:36

Leiningen is a "sledgehammer": it starts a REPL, it runs programs, it bakes, it broils, it takes out the trash.

seancorfield03:03:18

I see beginners tripping over problems with Leiningen all the time because of plugins they've added to ~/.lein/profiles.clj because it's "easy" rather than "simple".

💯 4
wizard 4
👍 4
4
devn03:03:21

I expect to be on the losing end of this, but I have to really underscore the confusion for new users. I feel it’s my duty. I know that may sound hyperbolic, but this is coming from a place of love.

devn03:03:48

New users do not understand this subtle distinction you’re making.

Bobbi Towers03:03:56

Not needing Leiningen to get started is a huge plus for new users

seancorfield03:03:58

A lot of us here are on the front lines in #beginners all the time, helping beginners.

devn03:03:36

yes, and I’ve heard directly from some of them who wonder why anyone is bringing up http://tools.deps, because all of their work is in lein world

devn03:03:42

That’s exactly the confusion.

seancorfield03:03:27

The official Clojure Getting Started docs lead people to the CLI and deps.edn.

devn03:03:00

yes, why? The vast majority of existing production clojure projects don’t use it.

seancorfield03:03:07

The latest edition of Programming Clojure uses the CLI and deps.edn I believe (right @alexmiller?) and the ClojureScript Getting Started guide uses the CLI and deps.edn.

seancorfield03:03:20

Tech changes over time. There's nothing strange about that.

seancorfield03:03:53

Look at all the changes in the Java world over the years... beginners now do stuff completely differently than beginners five, ten, fifteen years ago...

devn03:03:03

I’ve said my piece. I don’t think it’s helpful. I think it’s confusing for new users.

seancorfield03:03:23

Would you prefer tooling never changed and never improved?

devn03:03:38

that’s hyperbolic

devn03:03:54

I’m telling you what I know here. Anecdotal as it may be...

seancorfield04:03:32

I have anecdotes too -- that many beginners find CLI and deps.edn to be much simpler to learn and use.

metal 8
devn04:03:16

Ok, what is depstar, and why does it exist?

seancorfield04:03:42

Ask the folks who wrote it? 🙂

devn04:03:55

You maintain a fork of it

devn04:03:33

I’m not trump to gotcha anybody

devn04:03:50

But that project alone should act as sufficient evidence of the confusion

seancorfield04:03:50

Yes, I found the initial version useful enough that I wanted to fix a few bugs in it, and folks seem to like my fork of it.

devn04:03:03

It uberjars.

devn04:03:12

It’s. A. Build. Tool.

seancorfield04:03:56

It's not ant, not maven, nor gradle etc -- those are build tools.

seancorfield04:03:16

Lein and Boot are build tools.

devn04:03:51

again, this isn’t for me

devn04:03:58

I know what you’re trying to say

devn04:03:07

But for beginners this is nonsense

devn04:03:17

I’ll leave it there

Alex Miller (Clojure team)04:03:13

I'm confused what you would find a satisfactory answer to these queries?

seancorfield04:03:30

@alexmiller Programming Clojure uses the CLI and deps.edn in the latest edition, right?

seancorfield04:03:13

And it doesn't mention "uberjar" anywhere in the book... and only has a couple of mentions of "jar" (I just looked).

seancorfield04:03:09

So a beginner can read that book end-to-end and not worry about build tools...

seancorfield04:03:27

(that was what I suspected but wanted to confirm by searching the PDF)

pyr07:03:07

You could look at it from the angle that there's value in having a lower level toolbox for builds. There is tremendous value as well in having a build tool with conventions (a-la lein). We can hope for a new tool or an evolution of lein, bringing back the baked in conventions which relies on tools-deps

sogaiu07:03:10

check out the latest insideclojure?

vlaaad08:03:56

“tools.build - shh, top secret”? 🙂

👍 12
vlaaad08:03:42

A bit late to the flamewar of tools-deps vs build tools, I wanted to repeat once again how much I love clj compared to lein. It all boils down to how you run java programs. To answer the question “How do you run java programs?” one needs to have a look at java command: what arguments it accepts, what are actual, real semantics of running java programs. Actual, real semantics of running java are: • jvm options like maximum memory (`-Xmx512`) or system properties (`-Dclojure.spec.skip-macros=true`) • classpath: sequence of directories to look up resources (both .class files and any other type of file that might be needed for running programs) — either as a directories on file system or as archived directories (aka jar files) • main class to run • arguments to main class Everything else is non-essential complexity that is imposed by a build tool. Some of this added complexity has a lot of value: specifying dependencies to automatically build classpath, for example. Some of it does not. For example, lets look at source path and resource path separation in leiningen. It’s accidental: you can put your source code in resource files and it will work. What good does it give you besides obscuring actual, real semantics of running java program? Lein plugins, for example, are just wrong and unnecessary abstraction: you could create a separate profile that has a dependency on a tool of your choice and run it. Automatically picked up leiningen profiles for me are still hard to understand: clj’s aliases are much more intuitive and predictable. clj command line options map much better to java command line options. Case in point: yesterday I wanted to run a lein command with system property provided from command line. I could not do it: you have to specify jvm options in :jvm-opts in project.clj, there is no way (at least that I know of) to specify them for lein run invocation. Beginners coming from java land with equally complected tooling might have a hard time to adapt to tools-deps, but that’s because that tooling equally obscures the actual, real semantics of running java programs. Break the cycle. Rise above. Focus on simplicity.

👍 8
dominicm08:03:10

https://clojurians.slack.com/archives/C6QH853H8/p1583465670122100 This message was the one I think is quite interesting. The answer is quite easy "there are 2 major competing entrypoints into clojure". I'm sure anybody can understand that. Why is there multiple battery brands that exist, etc. Because they each believe they offer something better than the other. It's true that there's a cost to having 2 things that both provide the entrypoint. Enough people seem to think that cost pales in comparison to the benefits provided by tdeps. But those benefits are not easily rendered to someone who isn't versed enough in clojure to understand the bigger picture of elements playing out.

teodorlu09:03:52

Hey @devn, I think you're bringing some good points. I read your comments above, and I'd like to respond. I hope to be able to bring some civility. • Personal opinion: getting started with Clojure isn't a perfect experience for new users. That you might encounter project.clj and deps.edn is evidence of that -- learning a single tool would be easier. • Personal judgement: A common Clojure opinion is that "build tool" is a too broad category, in the sense that Gradle, Maven or NPM works. deps.edn is part of that solution. The "full solution" isn't known yet. deps.edn implementing part of that solution opens the space for other actors. • Personal experience: A simple / pure data deps.edn is great to work with. I can download the dependencies without having to buy into how you'd like to build your project. Too much complex stuff in the project build pipeline can cause the project build pipeline to become opaque / magic. Some questions: 1. Do you feel like the above points bring any value to the discussion you'd like? 2. Do you have a suggestion on what work should be prioritized in order to improve new user experience? Thanks! Teodor

devn13:03:49

Oof, hearing “flame war” and “bring some civility” makes me feel pretty terrible. I talked this out with Alex for awhile last night, and Teodor, I think the second question is one I need to pose to those that have expressed their frustration to me.

devn15:03:44

I sincerely apologize if I came off like a jerk. I was out of sorts last night and did a poor job of broaching the topic and keeping it constructive. facepalm

❤️ 12
teodorlu11:03:27

Yet, I think digging into frustration can be fruitful for improving the state! I appreciate that you're spending the effort 🙂

seancorfield15:03:30

It's definitely unfortunate that Clojure didn't have something like the CLI and deps.edn from "day one" and we ended up with lein as the default : it is easy but not simple in the Clojure sense. At work we started with Leiningen because there really was no choice back in 2011. We switched to Boot in 2015, and the CLI and deps.edn at the end of 2018 I think. It's a much simpler setup, even managing a 90k line monorepo with thirty subprojects. I'm very glad that our tooling continues to evolve. Yes, it means that beginners who start from most books and tutorials today see Leiningen there but then see CLI and deps.edn in new books, new tutorials and on the official site. That's why I consider this a transition : at some point Leiningen will be the minority tooling I suspect or will at least be seen as "the old way" and the question of "which should I use?" will go away, or at least have an obvious answer.

seancorfield15:03:21

The transition from monolithic Contrib to new Contrib was similarly confusing to beginners (but the community was much smaller then so the pain was less).

seancorfield15:03:34

The introduction of Spec caused similar confusion I think (should I use Schema? Spec? Typed Clojure?).

seancorfield15:03:59

And it's not like we don't have a bunch of options in other areas : which JSON library? Which routing library? Which web server? Learning Clojure is littered with such choices

devn18:03:04

I think I may just fundamentally disagree that simple > easy for new users. I agree there are a lot of choices and that this is just the nature of the beast, but for instance, getting a basic project structure with a working test runner is a whole thing, and IMO it needn’t be this way. test-runner and clj-new exist, but they both are extra steps to simply having a basic working project structure, and then people hit the “ok how do I build an uberjar” step and now they have 3 more options to choose from that I’m aware of. I hadn’t heard about tools.build and am interested in learning more if anyone has a link handy. lein has its warts certainly. At a certain point you definitely begin fighting with the tool. For instance, checkouts, tasks, the craziness of cljsbuild’s 9000 options, trampoline, passing java opts, and many more, but for new users you could type lein new to get a common project structure, add a dependency, write a test, type lein test, and it just worked.

devn19:03:06

I know this isn’t what http://tools.deps wants to be, but I think introducing it to new users on getting started is a rougher-than-necessary entry point without a bit of ease sprinkled in.

Alex Miller (Clojure team)19:03:25

I think there are multiple phases. clj is easy, probably easier than lein imo, if you want to just use a repl and write some clojure and use some deps, etc. Beyond that is "make a simple lib/app". Lein is easier in that zone (for now). And then beyond that you get to more bespoke builds, and actually I would again say clj is easier to fully customize because you are not constrained to whatever flags and switches and lifecycles lein makes available.

👍 12
Alex Miller (Clojure team)19:03:12

more to come in the near future for the middle zone

Alex Miller (Clojure team)19:03:44

in the interim, something like sean's template (and others) can give you a slightly opinionated set of choices for making a new project based on clj. It's harder, but it's not like 10x harder, it's like 1.5x harder.

dominicm19:03:57

You can make easy from simple, can't make simple from easy.

👍 4
👆 4
Alex Miller (Clojure team)19:03:33

more importantly, simple and easy are different dimensions not a vs b.

👍 4
Alex Miller (Clojure team)19:03:25

so really, you can go simple+hard -> simple+easy, but you can't go complex+easy -> simple+easy.

👍 4
devn19:03:02

Underneath some of what I said last night, there is some personal fatigue from conversations I’ve had where people seem to use simple v easy as a way of downplaying aesthetic or ergonomic concerns by tossing those two words out without addressing the human side. New users should have it easy, full stop. To the detriment of all other things? Of course not. It’s not an either/or. For example, I have a really difficult time imagining how having clj new and clj test would destroy the aims of the project.

devn19:03:48

I’m not saying this is what should be done necessarily, but there’s no point in being pure to the detriment of new users.

Alex Miller (Clojure team)19:03:28

I certainly agree that simple does not mean hard

Alex Miller (Clojure team)19:03:56

clj new and clj test seem like making clj into something that it's not though (a build tool). it's a tool for running programs. template gen and test runners are programs.

devn20:03:36

totally agree

Alex Miller (Clojure team)20:03:45

having ways to discover useful programs to run or templatize/kickstart how to do so seem like good things

👌 4
devn20:03:25

but like, should I just make a thing called dmk that does all of the clj stuff and adds two commands for new users?

devn20:03:37

that seems bizarre to me?

Alex Miller (Clojure team)20:03:08

well I wouldn't do that :)

devn20:03:37

haha, I’m asking then honestly, where would something like that go? It’s never in the cards?

💯 4
Alex Miller (Clojure team)20:03:03

I don't have a pat answer to that - finding one is a design process

Alex Miller (Clojure team)20:03:29

these things literally exist now

Alex Miller (Clojure team)20:03:51

as programs that you can run now with clj

Alex Miller (Clojure team)20:03:10

and as templates that you can create/install

Alex Miller (Clojure team)20:03:21

is the problem that we have not done a good job explaining that or that that is not easy enough. and if the latter, what would make it so?

Alex Miller (Clojure team)20:03:41

like if this page included those examples https://clojure.org/guides/deps_and_cli would that be sufficient?

Alex Miller (Clojure team)20:03:12

if you added a bit of stuff to your ~/.clojure/deps.edn, then it's literally clj -A:test or clj -A:new etc which seems pretty close to what you have above

devn20:03:33

It kind of reminds me of clj-rewrite and the evolution of “just update my deps” or “just add the damn test runner”

mike_ananev20:03:55

@devn try this https://github.com/redstarssystems/pbuilder here you can find templates, easy commands and more

mike_ananev20:03:50

in templates I use make utility, so it is very easy to perform different tasks

seancorfield20:03:58

If someone does add the :new alias to create a new app or lib project, the generated project includes the :test, :jar/`:uberjar`, :install/`:deploy` aliases so it all becomes as "easy" as lein for basic usage. So the missing piece is really only that initial project creation piece I think.

seancorfield20:03:21

Strangely, when we switched from lein to boot at work, "new" wasn't a thing for Boot either and I ended up forking lein-new (which wasn't a thing for Leiningen in the very early days, by the way!) into boot-new and it was quite some time before that was integrated into the core tool.

devn20:03:25

Heh, I mean, here we are. http://tools.deps is not a build tool but there are 3 choices for making an uberjar. It’s too simple to add any porcelain commands so there are at least three things that add that type of functionality. That’s my gripe I guess.

seancorfield20:03:44

So there's a history of tooling not including an "easy" way to start a new project from scratch 🙂

devn20:03:15

Oh yeah! I actually worked on the boot lein template thing I think...

seancorfield20:03:17

> 3 choices for making an uberjar Oh, c'mon! There are far more than just three choices! :rolling_on_the_floor_laughing:

devn20:03:18

Long time ago

seancorfield20:03:49

I think far more beginners are stumped by the "how do I build a simple web app" question -- given the plethora of choices they get offered at that point. And heaven help them if someone says lein new luminus myapp and then they have to figure out what that produces!

8