Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2022-01-05

Channels

#tools-deps
<
2022-01-05
>
kennytilton00:01:04

I started from https://github.com/lilactown/helix-todo-mvc, @hiredman. That manages to put up a web page supported by deps.edn and shadow-cljs.edn via npm i and npm start. It also works now that I have ripped out Helix and substituted Matrix. npm start seems to lead to npx shadow-cljs watch app via package.json: 

{
  "private": true,
  "scripts": {
    "start": "npx shadow-cljs watch app"
  },
  "dependencies": {
    "react": "^16.13.1",
    "react-dom": "^16.13.1",
    "react-refresh": "^0.8.1",
    "react-router-dom": "^5.1.2",
    "react-slider": "^1.3.1",
    "shadow-cljs": "^2.8.94",
    "todomvc-app-css": "^2.4.1",
    "todomvc-common": "^1.0.5"
  }
}
I wonder if I should have add the Matrix dependency there? ^^^ Though I should think shadow would find its way to deps.edn thanks to the :deps true in shadow-cljs.edn. I will see what the shadow crew thinks. Thx! 🙏

seancorfield00:01:59

@hiskennyness package.json is going to be just NPM dependencies, not Clojars/Maven. deps.edn is going to be just Clojars/Maven (unless addition repos are declared in the deps.edn file).

seancorfield00:01:26

(as for what, if anything, Shadow-cljs does with the deps.edn file, you'll have to ask in #shadow-cljs)

slimslenderslacks00:01:11

fwiw, I think npx shadow-cljs watch app will start up a jvm pretty much like what you'd get with clj -M:app and an alias of 

{:aliases 
  {:app
    {:extra-deps {org.clojure/clojure {:mvn/version "1.10.3"},
                  org.clojure/clojurescript {:mvn/version "1.10.866"},
                  thheller/shadow-cljs {:mvn/version "2.14.2"}},
     :main-opts ["-m" "shadow.cljs.devtools.cli" "watch" "app"]}}

👍 1
thheller06:01:04

that is indeed very similar with the exception of server mode as described here https://code.thheller.com/blog/shadow-cljs/2017/11/18/the-many-ways-to-use-shadow-cljs.html. clj will always start a new JVM and never take advantage of an already running instance.

thheller06:01:24

note that nowadays the shadow-cljs command just uses a precompiled AOT dependency and does not AOT on demand anymore. you could get the same with clj using thheller/shadow-cljs$aot I believe

👍 1
scarytom11:01:17

I have a question about packaging libraries, and I hope this is the right channel to ask it in. Say I am building a Clojure library that happens to have some Java bits (e.g. some interfaces or some AOT compiled classes). My build process will do the compilation steps and create a Jar containing the Java classes and Clojure clj files. I will then push that to Clojars or Maven Central, and other people can depend on my library. If my lib depends on other libraries, expressed as :mvn/version deps in my deps.edn file then, when I build my library's jar, I create a pom file that expresses these dependencies and downstream users of my lib get these "transitive" deps pulled in automatically. Now what if my library has a dependency on something that is only packaged as a git ref, described in my deps.edn file as, for example {io.github.clojure/tools.build {:git/tag "v0.7.4" :git/sha "ac442da"}. How does this get expressed in my pom file? How do downstream users of my lib get that dependency automatically?

scarytom11:01:19

I should add that the downstream consumers of my lib might be Java codebases using maven not Clojure codebases using tools-deps

rickmoynihan11:01:21

Essentially you can’t. If you’re publishing artifacts to maven all transitive dependencies MUST also be published in maven. However tools.build is a build tool; so downstream users typically won’t need that as a dependency… i.e. that’s normally something only you need to build/deploy your project. Consumers won’t typically need it; unless you were also distributing build tools.

rickmoynihan11:01:26

There are some workarounds, e.g. you can package the git/deps into your libraries jar… However you need to be very careful if you do that, as it could cause classpath hell for consumers if someone else were to also do the same, as it would be hidden from tools.deps classpath resolution, and it would be hard for consumers to control which version they got.

scarytom12:01:19

tools.build was just an example dep, sorry if that was distracting.

scarytom12:01:52

that workaround sounds undesirable to me

scarytom12:01:03

I guess the other alternative is to package the git/dep into a jar myself and publish to my own maven repo under the same group/artifact, so that resolution conflicts between that artifact and other direct use of the git/dep are highlighted.

scarytom12:01:19

but that doesn't work if I want to make my lib public

scarytom12:01:53

I think this is a fairly horrible consequence of using git/deps. If more people choose to package their clojure libs in this way it will make it very hard to create java-friendly libs downstream.

rickmoynihan12:01:12

Yes; the best thing to do is either: 1. contribute a maven deployment to the upstream project 2. fork it and release under a coordinate you control; but also repackage the namespaces into a new top-level ns to avoid collisions with the original. I don’t think it’s that big a deal… git/deps let you get started quickly, and are good for sharing top-level deps immediately across a team/org without having to deal with extra infrastructure; and they’re also good for tooling, where you already know people are using tools.deps. If you want to make them a proper library for others to consume, you should release them as a maven lib, as that has the broadest audience for consumption.

rickmoynihan12:01:33

tools.build is also getting easier. If people start from something like a clj-new template etc, they’ll be taken 80% of the way there.

scarytom12:01:50

so why is tools.build not doing the latter then? It is a proper library, and it is legitimate for me to depend on it if I am building my own tool that wishes to make use of its utility functions for compilation etc. I notice the deps.edn for tools.build expresses a dependency on tools.deps, which uses a maven artifact not a git sha (presumably to solve this problem). Is there a plan for tools.build to be published as a maven artifact?

👆 1
rickmoynihan13:01:24

You’d need to ask @U064X3EF3 but whilst I agree it’s legitimate to use it as a library, I think it’s also legitimate for tools.build to assume you’re not going to be consuming from maven, but via tools.deps and the clojure CLI tools. I’m struggling to think of use cases where you wouldn’t be doing that.

rickmoynihan13:01:52

The same is true of @U04V70XH6’s build-clj which also uses tools.build as a library, and doesn’t ship a maven artifact. I’d expect 99.99% of possible users who have this as a dep will be consuming it in this manner; so I suspect there’s little point in doing it.

Alex Miller (Clojure team)13:01:37

As a build time tool, you don't really need a maven artifact, so I haven't published one. Presumably a lib depending on it would also be used at build time and can also use a git dep. I'm not opposed to doing so and may in the future if there is a need.

scarytom13:01:53

thanks both for your answers. I'm building an internal library that depends on tools.build, and we do most of our library publishing from a monorepo, so it doesn't make sense to package using git deps (especially considering all our current tooling works around maven deps). I'm not trying to fight the tide here, but it would be awfully convenient if tools.build were available as a maven dep too.

rickmoynihan13:01:50

Will that library not also always be run via tools.deps / clojure CLI? What is consuming it?

scarytom14:01:09

it will, but I don't want everyone to have to specify the library AND tools.deps tools.build itself in their deps.edn. The latter should come transitively by virtue of using the library.

scarytom14:01:02

This ensures the library comes with a compatible version of tools.deps tools.build.

rickmoynihan14:01:37

you mean tools.build right?

rickmoynihan14:01:12

what is “the library” doing here? If it’s truly a build tool, doesn’t :deps/prep-lib sufficiently solve the problem of creating those interfaces/classes for you?

rickmoynihan14:01:50

i.e. why can’t the outermost library dep, be a git/dep?

scarytom14:01:18

the repo for the library contains other things too, so it is hard to publish it as a git/dep. We publish things as jars internally.

rickmoynihan14:01:29

do you know about :deps/root?

scarytom14:01:50

yes, but that's a smell because it distributes knowledge about the packaging of your library amongst all downstream consumers.

rickmoynihan14:01:00

arguably it’s just part of the coordinate

rickmoynihan14:01:16

as the root belongs to the sha

scarytom14:01:52

you could make that case I guess.

scarytom14:01:41

To take a step back here, and attempt to summarise, there are basically two ways of distributing deps in clojure now. Some people will want to continue to use mvn deps and others will use git deps. For those using the former, they will find themselves unable to use the latter without taking the packaging task on themselves. I think your statement way back at the start of this thread is insightful: > If you want to make them a proper library for others to consume, you should release them as a maven lib, as that has the broadest audience for consumption.

rickmoynihan14:01:13

There are trade offs between the two. I use both, as do many others.

scarytom14:01:03

In the meantime. I have packaged tools.build v0.7.4 as a jar and pushed it into our internal mvn repo. This solves my immediate problem, but will make it harder to keep up-to-date with new tools-build releases. I hope that an official tools.build jar will be made available at some point, but I accept that this probably isn't a priority for the maintainers.

scarytom14:01:36

I want to say a big thank you to you for engaging on this @U06HHF230 -- It has been a very helpful conversation.

Alex Miller (Clojure team)14:01:23

I'd appreciate it if you made a request for this at https://ask.clojure.org and then I will (eventually) turn that into a ticket etc

scarytom14:01:56

I will do that @U064X3EF3, thanks.

seancorfield17:01:04

@UCQV87CN7 Despite your folks using Maven for "everything", I assume your projects still have deps.edn under git and therefore also your build.clj file -- and you have one or more git repos that are accessible to everyone? So there are two options for you: 1) if you have a monorepo containing all the projects this might affect, your build library will be in that repo and you could use a :local/root dep to depend on it (which would pull in any deps it uses, such as tools.build via git) since you're using the CLI to run the build or 2) put your build library under git internally and update your deps.edn to refer to it. If you were using my wrapper, you'd depend on that from GitHub and it would pull in tools.build via git -- and it can rely on that because the CLI is being used to drive the build.

seancorfield17:01:48

(I'd had requests to release build-clj as an artifact but pushed back because tools.build is not available as an artifact)

rickmoynihan15:01:53

I guess clj -X:deps tree can’t really be a -T tool as tools run outside of the projects classpath, but “deps tree” needs to run within it… Is that correct? I just wondered for a second why it was -X not -T…

Alex Miller (Clojure team)15:01:48

it could be a tool - it doesn't use the project classpath (like -Stree does)

rickmoynihan15:01:38

ahh cool, ok, that was what made me wonder… In that case would it not make sense to document it as a tool instead? My first thought was because it predates tools.

rickmoynihan15:01:28

I appreciate it doesn’t make a lot of difference; it just might be more consistent with usage now we have tools.

Alex Miller (Clojure team)15:01:41

just haven't gotten around to it

👍 1
Joshua Suskalo20:01:54

How does tools-deps handle git submodules when procuring a git dep?

Alex Miller (Clojure team)20:01:23

will not work correctly right now

Joshua Suskalo20:01:31

alright, good to know

Alex Miller (Clojure team)20:01:14

submodules are deeply weird and I did hack on it for a while to try to get them to work properly but the working tree strategy that gitlibs uses is I think fundamentally incompatible with submodules

Joshua Suskalo20:01:35

alright, so if I need a submodule then I should procure it myself as a part of the deps/prep step?

Alex Miller (Clojure team)20:01:03

I think you are likely to be affecting all other working trees of the repo in that case

Joshua Suskalo20:01:12

ah, that's no fun

Alex Miller (Clojure team)20:01:23

so it might appear to work until it is spectacularly broken

Joshua Suskalo20:01:24

I'll try to solve this with just a straight extra git dep then

Alex Miller (Clojure team)20:01:56

I did spend a day or so trying to making this work and eventually shelved it, it is not easy

Joshua Suskalo20:01:20

I believe you. I think in this case I can get away with just an extra git dep.

Joshua Suskalo20:01:36

I think the main real usecases for it would also be solved by aliases on deps

Joshua Suskalo20:01:50

since the only reason I can imagine to prefer submodules to a git dep is to allow you to specify which paths to include based on aliases