This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-01-15
Channels
- # aws (25)
- # babashka (2)
- # babashka-sci-dev (3)
- # beginners (39)
- # clj-kondo (69)
- # cljdoc (2)
- # clojars (6)
- # clojure (13)
- # clojure-dev (8)
- # clojure-europe (3)
- # clojure-uk (1)
- # clojurescript (6)
- # datomic (3)
- # honeysql (5)
- # introduce-yourself (3)
- # lsp (10)
- # malli (8)
- # membrane (16)
- # off-topic (8)
- # pedestal (6)
- # re-frame (28)
- # releases (3)
- # shadow-cljs (10)
- # tools-deps (38)
Is it possible to build tools.deps.alpha without a transitive dependency on clojurescript? I recently pacakged clojure-tools for Guix, which requires that all pacakges be built from source. However, building ClojureScript (and it's dependencies) from source is quite the undertaking. To avoid this, I simply strip out the S3 transporter, which is the only part of tools deps that depends transitively on clojurescript. However, this is for obvious reasons not really ideal in the long run. Is there a way to break this dependency chain somewhere? Currently we have tools.deps -> cognitect.aws -> core.async -> clojurescript.
For context, here is how I strip s3-transporter (in scheme, for version 0.12.1104):
(lambda _
(for-each delete-file
(list
(string-append
"src/main/clojure/clojure/"
"tools/deps/alpha/util/s3_aws_client.clj")
(string-append
"src/main/clojure/clojure/"
"tools/deps/alpha/util/s3_transporter.clj")
(string-append
"src/test/clojure/clojure/"
"tools/deps/alpha/util/test_s3_transporter.clj")))
(substitute*
"src/main/clojure/clojure/tools/deps/alpha/util/maven.clj"
(("clojure.tools.deps.alpha.util.s3-transporter")
"")))
core.async
does not depend on ClojureScript:
org.clojure/core.async 1.5.648
. org.clojure/tools.analyzer.jvm 1.2.2
. org.clojure/tools.analyzer 1.1.0
. org.clojure/core.memoize 1.0.253
. org.clojure/core.cache 1.0.225
. org.clojure/data.priority-map 1.1.0
. org.ow2.asm/asm 9.2
. org.clojure/tools.reader 1.3.6
tools.deps.alpha
does not depend on ClojureScript either:
clojure -Sdeps '{:deps {org.clojure/tools.deps.alpha {:mvn/version "RELEASE"}}}' -Stree|fgrep script
<empty result>
Here's the list of org.clojure
deps that get pulled in for tools.deps.alpha
:
(! 755)-> clojure -Sdeps '{:deps {org.clojure/tools.deps.alpha {:mvn/version "RELEASE"}}}' -Stree|fgrep org.clojure
org.clojure/clojure 1.10.3
. org.clojure/spec.alpha 0.2.194
. org.clojure/core.specs.alpha 0.2.56
org.clojure/tools.deps.alpha 0.12.1109
. org.clojure/data.xml 0.2.0-alpha6
. org.clojure/data.codec 0.1.0
. org.clojure/tools.gitlibs 2.4.172
. org.clojure/tools.cli 1.0.206
. org.clojure/data.json 2.4.0
. org.clojure/tools.logging 1.2.1
. org.clojure/core.async 1.5.644
. org.clojure/data.xml 0.2.0-alpha6
. org.clojure/core.async 1.5.644
. org.clojure/tools.analyzer.jvm 1.2.1
. org.clojure/tools.analyzer 1.1.0
. org.clojure/core.memoize 1.0.253
. org.clojure/core.cache 1.0.225
. org.clojure/data.priority-map 1.1.0
. org.clojure/tools.reader 1.3.6
What makes you think you have a dependency on ClojureScript @U6NJBB596?
(if you built clojure-tools from source, that depends on t.d.a so you would have already run into this issue?)
I second Sean’s confusion. core.async has a project.clj that depends on clojurescript for testing but that's not the project file we use to build core.async (built with Maven from the pom.xml)
Ah, I found the problem. It was attempting to compile macros for clojurescript, which depend on cljs.analyzer. removing macro namespaces from the list of aot compiled namespaces should fix this.
Again, there should be no clojurescript anywhere in this deps tree
The problem here stemming from the fact that guix's clojure-build-system tries to aot compile every clj file.
Guix doesn't use maven for dependency resolution, it uses its own packages, and trying to aot ClojureScript macros (in clj files) without ClojureScript failed.
Thank you for helping me realize something obvious after staring at what was ultimately a stupid problem for way too long.
Which namespaces are you talking about here?
cljs.core.async.impl.ioc-macros
Well, guix’s system is wrong to do that
Yeah, definitely. And packaging a library as AOT'd code is likely to cause horrible problems for consumers of that library.
Lots of Clojure libs have clj files that are not part of the lib
Do you think it would be worh it to get that default changed in guix? Currently (afaict) it calls clojure.core/compile on every clj file that isn't excluded by default, unless specifically disabled. I am not super familiar with guix clojure-build-system, just trying to get a few things packaged. Currently the number of Clojure libraries packaged is abysmally small.
@U6NJBB596 You're aware that lots of Clojure libraries are packaged as JAR files with no compilation of source code?
"Packaging" Clojure libraries seems kinda wrong on several levels. The normal way to get Clojure libraries is as dependencies in a project -- fetched by the Clojure tooling itself, when it is run.
I am aware of this, yes. I am neither familiar with the details of maven distribution, nor why guix's clojure-build-system was written like that.
I can sort of understand wanting the clojure
/`clj` tool itself as a package that one of these package managers can fetch and install for you, but there's no reason to package Clojure libraries in general.
I personally just don't get the “rebuild” everything from source obsession
Yeah, that is the normal way, which is why my first target for packaging was clojure-tools, to allow that use case. However guix likes to have everything packaged in its own system to facilitate reproducible builds of everything. And because guix as a package manager follows GNU guidelines on only providing software built from source, we get into situations where (possibly pre-compiled) maven artifacts are an issue.
What could be more reproducible than using the exact globally published bits everyone else uses?
It's like these package managers are stuck in the 1990s before the Java ecosystem appeared...
I don't necessarily disagree, it /is/ a GNU project...
But packaging clojure-tools solves things for 99% of cases, and for when people want to integrate a Clojure codebase with guix tools, they can package libraries themselves.
Seems like GNU's guidelines are at odds with how the Java ecosystem works. Does Guix has brew
as a package (linuxbrew) that can be installed via the package manager?
(b/c once you have brew
you can use that to install the Clojure tools directly from the Clojure repos)
It does not. My patch for clojure-tools (sans S3 transport) was already accepted, Ill just have to send a new patch lifting that restriction once I sort the AOT issues
I'm curious about the process for upgrading stuff? It seems like packaging clojure-tools relies on community volunteers and that means they have to repackage every time there's a new release of the CLI?
What happens if a community volunteer moves on, or even is off on vacation for a while? Doesn't that just block people from upgrading their clojure-tools in a timely manner?
What I like about brew
on Linux is that I can install any version of the Clojure CLI, as soon as it is released by the core team -- and I can install prerelease versions just as easily as stable versions, and I can switch back and forth between them easily.
This is all basically same as Debian, that redoes all this same work with their own slightly different build system that makes slightly different versions of libs too. They chunk it up into periodic release trains
But what seems weird to me is that once you start using the custom Clojure cli, you are back again to using maven libs etc so …
Fwiw, reproducible builds are perhaps not the best name for the concept: verifiable builds would perhaps be a better description. But it's a short ddg search away if you want to read up on it https://reproducible-builds.org. If a person does not experience (or care about) a problem, naturally they won't appreciate the trade-offs made to solve the problem. Edit: which is not a dig at anyone or any point made here, but just an almost tautological observation.