This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-05-29
Channels
- # announcements (15)
- # babashka (17)
- # beginners (66)
- # calva (18)
- # clojure (20)
- # clojure-dev (2)
- # clojure-europe (27)
- # clojure-nl (1)
- # clojure-norway (24)
- # clojure-sweden (5)
- # clojure-uk (4)
- # clr (1)
- # cursive (3)
- # datomic (10)
- # dev-tooling (9)
- # gratitude (1)
- # honeysql (5)
- # hyperfiddle (13)
- # malli (4)
- # music (1)
- # nrepl (1)
- # off-topic (31)
- # polylith (11)
- # portal (6)
- # re-frame (3)
- # reitit (4)
- # releases (1)
- # shadow-cljs (8)
- # squint (4)
- # tools-build (26)
- # vim (1)
- # yamlscript (52)
I have a monorepo project with several deps.edn modules. Some of these modules are depended upon by an application which is shipped to AWS lambda. AWS lambda has a size limitation on the uploaded jar, so I’m running into the limit when I create an uberjar. To remedy this, we’ve been using b/write-pom
to package the jar dependencies into a lambda layer, then using b/jar
(not uber) to compile only our src into the shipped jar. It looks like b/write-pom
does not roll up dependencies from :local/root
so we’re starting to see issues where some required deps are not being included. I’m guessing I can generate a zip of jars per module, but wanted to check and see if there’s a better solution to get the full module tree pom created. Anyone have this issue?
write-pom can't include local deps, because there is no way to specify that in the pom
the basis includes information on the local libs and their location, so you could copy source paths from local jars into your target dir for inclusion in the final jar
that code effectively exists inside the uber jar, I guess you could also uber just the local libs with a custom basis and then unjar that into your target dir
if some subset or internal part of tools.build would be useful as a task, I'm happy to look at breaking it out
amazing thanks @U064X3EF3. Our :local/root
dependencies are pointing to deps.edn
projects, not jars. It definitely seems like clojure.tools.build.tasks.write-pom
has the parts we need, since the deps were interested in are in the basis
write-pom just writes the pom file, I assume you actually need the contents of the local project paths?
we want the local project path jars to be assembled separately from the compiled src code due to the lambda limitation
the code is kind of spread throughout the uber task, but it's essentially using :libs in the basis, looking for lib paths that point to a directory (will be the :paths from local deps), then copying those libs to the target dir where the uber is built. the latter part is not hard but is careful about things like retaining file timestamps (which can affect whether compilation happens)
looks like I may not really need to write a pom, ultimately I just need a zip of all the lib
entries in the basis, don’t even need to compress them into a jar since lambda can work with a zip of jars to add to the classpath
yeah, the pom is probably unimportant if you're not deploying it to maven
I'm trying to think about what sensible part of this uber task could be extracted for reuse
https://github.com/clojure/tools.build/blob/master/src/main/clojure/clojure/tools/build/tasks/uber.clj#L188-L207 is the key bits there
that's kind of bound up with all of the additional handlers that uber provides though
is there a reason I can’t iterate over the :libs
entry and copy each into a dir and zip those? this looks like its packing up the jar which may not be needed for me
you can feel free to remix whatever you need from here, there's no magical stuff happening, it's all just basically copying files from jars and libs into a dir and zipping it
not readily apparent to me without doing the work, but if you identify something like that, happy to look at it. lambda layer building is something that people have built (I know @U050ECB92 did a version of this, but don't think it was ever made public), would be great to consider something reusable