Clojurians Log v2
tools-build
I have a monorepo project with several deps.edn modules. Some of these modules are depended upon by an application which is shipped to AWS lambda. AWS lambda has a size limitation on the uploaded jar, so I’m running into the limit when I create an uberjar. To remedy this, we’ve been using b/write-pom to package the jar dependencies into a lambda layer, then using b/jar (not uber) to compile only our src into the shipped jar. It looks like b/write-pom does not roll up dependencies from :local/root so we’re starting to see issues where some required deps are not being included. I’m guessing I can generate a zip of jars per module, but wanted to check and see if there’s a better solution to get the full module tree pom created. Anyone have this issue?
Iterating over the :libs from basis appears to work great
write-pom can't include local deps, because there is no way to specify that in the pom
the basis includes information on the local libs and their location, so you could copy source paths from local jars into your target dir for inclusion in the final jar
that code effectively exists inside the uber jar, I guess you could also uber just the local libs with a custom basis and then unjar that into your target dir
if some subset or internal part of tools.build would be useful as a task, I'm happy to look at breaking it out
amazing thanks @alexmiller. Our :local/root dependencies are pointing to deps.edn projects, not jars. It definitely seems like clojure.tools.build.tasks.write-pom has the parts we need, since the deps were interested in are in the basis
write-pom just writes the pom file, I assume you actually need the contents of the local project paths?
we want the local project path jars to be assembled separately from the compiled src code due to the lambda limitation
that's more similar to code in the current uber task
but I guess we could copy them in first prior to compilation….
the code is kind of spread throughout the uber task, but it's essentially using :libs in the basis, looking for lib paths that point to a directory (will be the :paths from local deps), then copying those libs to the target dir where the uber is built. the latter part is not hard but is careful about things like retaining file timestamps (which can affect whether compilation happens)
I see
looks like I may not really need to write a pom, ultimately I just need a zip of all the lib entries in the basis, don’t even need to compress them into a jar since lambda can work with a zip of jars to add to the classpath
yeah, the pom is probably unimportant if you're not deploying it to maven
I'm trying to think about what sensible part of this uber task could be extracted for reuse
it was a mechanism by previous dep to achieve the packaging
yea if some of the uber ns could be depended upon that would be great
https://github.com/clojure/tools.build/blob/master/src/main/clojure/clojure/tools/build/tasks/uber.clj#L188-L207 is the key bits there
that's kind of bound up with all of the additional handlers that uber provides though
right
is there a reason I can’t iterate over the :libs entry and copy each into a dir and zip those? this looks like its packing up the jar which may not be needed for me
you can feel free to remix whatever you need from here, there's no magical stuff happening, it's all just basically copying files from jars and libs into a dir and zipping it
totally, thanks for the support, this makes sense and I can work with those bits
if there’s something particular to extract I can let you know
not readily apparent to me without doing the work, but if you identify something like that, happy to look at it. lambda layer building is something that people have built (I know @ghadi did a version of this, but don't think it was ever made public), would be great to consider something reusable