Clojurians Log v2

not directly, although it has all the pieces there. the create-basis task returns the basis, which includes :libs, which is a map that includes all of the jars and paths that would end up on the classpath and their source type. The copy-file and copy-dir task exist to move files around, and jar and zip exist to make archives.

Ah, so create-basis sounds like the missing piece from the last time I attempted to port this build 😊 Then I think it'll be fairly straight-forward to port this.

or the uber task builds an uber jar, which is a standalone bundle that combines libs and paths from git or local deps and does resource conflict management

create-basis has always been part of tools.build as that is the heart of tools.deps, which tools.build depends on

Yeah, this is more a case of me not finding it!

uber is close to what I want, I just don't want the extra level of jar-ing. We wrap things up in a Docker image anyway, and I find it easier to troubleshoot without the extra level of packaging.

so you want a set of jars, one of which is the file stuff?

how do you actually use the bundle to start your app, out of curiosity- is it a classpath wildcard kind of thing?

FROM eclipse-temurin:21-jre-alpine

ADD target/jars /app/lib
ADD target/classes /app/classes
COPY config /app/config

WORKDIR /app

CMD exec java $JVM_OPTS -cp "classes:lib/*" ourapp.prod

target/jars is the result of badigeon's bundle operation, and target/classes is the result of compile.

This is basically as low tech of a production build I can imagine. Very "inspectable" and easy to verify locally.

yep, I get it

it's very similar to what datomic ions does under the hood

😊 👍

I think they do some additional stuff to only deploy the delta jars rather than all the jars to reduce traffic

I can see how that would be very useful for them.

I take it there should be some conflict handlers for this. I guess we should write our own? Anyone know of any prior art here?

I’ve just searched and found the solution! I’m sorry i should have searched first

I gather you found this? https://github.com/seancorfield/build-uber-log4j2-handler

yes i did! thank you!

hadn’t heard any chatter of it. i should have searched first

Any suggestions on how I could have made it more discoverable?

i’m not sure. my steps were i googled part of the weird logging message and ended up here: https://stackoverflow.com/questions/48033792/log4j2-error-statuslogger-unrecognized-conversion-specifier Then checked depstar source and then went to tools.build to see if there was something built in there.

This talks about it, and is linked from the tools.build guide: https://clojure-doc.org/articles/cookbooks/cli_build_projects/

that’s good to know! I’ll need to reread it.

The exact thing i googled was “ERROR Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.” I wonder what i might have tried? I didn’t think the error message would hit with any clojure keywords. And the error doesn’t really point to what might have caused it.

The actual error you get depends on a lot of things, unfortunately.

yeah. it’s a very “upstream” type of bug

What happens, without the cache merging, is that you randomly get one of the multiple plugin caches from the classpath, which means you get a random subset of the log4j features you expect -- so any particular feature could be the one that fails 😞

I get why this isn’t part of tools.build because of the dep. But it almost seems such a critical piece maybe it merits a callout in the readme there?

Yeah, it's one of those things that you only need if you use log4j2 and only if you depend on multiple libraries that supply plugins... which is a pretty narrow set of apps... but if you need it, you must use it!

i would expect almost every uberjar these days to need this? I wish i could easily see which jars brought in plugins

The only consolation is that this will all go away in log4j 3.x 🙂

haha. yeah i saw the todo in depstar

"Log4j2 has a very problematic binary plugins cache file that needs to
  be merged -- but it's going away in 3.0.0 apparently because it keeps
  breaking build tools... We use log4j's plugin processor to merge it."

The depstar README says "The missing piece (for us at work) had been the merging of the Log4j2Plugins.dat files across libraries which depstar had supported since December 2020 (version 2.0.160) and that is now possible through the :conflict-handlers option added to uber in v0.4.0 of tools.build, along with my log4j2 conflict handler library."

ooh. that’s a great bit. I think if it had been in a big “migration guide” section or some kind of alert callout that everything works the same except this one piece we might have noticed it. But not sure 🙂 Sometimes you just jump in despite so much helpful documentation 🤠

It's archived so I can't easily edit that... I'd have to unarchive, edit it, and re-archive 🙂

dang. you’d think a change to a readme would be easy for archived stuff

I unarchived it and edited it. Does this help? https://github.com/seancorfield/depstar/blob/develop/README.md

(I also took the opportunity to remove the recommendation to use my build-clj library, which I've since archived too!)

yeah that looks super helpful!