Fork me on GitHub
#leiningen
<
2023-03-01
>
Mor Gazith16:03:26

hello 👋 channel, i have an issue related to lein dependencies with :aot :all. i have an app built with lein that depends a library called https://github.com/fullcontact/full.cache that serializes objects using https://github.com/ptaoussanis/nippy. my app is using full.cache version 1.2.1 which depends on nippy version 2.14.0 the cache is shared among several other apps that might depend on different versions of full.cache/nippy, so full.cache solves this by adding a prefix with nippy’s version to the cache key: n2.14.0- however when reading some keys from cache i still get an exception from nippy suggesting the object was serialized on a different version:

Thaw failed: Decryption/decompression failure, or data unfrozen/damaged
...
Thaw failed against type-id: 112
...
No reader provided for custom type with internal id: 112
took me a while figuring out that the exact string
No reader provided for custom type with internal id
does not exist in nippy 2.14.0, it was changed to:
No reader provided for custom type id:
on version ~2.12 that means my app is actually using an older version of nippy, even though i’m confident its using the 1.2.1 version of full.cache. can this be caused by :aot :all in one of my libraries? how can I investigate and resolve this issue? (sorry for the long post… :face_palm: )

lread17:03:21

I think your guess is good! If you are in control of these libraries I'd recommend that you not use :aot :all when creating the library jars.

Mor Gazith17:03:14

thanks @UE21H2HHD it's the second time you're jumping to help, I might have to send your a beer 🙂 some libraries are in my control, some are not. any idea how do I find the culprit library? I went through some manually, and checked lein deps, but no luck so far

lread17:03:03

Ah yeah, MrAnderson stuff right @U04H5V7URR9? Hiya! I think aot gunk in fullcontact libs was causing you grief there as well.

lread18:03:28

So if you are actively working on these libs, they'll likely be downloaded to your local maven repository. If you are on mac or linux this will be ~/.m2/repository/. You can use unzip -l or jar tf to take a peek at what files are in the jars, I used unzip -l above on fullcontact/full.core v1.1.1 which happened to be in my local maven repo.

lread18:03:39

The .class files are likely all due to aot compilation of Clojure source. And that's not good, because the result is that it brings in a specific compiled version of a dependency which will often result in a conflict of that same dependency brought from a different lib.

lread18:03:46

There are times when you'll need .class files in a lib jar. Like when your lib has some Java source, you'll need to compile those Java sources into .class files. But, for a lib, including Clojure compiled sources is going to cause grief.

lread18:03:31

For an application uberjar, for faster startup, some folks decide to :aot :all. This does not cause conflicts because nothing is using the application as a lib dependency.

Mor Gazith18:03:40

I see, so I guess I'll update all the libs that I can change removing aot and checking to see if the issue resolved appreciate the guidance

lread18:03:55

Yeah, I think that would solve at least one big problem, @U04H5V7URR9. And then maybe you can focus on more interesting problems! simple_smile

Mor Gazith18:03:47

I've been in dependency hell for months

lread18:03:50

Getting rid of :aot :all for your libs will solve many, if not all, of those problems for you.

lread18:03:55

Well, and then there are just sometimes general dependency headaches from libs like jackson. But you'll be well on your way, you'll find that missing spring in your step, and you might even occasionally and uncontrollably break out into song.

Mor Gazith18:03:16

:rolling_on_the_floor_laughing: :rolling_on_the_floor_laughing: :rolling_on_the_floor_laughing:

simple_smile 1
Mor Gazith07:03:59

hey @UE21H2HHD I removed aot from all of our libraries and so far it seemed to fix issues with two services that had been upgraded to the new library versions! thank you so much

lread12:03:38

I thought I heard you singing, congrats on finding a better path!