How do you deal with Bill of Materials? https://cloud.google.com/java/docs/bom
com.google.cloud
libraries-bom
26.22.0
pom
import
com.google.cloud
google-cloud-pubsub
I know there is https://clojure.atlassian.net/browse/TDEPS-202
Because BOM is not supported I have deps manually, and I have Bad type on operand stack
on (.build (.setData (PubsubMessage/newBuilder) (ByteString/copyFromUtf8 (json/write-value-as-string body)))), but only in JAR.
I see ByteString/copyFromUtf8 is com.google.protobuf.LiteralByteString in jar, but com.google.protobuf.ByteString$LiteralByteString in REPL. Why it works in REPL, but doesn’t in jar.
I tried clojure.tools.build.api/write-pom and versions look the same as in deps.edn.
Any hints what I can do to fix this? How to debug this to find the issue?
PS I can’t be sure if the issue is luck of BOM or something else.if you're not using BOM, then that can't be the problem - just work this as a java interop issue - what is the full error when it occurs?
#error {
:cause Bad type on operand stack
Exception Details:
Location:
com/google/api/AnnotationsProto.registerAllExtensions(Lcom/google/protobuf/ExtensionRegistryLite;)V @4: invokevirtual
Reason:
Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current frame, stack[1]) is not assignable to 'com/google/protobuf/ExtensionLite'
Current Frame:
bci: @4
flags: { }
locals: { 'com/google/protobuf/ExtensionRegistryLite' }
stack: { 'com/google/protobuf/ExtensionRegistryLite', 'com/google/protobuf/GeneratedMessage$GeneratedExtension' }
Bytecode:
0000000: 2ab2 0002 b600 03b1
:via
[{:type java.lang.VerifyError
:message Bad type on operand stack
Exception Details:
Location:
com/google/api/AnnotationsProto.registerAllExtensions(Lcom/google/protobuf/ExtensionRegistryLite;)V @4: invokevirtual
Reason:
Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current frame, stack[1]) is not assignable to 'com/google/protobuf/ExtensionLite'
Current Frame:
bci: @4
flags: { }
locals: { 'com/google/protobuf/ExtensionRegistryLite' }
stack: { 'com/google/protobuf/ExtensionRegistryLite', 'com/google/protobuf/GeneratedMessage$GeneratedExtension' }
Bytecode:
0000000: 2ab2 0002 b600 03b1
:at [com.google.pubsub.v1.PubsubProto <clinit> PubsubProto.java 570]}]
:trace
[[com.google.pubsub.v1.PubsubProto <clinit> PubsubProto.java 570]
[com.google.pubsub.v1.PubsubMessage$AttributesDefaultEntryHolder <clinit> PubsubMessage.java 112]
[com.google.pubsub.v1.PubsubMessage$Builder internalGetAttributes PubsubMessage.java 910]
[com.google.pubsub.v1.PubsubMessage$Builder buildPartial PubsubMessage.java 694]
[com.google.pubsub.v1.PubsubMessage$Builder build PubsubMessage.java 682]
...Why it work in REPL and doesn’t in jar?
I don’t see any conflicts or double versioning etc. in pom.xml or -Stree
not sure, but "Type 'com/google/protobuf/GeneratedMessage$GeneratedExtension' (current frame, stack[1]) is not assignable to 'com/google/protobuf/ExtensionLite'" makes me think of either getting mismatched lib versions, or a classloader issue
Any actions which I can do to verify it?
a VerifyError generally means the class bytecode is invalid. I assume here it's a class generated by protobuf
when you say "in jar" - what does that mean?
are you building an uberjar out of this?
I assume it is about: > I see ByteString/copyFromUtf8 is com.google.protobuf.LiteralByteString in jar, but com.google.protobuf.ByteString$LiteralByteString in REPL. Why it works in REPL, but doesn’t in jar.
I don't understand what you mean by that sentence
(b/uber
{:class-dir class-dir
:uber-file jar-file
:basis basis
:main 'system.main})
To be precise, this is not my project and I am still learning this project.I mean (ByteString/copyFromUtf8 (json/write-value-as-string body)) in REPL is com.google.protobuf.ByteString$LiteralByteString, but in jar is com.google.protobuf.LiteralByteString which suggest me there is an issue about type.
and this is the input for later function which fail
don't know why that would be unless the libs are different. could be you have two jars on your classpath with the same classes and on the repl, the ordering puts the "correct" one first, but after building the uber jar, the "wrong" one ends up in the uberjar
this was my first idea, but I don’t see any duplicate of libs in -Stree or pom.xml
Do you suggest any actions, code, commands, check which I can do?
it would not be duplicate lib, it would be two different jars that include the same classes. since you have two classes above, I would look for those in the jars you have (com.google.protobuf.LiteralByteString and com.google.protobuf.ByteString).
maybe something in the google-cloud-pubsub includes protobuf instead of depending on it
that kind of problem could definitely cause the error you're seeing
> com/google/protobuf/ByteString$LiteralByteString.class > com/google/protobuf/LiteralByteString.class I see both in Jar, but still I don’t know what actions I can do to fix it.
> maybe something in the google-cloud-pubsub includes protobuf instead of depending on it Do you mean the pubsub (or dependency of pubsub) is builded as uberjar instead of jar and include other lib jar with protobuf while at the same time have protobuf as dependency?
Yes
I don’t think there is any way to fix it in deps.edn, is it? Perhaps BOM fix it.
Bom has nothing to do with this - it’s just a way to specify multiple deps together
I mean maybe BOM set the same version in dependency as it is in jar
Ultimately you’re getting some set of jars that is on your classpath in repl case. In uberjar, that set of jars is being collapsed into a single jar. If the jars do not overlap, this identical from classpath perspective. If jars overlap, then ordering matters, in both cases.
interesting, I have tried to isolate problem (libraries and small code) to separate in small project:
1) with deps.edn clj -T:build ci I have
> Cannot write META-INF/license/LICENSE.aix-netbsd.txt from io.grpc/grpc-netty-shaded as parent dir is a file from another lib. One of them must be excluded.
2) with lein uberjar it compiles and work in jar as expected
both have the same versions of the same libraries in the same order
also -Spath show io.grpc/grpc-netty-shaded only once
everything is only once
Can it be clj build bug somehow?
that error means probably means you have both a META-INF/license/ dir and META-INF/license file from different jars during uber jar'ing - in uber task you can add :exclude ["META-INF/license"] to exclude the file
oh sorry, I misinterpret it as a part of the libraries conlict
ok, it works both from lein and clj when isolated from the project.
finally, something
thank you for your help
you were right. This is one of third party library which has to include protobuf in hidden way. Just the presence of the library in deps.edn make it fail after build to jar.
I did it by commenting half of the deps, later other half etc. in experimental project. Is there any other way to detect such conflicts in the real project and real code?
I mean in the real project I can’t comment any library, because system will not build, because code needs each of them.
So I am thinking if there is any reasonable way to detect such conflicts and display them.
jars have files, jars with the same .class file are in conflict - there are some java tools out there that can help identify stuff like this
Can you give any name of such tool to start reading in google?
don't know one offhand
thank you, I will find something