Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2024-05-16

Channels

#clojure
<
2024-05-16
>
Eric Dvorsak13:05:29

I am getting some sort of dependencies conflict while using: 

com.draines/postal {:mvn/version "2.0.5"} ;; sending emails
           org.apache.tika/tika-core {:mvn/version "3.0.0-BETA"}
           org.apache.tika/tika-parsers-standard-package {:mvn/version "3.0.0-BETA"}
postal was working fine before, once I added tika I started getting the following exception when sending emails: 
no object DCH for MIME type multipart/mixed; 
	boundary=\"----=_Part_0_336296515.1715862070244\

{"classLoaderName":"app","className":"javax.activation.ObjectDataContentHandler","fileName":"DataHandler.java","lineNumber":909,"methodName":"writeTo","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"javax.activation.DataHandler","fileName":"DataHandler.java","lineNumber":330,"methodName":"writeTo","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"javax.mail.internet.MimeBodyPart","fileName":"MimeBodyPart.java","lineNumber":1670,"methodName":"writeTo","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"javax.mail.internet.MimeMessage","fileName":"MimeMessage.java","lineNumber":1880,"methodName":"writeTo","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"postal.message.proxy$javax.mail.internet.MimeMessage$ff19274a","fileName":null,"lineNumber":-1,"methodName":"writeTo","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"com.sun.mail.smtp.SMTPTransport","fileName":"SMTPTransport.java","lineNumber":1300,"methodName":"sendMessage","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"postal.smtp$smtp_send_STAR_","fileName":"smtp.clj","lineNumber":36,"methodName":"invokeStatic","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"postal.smtp$smtp_send_STAR_","fileName":"smtp.clj","lineNumber":29,"methodName":"invoke","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"postal.smtp$smtp_send","fileName":"smtp.clj","lineNumber":58,"methodName":"invokeStatic","moduleName":null,"moduleVersion":null,"nativeMethod":false}
{"classLoaderName":"app","className":"postal.smtp$smtp_send","fileName":"smtp.clj","lineNumber":39,"methodName":"doInvoke","moduleName":null,"moduleVersion":null,"nativeMethod":false}
it only happens with the uberjar build, not in development which makes it even harder to debug

oyakushev13:05:38

Try doing clj -Srepro -Stree in the project. What does it say?

Eric Dvorsak13:05:44

I have a lot of deps but I narrowed down the issue to the addition of tika. Tika has the following deps: 

org.apache.tika/tika-parsers-standard-package 3.0.0-BETA
  . org.apache.tika/tika-parser-apple-module 3.0.0-BETA
    . org.apache.tika/tika-parser-zip-commons 3.0.0-BETA
      . org.apache.commons/commons-compress 1.25.0
    . com.googlecode.plist/dd-plist 1.27
  . org.apache.tika/tika-parser-audiovideo-module 3.0.0-BETA
    . com.drewnoakes/metadata-extractor 2.19.0
      . com.adobe.xmp/xmpcore 6.1.11
  . org.apache.tika/tika-parser-cad-module 3.0.0-BETA
    . org.apache.tika/tika-parser-microsoft-module 3.0.0-BETA
    X com.fasterxml.jackson.core/jackson-core 2.16.0 :older-version
    X com.fasterxml.jackson.core/jackson-databind 2.16.0 :older-version
  . org.apache.tika/tika-parser-code-module 3.0.0-BETA
    . org.apache.tika/tika-parser-text-module 3.0.0-BETA
    . org.codelibs/jhighlight 1.1.0
      X commons-io/commons-io 2.7 :older-version
    . org.ccil.cowan.tagsoup/tagsoup 1.2.1
    . org.ow2.asm/asm 9.6 :newer-version
    . com.epam/parso 2.0.14
      X org.slf4j/slf4j-api 1.7.5 :older-version
    . org.tallison/jmatio 1.5
      X org.slf4j/slf4j-api 1.7.25 :older-version
  . org.apache.tika/tika-parser-crypto-module 3.0.0-BETA
    . org.bouncycastle/bcjmail-jdk18on 1.77
      . org.bouncycastle/bcprov-jdk18on 1.77
      . org.bouncycastle/bcutil-jdk18on 1.77 :newer-version
        . org.bouncycastle/bcprov-jdk18on 1.77
      . org.bouncycastle/bcpkix-jdk18on 1.77 :newer-version
        . org.bouncycastle/bcprov-jdk18on 1.77
        . org.bouncycastle/bcutil-jdk18on 1.77
    . org.bouncycastle/bcprov-jdk18on 1.77 :newer-version
  . org.apache.tika/tika-parser-digest-commons 3.0.0-BETA
    X commons-codec/commons-codec 1.16.0 :older-version
    . org.bouncycastle/bcjmail-jdk18on 1.77
    . org.bouncycastle/bcprov-jdk18on 1.77
  . org.apache.tika/tika-parser-font-module 3.0.0-BETA
    X org.apache.pdfbox/fontbox 3.0.1 :older-version
  . org.apache.tika/tika-parser-html-module 3.0.0-BETA
    . org.jsoup/jsoup 1.17.1
    X commons-codec/commons-codec 1.16.0 :older-version
  . org.apache.tika/tika-parser-image-module 3.0.0-BETA
    . com.drewnoakes/metadata-extractor 2.19.0
    . org.apache.tika/tika-parser-xmp-commons 3.0.0-BETA
    . com.github.jai-imageio/jai-imageio-core 1.4.0
    . org.apache.pdfbox/jbig2-imageio 3.0.4
  . org.apache.tika/tika-parser-mail-module 3.0.0-BETA
    . org.apache.tika/tika-parser-mail-commons 3.0.0-BETA
      . org.apache.james/apache-mime4j-core 0.8.9
        X commons-io/commons-io 2.11.0 :older-version
      . org.apache.james/apache-mime4j-dom 0.8.9
        . org.apache.james/apache-mime4j-core 0.8.9
        X commons-io/commons-io 2.11.0 :older-version
    . org.apache.tika/tika-parser-text-module 3.0.0-BETA
    . org.apache.tika/tika-parser-html-module 3.0.0-BETA
  . org.apache.tika/tika-parser-microsoft-module 3.0.0-BETA
    . org.apache.tika/tika-parser-html-module 3.0.0-BETA
    . org.apache.tika/tika-parser-text-module 3.0.0-BETA
    . org.apache.tika/tika-parser-xml-module 3.0.0-BETA
    . org.apache.tika/tika-parser-mail-commons 3.0.0-BETA
    . com.pff/java-libpst 0.9.3
    . org.apache.tika/tika-parser-zip-commons 3.0.0-BETA
    X commons-codec/commons-codec 1.16.0 :older-version
    . org.apache.commons/commons-lang3 3.14.0
    . org.apache.poi/poi 5.2.5
      X commons-codec/commons-codec 1.16.0 :older-version
      . org.apache.commons/commons-collections4 4.4
      . org.apache.commons/commons-math3 3.6.1
      X commons-io/commons-io 2.15.0 :older-version
      . com.zaxxer/SparseBitSet 1.3
      . org.apache.logging.log4j/log4j-api 2.21.1
    . org.apache.poi/poi-scratchpad 5.2.5
      . org.apache.poi/poi 5.2.5
      . org.apache.logging.log4j/log4j-api 2.21.1
      . org.apache.commons/commons-math3 3.6.1
      X commons-codec/commons-codec 1.16.0 :older-version
    . org.apache.poi/poi-ooxml 5.2.5
      . org.apache.poi/poi 5.2.5
      . org.apache.poi/poi-ooxml-lite 5.2.5
        . org.apache.xmlbeans/xmlbeans 5.2.0
      . org.apache.xmlbeans/xmlbeans 5.2.0
        . org.apache.logging.log4j/log4j-api 2.21.1
      . org.apache.commons/commons-compress 1.25.0
      X commons-io/commons-io 2.15.0 :older-version
      . com.github.virtuald/curvesapi 1.08
      . org.apache.logging.log4j/log4j-api 2.21.1
      . org.apache.commons/commons-collections4 4.4
    X commons-logging/commons-logging 1.2 :older-version
    . com.healthmarketscience.jackcess/jackcess 4.0.5
      X org.apache.commons/commons-lang3 3.10 :older-version
      X commons-logging/commons-logging 1.2 :older-version
    . com.healthmarketscience.jackcess/jackcess-encrypt 4.0.2
      X org.bouncycastle/bcprov-jdk18on 1.72 :older-version
    . org.bouncycastle/bcjmail-jdk18on 1.77
    . org.bouncycastle/bcprov-jdk18on 1.77
  . org.slf4j/jcl-over-slf4j 2.0.9
    . org.slf4j/slf4j-api 2.0.9
  . org.apache.tika/tika-parser-miscoffice-module 3.0.0-BETA
    . org.apache.tika/tika-parser-zip-commons 3.0.0-BETA
    . org.apache.tika/tika-parser-text-module 3.0.0-BETA
    . org.apache.tika/tika-parser-xml-module 3.0.0-BETA
    . org.apache.commons/commons-lang3 3.14.0
    . org.apache.commons/commons-collections4 4.4
    . org.apache.poi/poi 5.2.5
    X commons-codec/commons-codec 1.16.0 :older-version
    . org.glassfish.jaxb/jaxb-runtime 4.0.4
      . org.glassfish.jaxb/jaxb-core 4.0.4
        . jakarta.xml.bind/jakarta.xml.bind-api 4.0.1
          . jakarta.activation/jakarta.activation-api 2.1.2
        . jakarta.activation/jakarta.activation-api 2.1.2
        . org.eclipse.angus/angus-activation 2.0.1
          . jakarta.activation/jakarta.activation-api 2.1.2
        . org.glassfish.jaxb/txw2 4.0.4
        . com.sun.istack/istack-commons-runtime 4.1.2
    . org.apache.tika/tika-parser-xmp-commons 3.0.0-BETA
  . org.apache.tika/tika-parser-news-module 3.0.0-BETA
    . com.rometools/rome 2.1.0
      . com.rometools/rome-utils 2.1.0
        X org.slf4j/slf4j-api 2.0.6 :older-version
      . org.jdom/jdom2 2.0.6.1
      X org.slf4j/slf4j-api 2.0.6 :older-version
    . org.slf4j/slf4j-api 2.0.9
  . org.apache.tika/tika-parser-ocr-module 3.0.0-BETA
    . org.apache.commons/commons-lang3 3.14.0
    . org.apache.commons/commons-exec 1.3
  . org.apache.tika/tika-parser-pdf-module 3.0.0-BETA
    . org.apache.tika/tika-parser-xmp-commons 3.0.0-BETA
    X org.apache.pdfbox/pdfbox 3.0.1 :use-top
    . org.apache.pdfbox/pdfbox-tools 3.0.1
      X commons-io/commons-io 2.15.0 :older-version
      . info.picocli/picocli 4.7.5
      . org.junit.jupiter/junit-jupiter 5.10.1
        . org.junit.jupiter/junit-jupiter-api 5.10.1
          . org.opentest4j/opentest4j 1.3.0
          . org.junit.platform/junit-platform-commons 1.10.1
            . org.apiguardian/apiguardian-api 1.1.2
          . org.apiguardian/apiguardian-api 1.1.2
        . org.junit.jupiter/junit-jupiter-params 5.10.1
          . org.junit.jupiter/junit-jupiter-api 5.10.1
          . org.apiguardian/apiguardian-api 1.1.2
        . org.junit.jupiter/junit-jupiter-engine 5.10.1
          . org.junit.platform/junit-platform-engine 1.10.1
            . org.opentest4j/opentest4j 1.3.0
            . org.junit.platform/junit-platform-commons 1.10.1
            . org.apiguardian/apiguardian-api 1.1.2
          . org.junit.jupiter/junit-jupiter-api 5.10.1
          . org.apiguardian/apiguardian-api 1.1.2
    . org.apache.pdfbox/jempbox 1.8.17
    . org.bouncycastle/bcjmail-jdk18on 1.77
    . org.bouncycastle/bcprov-jdk18on 1.77
    . org.glassfish.jaxb/jaxb-runtime 4.0.4
  . org.apache.tika/tika-parser-pkg-module 3.0.0-BETA
    . org.tukaani/xz 1.9
    . org.brotli/dec 0.1.2
    . org.apache.tika/tika-parser-zip-commons 3.0.0-BETA
    . com.github.junrar/junrar 7.5.5
      X org.slf4j/slf4j-api 1.7.36 :older-version
  . org.apache.tika/tika-parser-text-module 3.0.0-BETA
    . com.github.albfernandez/juniversalchardet 2.4.0
    X commons-codec/commons-codec 1.16.0 :older-version
    . org.apache.commons/commons-csv 1.10.0
  . org.apache.tika/tika-parser-webarchive-module 3.0.0-BETA
    . org.netpreserve/jwarc 0.28.3
    . org.apache.commons/commons-compress 1.25.0
  . org.apache.tika/tika-parser-xml-module 3.0.0-BETA
    X commons-codec/commons-codec 1.16.0 :older-version
  . org.apache.tika/tika-parser-xmp-commons 3.0.0-BETA
    . org.apache.pdfbox/jempbox 1.8.17
    . org.apache.pdfbox/xmpbox 3.0.1
      . org.junit.jupiter/junit-jupiter 5.10.1
      X commons-logging/commons-logging 1.2 :older-version
  . org.gagravarr/vorbis-java-tika 0.8
  . org.gagravarr/vorbis-java-core 0.8
postal only has these 2 
[com.sun.activation/jakarta.activation "1.2.1"]
[com.sun.mail/jakarta.mail "1.6.7"]
They don't have any deps in common, I know very little about the java world but it looks like it could be a conflict is the implementation of javax.mail spec

oyakushev13:05:56

jakarta.activation appears in both (howerer, the artifacts are different). Maybe try excluding those dependencies in Tika.

Eric Dvorsak13:05:26

I already tried that

Eric Dvorsak13:05:45

idk I might just need to find an alternative to tika this library is a dependency hell

jpmonettas13:05:30

I would try to run both instances with the java option of -verbose:class and log the class loading sequence to both files, and then compare.

jpmonettas13:05:39

I also find running with clj -Srepro -Spath | tr : '\n' better than clj -Srepro -Stree` to compare, since you want to compare the final classpaths

Eric Dvorsak13:05:58

I will try, right now I'm trying a newer version of bcjmail, maybe fixing https://github.com/bcgit/bc-java/issues/704

jpmonettas13:05:45

if you log both classpaths (with newlines) to different files, then a diff should show you the issue

Eric Dvorsak13:05:50

but this whole lib is getting on my nerves, it has so many deps, I was also using pdfbox and it uses a version with a bug

Eric Dvorsak13:05:22

if you log both classpaths (with newlines) to different files, then a diff should show you the issue by both you mean one with and one without tika?

Eric Dvorsak13:05:14

so org.bouncycastle/bcjmail-jdk18on {:mvn/version "1.78.1"} fixes the issue

šŸ™Œ 1
Eric Dvorsak13:05:08

but I'll still try the comparison

jpmonettas13:05:27

This looks like it could introduce an issue : 

āžœ  ~ clj -Srepro -Sdeps '{:deps {com.draines/postal {:mvn/version "2.0.5"}}}' -Spath | tr ':' '\n'
src
...
/home/jmonetta/.m2/repository/commons-codec/commons-codec/1.9/commons-codec-1.9.jar
/home/jmonetta/.m2/repository/com/sun/activation/jakarta.activation/1.2.1/jakarta.activation-1.2.1.jar
...


āžœ  ~ clj -Srepro -Sdeps '{:deps {org.apache.tika/tika-parsers-standard-package {:mvn/version "3.0.0-BETA"}}}' -Spath | tr ':' '\n'
src
...
/home/jmonetta/.m2/repository/commons-codec/commons-codec/1.16.0/commons-codec-1.16.0.jar
...
/home/jmonetta/.m2/repository/jakarta/activation/jakarta.activation-api/2.1.2/jakarta.activation-api-2.1.2.jar
...

jpmonettas13:05:37

so postal and tika-parsers-standard-package both uses different versions of commons-codec and jakarta.activation-api

Eric Dvorsak14:05:13

yeah but I tried excluding them and explicitely requirin them and still had the issue

Eric Dvorsak14:05:34

I tried now with nothing but adding explicitely org.bouncycastle/bcjmail-jdk18on {:mvn/version "1.78.1"}

Eric Dvorsak14:05:46

and it fixes the issue without any other exclusion

Eric Dvorsak14:05:02

< /home/yenda/.m2/repository/org/bouncycastle/bcjmail-jdk18on/1.78.1/bcjmail-jdk18on-1.78.1.jar
212,214d210
< /home/yenda/.m2/repository/org/bouncycastle/bcpkix-jdk18on/1.78.1/bcpkix-jdk18on-1.78.1.jar
< /home/yenda/.m2/repository/org/bouncycastle/bcprov-jdk18on/1.78.1/bcprov-jdk18on-1.78.1.jar
< /home/yenda/.m2/repository/org/bouncycastle/bcutil-jdk18on/1.78.1/bcutil-jdk18on-1.78.1.jar
368a365,366
> /home/yenda/.m2/repository/org/bouncycastle/bcjmail-jdk18on/1.77/bcjmail-jdk18on-1.77.jar
> /home/yenda/.m2/repository/org/bouncycastle/bcprov-jdk18on/1.77/bcprov-jdk18on-1.77.jar
414a413,414
> /home/yenda/.m2/repository/org/bouncycastle/bcpkix-jdk18on/1.77/bcpkix-jdk18on-1.77.jar
> /home/yenda/.m2/repository/org/bouncycastle/bcutil-jdk18on/1.77/bcutil-jdk18on-1.77.jar

jpmonettas14:05:48

dang dependency hell, at least you found a solution

Eric Dvorsak14:05:36

yeah thanks for the help!

Alex Miller (Clojure team)21:05:08

on 2, they changed formats and http://mvnrepository.com shows them to you in lexicographic order. fyi, the Clojure CLI has a tool to tell you the https://clojure.org/reference/clojure_cli#deps_find_versions of a dep. by default it will give you the newest 8 versions (see https://clojure.github.io/tools.deps.cli/clojure.tools.deps.cli.api-api.html#clojure.tools.deps.cli.api/find-versions for flags) 

~  $ clj -X:deps find-versions :lib com.google.apis/google-api-services-sheets
Downloading: com/google/apis/google-api-services-sheets/maven-metadata.xml from central
{:mvn/version "v4-rev20230526-2.0.0"}
{:mvn/version "v4-rev20230808-2.0.0"}
{:mvn/version "v4-rev20230815-2.0.0"}
{:mvn/version "v4-rev20240312-2.0.0"}
{:mvn/version "v4-rev20240319-2.0.0"}
{:mvn/version "v4-rev20240402-2.0.0"}
{:mvn/version "v4-rev20240416-2.0.0"}
{:mvn/version "v4-rev20240423-2.0.0"}

Alex Miller (Clojure team)21:05:32

works with both maven deps (from maven repository metadata - which CAN be missing or wrong) or git sha

Alex Miller (Clojure team)21:05:04

the best way to solve this kind of error is to understand it. but failing that, commenting things out is an approach. :) protobuf compiles the defiitions to classes, which are often included in libs. it is not uncommon for two libs to include the same classes and/or use different versions of protobuf and yield something like this. I don't think I really buy it's babashka/fs, probably more likely something in a transitive dep chain

Alex Miller (Clojure team)21:05:20

grpc has a notoriously awful chain of circular libs (I used it for testing pathological cases in tools.deps). of all the things mentioned here I immediately suspect that :)