Fork me on GitHub
#shadow-cljs
<
2023-08-23
>
anthony-galea07:08:43

Hi, has anyone had success using AWS SDK for JavaScript v3 in a ClojureScript webapp? Running into the following in steps 4+5 here: https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/getting-started-react-native.html#getting-started-react-write-native-code

npm install @aws-sdk/client-s3

(ns foo.bar
  (:require
   ["@aws-sdk/client-s3" :as s3]))
As soon as I eval the ns form:
The required JS dependency "http2" is not available, it was required by "node_modules/@smithy/node-http-handler/dist-cjs/node-http2-handler.js".

Dependency Trace:
	...
	node_modules/@aws-sdk/client-s3/dist-cjs/index.js
	node_modules/@aws-sdk/client-s3/dist-cjs/S3Client.js
	node_modules/@smithy/smithy-client/dist-cjs/index.js
	node_modules/@smithy/smithy-client/dist-cjs/collect-stream-body.js
	node_modules/@smithy/util-stream/dist-cjs/index.js
	node_modules/@smithy/util-stream/dist-cjs/sdk-stream-mixin.js
	node_modules/@smithy/node-http-handler/dist-cjs/index.js
	node_modules/@smithy/node-http-handler/dist-cjs/node-http2-handler.js

thheller07:08:30

that package is absolut horror and I do not recommend using it with shadow-cljs

thheller07:08:44

also don't follow a react-native guide when you are building a webapp?

anthony-galea07:08:04

thanks for the link @U05224H0W I’ll try that

Dustin Getz14:08:22

I've made progress on the guava conflict in certain builds. In my case (every case?) it appears to be a conflict with tools.build. Thread

Dustin Getz14:08:11

Summary of issue: java.lang.NoClassDefFoundError: com/google/common/collect/Streams

Dustin Getz14:08:45

Relevant classpath trace:

Dustin Getz14:08:16

So, the conflict is between

io.github.clojure/tools.build 0.9.5
  . org.clojure/tools.deps 0.18.1354
    . com.google.guava/guava 31.1-android :newer-version
and
thheller/shadow-cljs 2.25.2
  X org.clojure/clojurescript 1.11.60 :use-top
  . com.google.javascript/closure-compiler-unshaded v20230411
    X com.google.guava/guava 31.0.1-jre :superseded

Dustin Getz14:08:52

In the past, I have worked around by including com.google.guava/guava {:mvn/version "31.1-jre"} in project deps

Dustin Getz14:08:49

However, this also works, and seems to be the better fix: io.github.clojure/tools.build {:mvn/version "0.9.5" :exclusions [com.google.guava/guava #_ 31.1-android]}

Dustin Getz14:08:28

Commenting here because I've read all the historical threads and don't recall anyone getting this far. Maybe I have made a mistake

thheller14:08:13

well the solutions vary because the source of guava differs

👍 2
thheller14:08:38

but I have advised with solution many times in many threads as that is a rather frequent problem

👍 2
Dustin Getz14:08:46

For the record, the theller guidance is to shell out to shadow to limit classpath contaminataion

thheller14:08:17

I still don't know why none android libs depends on the guava android variant

👍 2
Dustin Getz14:08:46

this one looks to me like a mistake in tools.build

thheller14:08:56

I have also suggested pinning the specific version in your own deps or adding exclusions, but yes best option is to keep the classpath clean

thheller14:08:17

pretty sure it isn't tools.build depending on it directly, rather some maven lib

Dustin Getz14:08:37

I can't tell yet if clean classpath is a real solution because, due to clojurescript macroexpansion, shadow requires app classpath to build

thheller14:08:03

correct, but that likely doesn't need tools.build? 😛

Dustin Getz14:08:07

what the fuck https://github.com/clojure/tools.deps/blob/master/deps.edn#L14 com.google.guava/guava {:mvn/version "31.1-android"} ;; update transitive dep due to CVE-2020-8908 https://nvd.nist.gov/vuln/detail/CVE-2020-8908

thheller14:08:25

oh yikes it does

thheller14:08:36

and the reason I guess

thheller14:08:27

I wonder if its an acceptable fix to just have shadow-cljs depend on a newer guava version again 😛

💡 2
Dustin Getz14:08:58

that CVE is years old, the override can likely be removed, and it's unclear if the override was ever correct

thheller14:08:14

it used to when the closure compiler didn't declare it correctly, but thats not really a fix as long as a android variant is arround

👀 2
thheller14:08:40

just checked https://mvnrepository.com/artifact/org.apache.maven/maven-core/3.9.4 and it still has a bunch of CVEs, not sure if they are the same though

Dustin Getz14:08:43

afaict shadow depends on the correct guava version right now, shadow is doing the right thing

thheller14:08:12

shadow currently doesn't depend on it, the closure compiler does

thheller14:08:24

but with an "older" version, thus getting replaced by the android variant

Dustin Getz14:08:32

thheller/shadow-cljs 2.25.2
  X org.clojure/clojurescript 1.11.60 :use-top
  . com.google.javascript/closure-compiler-unshaded v20230411
    X com.google.guava/guava 31.0.1-jre :superseded

Dustin Getz14:08:38

copy paste error

thheller14:08:30

guava is the #1 dependency conflict source by a very wide margin, like 99%

Dustin Getz14:08:08

Now I understand - yes, if shadow depends directly on an upgraded guava (to a backwards compat version that is newer) - it would take precedence

thheller14:08:12

so might make sense for shadow-cljs to force a known compatible version that "defeats" the common known android variants

👍 2
thheller14:08:53

datomic also brings an incompatible guava, so that was the other common source

👍 2
jrwdunham15:08:41

Hi shadow-cljs folks. Looking for some help building a release for my browser-targeting app. Running npm run release works locally but not on my server. I have been able to reproduce this with https://github.com/minimal-xyz/minimal-shadow-cljs-browser. As shown below, it works on local but fails on server. Any thoughts to help me debug this? Thanks! Local machine (macbook):

$ npm install
  $ npm run release
  [:app] Compiling ...
  [:app] Build completed. (46 files, 1 compiled, 0 warnings, 12.35s)
  $ tree target/
  target/
  ├── index.html
  ├── main.js
  └── manifest.edn
Server (ubuntu):
$ npm install
  $ npm run release
  [:app] Compiling ...
  $ tree target/
  target/
  └── index.html

thheller15:08:54

looks like it is getting killed by OOM killer

thheller15:08:37

should be something in your system looks about the process getting killed?

jrwdunham15:08:08

Yeah, that looks like it's the issue. /var/log/kern.log is showing plenty of Out of memory: Killed process 6432 (java)

thheller15:08:46

is your server a virtual machine? which java version do you use?

jrwdunham15:08:43

yeah, it's the minimal DO droplet; java 11.0.20

jrwdunham15:08:53

tried adding :jvm-opts ["-Xmx512M"] to [:builds :app]

thheller15:08:29

that is not a valid option in the build config itself, only at the shadow-cljs.edn root

thheller15:08:05

try upgrading the java version, that might also just fix it

thheller15:08:19

11 still had the bug of allocation memory based on host not vm

jrwdunham15:08:22

thanks, moved jvm-opts to root and upgraded java to 18

jrwdunham15:08:02

Still failing due to OOM, using

{:source-paths ["src"]
 :dependencies []
 :dev-http {8080 "target/"}
 :jvm-opts ["-Xmx256M"]
 :builds {:app {:output-dir "target/"
                :asset-path "."
                :target :browser
                :modules {:main {:init-fn app.main/main!}}}}}

thheller15:08:44

why compile on the server in the first place?

thheller15:08:01

compile locally and rsync the output to the server or something?

thheller15:08:29

how much memory do you have? I mean 256M is already probably too low

jrwdunham15:08:35

I want this compile to happen as part of CI. I guess I could build on CI and then rsync the result to the server ...

thheller15:08:31

well the CI already probably uses 50% of that RAM for itself, so you are kinda fighting the limited resources a bit 😛

jrwdunham15:08:14

sorry, I meant a CI service (I'm using circleci)

thheller15:08:24

ah, sure that works. also a known OOM killer though 😉

jrwdunham16:08:02

ok, resizing the droplet to 4G memory did the trick! Thanks for the advice here