Fork me on GitHub
#shadow-cljs
<
2022-01-13
>
tony.kay00:01:24

Any idea what this means?

Multiple files failed to compile.
aborted par-compile, [:shadow.build.classpath/resource "dataico/model/bank.cljc"] still waiting for #{dataico.model.payment}
I’m getting these in my CI system, but the compile is fine locally. No other info. The exception does not show a cause.

tony.kay00:01:41

I’m guessing perhaps RAM?

thheller06:01:04

@tony.kay compilation aborts if no progress is made for 60 seconds. par-compile is the parallel compile and threads wait if dependencies they need are still compiling. after 60sec they abort since that usually means something went wrong and compilation cannot finish. is there something in dataico.model.payment that might cause a huge wait time? eg. some macro expanding into something gigantic or just taking very long?

thheller06:01:41

you can set :build-options {:par-timeout 120000} to increase the timeout to 2min (its msec) but unlikely that changes much unless you CI system is running on a toaster 😉

tony.kay16:01:46

I run parallel builds and tests on a 16GB/4CPU node, but in docker containers. Was using 2G/4G ram/swap limits on those nodes, so it is possible that was just too little ram to allocate to the build for CLJS. I upped it some, and we’ll see how it goes.

tony.kay16:01:08

Thanks a lot for the response. Now that I understand what the error means I can handle it 😄

tony.kay06:01:52

It's a large build, and the runner is a bit taxed. 60s seems like a long time with no progress. It fails often, but not always. I'll try the larger timeout.

thheller06:01:28

note that "progress" is counted based on top-level forms processed. so there must be a single form taking more than 60seconds to compile?

tony.kay16:01:12

Wow…that’s kind of hard to believe. Wonder if I’m thrashing the CI node

simongray14:01:26

trying to set up shadow-cljs for an existing project and I keep getting this error when starting the watch process:

[:app] Build failure:
------ ERROR -------------------------------------------------------------------
 File: jar:file:/Users/rqf595/.m2/repository/thheller/shadow-cljs/2.16.12/shadow-cljs-2.16.12.jar!/shadow/remote/runtime/obj_support.cljc:1:1
--------------------------------------------------------------------------------

   1 | (ns shadow.remote.runtime.obj-support
-------^------------------------------------------------------------------------
Invalid :refer, var cljs.repl/error->str does not exist
--------------------------------------------------------------------------------
   2 |   (:require
   3 |     [clojure.datafy :as d]
   4 |     [clojure.pprint :refer (pprint)]
   5 |     [clojure.spec.alpha :as spec]
--------------------------------------------------------------------------------

simongray14:01:05

Otherwise no other errors reported. The only other annoyance is about a novel’s worth of text in my terminal telling me [2022-01-13 15:53:12.679 - INFO] duplicate resource goog/....

thheller18:01:39

this is coming from one of you dependencies I guess. since you truncated the error message I cannot say more about that

thheller18:01:42

run shadow-cljs clj-repl and ( "cljs/repl.cljs"). I'm guessing that is not yielding the expected clojurescript jar

simongray08:01:56

That gives me

shadow-cljs - HTTP server available at 
shadow-cljs - HTTP server available at 
shadow-cljs - server version: 2.16.12 running at 
shadow-cljs - nREPL server started on port 57233
shadow-cljs - REPL - see (help)
To quit, type: :repl/quit
shadow.user=> ( "cljs/repl.cljs")
#object[java.net.URL 0x418341f9 "jar:file:/Users/rqf595/.m2/repository/com/owoga/tightly-packed-trie/0.2.3/tightly-packed-trie-0.2.3.jar!/cljs/repl.cljs"]
shadow.user=> 

simongray08:01:39

And a novel’s worth of

[2022-01-14 09:31:54.403 - INFO] duplicate resource goog/mochikit/async/deferred.js on classpath, using jar:file:/Users/rqf595/.m2/repository/com/owoga/tightly-packed-trie/0.2.3/tightly-packed-trie-0.2.3.jar!/goog/mochikit/async/deferred.js over jar:file:/Users/rqf595/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20211011-0726fdeb/google-closure-library-third-party-0.0-20211011-0726fdeb.jar!/goog/mochikit/async/deferred.js
and similar messages before it.

simongray08:01:10

So I wonder if this means that tightly-packed-trie is overruling my ClojureScript version? Shouldn’t it be using the most recent version when there’s a conflict…?

thheller11:01:30

all this means is that tightly-packed-trie is actually itself containing the cljs sources and others it shouldn't contain

thheller11:01:26

doesn't matter what other cljs version you specify. classpath only picks the first one

thheller11:01:21

this also has nothing to do with dependency resolution or conflicts. the jar contains files it should not contain that is all.

simongray11:01:32

Well, that’s annoying :S

simongray11:01:19

thanks a lot for clearing that up for me. You don’t happen to know if there’s a way to exclude those CLJS sources?

thheller11:01:46

this also has nothing to do with exclusions. this can only be fixed by the library itself cleaning up the release

thheller11:01:11

who knows what else it contains

simongray11:01:51

damnit. I’ll work around it then, gotta find some other trie implementation.

simongray12:01:02

Thanks a lot, Thomas. I really appreciate it.

thheller12:01:37

why are you activating the :backend alias in your shadow-cljs.edn? :deps {:aliases [:frontend :backend :test]}?

thheller12:01:51

if you remove that the lib won't be on the classpath so the problem will be gone?

simongray12:01:01

I have a custom handler

simongray12:01:21

located in the backend code

thheller12:01:03

why not run the backend separately? thats what I typically recommend because of issues such as this

simongray12:01:04

I want to make a progressive app, so the integration is sort of the point.

simongray12:01:04

I’ll work around it, doesn’t have to be that trie implementation. 🙂 I can use some other or make my own.

simongray12:01:23

Nice! Well, hopefully they fix it and I can proceed as before. I just commented out the trie code for now and everything’s working fine!

simongray14:01:41

Not really sure where to go from here since the error is reported to originate from within shadow-cljs itself :S

Aron15:01:10

which version?

Aron15:01:23

weird, I saw something similar long time ago, but I don't remember what did I wrong

simongray15:01:46

yeah, also pretty much at a loss here… basically just copied in my shadow-cljs setup from a different project, trying to hook up the paths with new namespaces…

Kurt Sys15:01:48

Translating fs.readFile to cljs seems to be not as straight forward (I think I'm missing something):

(ns ...
  (:require ...
            ["fs" :as fs]))

(fs/readFile ...)
doesn't seem to work: react-dom.development.js:11341 Uncaught TypeError: shadow$empty.readFile is not a function What am I missing? (I'm using shadow-cljs, obviously and I'm using cljs with a browser target)

Aron15:01:45

browser doesn't have fs, that's nodejs api

Kurt Sys17:01:50

I'm trying to translate this to cljs (https://github.com/alonrbar/easy-template-x#template-handler-options )

import { TemplateHandler } from 'easy-template-x';

const response = await fetch('');
const templateFile = await response.blob();

const data = {
    posts: [
        { author: 'Alon Bar', text: 'Very important\ntext here!' },
        { author: 'Alon Bar', text: 'Forgot to mention that...' }
    ]
};

const handler = new TemplateHandler();
const doc = await handler.process(templateFile, data);
Somehow, whenever I fetch a docx template, it seems it's not valid docx.
(-> (fetch/get filePath {:headers {:accept "application/vnd.openxmlformats-officedocument.wordprocessingml.template"} })
        (.then (fn [response] (:body response)))
        (.then (fn [template]
                (.process templateHandler template #js{}) ))
results in: Uncaught (in promise) Error: Malformed file detected. Make sure the file is a valid docx file. It's a fresh docx file. I suppose there's something wrong with fetching and reading it. It doesn't matter how, but did anyone ever generate a docx in-browser (from a template)?

Aron17:01:11

have you tried debugging? like you can just step through the code in the browser and see what the problem is

Kurt Sys17:01:38

well, the blob is not recognized as a valid zip file. The Zip.load line fails. It fails when I fetch it first, so there must be something in making the response a valid zip.

async loadDocx(file) {
    // load the zip file
    let zip;

    try {
      zip = await Zip.load(file);
    } catch (_unused) {
      throw new MalformedFileError('docx');
    } // load the docx file

Kurt Sys17:01:56

well, solved it by just using js/fetch:

(-> (js/fetch filePath)
        (.then (fn [response]
                 (.blob response)))

thheller17:01:23

@kurt.sys you need to use JS interop for many things. (.then (fn [response] (:body response))) this doesn't work because response is a JS object and keyword looks only work for clojure maps

Kurt Sys17:01:02

Yeah, well, the (:body response) returned something, but not something valid 🙂 . Just plain js interop seems to work fine.

Derek17:01:20

Are you using lambdaisland’s fetch there?

Kurt Sys17:01:10

Yeah, I was, but I removed it and just use plain js/fetch

thheller17:01:38

ah that was a library. though it was just fetch. then I don't know what response is

Derek17:01:45

Same, but it explains why :body returned something

Kurt Sys17:01:02

Well, plain js/fetch works, with the lib, the body doesn't seem to have a valid response, and .blob doesn't exist in the library.