Fork me on GitHub
#shadow-cljs
<
2020-01-17
>
samedhi01:01:48

Hello. I am attempting to write a macro in cljs using shadow-cljs. https://gist.github.com/samedhi/f8c6aa8eb5ca435ffac808da27e85978 . I have sidestepped most of the issues with referring to clojurescript ns by using their full names rather than using :as and a symbols to the respective ns. However, I am just not sure how to refer to a a NPM package "firebase-admin" from within my cljs macro. Two questions: 1. Is there a better way of referring to cljs ns from my macro? Some seem to work and some don't, quick summary on why that is? 2. How do I refer to node packages from a clojure macro? In my case I am attempting to use ["firebase-admin" :as admin], but that does not work within my clj macro namespace.

samedhi01:01:29

End goal is to have a testing helper that lets me write expressions like the following.

(deftest-function create-example-with-macro-test
  sut/create-example-cloudfx
  [doc1 "messages/message-1" {"foo" "bar"}
   doc2 "messages/message-2" {"foo" "baz"}]
  (t/is (not= (.get doc1 "foo") (.get doc1 "foo")))
  (t/is (not= "bar" (.get doc1 "foo")))
  (t/is (not= "baz" (.get doc2 "foo")))
  ;; And presumably also testing some side effect of the sut/create-example-cloudfx running
  )

thheller09:01:29

@samedhi you can't reference JS namespaces directly in macros without using resolve-var. the common thing I recommend is doing something like (ns foo.bar (:require ["thing" :as x])) (def thing-you-need-in-macro x/foo)

thheller09:01:41

and then using foo.bar/thing-you-need-in-macro in the actual macro

thheller09:01:40

then you have the place where to put it right there

thheller09:01:28

also very little of this macro would actually need to be written by this macro

thheller09:01:58

I'd advise moving most of it out into a helper function and only use the macro to generate the boilerplate needed

darwin12:01:40

@thheller just by chance do you remember how you added support for :default keyword into your ns analyze phase? as documented here: https://shadow-cljs.github.io/docs/UsersGuide.html#_using_npm_packages

darwin12:01:59

cljs compiler’s analyze is choking on it and complains about it

darwin12:01:18

I tried to look into shadow-cljs sources, but I don’t see any explicit mention how it is handled

darwin12:01:05

when I try to pass shadow-cljs ns-form with :default into vanilla cljs analyze it gives me “Only :as, :refer and :rename options supported in :require / :require-macros…”

darwin12:01:49

ok, thanks, are you aware of anything else like :default which would cause error in ana/analyze in normal cljs?

darwin12:01:14

if it is just :default, I can easily preprocess ns-forms before feeding them into cljs analyze

thheller12:01:41

thats the only non-standard thing yes

darwin12:01:52

ok, great, thanks

bbss13:01:50

I'm messing about with yarn link to test some third party library feature that's not pushed to npm yet. Am I right to conclude shadow-cljs doesn't deal with links?

thheller13:01:40

dunno what yarn link does exactly. symlinks are fine

bbss13:01:05

I think they create symlinks, but have some extra registry in ~/.config/yarn/...

bbss13:01:20

will try my hand at a manual symlink then 🙂

bbss13:01:48

made a symbolic link to the directory node_modules but still getting

Search in:
	/Users/baruchberger/work/gsv/node_modules
You probably need to run:
  npm install 

See: 

thheller14:01:38

so does /Users/baruchberger/work/gsv/node_modules/deck.gl/package.json exist or not?

thheller14:01:00

I think http://deck.gl was one of those packages the closure compiler doesn't like? its from uber right?

simongray14:01:15

If I want to develop a library in shadow-cljs that uses an NPM package, is there simply no way to distribute this as a jar that can be used in a non-shadow-cljs project?

simongray14:01:35

The official documentation is a bit lacking in this area

simongray14:01:37

> Please note that currently only `shadow-cljs` has a clean automatic interop story with `npm`. That may represent a problem for users of your libraries using other tools. You may want to consider providing a CLJSJS fallback and/or publishing extra documentation for `webpack` related workflows.

simongray14:01:01

So what does this mean in practice? Either I create a CLJSJS package in which case the business value of shadow-cljs diminishes or I distribute some kind of setup instructions with the library?

simongray14:01:19

What would that "extra" documentation look like, though?

bbss15:01:46

Yes, it's from uber, huge repo. https://github.com/uber/deck.gl/blob/master/package.json they seem to work with lerna for sub modules.

bbss15:01:22

Either way no issue if this isn't supported for now 🙂.

wilkerlucio15:01:29

@simongray there is a trick, you can add a deps.cljs in your source root, and there you can declare NPM deps, example: https://github.com/wilkerlucio/pathom-viz/blob/master/src/core/deps.cljs

wilkerlucio15:01:00

them, if you use the CLJS library with Shadow, it will automatically install those deps

wilkerlucio15:01:01

oh, sorry, just noticed you asked for non-shadow, in that case will be hard considering that no other cljs build tools have proper support for NPM directly

simongray15:01:35

Exactly. Hm, I think will investigate making a CLJSJS package instead then and probably just stick with figwheel.

simongray15:01:49

I was mainly interested in shadow-cljs for its NPM support.

simongray15:01:36

BTW I just saw your Conj talk last week! Really cool stuff :)

🙏 4
thheller15:01:45

@simongray yeah you pretty much have to create a matching cljsjs package. if you configure things correctly shadow-cljs will use npm directly and the others will fall back to cljsjs

thheller15:01:16

or you can skip the cljsjs and provide manual instructions for people using webpack+cljs

thheller15:01:18

I don't know how many people actually use webpack+cljs though

simongray15:01:42

Ok. Thanks for taking the time to answer!

darwin16:01:56

aynone seen this error? “Requested module does not have an export <name>” it gets printed afte “Closure compilation failed with 2 errors”

darwin16:01:33

I’m trying to convert react-three-fiber examples into cljs with shadow-cljs, and wanted to require their shader js files as is by following https://shadow-cljs.github.io/docs/UsersGuide.html#_requiring_js (using absolute path)

darwin16:01:05

I put this file[1] on classpath and then require it as ["/resources/shaders/Backface" :as BackfaceMaterial], shadow-cljs then complains ` [1] https://github.com/react-spring/react-three-fiber/blob/master/examples/src/resources/shaders/Backface.js

darwin16:01:51

btw. three.js is properly specified in package.json

thheller17:01:32

.d.ts files are irrelevant. they aren't used in any way.

thheller17:01:44

closure is picky about commonjs -> esm interop

thheller17:01:07

since it doesn't understand the way shadow-cljs processes JS it seems three as a commonjs dependency

thheller17:01:18

thus you can only do import Three from "three"

thheller17:01:29

and then use Three.BackSide in the code

darwin17:01:45

ok thanks, I think I can live with that

darwin17:01:51

let me try it

thheller17:01:04

but .. given that you can't provide type hints in JS for inference it'll likely have problems with :advanced

thheller17:01:25

so you might need to provide externs

darwin17:01:33

btw. in the meanwhile I tried to attach Intellij Java debugger to the shadow-cljs process, I was able set breakpoints and break on them but no debug inspections were possible, it always displayed “Debug information is inconsistent”

darwin17:01:44

not sure how to fix this, google doesn’t help

darwin17:01:17

then I followed your “Hacking” docs and was able to replace some shadow-cljs sources with my copy, so I was able to work around it

thheller17:01:25

no clue. I have never tried that

darwin17:01:45

I wonder if it is not related to the way how you prioritize “The shadow-cljs compiler ensures that things on your source paths are compiled first”

thheller17:01:13

thats regular JVM logic

thheller17:01:37

the classpath is just constructed by listing the source paths first

thheller17:01:55

it works like that is probably all other tools as well

thheller17:01:00

nothing special about that

darwin17:01:23

btw. your suggestion worked, now it got compiled without issues

darwin17:01:59

want to get it working in dev mode first, will be solving advanced mode later

darwin17:01:42

btw. this is my first real shadow-cljs project and I’m impressed with the whole experience. Kudos!

👍 4
darwin18:01:32

I still seem to be unable to make it work. Cannot access Three.something , because it generates Three.default.something instead of Three.something in js

thheller18:01:29

just use require instead of import

thheller18:01:41

the ESM import/export handling in closure is rather strict

thheller18:01:30

commonjs is a lot easier to work with 😛

darwin18:01:38

hmm, const Three = require("three"); results in similar issue,

thheller18:01:47

maybe import * as three from "three" works

thheller18:01:23

hmm looks like closure still recognized the file as ESM

thheller18:01:43

need to replace the export too

thheller18:01:52

or just add module.exports = 'foo'; or so

darwin18:01:58

import * as three from "three" works!

thheller18:01:01

not sure what the detection of closure is looking for

darwin18:01:46

thanks a lot, I wouln’t be able to solve this - I’m unfamiliar with all these details of JS modules

darwin18:01:38

I almost wanted to hack it by storing three object in some global js var in cljs code and access it in the js file

thheller18:01:49

wouldn't that be the same as you have not with import *?

darwin18:01:16

yep, now the generated code looks the same (just decorated to prevent clashes) not sure about advanced mode tough

darwin18:01:00

but it is dirty to go explicitely via globals, so I’m happy that at least one form of import worked 🙂

thheller18:01:11

yeah CLJS -> JS -> npm isn't something I spent much time on so it has rough edges

p-himik19:01:22

I decided to give the AdoptOpenJDK 13 a try and it seems to work for the most part. But there are some bizarre errors that pop up intermittently during compilation. It seems that it always involves clojure.lang.KeywordLookupSite$1. Example 1:

------ WARNING #1 - :invalid-arithmetic ----------------------------------------
 File: /home/p-himik/dev/git/ensemble/src/hgs/strayhorn/named_expressions/views.cljs:26:5
--------------------------------------------------------------------------------
  23 |                                       (cst/split model ",")
  24 |                                       [])))
  25 |         roman-on-change #(on-change (cst/join "," %))]
  26 |     (fn [_ & {:keys [error]}]
-----------^--------------------------------------------------------------------
 cljs.core/unsafe-bit-and, all arguments must be numbers, got [#object[clojure.lang.KeywordLookupSite$1 0xae194d44 "clojure.lang.KeywordLookupSite$1@ae194d44"] number] instead
--------------------------------------------------------------------------------
  27 |       [rc/v-box
  28 |        :gap "10px"
  29 |        :children [[rc/horizontal-bar-tabs
  30 |                    :tab-style tabs-style
--------------------------------------------------------------------------------
Example 2:
------ ERROR -------------------------------------------------------------------
 File: jar:file:/home/p-himik/.m2/repository/com/cognitect/transit-cljs/0.8.256/transit-cljs-0.8.256.jar!/cognitect/transit.cljs:295:6
--------------------------------------------------------------------------------
 292 |   ([tag-fn rep-fn str-rep-fn]
 293 |      (write-handler tag-fn rep-fn str-rep-fn nil))
 294 |   ([tag-fn rep-fn str-rep-fn verbose-handler-fn]
 295 |      (reify
------------^-------------------------------------------------------------------
Syntax error macroexpanding cljs.core/deftype.
ClassCastException: clojure.lang.KeywordLookupSite$1 incompatible with clojure.lang.IFn
	clojure.spec.alpha/spec-impl/reify--2059 (alpha.clj:923)
Example 3:
------ ERROR -------------------------------------------------------------------
 File: /home/p-himik/dev/git/ensemble/src/hgs/ella/score_details/sequences/views.cljs:297:3
--------------------------------------------------------------------------------
 294 |     :children (mapv (fn [w] [rc/label :label w]) issues)]])
 295 | 
 296 | (defn question-panel [_game-id changeset-id q args]
 297 |   (let [{:keys [events subs]} args
---------^----------------------------------------------------------------------
Syntax error macroexpanding cljs.core/nil?.
ClassCastException: clojure.lang.KeywordLookupSite$1 incompatible with clojure.lang.IFn

p-himik19:01:44

Can you make any sense of it?

p-himik19:01:33

Couldn't find anything on the warning about unsafe-bit-and. But the warning about ClassCastException is also mentioned here: https://github.com/thheller/shadow-cljs/issues/443 As well as OpenJ9 (aka AdoptOpenJDK I believe). I do embed shadow-cljs, but I definitely don't interact with the REPL in any way when these issues occur.

sogaiu21:01:24

re: "...as OpenJ9 (aka AdoptOpenJDK..." - iiuc, adoptopenjdk does have openj9 builds, but adoptopenjdk != openj9 adoptopenjdk also provides openjdk builds or may be i misunderstand?

p-himik06:01:31

Thanks, seems like you are correct.

thheller20:01:56

@p-himik not a clue. I've been on openjdk 13 since it came out without issues

mauricio.szabo21:01:15

Yes, in my experience unless it's a simple project, is close to impossible to run shadow-cljs on OpenJ9. These errors happen sometimes, specially when I'm using macros or core.async

thheller21:01:20

never tried openj9. do other clojure apps work? don't know why shadow-cljs would break. its just normal clojure.

mauricio.szabo21:01:10

Yes, everything works fine. I remember that in the past I opened another bug on shadow because of OpenJ9 (something about detecting if a file was stale, if I'm not mistaken)

mauricio.szabo21:01:27

To be honest, I did not try other CLJS implementations like Figwheel, so it can be a ClojureScript compiler issue too...

Eric Ihli21:01:11

I'm curious if anyone would know of this to be a problem.

(defmacro slurp [file]
  (clojure.core/slurp (io/file (io/resource file))))
(ns ezmonic.e-data
  (:require [cljs.reader :as reader])
  (:require-macros [ezmonic.util :refer [slurp]]))

(defonce data
  (reader/read-string (slurp "number-to-word-tree.edn")))
I'm using that to read a 6MB file. Everything works fine in the emulator. But, when I create a release bundle and install it on my phone, the app freezes for 10 seconds when going to and coming from the background. (React Native)

thheller21:01:47

parsing 6MB of EDN is really slow. the emulator is just too fast.

Eric Ihli22:01:18

I would not expect it to have to parse the EDN when going to/from the background though. I would only expect that on startup, no?

thheller22:01:26

maybe you aren't allowed to use that much memory?

thheller22:01:45

I don't do react-native development. maybe try asking in #cljsrn

👍 4
thheller22:01:47

:require-macros is so ugly 😛

thheller22:01:13

6MB of EDN source will be much more are runtime, do you have stats about memory use for the app?

Eric Ihli22:01:16

Good ideas. I'll take a look at the memory, and thanks for the link.