Fork me on GitHub
#shadow-cljs
<
2018-03-14
>
mhuebert09:03:18

weird edge case I just found: I had a .cljc namespace which used defmacro, but did not include a (:require-macros [...self...]) in the ns form. this was in a utility library I used in Maria. Whenever I would compile the ‘browser’ build and the ‘bootstrap’ build together, this still worked as expected, so I had no idea I had even made a mistake. But if I would compile the ‘browser’ build independently from the ‘bootstrap’ build, I wouldn’t get a warning, but cljs would treat the defmacro as a function and return the unevaluated form.

mhuebert09:03:00

It only showed up when I split these builds for circleci (I’m using tools.deps now, and that increases memory usage, so I can’t compile all the builds in one step anymore)

thheller09:03:08

why does tools.deps increase memory usage?

mhuebert09:03:28

the good news is that you get a proper exception on circle when it fails

thheller09:03:49

ah right you limited the memory via jvm-opts. those are probably not yet passed onto tools.deps

thheller09:03:24

I don't know about the defmacro thing. one build should not affect the other since they are completely isolated

mhuebert09:03:25

[:bootstrap] Compiling ...[:live] Compiling ...[:trusted] Compiling ...


/usr/local/bin/clojure: line 342:   316 Killed                  "$JAVA_CMD" "${jvm_cache_opts[@]}" "${jvm_opts[@]}" -classpath "$cp" clojure.main "${main_cache_opts[@]}" "$@"
Exited with code 137

Hint: Exit code 137 typically means the process is killed because it was running out of memory
Hint: Check if you can optimize the memory usage in your app
Hint: Max memory usage of this container is 4287946752
 according to /sys/fs/cgroup/memory/memory.max_usage_in_bytes

mhuebert09:03:54

I really have no idea, but I could reproduce it

mhuebert09:03:05

locally and on the server

mhuebert09:03:09

after I finally figured out what was happening

thheller09:03:17

well they do share the same clojure vm instance. so if one defines a var the other can see it

thheller09:03:47

but both builds should always check if the defmacro is properly required

thheller09:03:30

defmacro in .cljc files without being hidden in a conditional is scarily common

thheller09:03:29

ah right ... the warning does only check if the var exists

thheller09:03:36

but the var exists

mhuebert09:03:52

the macro is in util, a cljc file, and I was requiring it from another cljc file

thheller09:03:54

so it doesn't warn

mhuebert09:03:52

so basically macros in cljc files should always be in a #?(:clj …)

thheller09:03:58

so if you (util/the-thing ...) it just calls it as a function

thheller09:03:20

yes, unless you care about self-host in which case I don't have an easy answer for oyu

mhuebert09:03:35

i think that’s fine with self host?

thheller09:03:50

no self host reads .cljc files with :cljs

mhuebert09:03:35

macrovich is probably the answer

thheller09:03:51

not really. same problem.

mhuebert09:03:03

(defmacro deftime
  "This block will only be evaluated at the correct time for macro definition, at other times its content
   are removed.
   For Clojure it always behaves like a `do` block.
   For Clojurescript/JVM the block is only visible to Clojure.
   For self-hosted Clojurescript the block is only visible when defining macros in the pseudo-namespace."
  [& body]
  (when #?(:clj (not (:ns &env)) :cljs (re-matches #".*\$macros" (name (ns-name *ns*))))
    `(do ~@body)))

mhuebert09:03:24

would that not work?

thheller09:03:46

ah right. its not doing this via read-cond

thheller09:03:49

so yes that works

mhuebert09:03:37

incidentally this util macro itself uses macrovich, to use keyword-identical? for cljs and identical? for clj

thheller09:03:01

does your build actually work when running both builds? or does it just not complain?

mhuebert09:03:31

expands the macro

thheller09:03:44

ah right thats the cljs compiler doing that

thheller09:03:45

I don't think I can do anything here

thheller09:03:10

I don't know if you meant to call a fn or a macro when doing (util/the-thing ...)

thheller09:03:22

the var exists because defmacro creates it

thheller09:03:43

I can probably complain when you defmacro in cljs though

thheller10:03:11

thats just never correct

mhuebert10:03:18

except in self-host?

thheller10:03:45

well I can just ignore it when compiling normal cljs

thheller10:03:19

I mean just don't compile it. so you'd at least get a undeclared var warning

mhuebert10:03:01

because you know when you are compiling normal cljs vs. bootstrapped-macro-mode

thheller10:03:55

------ WARNING #1 --------------------------------------------------------------
 File: C:\Users\thheller\code\shadow-cljs\src\dev\demo\browser.cljs:24:2
--------------------------------------------------------------------------------
  21 | (defmacro dummy-macro [& body]
  22 |   (prn :foo))
  23 | 
  24 | (dummy-macro 1 2 3)
--------^-----------------------------------------------------------------------
 Use of undeclared Var demo.browser/dummy-macro
--------------------------------------------------------------------------------

thheller10:03:15

makes sense I guess although the warning is a little bit weird since the defmacro is right there 😛

thheller10:03:02

kinda weird CLJS doesn't already do this by default

mhuebert10:03:56

yeah that looks good

mhuebert10:03:09

would have saved me a couple hours of debugging

mhuebert10:03:58

it’s the non-locality that made it so weird. not requiring a clj namespace in one part of the project suddenly means that a cljs namespace somewhere else treats a macro like a function

thheller10:03:11

yeah it is definitely a weird issue. I blame https://dev.clojure.org/jira/browse/CLJS-2454 for not properly checking if macros are actually referred and blindly using them. defmacro is also to blame though.

mhuebert10:03:46

yes i added a vote to that issue

thheller10:03:44

I wonder if there a legit reasons for keeping the defmacro but I can't think of any

thheller10:03:17

@mhuebert just pushed [email protected]. it probably breaks all the self-host stuff. please test.

thheller10:03:12

I mean it shouldn't break anything but self-host has proven me wrong more than once

mhuebert10:03:27

anything in particular to look out for?

thheller10:03:51

compile warnings I guess

mhuebert10:03:18

all looks good

mhuebert10:03:32

just the usual:

------ WARNING #1 --------------------------------------------------------------
 File: cljs/spec/test/alpha$macros.cljc:113:35
--------------------------------------------------------------------------------
 110 |   ([xs]
 111 |    `(instrument ~xs nil))
 112 |   ([sym-or-syms opts]
 113 |    (let [syms (sym-or-syms->syms (eval sym-or-syms))
-----------------------------------------^--------------------------------------
 Use of undeclared Var cljs.spec.test.alpha$macros/eval
--------------------------------------------------------------------------------
 114 |          opts-sym (gensym "opts")]
 115 |      `(let [~opts-sym ~opts]
 116 |         (reduce
 117 |           (fn [ret# [_# f#]]
---------------------------------------

mhuebert10:03:46

can they not declare that?

thheller10:03:23

is that a known issue? I mean this doesn't work right?

mhuebert10:03:45

they set! eval later

mhuebert10:03:14

is what is said on a slack log somewhere

thheller10:03:31

no idea what that means

thheller10:03:42

can't find where eval is set!

thheller10:03:16

@fbielejec I think cider-jack-in starts a new lein process, so it won't find the one started by shadow-cljs watch

thheller10:03:37

you can instead start the embedded version in the cider repl

fbielejec10:03:54

right, it's equivalent to lein repl basically, starts an nrepl session

thheller11:03:06

(require '[shadow.cljs.devtools.server :as server])
(server/start!)
(require '[shadow.cljs.devtools.api :as shadow])
(shadow/watch :foo)
(shadow/nrepl-select :foo)

fbielejec11:03:20

can I invoke a watcher like this too? e.g.

(shadow.cljs.devtools.server/start!)
      (shadow.cljs.devtools.api/watch :dev)
      (shadow.cljs.devtools.api/nrepl-select :dev)

fbielejec11:03:35

wrote at the same time, nevermind

thheller11:03:19

btw :dev is a built-in concept of shadow-cljs. every build has a dev and release mode

fbielejec11:03:42

so I better not call the build-id :dev?

thheller11:03:59

you can call it whatever but it doesn't make much sense yes

thheller11:03:11

in figwheel&co you actually have a different build for release

thheller11:03:21

in shadow-cljs its just shadow-cljs release the-build or shadow-cljs watch the-build

thheller11:03:41

I need to document this better, I see this a lot 🙂

fbielejec11:03:43

it's just scattered a bit, but the fact I can get npm packages working out of the box is worth spending some time searching 🙂 It's awesome.

fbielejec11:03:43

Ok, so I killed the watcher in the terminal, create a user ns with these invocations

fbielejec11:03:47

shadow-cljs - HTTP server for :app available at 
shadow-cljs - server running at 
[:app] Configuring build.
[:app] Compiling ...
[:app] Build failure:
The required namespace "cljs-0x-connect.dev" is not available.

To quit, type: :cljs/quit
[:selected :app]
mar 14, 2018 12:02:49 PM clojure.tools.logging$eval8628$fn__8631 invoke
WARNING: stale websocket client, no worker for build :dev

fbielejec11:03:11

but I do get the build served in the browser

fbielejec11:03:22

no cljs REPL though

thheller11:03:27

did you use :lein true in your shadow-cljs.edn?

thheller11:03:36

the http server is independent of the build

fbielejec11:03:55

yes, it''s probably a stale version served

fbielejec11:03:26

yes I have :lein true.

fbielejec11:03:43

What's curious is that it keeps me in clj REPL

thheller11:03:10

wondering why you get the build failure? did it work when running shadow-cljs watch?

fbielejec11:03:44

me too, when running the same build using CLI tools it works as expected

fbielejec11:03:12

source paths maybe

thheller11:03:32

please don't ever clean the .shadow-cljs dir. :clean-targets ^{:protect false} ["target" "public/js" "node_modules" ".shadow-cljs"]

thheller11:03:03

cache actually works in shadow-cljs so there is no reason to do it ever.

fbielejec11:03:37

gotcha. Ok so the build failure happened because "src" was not in :source-paths ["src" "dev"]

fbielejec11:03:39

I incorrectly assumed it is handled by the shadow-cljs, since I specify the :source-paths ["src"] there too

thheller11:03:08

as soon as you set :lein :dependencies and :source-paths are managed by lein and no longer mean anything in shadow-cljs.edn

thheller11:03:44

I should really start writing a proper migration guide

fbielejec11:03:58

ok so very close now:

user> (start-dev!)
mar 14, 2018 12:17:48 PM org.xnio.Xnio <clinit>
INFO: XNIO version 3.3.8.Final
mar 14, 2018 12:17:48 PM org.xnio.nio.NioXnio <clinit>
INFO: XNIO NIO Implementation Version 3.3.8.Final
shadow-cljs - HTTP server for :app available at 
shadow-cljs - server running at 
[:app] Configuring build.
[:app] Compiling ...
[:app] Build completed. (226 files, 124 compiled, 0 warnings, 29,07s)
To quit, type: :cljs/quit
[:selected :app]
user> JS runtime connected.

fbielejec11:03:23

only problem is I'm kept in clj REPL in the user namespace, no cljs REPL

thheller11:03:14

@mitchelkuijpers has cider experience. I don't really know anything about cider

thheller11:03:35

I think you are supposed to call something from cider to make the switch

thheller11:03:38

but I don't know what

fbielejec11:03:44

ok I'll experiment with this, probably a middleware problem

fbielejec11:03:51

many thanks for your time and help!

thheller11:03:34

@fbielejec if you figure it out let me know please. want to finally have something I can put into the docs. 🙂

fbielejec11:03:20

I will, off course. Btw I notice the same behaviour (no cljs REPL) when just calling lein repl and then starting the server, watcher and repl using the API.

thheller11:03:48

I think you can't purely do this via clojure

thheller11:03:05

something in cider is supposed to clone the session so you have one for clj and one for cljs

fbielejec11:03:20

you probably mean cider-create-sibling-cljs-repl. That could work, I'll try.

thheller11:03:29

just replace figwheel with the shadow-cljs stuff and it should work

mitchelkuijpers14:03:55

@fbielejec did you figure it out?

fbielejec14:03:53

hey - yes, I'll be writing up a quick how-to post, should be done by evening

fbielejec14:03:05

Unless it's urgent, then I'll paste the instructions here 🙂

mitchelkuijpers14:03:20

No, we have been working with cider for 2 months now 😛

fbielejec14:03:56

then you should be doing the writing I guess 😄

mitchelkuijpers14:03:07

Yes you are completely right

justinlee21:03:12

is the emacs/cider integration fully functional with a cljs nrepl? i.e., does it allow you to send forms, get documentation strings, do tab completion, find definitions, stuff like that?

justinlee21:03:25

that is the worst news ever: I really do not want to go back to emacs 🙂

fbielejec21:03:15

@lee.justin.m fairly certain the same setup should work for Cursive too

justinlee21:03:38

probably. i use atom. i tried to get cursive installed and it was super confusing.

justinlee21:03:31

i guess cursive would be better than emacs so i should give it another shot

fbielejec21:03:26

nothing's better than Emacs 😁

wilkerlucio21:03:53

expect Cursive troll

justinlee21:03:55

i spent 15 years on emacs and i don’t want to go back

fbielejec21:03:22

that's the minimum time span one should spend before switching editors 😛. It's really paramount for productiveness imho, whatever the choice might be in the end.

wilkerlucio21:03:03

agreed, I spent almost 8 years on Vim before moving to intellij stuff, I used to spend around 10 ~ 20% of my time tweeking my editor, in Intellij I don't have as much customizations, instead I just use it as is mostly (except for snippets and some repl commands) and I'm very happy with not having to configure things all the time

thheller21:03:05

I'm very happy with Cursive for the time being. Never want to back to emacs as well.

justinlee22:03:59

It asked if I wanted to index the remote repositories and I said yes. I’m worried i just told cursive to go index everything on maven and clojars.

thheller22:03:11

I honestly don't know what that is about but it eventually finishes 😉

wilkerlucio22:03:30

the clojars index in a decent time, the maven can take forever

justinlee22:03:39

so do i need to do anything special to get an nrepl to connect to shadow? I put in the localhost/9000 (which is what my nrepl is configured to), it just hangs on `Connecting to remote nREPL server... Clojure 1.9.0`. i confirm that it works if i connect using the terminal.

thheller22:03:07

it doesn't print anything else. you should be connected. just eval something to test

justinlee22:03:58

that still doesn’t work.

Connecting to remote nREPL server...
Clojure 1.9.0
(js/console.log "HELLO")
CompilerException java.lang.RuntimeException: No such namespace: js, compiling:(null:1:1) 
(+ 1 1)
CompilerException java.lang.RuntimeException: Unable to resolve symbol: + in this context, compiling:(null:1:1) 

thheller22:03:18

its a clojure repl

thheller22:03:33

no idea about the null though

thheller22:03:48

Connecting to remote nREPL server...
Clojure 1.9.0
(+ 1 2)
=> 3