This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2017-07-03
Channels
- # aws (1)
- # beginners (5)
- # boot (46)
- # cider (6)
- # cljs-dev (421)
- # cljsrn (74)
- # clojure (77)
- # clojure-greece (1)
- # clojure-italy (7)
- # clojure-russia (12)
- # clojure-serbia (42)
- # clojure-sg (1)
- # clojure-spec (10)
- # clojure-uk (38)
- # clojurescript (40)
- # core-async (14)
- # data-science (1)
- # datomic (18)
- # emacs (2)
- # events (1)
- # garden (4)
- # gorilla (4)
- # graphql (5)
- # hoplon (69)
- # luminus (1)
- # lumo (1)
- # off-topic (31)
- # om (31)
- # om-next (2)
- # overtone (3)
- # pedestal (1)
- # precept (4)
- # re-frame (23)
- # reagent (2)
- # remote-jobs (1)
- # ring-swagger (23)
- # rum (7)
- # spacemacs (7)
- # sql (4)
- # unrepl (9)
- # untangled (5)
- # vim (11)
- # yada (5)
merged the module stuff to master since I couldn’t find a way to break it here - feel free to do your worst and report back 🙂
random thought - @anmonteiro, @mfikes and I did do some work around registering specs across incremental compiles
the stacktrace didn’t show that
could it be that the exception is swallowed somehow?
yeah I see the try catch
so it would show up somewhere else?
cljs.spec.alpha
does depend on gen
I wonder if that could be it
let me build a version of CLJS with a lock around that require
and see if I can repro
@dnolen > you may see an error about goog.require but this can be ignored
what is this about?
or rather, why does it happen?
also noticed that under :verbose
or :compiler-stats
we don’t output diagnostic information so created & attached a patch to https://dev.clojure.org/jira/browse/CLJS-2154
one more q: does loader/set-loaded!
need to be called at the end of the namespace?
^ it looks like under advanced it doesn’t really matter (except the order in which the code executes will be slightly different - e.g. for toplevel code)
but with :none
we need to call set-loaded!
at the end of the file
I assume because of goog.require
calls?
or like, code will stop being evaluated once you call set-loaded!
?
@dnolen FWIW I made a quick comparison between cljs.loader
and shadow.loader
, the only “significant” difference is that shadow-cljs
covers the set-loaded!
call as well
@thheller no reason other than I wasn’t really interested in writing injection code for it
that said I’m not against it - but it should probably be done via some generic simple feature we don’t currently have to append to the end of a source file like postamble
yeah I have :prepend
:prepend-js
:append-js
:append
per :module
and the loader is done entirely via that
@anmonteiro I haven’t had time to look into the error that comes up in the guide, only happens under browser REPL as far as I could - probably something simple
reviewing the Quick Start I realized the browser REPL section is much simpler now due to :browser-repl true
compiler option (which just converts to a preload)
really what I’m wondering is does anybody actually have a problem with this just always being set to true?
I mean pretty much everyone is targeting browser or Node.js, where setting it to true isn’t an issue
we could probably cover Nashorn / Rhino users via :target
so this being set to true isn’t a problem
So: Re apply
optimization: I'm thinking about doing a specialized implementation for IndexedSeq
, which is by FAR the most common data type to be passed in.... For me. If anybody could run this little snippet in a real world CLJS app, that would be greatly appreciated. Then report back the types:
(def seen-types (atom {}))
(set! js/print_appy_usage #(js/console.log @seen-types))
(let [oapp apply
new-apply (js/Object.assign
(fn []
(let [a (js-arguments)
xs (aget a (dec (alength a)))]
(swap! seen-types update-in
[(alength a)
(cond
(nil? xs) "nil"
(array? xs) "array"
:else (.-cljs$lang$ctorStr (.-constructor xs)))]
(fnil inc 0))
(.apply oapp nil (js-arguments))))
apply)]
(set! apply new-apply))
Just paste the snippet, load the app and then either inspect seen-types
or call print_apply_usage()
in the console.For me, it's:
{3 {"cljs.core/IndexedSeq" 88, "cljs.core/List" 5},
2 {"array" 127,
"cljs.core/IndexedSeq" 1286,
"cljs.core/PersistentVector" 213,
"cljs.core/LazySeq" 416,
"cljs.core/Cons" 1},
4 {"cljs.core/LazySeq" 300}}
@rauh that seems to match my intuition - IndexedSeq is common in apply
scenarios due to wrapping of arrays
@petterik Thanks a lot. That helps. The LazySeq
will get a little faster for higher arity invokes. it'll safe on some first/next calls. But good to see that you're also having a bunch of IndexedSeq
which are super easy to optimize.
@anmonteiro I fixed up those issues from your testing the modules yesterday - thanks for the testing/feedback
@dnolen awesome, I'll give it another spin in the next hour. Can you explain the use case for set-loaded!
now that it's automatically emitted?
@anmonteiro I don’t know but it’s just a hook into the real API so I don’t see a problem with it
Did you also fix the console errors in the browser? I can probably look at those when I get a chance
@anmonteiro yes already fixed
Right, I also couldn't come up with anything. But one thing I wonder is: should we emit that call to set-loaded!
if it's already present (manually called)?
I don’t think it matters, @thheller believes it’s not idempotent but I couldn’t confirm that here
and I can’t think of a way multiple set-loaded!
calls could ever be problem - need an example
it seems to me to make that not idempotent would be unwise, so I suspect this is handled
I don't know exactly what it's doing, but yesterday I had my code evaluating differently based on whether the call was at the top or the bottom of my namespace
I guess if it's idempotent it wouldn't matter even for that case
and I think relying on order of leaf entries in a module doesn’t make much sense to me
but if someone can come up with a real problem case worth worrying about - will consider it
Gotcha
@dnolen I see there's still a mention of the console errors in the gist. Probably confusing now that it's been fixed
Also java -cp cljs.jar clojure.main -m cljs.repl.browser
was an interesting realization
@anmonteiro yeah @mfikes version is neater 🙂
@dnolen :modules {:bar {:entries [bar.*]}}
bar/a.cljs
triggers set-loaded!
but bar/b.cljs
contains (defmethod some-multi :bar [_])
.. user triggers loading of :bar
and calls (some-multi ...)
in callback which isn’t defined since bar/b.cljs
isn’t loaded yet
@thheller right I would not consider that a problem since you did not express a dependency between leaf entries
I’ll test it out when I have more time … I think there is a problem but could be wrong …
relying on side effects in unordered entry points is just not something we are going supoprt
what the heck … bar.a
correctly loads before bar.b
but incorrectly triggers set-loaded
but ok now I see what you mean talking about code here is just making it more confusing 🙂
@dnolen So I was just fixing to file 2 issues with how the browser repl serves files, and then I wanted to ask if a patch would be welcome on it. Then I noticed this issue: https://dev.clojure.org/jira/browse/CLJS-2150 which could solve both of the issues I found.
@potetm well what did you have in mind? I was going to close that ticket actually - I just changed what the browser REPL emits when starting up so it’s not confusing
The issues I found were: 1) the browser repl shouldn't fail when query params are passed (this is useful for cache busting) and 2) that it should be possible to start a browser repl from opening a static file.
So it seemed those could be accomplished by pulling out a proper standalone webserver. But I haven't tried either of those things on latest, so maybe it's fixed?
It's been a minute since I was in the guts of the repl, but IIRC opening an index.html from the filesystem was disallowed by the browser repl. The content had to be served from the repl process.
And IIRC that's not a hard requirement. It's just fallout of how that was implemented.
I think the deal was in send-repl-client-page
. It blindly pulls the host off the request, which doesn't work for filesystem files. This could be fixed by either defaulting to localhost or allowing the user to pass a hostname (or both).
as far as pulling out a proper standalone server - I’m kind of meh about that - what’s there is just a super lighweight thing for quick testing
Fair enough. If nothing else I may file a bug report and patch for allowing query params then.
@thheller after some more thought I do see your point if we were just generating setLoaded
for every entry but we don’t actually do that. Also given how Google Closure Modules works it seems to me the design was always for setLoaded call to be explicitly handled by the user in the ns involved in loading. But that’s just it, in the case of this ClojureScript enhancement you’re only going to get a cljs.loader/set-loaded!
call in a namespace that actually requires cljs.loader
, we don’t do it for any old entry. So my hunch is this is fine for typical use cases.
but time will tell - easy to revisit if people just absolutely want a different behavior
but only the the :modules
that want to load modules will include cljs.loader
, the modules that are loaded don’t have to
> you’re only going to get a cljs.loader/set-loaded!
call in a namespace that actually requires cljs.loader
sorry .. I’m confused trying to juggle work and this … I’ll take some time tomorrow and see if this is an actual problem or just in my head
I just remembered my first :module-loader
impl had issues with calling setLoaded
too early
@dnolen I realized afterward: I blindly copied some of the numeric test that appears to be dealing with sets—I didn't understand that part and it might not be appropriate to copy
Also, realized: the fix might slow some things down given the extra checks in goog.object/get‘ (
simple-benchmark` shows it is an order magnitude slower)
hrm actually immediately, I know that native protocols & string cache will get hit for this
@dnolen no specific ops identified that it could matter. Just observing
(simple-benchmark [o #js {:a 1}]
(aget o "a")
1000000)
Is faster than
(simple-benchmark [o #js {:a 1}]
(goog.object/get o "a")
1000000)
Regarding cljs.loader
. Is there any interest in adding (defn prefetch [module-name])
and/or (defn loaded? [module-name])
?
@dnolen the "set logic" part of the patch I didn't understand (and you suggest is type union stuff):
+ (when #?(:clj (set? t)
+ :cljs (cljs-set? t))
+ (or (contains? t 'array)
+ (contains? t 'any)
+ (contains? t 'js))))))
Okay. Just to be clear: This is about starting a browser repl when you open a file from the filesystem directly. I'm not sure should be a supported use case, because I'm not sure what it provides beyond the use case itself. But the first issue you run into when trying to do this is an invalid ppu
(`PEER_POLL_URI`) field. This is because the hostname is not propagated to repl/start-evaluator
.
the whole point of having the browser REPL start a webserver is to get around all those problems
@mfikes gonna test with string cache - that should give use a accurate picture, we don’t currently test it
No yeah. I think that's a valid call. I just noticed that it could be made to work without a webserver at all. And some other repls support that use case. (Also, note that I was wrong about this involving making a standalone webserver. This is a separate issue that I got tangled in my head.)
@potetm yeah I think that kind of enhancement doesn’t seem necessary - I think the query params stuff is more impactful
Awesome. I'll get that bug filed and see if I can get a patch in. Should be an easy fix, but it's gonna take a minute for me to figure the whole process out.
why make it private though?
I suggest making it “internal”, but not private 🙂
we have some of those cases
@dnolen I could attach a revised "fix" patch that adds an unsafe-get
/ unsafe-set
. If so, an open question: Replace all the detected invalid uses of aget
/ aset
to these new variants, or only the few for which perf matters (with the others using goog.object
as the current "fix" patch does)?
@dnolen just got bitten by adding an entry which doesn’t require cljs.loader
does it make sense to issue a warning if an entry doesn’t require cljs.loader
, or we can’t because of backwards compat?
@anmonteiro I don’t know what there is to do here - you can have modules that don’t need to worry about loading
I might be easier to understand the problem if you explain your expectations a bit and what led to the confusion
gotcha, I guess I just need to understand the use case for modules that don’t need to worry about loading
reagent
isn’t involved in loading - it must be a script tag along with cljs_base on the pages that need it
@dnolen my next question was gonna be about “vendor splitting” but I guess you just answered it 🙂
but that brings up another question: how does page1
get reagent
?
resolve
?
like an empty namespace that just has a bunch of requires?
“pull reagent, react, react-dom into this module”
{:shared {:entries []} :foo {... :depends-on [:shared} :bar {... {:depends-on [:shared]}}}
@anmonteiro no need for empty namespace, just empty module
every input has to be assigned to a module - but we try to keep each compiler input as deep as possible in the module graph
this is pretty cool, just tried it
module manager stuff and a bunch of goog
libs got pulled into that shared thing
yeah this is what JSModuleGraph does, and I just copied it - it’s pretty slick, shadow-cljs has a variant too
@dnolen I’m guessing this warrants a blog post in the new & shiny CLJS blog
or maybe just a guide
or like a series of guides
from beginner to advanced stuff
I’m happy to help writing something but I don’t think I fully grasp the more advanced stuff
I think the guide can use my thing as jumping off point - and have a trick/tips section at the end
so far it looks really good and I haven’t found major problems
but I don’t have a major codebase I can try this on
Allen Rohner’s post was the best written reference and you can’t even get to it anymore 😛
I have a static copy of that post
let me find it
I should probably buy that too
other issue was that :none
and :advanced
were different here for Modules based projects
shadow-cljs did the right here in blurring that distinction as otherwise it’s so cumbersome
^ Allen’s post
I predicted the website would go down someday.. that info was far too valuable to go to waste 🙂
hey yeah and that post more less shows how much work/knowledge it required to do things correctly 😛
I agree that shadow-cljs is probably a bit more “dummy-proof” than my approach - but I just didn’t want to go down the path of code injection or adding stuff to append code generically
@anmonteiro something that would be pretty cool/slick to try - splitting + code motion of NPM modules
I think how Google Closure works is way better than Webpack in this regard - it just needed a easier interface
yeah, I’d love to see that
I think we’re not too far from that working
I think it would even work today with some tricks
@anmonteiro is there something off the top of your head that’s missing?
I was gonna ask you that
@anmonteiro I don’t think anything is missing
@dnolen I’m happy to put that together
I can probably work on that before the next release
is there a ticket for it?
yeah I don’t think there’s any issues with code splits and :npm-deps
- it should just work
I might forget otherwise
@dnolen yeah, my line of thinking is: if Closure understands the modules, it can probably do splitting too
@anmonteiro https://dev.clojure.org/jira/browse/CLJS-2061 major enhancement ticket here
thanks
also if people don’t come with a better name than js_transforms.clj
that’s what we’re going to go with 🙂 \cc @juhoteperi
@dnolen That name is fine
Will those files be normal namespaces or "special"? I think they will need to require other namespaces.
I guess if they will reside in classpath root like data_readers.clj
, they will be special
@juhoteperi yes just planning on copying data_readers.clj
Trying to test cljs.loader
with our 18LOC .cljc + 2k .cljs project and 18 modules, compilation of cljs.loader
hangs (runs forever).
Gist with output:
https://gist.github.com/petterik/ee058c53a0fc3a1f1a328b10e35d325f
Sounds good
are you using :parallel-build
?
I think we had a problem where :parallel-build
didn’t detect cycles
right, so that might tell if you have cycles
(if your :modules
is messed up somehow, we should error out before doing all this work)
so if it’s not a leaf entry, or something you want to pin to a module, don’t worry about it
updated the site doc issue to reflect this pitfall https://github.com/clojure/clojurescript-site/issues/85
@dnolen updated Closure Compiler to latest release https://dev.clojure.org/jira/browse/CLJS-2161
more (commonJS) module processing fixes
they haven’t put out the changelog, I was mostly looking at https://github.com/google/closure-compiler/compare/v20170521...v20170626
something specific to CLJS is they renamed a class that we import in cljs.closure
@anmonteiro hrm that patch doesn’t look like it’s sufficient then right?
but the API didn’t change
we don’t use it
I don’t really know why we import it anyway 🙂
@dnolen I actually think those imports could go away
Maria used them to check if we were compiling aginst old vs new version of Closure
and we eliminated all those checks when we migrated
I’ll do a follow up ticket cleaning up the imports
@anmonteiro I’m there testing the patch
@anmonteiro applied w/ removed imports
awesome
Nice, got further. One of the problems I had was that I had defined a module for :cljs-base
with {:entries #{<my-main-ns>} :output-to "<main.js>"}
Don't remeber why I did this earlier, but removing it got me to the next error, saying Assert failed: Module <my-module> does not define any :entries
.
Trying to compile again without that module defined.
I had defined <my-module>
with {:entries []}
, and it's not unbelievable that it wouldn't get any entries in it.
this means no entries ever got assigned to that module even after assigning everything
Here is the new unsafe-get
macro in the CLJS-2148 patch:
(core/defmacro unsafe-get
"Efficient alternative to goog.object/get which lacks opt_val and emits
unchecked property access."
[obj key]
(core/list 'js* "(~{}[~{}])" obj key))
@dnolen compilation succeeded! Output:
Optimizing 333 sources, elapsed time: 444366.645424 msecs
Successfully compiled "resources/public/release/js/out/main.js" in 487.734 seconds.
Quick look at the splits show that it's done optimizations I had to do by hand before, which is really cool!My resources/public/releases/js/out/main.js is empty, but that's expected, right? My "main namespace" ends up in :cljs_base, as it should?
However, when trying to load cljs_base.js
in my browser I get error:
React is not defined
Even thought I see adding entry (cljsjs.react)
to :cljs_base
I don't see minified React at the top of cljs_base.js (which there usually is with foreign-libs if I'm not mistaken?).
@petterik that’s nice to hear that placements of the splits at least appear to be working in a large project
@petterik how long were your builds before? (curious if module computation is adding significant time, though I would be surprised)
so nice, https://github.com/clojure/clojurescript/commit/fb6c04f4ae16877ba7fa797e9378d8a1ba74fd89
@petterik I have a feeling a little bit of strategic memoization could go a long way here
if I could just get at your inputs
and your :modules
I could reproduce, I don’t need your whole project
I’m sure with some info we can make this take a couple of seconds, instead of hundreds
so you’re probably using a build tool instead of just calling the ClojureScript compiler directly
then I just need this sequence and your :modules
def and I can see what the algorithm does with that much info
feel free to email me privately at <mailto:[email protected]|[email protected]>
is there any way to blacklist dependencies through a compiler option?
some way that I can say: “I don’t want this namespace to be included in my build and I’m prepared to suffer the consequences”
I’m getting things in my bundle that I don’t want
and Closure can’t remove because they’re side-effecty
@anmonteiro I'm sure I'm not telling you anything new, but we have dev/prod files with the same namespace name that we include selectively depending on the build to avoid dev tools being included in prod. But it sounds like this might be a different use case?
yeah. this particular use case is timbre
-> taoensso/encore
-> cljs.test
-> cljs.pprint
so I’m getting 100KB of cljs.pprint
in my bundle because it has defmethod
s
similar problem here with cljs.pprint
, and no good solution:
https://github.com/binaryage/cljs-devtools/issues/37
and I actually want timbre
in the bundle for logging 🙂
btw. even some less-common closure library namespaces could be side-effecty and not 100% DCE-friendly, like goog.userAgent
btw this works incredibly well with CLJS: https://github.com/danvk/source-map-explorer
@anmonteiro I think a feature to elide certain namespaces sounds pretty scary
I did say “I’m prepared to suffer the consequences” 🙂
I think you could just write code to intercept build fns, but I guess you’re using a plugin
nah, I’m not using a plugin
I can write code
at Ladder we use Facebook’s Buck as our build system
which meant we had to write our own thing to compile ClojureScript 🙂
(for better caching)