This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-03-08
Channels
- # aws (3)
- # beginners (126)
- # boot (19)
- # cider (31)
- # cljs-dev (324)
- # clojure (96)
- # clojure-boston (2)
- # clojure-denver (9)
- # clojure-dusseldorf (2)
- # clojure-greece (4)
- # clojure-italy (5)
- # clojure-losangeles (1)
- # clojure-spec (18)
- # clojure-uk (59)
- # clojurebridge (1)
- # clojurescript (184)
- # community-development (29)
- # cursive (2)
- # datascript (2)
- # datomic (5)
- # emacs (1)
- # figwheel (6)
- # fulcro (270)
- # hoplon (2)
- # jobs (1)
- # jobs-discuss (1)
- # keyboards (2)
- # leiningen (2)
- # london-clojurians (2)
- # luminus (10)
- # mount (1)
- # off-topic (26)
- # onyx (8)
- # other-languages (1)
- # parinfer (1)
- # protorepl (6)
- # re-frame (23)
- # reagent (61)
- # reitit (5)
- # shadow-cljs (100)
- # spacemacs (3)
- # sql (19)
- # unrepl (90)
- # vim (25)
woah really
@mfikes https://github.com/clojure/clojurescript/commit/114f5b5ae76b04be31d008c93604773cb7664796
the concurrent queues don’t fix anything - you simply cannot have threads enter connection
and set-connection
at the same time
I tested this, no discernible perf difference, still fast - and I can’t break with my Socket REPL experiments
anyway, end result - very few changes and browser REPL is now significantly more capable 🙂
right! so this new lock
is effectively doing what the swap!
guarantees, sorry if I repeat the obvious, it just makes me feel good to know that I was on the right path 😄 lol a developer needs his/her certainties
Yeah, perhaps if you wanted to imitate the lock
approach you'd have to reach for ref
. TBH, though I understand locks better than STM.
Tried master again (with the lock
construct) on 2012 mac an repro’d CLJS-2629 after 3508 iterations.
Oh, hey @bhauman do you happen to know if Figwheel sends Content-Length
when talking with the browser? (That might explain why it works…. thinking about CLJS-2629)
Right with 2629, I never had any issues with connections (at least no obvious ones). It is that silly Content-Length
thing.
I finally managed to repro CLJS-2629 on a Late 2013 iMac, after 17000 iterations using the zippy.clj
version at https://gist.github.com/mfikes/765ac8e43d70b743e3bfbd02ff4703a1
So perhaps this is getting in the realm where others can repro. (I've now repro'd on 3 separate boxes ranging from 2009 through 2013 vintage)
I've definitely tried to ascertain if the content length might be wrong, and I've haven't found evidence of that yet.
My only theory so far https://clojurians.slack.com/archives/C07UQ678E/p1520456944000503
when I eval something, I see Content-Length that’s one less byte then when I copy and paste the response into a file
I added a bit of code to count the form before calling .getBytes
to see if anything there might be going wrong.
Anyway, without pasting images, I'm looking in the Network tab which has Size and Transferred numbers.
But, what could happen is that we send a Content-Length
saying that we the payload is, say 500 bytes, but only send 474, and then Safari will conclude that the network connection must have been "lost"
To repro on the 2013 box, zero delay. Using zippy.clj
https://gist.github.com/mfikes/765ac8e43d70b743e3bfbd02ff4703a1#file-zippy-clj
Right, I eliminated the delay, and also the JavaScript console printing, to make it go as fast as possible
yeah, I really couldn’t repro with this test case I have where I start two connecting pREPL clients in quick succession
I'm still interested in the other, slower, "long-lived" but with substantial delay, style test, to imitate what a real developer might do. (These tight loops are interesting but not the main use case.)
The only time where rapid evaluation might matter is right at startup, when a lot of stuff is getting evaluated in rapid succession during REPL environment bootstrap—maybe that's where started thinking connections were failing 20% of the time.
@mfikes hrm when running these tests are you making sure there’s only one browser window open?
@dnolen Yeah, for tickets like these, it is almost like we could close them as "assessed, but we aren't likely to fix", just so they don't clog up the JIRA
This was pretty cool, IMHO 🙂 https://twitter.com/functionalcave/status/971438971648692225
@mfikes it’s worth keeping these kinds of tickets open, nice history of the various challenges we encounter along the way
It represents AOT caching. The idea is that ClojureScript itself is expanding out to embrace the JARs that it pulls from Clojars.
the list is whittling down again - circular dep parallel build and one I’m not really sure what to do about - the Windows one
One way to look at the Windows path issue https://dev.clojure.org/jira/browse/CLJS-2588 is that it is not a regression introduced by 1.10.x, and it also appears to be a Closure issue perhaps (https://github.com/google/closure-compiler/issues/2611#issuecomment-345595935_
unless there’s something major I think the last thing I will fix is circular dep check and we should just push this out the door next Monday
Will try CLJS-2613 and confirm that it works
Also I would like to take a stab at applying the shared repl env to the node socket Repl (or more correctly to all socket Repl)
Or maybe there is no need, will play around
I have regression in npm-deps resolution in 1.10.x for material-components-web
. For sub-dependencies it doesn’t resolve the module path correctly. When I require eg. @material/snackbar
it fails with Uncaught Error: Undefined nameToPath for module$$material$base$component
. Only related issue seems CLJS-2451 but it worked fine in 1.9.946 for me.
I noticed a small bug in https://github.com/clojure/clojurescript/blob/d0be39660f3a65422c3de6a774ceec0b6a863ee2/src/main/clojure/cljs/analyzer.cljc#L3127-L3129
(case ext
"edn" (spit cache-file
(str (when
(str ";; Analyzed by ClojureScript " (util/clojurescript-version) "\n"))
(pr-str analysis)))
The extra when
is wrong here, if you reindent, you can see it better:
(case ext
"edn" (spit cache-file
(str (when
(str ";; Analyzed by ClojureScript " (util/clojurescript-version) "\n"))
(pr-str analysis)))
This means that the "Analyzed by ClojureScript 1.9.512" will never show in the cache file headerShould I open a ticket, or can someone with commit rights just fix it?
It was introduced in https://github.com/clojure/clojurescript/commit/296f2bc4311e6023e83d3f7cdfe7bd8fc336b982
Oh… if I can just convince Lee Byron to add a "browser": "index.js"
field to graphql-js and iterall, then nothing needs to be changed in CLJS/CLosure to to support both packages out of the box (for the browser target at least).
That would be good enough for my use case, although probably not a long term solution.
@jannis Which env are you targetting? Iterall at least already says main is index
. If you target browser I think Cljs should use browser
and then main
for nodejs module
then main
@juhoteperi Browser. For that, CLJS picks browser
, module
, then main
. Unfortunately both graphql-js and iterall have an .mjs
file in module
and no browser entries.
Ah no, for browser the order is browser
, module
, main
: https://github.com/google/closure-compiler/blob/6797f5b2932c4dcdc12a3d4f9580d7fb9ae4bbf8/src/com/google/javascript/jscomp/CompilerOptions.java#L1210
Regardless of the target, the module
entries with .mjs
will break CLJS and Closure. Easy win could be to convince them to add a browser
entry pointing to .js
, then at least we could import both packages for the :browser
target.
What does that .mjs
extension mean? Are they just normal es6 modules?
Yeah, a new extension for ES modules: http://2ality.com/2017/05/es-module-specifiers.html
Does Closure have issue about .mjs support yet?
Or does Closure-compiler need changes in addition to those?
Where Closure resolves import ... from "iterall"
to module$iterall
instead of module$path$to$node_modules$iterall($index)
.
i think it may be related to https://dev.clojure.org/jira/browse/CLJS-2633. It doesnt use .mjs
but it breaks on import ... from ...
same way as yours
I’m going back and forth between different approaches because I’m not sure I can get that to work in Closure.
If you open issue at Closure (with or without fixes) it is going to be good idea to provide them commands to test this with just Closure cli, similar to e.g. https://github.com/google/closure-compiler/issues/2634
@danielcompton I’d write a JIRA. It may not be critical because ClojureScript now directly depends on Transit so that branch may be less relevant now. https://github.com/clojure/clojurescript/commit/ff573d12d8f868c980585445ff39b957967b6dce
@jannis FWIW I would stay away from .mjs
until the node/webpack world has settled all the issues. currently closure only supports the "strict" version of https://medium.com/webpack/webpack-4-import-and-commonjs-d619d626b655
so it currently breaks if CJS is importing a ES6 that was converted by babel (most of the stuff on npm) since Closure only detects it as CJS and doesn't check the __esModule
prop
in shadow-cljs I just ignore module
and only use main
and browser
. too many issues with too many packages each doing weird things with module
I’ve been trying to understand and get this to work for over a week, going back and forth all the time. With a "browser": "index.js"
added to avoid pickup up the index.mjs
, everything worked immediately.
take this fun example https://unpkg.com/[email protected]/index.es.js it has export { default as AppBar } from './AppBar';
which is https://unpkg.com/[email protected]/AppBar/index.js
thats not the only package on npm doing this weird buggy mix of ES6/CJS and closure doesn't like this at all
for the module$iterall
issue you could just rename the filename passed to closure from .mjs
to .js
well until the solution is resolved by the closure compiler that is the solution I have chosen in shadow-cljs yes.
.mjs
should start getting more adoption now that webpack v4 has support for it although node
doesn't event support it without the flag.
the entire situation is pretty messy since the JS world hasn't figured out what to do yet
Another unexpected "feature composition synergy": You can use classpath-relative -i
and -r
to load code and run a REPL (without using -c
and -r
) and avoid dirtying the disk with out
(and also use the ability to use top-level require
to avoid the single-segment namespace warn) if running everything from a gist. The Bocko gist illustrates this https://gist.github.com/mfikes/e00202b2de7cc2352fedcf92b1fe60dc
@jannis we should let @anmonteiro weigh in when he is around
getting a whole bunch of
WARNING: Use of undeclared Var cljs.core/not at line 3 /Users/kommen/work/nextjournal.com/journal/src/editor/components/ui/icon.cljs
WARNING: Use of undeclared Var cljs.core/apply at line 3 /Users/kommen/work/nextjournal.com/journal/src/editor/components/ui/icon.cljs
WARNING: Can't take value of macro cljs.core/hash-map at line 3 /Users/kommen/work/nextjournal.com/journal/src/editor/components/ui/icon.cljs
the compiler eventually crashes then with
Caused by: clojure.lang.ExceptionInfo: Can't redefine a constant at line 19 file:/Users/kommen/.m2/repository/org/clojure/clojurescript/1.10.145/clojurescript-1.10.145.jar!/cljs/spec/alpha.cljs {:file "file:/Users/kommen/.m2/repository/org/clojure/clojurescript/1.10.145/clojurescript-1.10.145.jar!/cljs/spec/alpha.cljs", :line 19, :column 1, :tag :cljs/analysis-error}
@kommen I can only speak about what the symptoms mean: in the first case cljs.core
has not been analyzed, in the second case files are being compiled multiple times
@kommen without knowing about what build pipeline you are using I really can’t tell you anything more
it would be best if you try to repro your build or part of it with ClojureScript only - so there’s something to compare
@kommen oh one question - does everything work if not :nodejs
target? like your web stuff?
@dnolen also, we just build stuff with cljs.build.api, shouldn’t be much which can interfere
These “optimization” defaults change as ClojureScript evolves and their impacts are better understood, so there is no real question about potentially breaking anything I suppose, and it just boils down to “is this the right set of implied defaults”
just as data point, I always turn both static function and direct invoke on, independently from the optimizations
I don’t think we should move away from "module"
as I said yesterday, I’m pretty sure that would make many more packages stop working
there’s an issue that @juhoteperi is unfortunately too familiar with regarding UMD processing in Closure
if we stop taking "module"
into account we’re gonna be looking UMD bundles that Closure can’t process
UMD support might be okay currently
better than previously anyway
do you really wanna bet on that? 😛
definitely better than previously, but “okay” is not a word I’d use when different bundlers use different UMD checks
"how many different wrappers there can be..."
famous last words
If I switch the order of -c
and --optimizations
around it does a build w/o advanced (in clj -m cljs.main --optimizations advanced -c hello-world.core
). Bug?
There might be other benefits to using modules instead of UMD bundles also, maybe better DCE?
@anmonteiro Hm, ok. I don’t think I can make Closure work with .mjs
entries. I tried but there are still issues with dependencies between ES modules from different packages and I can’t fore the life of me figure out what’s going on. I think I’m going to give up for now if there’s no solution on the horizon.
sorry I can’t look into it either, I’ve been really busy with work the past few months
First draft Shared AOT Cache news post: https://github.com/mfikes/clojurescript-site/blob/9edfe32910f76798440461be6a2dbf0c5488bf66/content/news/2018-03-12-shared-aot-cache.adoc
> it is currently incompatible with running multiple ClojureScript compilers simultaneously on your dev box. Any plans on supporting this? I have quite often maybe 3 or 5 projects open
Another one: Doing (js/process.exit 0)
on a node repl with clj -m cljs.main --repl-env node
fails a little ugly
@rauh probably worth Jiras, I will look at the node
stuff as soon as I have some time
it’s like Maven - multiple things aren’t really going to cause problems except in the rare case where two things are starting at the same time and you don’t have the thing cached already
for now, yes - I’d like to see how often this is actually a problem in practice, so far I’ve been running lots of different REPLs, compiling things, and I haven’t encountered cache corruption yet
@jannis a fix I would be OK with would be some kind of resolution flag, for your case I wonder if you really need anything more than top level, i.e. what we have is :webpack
and we could offer plain :node
What would that flag do? Yes, the package.json -> index.mjs/js is the critical part, but it’s probably not enough to handle it in CLJS, Closure also needs to work. A flag to ditch "module"
from the entry fields (which we can also configure in Closure from cljs.closure) would solve it, I think.
^ which is what I had proposed too. but I might like @dnolen’s idea better actually
let’s us abstract over the nitty gritty of Closure if we ever want other resolution modes
i.e. not controlling the specific fields directly but offering a range of predefined options
this seems like a legitimate :npm-deps
regression but unclear where it happened https://dev.clojure.org/jira/browse/CLJS-2633
if somebody has time to at least sort out the revision where things went wrong that would be a good thing to add to the ticket
@jannis Closure doesn’t need to work, we imposed our own order, so just let’s make it work from our end - handling .mjs
is not under consideration here
I wonder if there are any other high-level (but not marquee) items to list for 1.10.x in here https://github.com/mfikes/clojurescript-site/blob/659a97a2159180cb344f459d147fd57856a0cbae/content/news/2018-xx-yy-release.adoc
Closure update is worth mention, the new version includes at least my UMD wrapper fixes and loads of general module processing changes
Hmm, update also fixes requiring ES6 modules from CJS code (or was it the other way...)
@jannis in this case :nodejs
would be just ["main"]
then
@mfikes enhanced socket REPL, and also alpha pREPL (requires Clojure 1.10.0-alpha4), I would definitely add @juhoteperi suggestions
package-json-resolution-type
?
or just package-json-resolution
?
@mfikes I added a suggestion for Closure/module processing section to the pull request.
@anmonteiro @jannis I like the last one too
@kommen can you file a bug for your issue - I think that & 2633 need to be fixed (or thoroughly investigated at least before release), critical priority is fine
@mfikes how did you implement that ClojureScript 1.10.145
header? I'm getting it at every prompt in piggieback
@mfikes yeah, so piggieback is going to be hit by this because it explicitly disables the quit-prompt, but doesn't disable the new title-prompt
oh boy
I have a PR in that would prevent this, but there is no telling when it will be accepted
I was thinking about this yesterday, probably it should be described with tests.
so many things to do, so little time
So it sounds like we can replace the default behavior of quit-prompt
to print the version or somesuch
@bhauman https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/server/browser.clj
https://dev.clojure.org/jira/browse/CLJS-2640 ^ quit prompt issue
in an idillic world you could have a browser AND a node AND a nashorn REPL running at the same time 😄
(on different threads)
yeah sorry 😄
so we just need to do the same thing for the others - Node.js being the obvious one to do next
should have thought of this before when I looked at what piggieback was doing - oh well
@dnolen re 2639: doesn’t seem to reproduce easily, couldn’t yet build a demo app with nodejs target which shows the same issue as I’m seeing in our project
started ripping out stuff, but it is getting late here though. I do see a different ex info now as before:
Caused by: clojure.lang.ExceptionInfo: Invalid :refer, var cljs.tools.reader.impl.utils/whitespace? does not exist in file file:/Users/kommen/.m2/repository/org/clojure/tools.reader/1.3.0-alpha3/tools.reader-1.3.0-alpha3.jar!/cljs/tools/reader/reader_types.cljs {:tag :cljs/analysis-error}
at clojure.core$ex_info.invokeStatic(core.clj:4739)
at clojure.core$ex_info.invoke(core.clj:4739)
at cljs.analyzer$error.invokeStatic(analyzer.cljc:697)
at cljs.analyzer$error.invoke(analyzer.cljc:693)
at cljs.analyzer$error.invokeStatic(analyzer.cljc:695)
at cljs.analyzer$error.invoke(analyzer.cljc:693)
at cljs.analyzer$check_uses.invokeStatic(analyzer.cljc:2192)
at cljs.analyzer$check_uses.invoke(analyzer.cljc:2187)
at cljs.analyzer$check_use_macros.invokeStatic(analyzer.cljc:2205)
at cljs.analyzer$check_use_macros.invoke(analyzer.cljc:2195)
at cljs.analyzer$check_use_macros_inferring_missing$fn__1975.invoke(analyzer.cljc:2217)
I've noticed that the last modified times on generated JS files are set to match the last modified times of their corresponding CLJS files. This can lead to unexpected caching behaviour: https://github.com/bhauman/lein-figwheel/issues/664
Is there a safer/better way to calculate a last modified time that also reflects the input dependencies to a ClojureScript namespace?
I was thinking of taking the max date of all last-modified's
@danielcompton that problem doesn’t seem to have to do anything setting last modified
it’s only about macro files changes and that not being part of invalidation (because they are not actually part of the build)
FWIW, I also don’t really think this is very simple to fix, so I don’t know if we should be involved
Figwheel sends the last modified time for any served files which is used for browser cache invalidation. I assumed that the last modified time of a JS file would be different after a recompile, but that's not a correct assumption.
What's the purpose of setting the last modified time of the generated JS files to match their corresponding CLJS files?
I'm not suggesting any of that, I'm saying that the last modified date of compiled JS files isn't able to be reliably used for web server caching headers
because it's still useful to be able to set caching headers in development to speed up reloads. It's just a confusing behaviour that I wouldn't have expected
The answer seems to be that you can't rely on the file modification date for caching CLJS development
which is fine, it just wasn't obvious to me
Yeah the caching is too aggressive, which leads to weird behaviour when you're developing
anyways, your idea needs to be reconciled with figuring out how to know the source is now different from the target
Now that I understand what is happening with last modified dates, I think it's safer just to not rely on them for caching purposes, as they aren't always going to reflect the latest changes
I'll take a look at the change tracking code, but it seems easier to leave it as is for now
The compiled JSON/EDN cache files also match the CLJS file modification date, I don't know much about that cache, but does it only cache information from the corresponding cljs file?
I understand, was just wondering if there were other issues lurking based on file modification time