This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2018-02-27
Channels
- # aleph (7)
- # beginners (80)
- # boot (1)
- # cider (3)
- # cljs-dev (277)
- # cljsjs (52)
- # cljsrn (1)
- # clojure (69)
- # clojure-gamedev (4)
- # clojure-italy (1)
- # clojure-losangeles (2)
- # clojure-russia (89)
- # clojure-spec (92)
- # clojure-uk (196)
- # clojured (1)
- # clojurescript (70)
- # cursive (5)
- # data-science (1)
- # datascript (84)
- # datomic (9)
- # defnpodcast (12)
- # docker (1)
- # emacs (4)
- # events (1)
- # fulcro (112)
- # graphql (1)
- # jobs (1)
- # lumo (1)
- # nrepl (21)
- # off-topic (2)
- # onyx (3)
- # protorepl (10)
- # re-frame (23)
- # reagent (66)
- # reitit (2)
- # rum (13)
- # shadow-cljs (144)
- # spacemacs (14)
- # sql (4)
- # unrepl (29)
- # vim (16)
also maybe a -ro
and -co
are maybe more in line with --repl-opts
and --compiler-opts
but I can put together a patch in case..
@richiardiandrea I was concerned that we were already using -O
-o
, but maybe not a big deal
oh yeah right, there is no clash in my head but it might be worth hearing someone else's opinion as well 馃槃
I was having a little trouble reproducing anyways. Was planning on playing with that quite a bit.
it's weird because I printed out the options that are merged and they look fine:
{:options {:output-dir out, :static-fns true, :install-deps false, :optimizations :none, :verbose true, :output-to out/ep-lambda-cqrs.js, :source-map-timestamp false, :fn-invoke-direct true, :source-map true, :target :nodejs, :compiler-stats true, :npm-deps {:aws-sdk 2.201.0, :bunyan 1.8.12, :dotenv 5.0.0, :lodash.camelcase 4.3.0, :lodash.kebabcase 4.1.1, :lodash.upperfirst 4.3.1, :pg
@mfikes if you have some time since I know your good at finding these things - testing that the browser REPL works with -i
-e
would be cool, I definitely found an issue and I think I fixed
Yeah, I'll mess with -i
and -e
some more (especially -i
, which I haven't put through a whole lot of paces)
First version of something new working in Windows is pretty awesome :)
@mfikes instead of fancy handshake I just increased the timeout to the value from the Closure book, maybe give that a spin with Safari, it seems more reliable
ok the only critical thing now is (caching) compile from JAR, I鈥檇 like any further work to focus on making sure cljs.main works great
@dnolen Perfect with the 1000 ms. I went to the same box where it would reliably fail (nearly deterministically), and I tried it 10 times with the new code and it never failed. I'm convinced it is solved. 馃憤
@dnolen I saw Alex mention something a few days ago about exposing some additional information from tools.deps
that sounded like it might be useful for figuring out things related to global caching if you have, say git deps at play. Let me see if I can dig up what he mentioned.
I just released 1.10.63 for testing purposes, a little more convenient than git URLs, maybe tomorrow start sharing cool snippets to get people to try it out
clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.63"}}}' -m cljs.main -r
Yeah, a whole flood of people who are using Lein and Boot can easily give 1.10.63 a spin
I can't find the stuff that Alex was mentioning. Perhaps it was the new -Sdescribe
option or its underpinnings.
probably clojure.tools.deps.alpha.reader/clojure-env
but that鈥檚 really just a clojure wrapper around a call to clj -Sdescribe
The overall goal is to globally cache compilation artifacts produced from (immutable) JARs. Now that git deps have come along, they feel like they fall into the same category.
My initial thinking was, if you start by looking for JARs in (System/getProperty "java.class.path")
, then you would miss out on the git deps, so there is probably something more sophisticated that can be done.
@mfikes hrm it鈥檚 getting late, but I鈥檓 thinking this Safari issue (I still see the case where files aren鈥檛 available yet on my iMac) is just revealing that browser REPL has a concurrency bug
the static file serving happens on a different thread than the normal REPL compilation process, so things are building in parallel but not in a safe way
interesting. I ran mfikes example of the geometry thing the other day. Now when i run clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.64"}}}' -m cljs.main -r
, it flashes the blue welcome page for about a second and then goes to the black page with the canvas
Yeah, @dpsutton the underlying technical reason is that the project itself doesn't provide an index.html
, and quickly revises the default one supplied by ClojureScript
(And the technical reason that approach is taken is so that it can be used as a git dep)
it showed the default blue one and then it showed the black canvas one from days ago
@dnolen https://github.com/clojure/clojurescript-site/pull/173 has been updated for 1.10.64. One idea to consider is releasing that page early for those that would like some cljs.main
documentation.
yeah i was doing it from random directories. i may have clobbered a work project's out directory in that case 馃檪
yeah. i think it saw an absence of index.html but then "found" some clojurescript to run on it. so it was a mis-mash of blank project and old project laying around
I don鈥檛 know that I would consider this a bug - you had a previous compile so of course that鈥檚 what you鈥檒l get.
Feedback: That -cf
option rocks. Using it here https://github.com/mfikes/shrimp#repl
and as beginners run a bunch of "show me what cljs can do" type scripts it might be surprising to see
For this particular case, it generates an out/main.js
only if one doesn't already exist, right? If it unconditionally generated one, the problem would go away, but perhaps introduce other undesired consequences?
of course for these examples they could include a pre clean with a clojure.main -e option?
@bhauman but what does figwheel do that鈥檚 different other than provide some way to clean beyond rm -rf out
?
Ahh, I was wrong in my statement earlier. It is more accurate to say that, for clj -m cljs.main
if either index.html
or out/main.js
don't exist, you get a synthetically generated one on the fly.
But, if you have either, I would see that a lot of people might want things to actually reflect the current out
if no out/main.js
we try load to something we compiled only when we see :output-dir
doesn鈥檛 exist
It feels like this problem might only arise for people running git-dep projects in random directories, and by chance seeing something stale. Perhaps for "proper projects", the implemented behavior matches what is desired.
right we can鈥檛 know what the user is trying to do - I don鈥檛 want to try to and figure that out
I suppose this presents no issue for new users running Quick Start鈥攖hey will be in the hello_world
directory working with that project.
Probably not intentional (and arguably not that big of a deal), but the main page of http://clojurescript.org now indicates "Current version: 1.10.64"
it鈥檚 intentionally automated :)
If you want me to temporarily change to something static, I could do that
Dang. I'm seeing issues again with Safari connecting. Trying clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.64"}}}' -m cljs.main
and also the latest commit.
(I'm getting 404 values on http://localhost:9000/out/goog/base.js and other files.
Ahh. Looking in out
, there is no goog
directory being generated. Will see if I can figure out why / file a JIRA.
It is not deterministic... but I have a stack trace now ... going to see where it is stuck
@mfikes previous we built in the static file thread - and that鈥檚 how I saw the Safari bug
Well, as far as I can tell, it is not stuck. https://gist.github.com/mfikes/0a809d16ca4e38f4e6958c3c973a1f06 When this happens my out directory is pretty anemic
$ find out
out
out/cljs
out/cljs/core.cljs.cache.edn
@mfikes hrm, that might just be a bug in the ad-hoc web server, nothing to do with Safari
@dnolen I got to the bottom of the "anemic" out directory. 馃檪 https://dev.clojure.org/jira/browse/CLJS-2579
just a heads up, I'm working on a 100 line minimal figwheel implementation that could work with cljs.main
. It has a figwheel-start
macro that adds a watch to env/*compiler*
for compilation signal and relies on a *repl-env*
var so it can eval reloads on the client. It's just a single cljc
file so you just (require '[figwheel.simple :include-macros true])
or preload it. The figwheel-stop
macro just removes the watch.
@dnolen a question, I am debugging the node-postgres
problem and I see that the indexing produces a map like this:
{:file /home/arichiardi/git/clojurescript/node_modules/pg/lib/index.js, :module-type :es6, :provides [pg/lib/index.js pg/lib/index pg/lib]}
should we create an entry like the above but that follows the convention of the package name for node and therefore provides becomes :provides [pg pg/lib/index.js pg/lib/index pg/lib]
?
it would probably solve the require problem and it would follow the node convention
@richiardiandrea I don鈥檛 know what you are asking me
basically I think we are not building the :provides
key correctly for that package
I am dumping all the maps and the closest I see is:
{:file /home/arichiardi/git/clojurescript/node_modules/pg/package.json, :module-type :es6}
but there is no :provides
...
ok gotcha, sorry I am exploring this for the first time
@richiardiandrea try this with something we test with like lodash
compare the results
yep I am exactly in that file
will try that
@dnolen sorry to bug you but basically modules
to index-node-modules
is empty with pg
...digging...
ok so it looks like it is not a top-level
dep
"pg/lib/index.js"
is
and node-require
is {pg goog.array goog.Uri goog.object goog.math.Long goog.string goog.math.Integer cljs.core goog.string.StringBuffer}
so there is no intersection, I will file an issue, but any hint on possible solutions are greatly appreciated
@richiardiandrea it needs to be top level
@dnolen it is in there, maybe my description in the bug is not that great
so this is my opts map:
{:verbose true
:npm-deps {:pg "7.4.1"}
:install-deps true
:output-dir (str out)}
the indexer does not return a :provides
that matches :pg
basically
see Jira
@richiardiandrea https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/module_deps.js
https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/module_deps.js#L117
some kind of debug dump function that easily lets you test a package might be useful for users trying to understand what鈥檚 going on
@richiardiandrea yeah that code shows why it doesn鈥檛 work
@anmonteiro probably just an oversight? ^ or is that not following npm semantics?
@dnolen yeah ok so there is a problem there
@richiardiandrea actually we probably need to be careful what is the contents of pg/package.json
well I am asking because I don't want to be the willy-nilly guy 馃槃
@dnolen yeah I know that's the issue
the indexing does some filtering
let me fetch the line
here we filter https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/closure.clj#L2620
so :provides
needs to be compatible with what you put in :npm-deps
and in this particular case it is not
https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/module_deps.js#L205 as the :main
path isn't single file but folder, this check is never true?
I think fix would be to add pkgJson.mainEntry + 'index.js'
to candidates
list here: https://github.com/clojure/clojurescript/blob/master/src/main/cljs/cljs/module_deps.js#L183-L191
or no, I read the for loop logic wrong
@juhoteperi it might actually be that one
maybe if pkgJson.mainEntry
ends in /
, index.js
should be appended to the value
no you can't do that
for instance pg
's entry is only ./lib
so in the candidates probably we just need an entry ./lib/index.js
?
Oh right. I guess the code could also check if the the value refers to a directory.
pkgJson.mainEntry
+ /index.js
?
yeah that's true
Maybe adding it to candidates list works, but I'm not sure what the for loop does.
I am trying
{"file":"/home/juho/Source/clojurescript/node_modules/pg/lib/index.js","provides":["pg","pg/lib/index.js","pg/lib/index","pg/lib"]}
@juhoteperi what's the change you made?
I added pkgJson.mainEntry` +
/index.js
to candidates
oh ok me too 馃檪
but i did not get that, let me try again 馃槃
I have [pkgJson.mainEntry, pkgJson.mainEntry + '.js', pkgJson.mainEntry + '/index.js', pkgJson.mainEntry + '.json']
Yeah, same
ok cool
weird, I cannot get that
I tested directly by calling node
doesn't work when calling Cljs function
sorry just now reading the backlog
@richiardiandrea you probably wanna add a test case for pg
and go from there
I did that myself with running directly
@anmonteiro yep I have been doing that this morning
test-first development for this case is actually really helpful
so I still feel
make sure you won鈥檛 break other tests too because we鈥檙e already working around many edge cases
that we need to add thing in the indexer
also the issue might not be in module_deps.js
but in the cljs.closure
function
we check if something is a node dep by indexing there first
I can have a quick look over here
yeah exactly
Yes, I think module_deps.js
doesn't affect this because index-node-modules-dir
doesn't use that, its just Clojure
I can confirm that
@richiardiandrea have you already written a test?
otherwise I鈥檓 just going to write one and look at it on my end
I have written a test but just for triggering printlns here
I mean a test in closure_tests.clj
nope there not
I am only doing:
(deftest cljs-2580-main-with-dir-test
(test/delete-node-modules)
(spit (io/file "package.json") "{}")
(let [out (.getPath (io/file (test/tmp-dir) "cljs-2580-main-with-dir-test"))
root (io/file "src" "test" "cljs_build")
opts {:verbose true
:npm-deps {:pg "7.4.1"}
:install-deps true
:output-dir out}]
(test/delete-node-modules)
(test/delete-out-files out)
;; (closure/maybe-install-node-deps! opts)
(build/build (build/inputs (io/file root "node_postgres_test")) opts)
#_(test/delete-out-files out)
#_(test/delete-node-modules)))
the problem seems to be in node-file-seq->libs-spec*
Yes, the fix is pretty much same as the one we tried with module_deps.js, (str main-path "/index.js")
@juhoteperi yes here? https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/closure.clj#L2416
line 2420
sorry yeah the line did not match my emacs 馃槃
cool works
there is a Error: Can't resolve 'util' in '/home/arichiardi/git/clojurescript/node_modules/pg/lib'
which is odd, but it might be something else
just need to hook the transitive dependencies probably
^ that happens when running module_deps.js
now
util
is built-in Node module
right so you just need to add :target :nodejs
cool lemme do that
awesome it works!
@richiardiandrea congrats, patch welcome, you鈥檙e now an :npm-deps
expert 馃槈
@richiardiandrea here鈥檚 a test
(deftest test-cljs-2580
(spit (io/file "package.json") "{}")
(let [opts {:npm-deps {"pg" "7.4.1"
"pg-native" "2.2.0"}
:target :nodejs}
out (util/output-directory opts)]
(test/delete-node-modules)
(test/delete-out-files out)
(closure/maybe-install-node-deps! opts)
(let [modules (closure/index-node-modules-dir)]
(is (true? (some (fn [module]
(= module
{:file (.getAbsolutePath (io/file "node_modules/pg/lib/index.js"))
:module-type :es6
:provides ["pg/lib/index.js"
"pg/lib/index"
"pg"
"pg/lib"]}))
modules))))
(let [modules (closure/index-node-modules ["pg"] opts)]
(is (true? (some (fn [module]
(= module {:module-type :es6
:file (.getAbsolutePath (io/file "node_modules/pg/lib/index.js"))
:provides ["pg"
"pg/lib/index.js"
"pg/lib/index"
"pg/lib"]}))
modules))))
(.delete (io/file "package.json"))
(test/delete-node-modules)
(test/delete-out-files out)))
wow awesome
it鈥檚 still complaining about not being able to find pg-native
for me though
will add the test and provide a group patch 馃槃
should also install pg-native
the full build does it
Other than pg-native, I can confirm the test case is good and the two fixes fix both checks, I wrote nearly same checks 馃槃
closure doesn't do dynamic requires so I would just give up on this package working completely
it鈥檒l work for target nodejs
which I think is the point here
@juhoteperi by two fixes you mean both in module_deps.js
and closure.cljs
right?
just making sure I am not missing anything 馃槃
edited the test case above
yep it failed 馃槃
@richiardiandrea if there鈥檚 a failure try printing modules
and see what the provide order is
might be just that
should we just ignore the error there?
with pg-native
maybe check (= (set (:provides module)) #{"pg" ..."}))
so the order doesn't matter?
I think it's ok, all the other tests use vectors anyways
it fails here with some unrelated stuff:
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
gyp WARN download NVM_NODEJS_ORG_MIRROR is deprecated and will be removed in node-gyp v4, please use NODEJS_ORG_MIRROR
You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application.
gyp: Call to '/usr/bin/pg_config --libdir' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.onCpExit (/home/arichiardi/.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:336:16)
gyp ERR! stack at emitTwo (events.js:126:13)
gyp ERR! stack at ChildProcess.emit (events.js:214:7)
gyp ERR! stack at Process.ChildProcess._handle.onexit (internal/child_process.js:198:12)
gyp ERR! System Linux 4.4.0-116-generic
gyp ERR! command "/home/arichiardi/.nvm/versions/node/v8.9.4/bin/node" "/home/arichiardi/.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /home/arichiardi/git/clojurescript/node_modules/libpq
gyp ERR! node -v v8.9.4
gyp ERR! node-gyp -v v3.6.2
gyp ERR! not ok
so probably something about the native depusing maybe an old node
look a the third line, you need to install postgres dev headers package
Yep that worked
oh ok let me try
patch attached to https://dev.clojure.org/jira/browse/CLJS-2580, thanks folks, great collaboration on this one 馃帀
(I would add as usual 馃槈)
In other news, no huge onslaught of bug reports regarding the 1.10.64 release. I only saw a single tweet about some code that is probably calling key
on a vector.
Wow today is a very rewarding day 馃槃 https://github.com/google/closure-compiler/pull/2829
@richiardiandrea applied!
@dnolen if you feel so generous I also added this one: https://dev.clojure.org/jira/browse/CLJS-2566 馃槃
@mfikes thanks for all the cljs.main
testing, I moved most of those into must fix before release status
Tried the pg
patch and my code now compiles smoothly