Fork me on GitHub
#cljs-dev
<
2018-03-27
>
dnolen02:03:47

We use all 32, perf benefits

shaunlebron02:03:33

I’ve been thinking about how to create JS generators and async functions in ClojureScript

shaunlebron02:03:25

(they are heavily used in https://observablehq.com notebooks, and I eventually want to see if it would be possible to use ClojureScript there)

shaunlebron02:03:17

I wonder if it would be a good idea to add :js-generator and :js-async metadata to functions so they can compile to function* and async function, respectively

shaunlebron02:03:19

and js-yield,`js-yield*`,`js-await` for emitting => yield,`yield*`,`await`

dnolen02:03:49

It’s been proposed before but I don’t see any benefit

dnolen02:03:19

Well not enough benefit to seriously consider their inclusion

shaunlebron03:03:07

js interop mainly, it’d be pretty ugly to use existing js generators and async functions inside cljs

shaunlebron03:03:16

i thought i could just reach for core.async instead, but I didn’t really understand generators/async functions until I saw them combined, which are now implemented in most browsers today: https://github.com/tc39/proposal-async-iteration#async-generator-functions

dnolen03:03:44

I don’t see why you would need them for interop

dnolen03:03:48

you don’t need core.async for interop

dnolen03:03:55

and they’re more or less like core.async - nobody uses that stuff unless they have to because callbacks are the baseline

dnolen03:03:25

they’re even less meaningful for React code (>70% of what ClojureScript users are doing) where async patterns are just callbacks

dnolen03:03:33

I’d like to see more compelling cases where they are solving some actual problem for end users before we start looking at adding more JS primitives

👍 4
rarous04:03:40

@shaunlebron ES await\`async` is just sugar over functions that return Promise. You can always use the Promise API on the result of async function. If you need to provide async function in your cljs public API, just return Promise. Generators are sugar for generating stateful Iterators and way to add some kind of laziness to JS. Cljs structures are lazy and implement Iterator for long time. Cljs is ahead in this.

shaunlebron14:03:52

thanks for replying. it would just be nice to drop down to JS things since clojure is about embracing the host (if I’m interpreting that goal correctly)

shaunlebron14:03:59

also, generators are pausable/cancelable coroutines that can send and receive values at yield points. not sure a lazy sequence can do that

shaunlebron14:03:19

as for just using the Promise API, the synchronous style of async/await is nicer, similar to why core.async was created

shaunlebron15:03:41

@U050B88UR my feeling is that it’s more important than we realize, so I’ll do the work to make a case for it. already drafting a notebook that is essentially describing how it makes cancelable “goblocks” possible

juhoteperi07:03:54

Are the tests passing on Windows CI? One report on Reagent mentioned problems with the change I made to module_deps.js

juhoteperi11:03:32

It is expected with the new release that this code breaks: https://github.com/frankiesardo/linked/blob/master/src/linked/map.cljc#L257-L259 (because the map-entry created by this custom type is not MapEntry or doesn't implement IMapEntry) ?

dnolen11:03:21

custom data structures that return vectors for map entries won’t work

juhoteperi11:03:36

Is the correct solution to call (MapEntry. k v nil) there? The hash__ parameter is different compared to Clojure

mfikes12:03:11

@juhoteperi I've shut off Windows CI on the ClojureScript clone I maintain in my GitHub account. The two issues with it currently (IIRC): It fails silently, so you don't know if a unit test failed. It started failing incessantly due to something else that I couldn't figure out, so I turned it off.

mfikes12:03:18

I'll see if I can find an old Windows CI (AppVeyor) run, but I think it was failing fast with something related to certificates and lein.

mfikes13:03:33

I've re-enabled Windows CI so we can see how it is failing

mfikes13:03:20

@juhoteperi Well, good news is that the fail-fast certificate thing has now been resolved (whatever it was), and Windows CI runs again 🙂 We can see failures https://ci.appveyor.com/project/mfikes/clojurescript

mfikes13:03:22

I think (apart from fixing any unit test failures there), the only other bit to be addressed is to make it cause the the AppVeyor build to fail if the unit tests fail (otherwise you have to manually look at them)

juhoteperi13:03:36

Okay, that was the issue that was reported on Reagent / https://dev.clojure.org/jira/browse/CLJS-2708

mfikes14:03:54

@juhoteperi I have access to Node on Windows. Let me know if there is a specific expression you’d like evaluated. I tried this, with the very last form (not involving a string) hanging for more intput I suppose.

> "abc".startsWith("a");
true
> "abc".startsWith("goog:");
false
> "abc".startsWith(goog:);
...

juhoteperi14:03:21

Could it be due to expression being split to two lines? And the file probably uses Unix Line-endings

juhoteperi14:03:05

Or hmm, could it be the quotation marks are lost for some reason

mfikes14:03:42

Yeah, it appears that, from the Windows CI errors, it thinks the call is syntactically intended to be id.startsWith(goog) with goog being a variable, but it sees a : and indicates a syntax error

mfikes14:03:23

@juhoteperi I have no clue why, but the above might be fixing it (using single quotes), and perhaps we are on to other errors that were masked by that one: https://ci.appveyor.com/project/mfikes/clojurescript/build/1.0.2

juhoteperi14:03:04

really strange

juhoteperi14:03:16

maybe they would need to be escaped or something

mfikes14:03:37

I can change those to single quotes as an experiment

mfikes14:03:04

Oh, actually that is Clojure code, nvm

mfikes14:03:21

Perhaps it is exhibiting something like

C:\Users\mfikes>node --eval console.log("goog:")
SyntaxError: Expected ')'
   at exports.runInThisContext (vm.js:53:3)
   at Anonymous function ([eval]-wrapper:6:1)
   at Module.prototype._compile (module.js:423:3)
   at Anonymous function (node.js:613:7)
   at nextTickCallbackWith0Args (node.js:452:9)
   at _tickCallback (node.js:381:13)

mfikes14:03:10

I’ll try this with ProcessBuilder from Clojure.

mfikes14:03:58

@juhoteperi This is probably equivalent to the theory you have:

user=> (shell/sh "node" "--eval" "console.log('goog:')")
{:exit 0, :out "goog:\n", :err ""}
user=> (shell/sh "node" "--eval" "console.log(\"goog:\")")
{:exit 1, :out "", :err "SyntaxError: Expected ')'\r\n   at exports.runInThisContext (vm.js:53:3)\r\n   at Anonymous function ([eval]-wrapper:6:1)\r\n   at Module.prototype._compile (module.js:423:3)\r\n   at Anonymous function (node.js:613:7)\r\n   at nextTickCallbackWith0Args (node.js:452:9)\r\n   at _tickCallback (node.js:381:13)\r\n"}

mfikes14:03:19

I’ll try actually using ProcessBuilder, but this is probably what shell/sh does anyway,.

mfikes14:03:50

Yeah, confirmed that you can repro directly with ProcessBuilder as you would expect:

user=>  (let [proc (-> (ProcessBuilder. ["node" "--eval" "console.log(\"goog:\")"])
 .start)
is (.getInputStream proc)
iw (StringWriter. (* 16 1024 1024))
         es   (.getErrorStream proc)
         ew   (StringWriter. (* 1024 1024))
         _    (do (.start
                    (Thread.
                      (bound-fn [] (pipe proc is iw))))
                  (.start
                    (Thread.
                      (bound-fn [] (pipe proc es ew)))))
         err  (.waitFor proc)]
  [err (str iw) (str ew)])
[1 "" "SyntaxError: Expected ')'\r\n   at exports.runInThisContext (vm.js:53:3)\r\n   at Anonymous function ([eval]-wrapper:6:1)\r\n   at Module.prototype._compile (module.js:423:3)\r\n   at Anonymous function (node.js:613:7)\r\n   at nextTickCallbackWith0Args (node.js:452:9)\r\n   at _tickCallback (node.js:381:13)\r\n"]

mfikes14:03:23

So, if we somehow can ensure all literal strings use single quotes, that would work around the issue.

user=> (let [proc (-> (ProcessBuilder. ["node" "--eval" "console.log('goog:')"])
 .start)
is (.getInputStream proc)
iw (StringWriter. (* 16 1024 1024))
         es   (.getErrorStream proc)
         ew   (StringWriter. (* 1024 1024))
         _    (do (.start
                    (Thread.
                      (bound-fn [] (pipe proc is iw))))
                  (.start
                    (Thread.
                      (bound-fn [] (pipe proc es ew)))))
         err  (.waitFor proc)]
  [err (str iw) (str ew)])
[0 "goog:\n" ""]

mfikes14:03:53

Or maybe there is some nice way to quote everything at the top level.

juhoteperi14:03:31

Those two double quotes I added are only ones in the file. Change those to single quites + add comment on the file about only using single quotes should be good enough.

mfikes14:03:24

Cool. I can add a JIRA patch. Then we can perhaps sort out the subsequent failures that occur even with that fix.

juhoteperi14:03:50

mainFields problems is probably very similar

mfikes14:03:02

Yep. OK. I’ll try that. Thanks!

dnolen15:03:43

@shaunlebron sure, but I think making a case for has to be grounded in the reality of the existing user base

dnolen15:03:22

this request comes up very infrequently and never with a actual use case that relates at all to how people are actually using ClojureScript

dnolen15:03:30

basically I’ve already rejected this at least 2 times over after past 3 years

dnolen15:03:59

and I haven’t seen anything happen in the JavaScript world that makes me think it’s delivering enough value to take it on

dnolen15:03:36

also the fact that we can process ES6 in build means you can do what you need in JS if it’s really a problem

dnolen15:03:52

just like Clojure there’s no reason to import all the primitives of the host, it’s not meaningful

dnolen15:03:16

sometimes you have to write some Java - so what?

shaunlebron15:03:16

yeah, that’s compelling

dnolen15:03:11

ES6 is really not that hard to write when you need it

dnolen15:03:15

goog require works

dnolen15:03:05

this was big part of the original idea of ES6 module processing in ClojureScript

dnolen15:03:12

write ES6 when you need to and it not suck

shaunlebron15:03:55

yeah, and that ES6 code can call cljs functions fine too

shaunlebron15:03:08

this may be the direction that I end up agreeing with

dnolen15:03:16

but also it’s gives an escape hatch

dnolen15:03:25

they’re going to keep adding sugar / keywords to JavaScript

dnolen15:03:32

I don’t want us to be chasing that all the time

juhoteperi15:03:29

If using ES6 with foreign-libs + :module-type :es6 there are some limitations currently calling cljs <-> es6 <-> cljs

shaunlebron15:03:43

would age and increased use of generators and async/await functions merit their inclusion eventually?

juhoteperi15:03:53

Not sure if ES6 is usable with Closure JS (`:libs`)

dnolen15:03:09

@juhoteperi it should be since i worked on that

dnolen15:03:20

that was one of the first things I tried

dnolen15:03:31

node_modules stuff becomes :libs

juhoteperi15:03:35

If es6 module is only module using a Cljs namespace, the namespace is not included in build

dnolen15:03:41

but you can supply :libs yourself

juhoteperi15:03:42

I think there is same problem with :libs

juhoteperi15:03:56

It is due to how build finds the necessary Cljs files to include in build

dnolen15:03:00

it’s possible there’s a regression

juhoteperi15:03:06

Oh, and this works with :none but not with optimizations

dnolen15:03:11

but anyways that’s neither here nor there

dnolen15:03:15

it should work

dnolen15:03:20

@shaunlebron of course if people start clamoring for this feature because it’s a common requirement I don’t see why not

dnolen15:03:24

but I’m not seeing that today

dnolen15:03:35

@juhoteperi right I think the issue is when you want to require a ClojureScript namespace

juhoteperi15:03:15

Yes, ES6 can import CLjs using import * from "goog:foo.cljs.namespace"; but this only works if this cljs ns is used by another cljs namespace

juhoteperi15:03:29

I'll open issue about this

dnolen15:03:07

@juhoteperi wouldn’t be so hard to fix, I guess we need to check ES6 modules for cljs imports as an earlier pass

juhoteperi15:03:35

I think add-js-sources adds files from :libs (including module processed files)

dnolen15:03:53

could fix it there

juhoteperi15:03:55

But add-dependency-sources is called before that

dnolen15:03:09

add-js-sources could be cljs aware

juhoteperi15:03:25

In :none mode all sources inside build source are compiled (if source is directory) but if optimization and :main is used, only that single file is used as entrypoint

dnolen15:03:26

I like to refer to this as legacy mode

dnolen15:03:30

but your point is?

dnolen15:03:52

(in fact I think maybe we should starting warning about the file system way of doing things)

juhoteperi15:03:29

Point is that this happens even without optimizations if source is single file

mfikes15:03:42

I’ve attached a patch to https://dev.clojure.org/jira/browse/CLJS-2708, tried confirming that it works, but honestly can’t figure out Node and Windows issues to even repro the original ticket description, so I’ve asked the original reporter to help see if the patch resolves it.

juhoteperi15:03:21

Though with :optimization :none there is no error about the missing namespace (because JS modules don't validate if required modules exist)

juhoteperi16:03:52

I fear add-dependencies step should somehow be recursive

juhoteperi16:03:24

Because we use intitial js-sources to find the used JS modules, and then those JS modules could add new Cljs namespaces, which use different JS modules etc.

dnolen16:03:37

you could easily make add-js-sources run to a fix point

dnolen16:03:56

recursion not needed

dnolen16:03:37

er well I guess it would be iterative

dnolen16:03:48

fix point meaning, we keep applying the source finding passes until the set of discovered things stops changing

juhoteperi16:03:32

I think it would have to also keep applying handle-js-modules (or we need to split / move this) so that new added JS modules get processed

juhoteperi16:03:21

handle-js-modules could be split into once function that can be called multiple times to add new JS modules to build, and then another function which will process files once all modules are found

👍 4
dnolen16:03:27

that’s probably better

dnolen16:03:49

yeah I would like for it to work that way

dnolen19:03:35

looks like I missed classpath libs REPL support - fixed in master - will probably do a spot release on Friday

dnolen19:03:45

if there’s anything you want me to look at let me know

richiardiandrea20:03:22

thanks for the transit-cljs fix as well

dnolen21:03:19

heh took a long time to really hit this one https://dev.clojure.org/jira/browse/CLJS-2710

dnolen21:03:14

related to CLJS-364/5