Fork me on GitHub
#cljs-dev
<
2016-08-16
>
mfikes01:08:29

Clojure got specs for the ns macro today. Perhaps for ClojureScript there’d need to be compiler-level support for specs on special forms to pull that off. :thinking_face:

dnolen01:08:19

@mfikes: special forms being a special case doesn’t seem like a problem to me

darwin14:08:21

just testing v1.9.216 and repl stopped printing warnings when using undeclared vars, e.g. WARNING: Use of undeclared Var cljs.user/abc at line 1 <cljs repl> https://gist.github.com/darwin/a737a966642f8942cf2c0b09bc22fc20 is this by design?

darwin14:08:36

last working version seems to be 1.9.93, broken since 1.9.183

dnolen15:08:44

@darwin: good catch, fixed in master

darwin15:08:37

just tested it, works as expected, thanks for quick fix

anmonteiro16:08:50

is it a hard requirement that CLJS sources need to be analyzed/compiled in dependency order?

anmonteiro16:08:53

if yes, why? so that the compiler env has the necessary information from dependents?

thheller17:08:36

@anmonteiro: yes, for the reason you mentioned

anmonteiro17:08:52

@thheller: can you point me towards an example in the codebase?

thheller17:08:05

example for?

thheller17:08:39

the compiler populates the env/*compiler* information (ie. :cljs.analyzer/namespaces)

anmonteiro17:08:44

@thheller: yea, just found it

thheller17:08:46

they contain the defs

anmonteiro17:08:49

it would break this case, for example

thheller17:08:33

the compiler needs all the defs for warning and optmized :invoke code

thheller17:08:39

bunch of other stuff as well

anmonteiro17:08:57

makes total sense, I was kinda expecting that wouldn’t be true though 🙂

thheller17:08:22

nah you always need that information

thheller17:08:44

caching just writes the analyzer information to disk so it can skip the compilation on the next go

dnolen18:08:23

@anmonteiro: any particular reason you were interested in the answer?

anmonteiro18:08:41

I thought it would be a very useful addition

anmonteiro18:08:07

was trying to figure out where to hook up loading dependencies that are not part of the ns form

dnolen18:08:26

@anmonteiro: that ticket is far less ambitious than it sounds

dnolen18:08:16

ideally we just assume cljs.user for the ns

dnolen18:08:43

require in arbitrary locations of the source is a non-goal

dnolen18:08:54

it would only appear at the beginning of the file

anmonteiro18:08:28

@dnolen: hrm, it seems that it would be easier to make it work if it only appeared at the beginning

anmonteiro18:08:57

I was going for require at arbitrary sources, and currently thinking of an approach like you did with preloads

anmonteiro18:08:14

where we’d hook up the missing dependencies there, and reorder sources again

dnolen18:08:41

yeah sounds like more than we need

dnolen18:08:29

that ticket is driven mostly by the desire to make more kinds of code portable

dnolen18:08:33

in this case typical user scripts

dnolen18:08:01

where you wouldn’t bother with an ns, or a data readers file etc.

anmonteiro18:08:01

@dnolen: yep, I think I understood that by the description. so in that case would require be a special form?

dnolen18:08:15

but some thought needs to given here to require-macros, import, refer-clojure etc.

dnolen18:08:23

the typical things you might use without a ns

anmonteiro18:08:43

that’s part of the reason why I was going for the extra mile

dnolen18:08:46

the simplest thing would be of course to gen a random ns

dnolen18:08:52

cljs.user.foo1

dnolen18:08:09

but that would break some expectations about all that stuff being in cljs.user

anmonteiro18:08:27

having macros in cljs.core and supporting require at arbitrary locations would probably solve the other cases

anmonteiro18:08:36

or am I missing something?

anmonteiro18:08:03

by macros in cljs.core I mean require, require-macros, etc defined in cljs.core

dnolen18:08:14

but not interested in require at arbitrary locations

dnolen18:08:19

it cannot be made to work at all

dnolen18:08:32

since it implies code loading

dnolen18:08:39

which you can’t do at arbitrary places

dnolen18:08:48

and not interested in any kind of code reshaping at all

anmonteiro18:08:56

what would the code reshaping be in this case?

dnolen18:08:34

well qualifying “it cannot be made to work at all”

dnolen18:08:41

I can think of really complicated ways to solve this problem

dnolen18:08:53

but we’re not going to go there

anmonteiro18:08:43

@dnolen: does an approach like I described earlier qualify as “really complicated ways”?

anmonteiro18:08:12

as in, save missing requires in the compiler env when we first process inputs

anmonteiro18:08:37

and add those to the sources later when we have expanded the eventual requires

dnolen18:08:43

the semantics seem wrong

dnolen18:08:49

in Clojure require means side effects

dnolen18:08:56

everything is present after the require

dnolen18:08:06

deferring loading is just introducing broken semantics

dnolen18:08:16

as opposed to just saying you can’t do that and require must come first

dnolen18:08:23

so all the usual expectations hold

anmonteiro18:08:49

OK that makes sense to me, but I don’t understand why that would be deferring loading

dnolen18:08:13

then I don’t know what you mean about allowing require in arbitrary locations

dnolen18:08:23

since you cannot satisfy what I just described

anmonteiro18:08:57

My idea was to have a require macro that when expanded during analysis would side-effect the compiler env

dnolen18:08:07

(require ‘foo) (set! foo.bar 1) (require ‘bar)

anmonteiro18:08:10

so this would not be deferring

dnolen18:08:26

side-effecting the compiler environment doesn’t do anything about runtime expecations

dnolen18:08:46

in Clojure I can do whatever I want right after a require against the loaded ns

dnolen18:08:55

I can interleave requires as much as I want

dnolen18:08:03

I can put (require …) in a fn

dnolen18:08:46

allowing require to appear in random places if you can’t support this stuff isn’t really useful since too many broken expecations

anmonteiro18:08:59

I don’t like the idea of introducing 5 new special forms, though

anmonteiro18:08:06

or is that not a problem?

anmonteiro18:08:27

require, use, refer-clojure, require-macros, use-macros would have to be supported, I believe

dnolen18:08:09

yes it’s annoying but not really a problem

dnolen18:08:19

for users require is probably going to be enough

dnolen18:08:25

but for completeness we need the others

thheller18:08:50

why introduce require if it is never going to work as in clojure?

dnolen18:08:12

@thheller: because lots of perfectly code fine is written without relying on it’s full power

dnolen18:08:46

there’s little point in excluding a large class of typical Clojure programs

thheller18:08:47

in clojure yeah but in cljs?

dnolen18:08:52

one off scripts etc.

dnolen18:08:13

data literal files - whatever

dnolen18:08:44

people have been asking for a way to do something like this since pretty much the beginning of ClojureScript

dnolen18:08:48

they don’t care about random require

anmonteiro18:08:05

@thheller: my main motivation for giving a look at this ticket was to have support for extensible tagged literals down the road

dnolen18:08:14

yes that’s the big one

dnolen18:08:24

a huge hole in ClojureScript functionality

dnolen18:08:06

we don’t support data_literals.cljc at runtime and compile time

thheller18:08:45

what does require have to do with that?

dnolen18:08:12

how are you going to load something without a namespace

thheller18:08:00

well you can't load it when you "discover" the tag

thheller18:08:17

since require is async in cljs

thheller18:08:24

can't really do sync as in clojure

dnolen18:08:44

I do not know what you are talking about now

thheller18:08:01

ah well I'm too focused on browser stuff I guess

thheller18:08:07

you can make it work in node

thheller18:08:27

as you said in clojure you can require and use it directly after

thheller18:08:44

in cljs require would need to do an async fetch since you can't do blocking IO in the browser

thheller18:08:58

so you cannot use it directly after

thheller18:08:45

I fail to see how require (if restricted to top level) is any different from putting it in the ns

dnolen18:08:52

I still don’t see what this has to do what @anmonteiro and I were discussing

dnolen18:08:00

I well aware of how require works

thheller18:08:26

you were discussing require as a special form

thheller18:08:58

so I assumed you were talking about being able to do (require 'some.thing) as in clojure

thheller18:08:19

but given how closure works that cannot work

dnolen18:08:38

this conversation is gone of the rails for me and I’m in the middle other things

thheller18:08:39

closure not clojure

dnolen18:08:48

we have tickets for this stuff for a very specific purpose

dnolen18:08:00

and if someone wants to implement, I’m interested

thheller18:08:24

yeah, just saying that given the constraints of a browser it cannot work unless I'm completely missing something

thheller18:08:32

but happy to be proven wrong

thheller18:08:51

ah well you basically made the exact same points I was aiming for .. slack didn't show the complete history .. sorry for the noise

dnolen18:08:23

@thheller: I thought maybe you were missing some of the backlog 🙂

anmonteiro18:08:49

@thheller: @dnolen if I understand well, GClosure requires in CLJS are static (compile-time), right? So a static, compile-time require could be made to work in arbitrary parts of a file, if I understand correctly

anmonteiro18:08:12

Does this imply, however, broken semantics?

anmonteiro18:08:26

Was this that you were referring to as broken semantics?

dnolen18:08:54

@anmonteiro: we need to be clear about Clojure semantics vs. what ClojureScript supports

dnolen18:08:02

in Clojure ns can appear anywhere

dnolen18:08:06

we don’t care about that

dnolen18:08:17

we only care about the typical case - at the top of the file

dnolen18:08:34

so we rule out a class of Clojure programs

dnolen18:08:44

but it’s not a particularly meaningful class

anmonteiro19:08:16

I think I'm not explaining myself well enough as to what I want to achieve

dnolen19:08:33

ok now that we understand that

dnolen19:08:56

we also want to support a small but important set of Clojure programs that use require

dnolen19:08:06

the problem is require -> goog.require

dnolen19:08:24

goog.require probably does not do what you think it does

dnolen19:08:18

it has several complected purposes

dnolen19:08:30

the simplest is for file concatenation statically

dnolen19:08:43

however it also has behavior at runtime

anmonteiro19:08:53

@dnolen: my intentions with this patch would be to support require at the top-level, but not at arbitrary locations of a file (e.g. inside functions)

anmonteiro19:08:03

I think this can be made to work

anmonteiro19:08:17

and IMO would be probably better than a require special form

dnolen19:08:20

it cannot be made to work as I said above

dnolen19:08:48

more importantly you need to ask the question

anmonteiro19:08:50

it would obviously not have the same semantics as in Clojure

dnolen19:08:58

what is the point of supporting it at the top-level?

dnolen19:08:11

vs. just at the top of the file

dnolen19:08:26

getting out of making special forms is not a good reason

anmonteiro19:08:58

e.g. things like this would work:

(ns cljs.require-test
  "Tests require support outside of the ns form")

(require '[clojure.test :refer [deftest is] :rename {is is?}]
         '[cljs.spec :as s :refer [spec? spec] :rename {spec foo}])

anmonteiro19:08:37

then again, not sure if desirable or not, also why I need this discussion to be clarified

dnolen19:08:32

but what if I call with-redefs or set! right after that require?

dnolen19:08:52

you’re suggesting something which requires more explanation about cases that don’t work

anmonteiro19:08:55

pretty sure they would work

dnolen19:08:12

because you don’t understand how goog.require works

dnolen19:08:47

goog.require doesn’t do what you think

dnolen19:08:50

it doesn’t trigger loads

anmonteiro19:08:10

@dnolen: that may be the case. But by making this require static, as if the require were in the NS form, wouldn’t it work?

anmonteiro19:08:23

this is what I was thinking about all along

dnolen19:08:28

if it doesn’t trigger loads

dnolen19:08:41

then user expectations about side effects will be wrong

anmonteiro19:08:46

which I why I wanted to insert it in the build pipeline

anmonteiro19:08:52

since it doesn’t trigger loads

dnolen19:08:00

putting in the build pipeline doesn’t fix side-effect order issues

anmonteiro19:08:03

we’d have to load the dependencies for them

dnolen19:08:22

anyways you need to think about what I’m saying here

dnolen19:08:26

I’m repeating myself

dnolen19:08:49

what I’m suggesting just gets rid of all expecations

dnolen19:08:58

you don’t need to think about it at all

anmonteiro19:08:53

I’ll make sure to re-read the conversation history to get the full picture

anmonteiro19:08:18

@dnolen: but I don’t think I understand what you mean by the expectations about side-effects

dnolen19:08:20

@anmonteiro: I also do not understand your example above

dnolen19:08:28

that isn’t solving any problem that I can see

anmonteiro19:08:44

my example above is supposed to be equivalent to having the :require in the NS form

dnolen19:08:53

but what problem is that solving?

dnolen19:08:03

just because you can do that isn’t a reason to do it

anmonteiro19:08:26

no it’s not solving any problem by itself

anmonteiro19:08:38

but wouldn’t it allow the NS form not to be mandatory then?

dnolen19:08:09

I’m just talking about assessing the value of something

dnolen19:08:22

we both want to solve the problem - I only care about something very narrow

dnolen19:08:43

you want to solve a wider problem - but I’m having difficulty seeing see how it doesn’t bring in a lot of orthogonal issues along for the ride

anmonteiro19:08:37

yea, that sounds about right

anmonteiro19:08:59

another problem is that I’m failing to see what orthogonal issues my solution would imply

dnolen19:08:30

@anmonteiro: what actual good is the increased flexibility buying you here?

dnolen19:08:38

(implementation details are not under consideration)

anmonteiro19:08:41

(I do understand that the semantics would be slightly different from Clojure’s)

anmonteiro19:08:00

but a lot of things in Clojurescript are different because of its compilation model

anmonteiro19:08:07

@dnolen: I think I’m arriving to an understanding. Having the increased flexibility of putting require wherever at the top level just doesn’t do anything besides just that, which is not good enough by itself

dnolen19:08:34

yes, there should be something else we’re getting for this flexibility

dnolen19:08:50

besides some confusion about what this does and doesn’t let you do

anmonteiro19:08:19

gotcha. everything would still be static because of goog.require

anmonteiro19:08:55

I had already understood that

anmonteiro19:08:41

@dnolen: OK thanks so much for taking the time to clarify all of this. I’ll pursue the special form approach

dnolen19:08:04

@anmonteiro: re side-effect problems

dnolen19:08:18

(require ‘[foo.core]) (def x foo.core/y) (set! foo.core/y 5) (require ‘[foo.util]) (def z foo.core/y)

dnolen19:08:31

the problem is even at top-level you have thorny stuff

anmonteiro19:08:17

@dnolen: in that case I suppose CLJS would only be able to require foo.core once

dnolen19:08:32

if you imagine the second require uses foo.core, sorry

dnolen19:08:48

anyways you can come up with lots of weird stuff

dnolen19:08:54

and my experience is that users will try everything

anmonteiro19:08:15

so limiting user options will ensure correct behavior

anmonteiro19:08:20

I can understand that

anmonteiro19:08:41

@dnolen: then again, doesn’t having support for require outside of NS imply that we can do what I did in my example above?

dnolen20:08:31

@anmonteiro: perhaps for the case where require appears right after the ns form, but this isn’t something we need to advertise or anything

dnolen20:08:05

that said I’m inclined to say no for now and relax later

dnolen20:08:13

it makes warning/error validation simpler

anmonteiro20:08:37

@dnolen: I imagine parse-ns needs to support both for multiple requires/uses to be additive

dnolen20:08:10

@anmonteiro: maybe, but maybe not

dnolen20:08:31

this ticket is tricky enough where it might be useful to write out a plan in the comments first

dnolen20:08:37

and I can give feedback about the plan

anmonteiro20:08:45

sounds good, I’ll do that

anmonteiro20:08:42

I’m probably missing some things that will only become clear after your feedback

dnolen20:08:22

@anmonteiro: thanks! will provide feedback later today, tomorrow morning at the latest

anmonteiro20:08:28

yeah, no rush