Fork me on GitHub
#cljs-dev
<
2020-05-29
>
plexus07:05:48

Something I've been meaning to bring up. shadow-cljs has this functionality for test builds where it scans the filesystem for test namespaces, and implicitly makes them dependencies of a main namespace (a test runner ns). This makes sure not just that they are included in the build, but that they are compiled before the runner, so that when the runner compiles it can inspect the compiler env to find all tests. This makes it possible for a third party runner like Chui to exist, where people can just point at a runner ns that's provided by the library, instead of having to write their own where they manually require all their tests. I brought this up with @bhauman and he's considering if it makes sense to have something similar in Figwheel, but having it directly in the compiler so it's available regardless of the tool you're using would be even better. Thoughts?

👍 8
dnolen12:05:53

this sounds like a divergence from Clojure? You have to require the test nses for run-tests to work

dnolen12:05:46

also I don't really understand the "manually require" - the test runners that don't require analysis stuff just support a simple naming pattern matching which works just fine.

dnolen12:05:14

so I don't see what the gain is for end users - it's already easy right now - but maybe I missing a subtle point

thheller12:05:35

the problem doesn't exist in clojure since the test-runner can just dynamically require all the tests it wants to run

thheller12:05:52

in CLJS the burden is on the user to create one namespace that requires all the other test namespaces

thheller12:05:28

so in shadow-cljs instead there is a generic test runner ns everyone can re-use and the requires are just provided by the build

dnolen12:05:39

yes I understand the dynamic require thing - but there is no such thing as a test-runner in Clojure

dnolen12:05:44

this is a tooling thing

dnolen12:05:36

a test-runner implies a lot more than just dynamically requiring - which implies you have to know what look for anyway (filtering)

thheller12:05:56

that is not the topic here

dnolen12:05:03

but regex based matching a la cljs test runner accomplishing the same thing

dnolen12:05:15

the conversation is about including something

dnolen12:05:28

and it has to be weighed against existing easy stuff

dnolen12:05:40

also the above is really problematic in that doesn't elaborate

dnolen12:05:47

so in shadow-cljs how does a user filter?

dnolen12:05:59

or control what sets of tests - or run a specific test

thheller12:05:59

:ns-regexp "-test$", and a couple more options. can also specify by name manually

dnolen12:05:32

right again this all smells of affordances

dnolen12:05:40

not something that Clojure does or cares about

dnolen12:05:47

thus probably not something we should care about

dnolen12:05:00

other than making it possible for someone else to solve the problem

thheller13:05:05

I think thats what the initial question is about. I think currently this isn't very easy to do.

thheller13:05:19

assume that the test tool already filtered and found all the namespaces it wants to test

thheller13:05:37

how does it tell the compiler to compile those before compiling the additional "test-runner" itself?

thheller13:05:03

ie. if the test-runner doesn't have the requires itself parallel-build may just decide to compile it whenever without actually waiting

dnolen13:05:11

I'm even more confused now by not easy

dnolen13:05:25

the compiler can compile forms, you can create test runner in memory and pass it into build no?

thheller13:05:10

sorry I don't really know what the state of the official APIs for this is. just wanted to make clear why this is useful to have from the users perspective.

dnolen13:05:54

gimme one second, browser REPL has an example

dnolen13:05:18

yes the compiler can take source forms directly

thheller13:05:47

that should be enough then

plexus13:05:02

does this allow me to solve this for my users in a way that is tool agnostic?

thheller13:05:43

probably no because neither shadow-cljs nor figwheel are "in the loop" for that build path

plexus13:05:33

I'm trying to imagine how this would work in practice, say someone is using cljs.main --compile, what do I tell them?

dnolen13:05:52

cljs.main doesn't need to be involved, we're talking about writing a test runner easily.

dnolen13:05:52

build can take a series of ClojureScript forms in memory

dnolen13:05:02

you don't even need to code gen (to disk)

dnolen13:05:13

you can fabricate the test runner

dnolen13:05:24

you can set :main

dnolen13:05:29

there's nothing else

plexus13:05:27

yes, I can invoke the compiler myself. I get that. I don't want that. People have their own setup with their own tools, they figured out how they're dealing with dependencies and externs and whatever. I just want to add something to their build.

plexus13:05:04

kaocha-cljs currently uses repl-env which means we implicitly invoke the compiler, this leads to no end of trouble because we don't replicate people's actual setup

dnolen13:05:10

this is going into the weeds about how you would like it work 🙂

dnolen13:05:24

I'm saying it works right now, maybe not how you want it work but it is already trivial to write a test-runner

dnolen13:05:52

adding test running features to cljs.main is not going to happen

dnolen13:05:07

I just don't understand what your concerns are

dnolen13:05:29

clj -m my.cool.test-runner-tool -co build.edn --run-tests

dnolen13:05:58

how does this not mean you can use your own tools, deps, etc.?

plexus13:05:21

so my.cool.test-runner-tool knows about shadow vs figwheel vs cljs.main?

dnolen13:05:38

why does it need to know about those things

dnolen13:05:48

if something has to know about those things - something is very, very wrong

dnolen13:05:07

all those tools already taking different options that are different from cljs.main

dnolen13:05:15

so there's nothing to reconcile

plexus13:05:53

kaocha-cljs currently doesn't work for a lot of shadow users because shadow's compilation is subtly different. that's why we want to get out of the job of invoking the compiler

dnolen13:05:22

I don't see how ClojureScript can solve this problem - the build tools are incompatible

dnolen13:05:43

I don't see how this is different from Lein, Boot, deps.edn

plexus13:05:21

the thing is I can write a library and people can include it using lein, boot, deps.edn. I'd like to write a test runner that people can include with shadow, figwheel, cljs.main

plexus13:05:36

but there is one aspect that I can't handle as a library

plexus13:05:50

and that brings us back to the top

dnolen13:05:51

let's rewind a bit

dnolen13:05:00

because I do not understand now what you are saying

plexus13:05:08

ok... let me try to rephrase or explain better the use case

dnolen13:05:35

well first let's elaborate what I see the issue w/ what you're saying is

dnolen13:05:41

and maybe you can tell me how you could solve it

dnolen13:05:44

to me you can't make a generic thing that works with all these build tools because they might inject something tooling specific

dnolen13:05:56

ClojureScript can't possibly know about these tooling specific things

dnolen13:05:22

you could write a library to create test-runner namespaces sure (but this is like ~100 lines of code for the basics)

dnolen13:05:34

but every tool actually needs to integrate that

dnolen13:05:44

and maybe they want to customize it

dnolen13:05:05

so a library doesn't appear to me to offer anything

dnolen13:05:05

the other option - which is perfectly fine and it's what I currently do is not care about these tooling in my testing scenario since these inevitably ends up as CI things

dnolen13:05:17

so who cares about shadow, figwheel, etc.

dnolen13:05:31

you just going compile w/ your test-runner and run Node.js or Puppeteer

plexus13:05:28

right, so that's all fine, I think we're getting off track here. I don't care about these tools apart from the fact that they can compile ClojureScript. They may have their own testing features, that's not what I'm concerned with.

plexus13:05:11

based on this chui offers an API for dynamically invoking tests. That API is used in browser testing UI, as well as in a component that gets tests to execute and reports on the results via a websocket

plexus13:05:39

this is all just standard ClojureScript, you just add it to your build and start it with a browser or node or whatever, we don't care.

dnolen13:05:04

this seems REPL-centric?

plexus13:05:08

but we can only know about the tests that have been compiled by the time that macro runs

dnolen13:05:12

since you're reflecting?

plexus13:05:49

currently there are two ways this can work, either you do

(ns foo (:require <all-the-tests>))
(capture-test-data!)
or you let shadow add those requires for you

plexus13:05:08

maybe there are better ways to solve this.... I don't know. As Thomas pointed out we lack the dynamic require so we can't scan the filesystem the way we do in Clojure, making it harder to provide a similar user experience

dnolen13:05:13

Some thoughts

dnolen13:05:27

it maybe we're missing something small - you need some dep graph for capture-test-data! to find/filter on

dnolen13:05:34

the REPL has a feature :analyze-path which just analyzes everything in a directory

dnolen13:05:54

this could be generalized to feed something like that macro

bhauman13:05:16

this also happens when you use a directory as a build input

dnolen13:05:19

finally you need require outside of ns but we already added that

dnolen13:05:28

@bhauman right good point

dnolen13:05:54

@plexus so that's probably enough a directory for build, and emitting require

plexus13:05:20

I did think of that but assumed I wouldn't be able to emit a require from a macro. Does that work?

dnolen13:05:49

we could probably make it work in (do ...) if we didn't already do that

bhauman13:05:50

the scan part is still missing?

dnolen13:05:14

like scanning for tests? analyzer apis should work here - not sure about doing test specific stuff unless Clojure has something similar

dnolen13:05:32

cljs.analyzer.api

plexus13:05:36

alright, this is promising. I can experiment with that.

dnolen13:05:20

@plexus if do doesn't work then a patch is welcome, I don't think this will be very hard

dnolen13:05:16

@plexus but also I don't think you need it?

dnolen13:05:33

(require ns-spec ...)

dnolen13:05:44

you can have N

bhauman16:05:20

my gosh getting simple re-bundle on npm_deps.js file change is much harder than I imagined, but almost done

dominicm16:05:05

What's the complexity? Curious more than anything :)

bhauman16:05:06

its mostly with extra-mains and stuff

bhauman16:05:48

had a simple-stateful file-changed function and it was failing because well its stateful

bhauman16:05:11

and a single npm_deps.js file is shared between the build and an extra-main build

bhauman16:05:45

so hard to detect a change twice

dominicm16:05:51

I see, so the state of bundle is external.

dominicm16:05:30

Hmm, maybe you just need to check if the bundle is newer than the npm_deps.js?

bhauman16:05:21

nope 🙂 npm_deps.js gets output on every run!!

bhauman16:05:51

so you may start to see these problems accumulating

bhauman16:05:21

so I have to have a checksum ie hashcode of the contents

dominicm16:05:27

Ah! So you'd need to do a contents based hash then.

bhauman16:05:53

yeah I thought this would take an hour, you know the story

dominicm16:05:07

If you don't know, there's a crc32 implementation built into the jvm. It's fairly nice to work with.

bhauman16:05:10

now I’m having to store that hash in a scoped manner

bhauman16:05:32

actually (.hashCode ) works just fine

bhauman16:05:55

not perfect but I’ve never seen it fail

dominicm16:05:02

I guess it's short and if you're slurping it... I've used crcs with very big files

bhauman16:05:42

yeah main.js and deps are both small files

dominicm16:05:06

But I have 10,0000 npm deps!

bhauman16:05:34

not in npm_deps.js you don’t 🙂

bhauman16:05:03

its only the ones you require directly

dominicm17:05:23

Haha. The fact you put right??? Let's me know you'd have believed me if I'd said that's how many I directly required.

bhauman17:05:30

hey we’re developers, anythings possible

😂 4