Fork me on GitHub
#shadow-cljs
<
2019-01-15
>
orestis14:01:52

I wonder how easy it is to add your own test runners for shadow-cljs, if e.g. I’d like to integrate Kaocha-CLJS, would it be possible? I’ve made an issue for Kaocha here: https://github.com/lambdaisland/kaocha-cljs/issues/2

thheller14:01:43

no that would most likely not be possible

thheller14:01:51

but shadow-cljs has testing stuff built-in

thheller14:01:00

no idea what kaocha offers over those

orestis14:01:30

I like the nice pretty-printed output of Kaocha for Clojure. The shadow test runner uses the default clojure.test reporting AFAICT?

thheller14:01:02

it does but you can switch that to whatever you like

orestis14:01:10

I’ve never used Kaocha-CLJS, but it seems it’s connecting to a CLJS repl via a websocket and gets the results out, then pretty-prints via Clojure.

thheller14:01:09

I have not used it any kaocha neither CLJ nor CLJS so I have no idea what even the goal is

thheller14:01:20

or the difference rather over normal lein test

orestis14:01:36

I haven’t used lein test so I don’t know what the “normal” is either 😄

thheller14:01:20

ok lets call it clojure.test

thheller14:01:24

that I know

orestis14:01:25

Let me rephrase my X/Y problem then — how can I get a pretty diff when my shadow-ran CLJS test fails?

thheller14:01:39

if you show me what a pretty diff is I can probably tell you

thheller14:01:53

dunno what that means

thheller14:01:53

that doesn't have a cljs impl. we can't use that easily without a whole bunch of extra crap we don't really want when running tests

thheller14:01:55

eg. I consider it vitally important to run tests against COMPILED builds

thheller14:01:07

not some REPL eval'd code

orestis14:01:23

For sure. I’m just showing that library as an example of what a “pretty diff” looks like

orestis14:01:09

I don’t have any strong preference on what should be the ideal pretty diff, but something that helps me narrow down the difference between two possible huge maps would be nice.

thheller14:01:13

I think thats the one most commonly used

orestis14:01:40

Yeah that’s what I’m after. Colors are nice, but secondary. Seems like I can just require that in my namespace and it’ll do the correct thing…

thheller14:01:24

don't like that it messes with the internals of cljs.pprint but I guess yes you just require it

thheller14:01:33

have not used that either. just saw it before somewhere

thheller14:01:31

this is the default test runner. you can write your own pretty easily

orestis14:01:35

Doesn’t seem to work under node:

ERROR in (select-tag-simple) (TypeError:NaN:NaN)
toggling a tag on and off works
expected: (= true (gamma.state/tag-selected? selected-state tag-id1))
  actual: #object[TypeError TypeError: Cannot read property 'append' of undefined]

thheller14:01:02

guess it expects a DOM? no clue

orestis14:01:27

Yeah I’ll keep hunting. Point taken about running tests against compiled builds — makes total sense.

thheller14:01:05

problem with koacha will be that is directly interops with the cljs analyzer and compiler

thheller14:01:27

that means that it won't understand any of the shadow-cljs features such as npm integration

orestis14:01:58

Seems like I should just look for a working pretty-printing library for cljs.test.

thheller14:01:23

dunno how hard porting deep-diff would be. probably pretty hard as pretty much all the libs used are CLJ only.

orestis14:01:37

And possibly trivial to fix — I’ll see if I can do it tomorrow.

thheller14:01:56

seems like the only thing really required is the pretty printing

thheller14:01:08

more than happy to to do the test integration stuff

thheller14:01:16

already applied

🙌 15
orestis14:01:24

I’ll keep you posted — ideally this should be something that shadow doesn’t need to change to accommodate (though perhaps in the spirit of reasonable defaults, could provide out of the box if needed).

thheller14:01:54

for the browser test target I already hooked up cljs-test-display which is great

☝️ 5
thheller14:01:09

something like that just for node would be golden

orestis14:01:08

I’d actually prefer to run my tests on a browser, but node was the easiest to start with (to run unit tests in a very light weight manner). How do you run CLJS tests in CI? Headless chrome?

thheller14:01:53

I don't at all but others use the karma stuff

orestis14:01:33

Ah, good to know. I’ll see how that looks like. Gotta run, thanks for everything (once more!)

richiardiandrea16:01:57

Agree that something like cljs-test-display for node would be awesome 😄

Daniel Hines19:01:36

I know hardly anything about how cljs works under the hood, so forgive me if this is a dumb question. I want to compile a cljs library into a single script file, load that file from a browser window into a script tag, and start using the library. Is that possible? Would I use shadow-cljs to do that?

lilactown19:01:18

@d4hines short answer, no

lilactown19:01:21

well, actually, I'm wrong

lilactown19:01:02

shadow-cljs (and the regular CLJS compiler) supports something called module splitting

lilactown19:01:26

so you could, as part of your build, have your library built as one module and your application built as another module

lilactown19:01:01

but if you want to be able to distribute that pre-built library and use it in other builds external to your application, it won't work

lilactown19:01:06

does that kind of make sense? 😬

Daniel Hines19:01:45

What you're saying makes sense. Why it is so does not make sense. What I'm trying to do sounds really trivial (in my head, anyway), and we do it in JS all the time: write a script that attaches a useful function to some global var, run that script from the browser, then reference that global var. Why is that so hard in cljs?

thheller19:01:46

@d4hines you mean use the library directly from JS?

thheller19:01:07

it is not hard in CLJS. you can totally do that

thheller19:01:33

the context matters though. how do you want to use it is the more important question?

Daniel Hines19:01:02

Yeah, context will help. Are you guys familiar with hyperfiddle? http://www.hyperfiddle.net It's basically a fiddle environment that sits on Datomic - write a query, write a quick view in cljs, and boom - you have a scrappy little app running on datomic cloud somewhere. It already can load custom script tags, which makes it pretty easy to load vanilla js stuff, but I'd like to consume cljs libs from this environment.

lilactown19:01:17

I have a desire for this same kind of capability (different context, same use case). I explored it a few weeks ago and wasn't able to find a way to do it easily

lilactown19:01:20

the problem is that when your CLJS code is compiled, it bundles with it things like the CLJS data structures and google closure code, that is incompatible with other separate CLJS bundles

lilactown19:01:44

so e.g. two CLJS applications, built from separate projects, will have their own copies of the vector/map/etc. data types. which means that if you try and pass a vector from one bundle to the other, the data types won't be compatible. the thing you passed it to will treat it as a foreign JS object

Daniel Hines19:01:22

Ok, so it's fundamentally a Google Closure thing, right?

Daniel Hines19:01:34

Not necessarily a Clojure thing?

lilactown19:01:21

there's definitely some interplay there. it's the way the CLJS compilation process works

lilactown19:01:02

what I would love is the ability to create a bundle that left out all of that stuff, and could be placed on a page with a foreign CLJS bundle. but when I asked dnolen about it, he said it wasn't supported at all

lilactown19:01:13

unfortunately for me, this was a fundamental requirement for the project I'm building. so what I ended up doing is serializing/deserializing the data across those bundle boundaries 😕

Daniel Hines19:01:38

... Well, our code is homoiconic. How terrible is that really?

Daniel Hines19:01:09

(granted, as soon as you reference anotehr npm lib, that's broken)

lilactown19:01:54

well, you can't pass code between the bundles without including self-hosted CLJS

lilactown19:01:09

I'm just sending EDN back and forth, without behavior

Daniel Hines19:01:18

Is this why self-hosted is a big deal?

thheller19:01:20

@d4hines are you going to use the library from JS code or from CLJS?

thheller19:01:42

so you are compiling CLJS? why have it pre-compiled then?

Daniel Hines19:01:03

That's a good question: maybe I don't have to?

lilactown19:01:26

@d4hines do you have to compile your CLJS code before you put it on hyperfiddle?

Daniel Hines19:01:51

No, you write it in the fiddle, like in JSFiddle or CodePen

thheller19:01:43

so the fundamental problem is that CLJS builds generally include all of cljs.core and some of the google closure library

thheller19:01:30

so if you already have CLJS in the page from hyperfiddle

thheller19:01:33

then CLJS from your lib

thheller19:01:39

then CLJS from your extra code

thheller19:01:49

means 3 separate cljs.core instances on the page

thheller19:01:51

which is bad

thheller19:01:07

you could strip out all of cljs.core and assume it provided

5
thheller19:01:18

but that only work when using no optimizations

thheller19:01:29

meaning huge builds

thheller19:01:39

clojurescript is different than JS because it has macros

thheller19:01:02

so even if you strip out cljs.core you still need access to the macros somehow when compiling your lib code

thheller19:01:17

so you could potentially end up compiling against a different CLJS version

thheller19:01:33

in short it is not advised at all to do this in CLJS

thheller19:01:46

you can make something work but its gonna suck big time

thheller19:01:53

if you just want a standalone library that provides a JS API thats not problem

lilactown19:01:05

it sounds like hyperfiddle already has self-hosting. so if your library is self-host compatible, you might be able to do what you were saying where you load the file and eval it

Daniel Hines19:01:39

Self-hosting in hyperfiddle means running the backend services yourself. You're talking about the host language right?

lilactown19:01:57

I mean "self-hosted ClojureScript"

lilactown19:01:15

i.e. you can (eval '(println "foo")) in ClojureScript

lilactown19:01:53

in "normal" CLJS projects, this isn't supported (and usually isn't needed 😉 )

lilactown19:01:12

there are ways to setup your project to use it though. it sounds like hyperfiddle already does, since it's executing your CLJS code in the browser

Daniel Hines19:01:33

Yes, that's all true.

Daniel Hines19:01:05

That will break down as soon as an external lib (like an npm lib) is referenced though, right?

lilactown19:01:46

right, if you need some external dependency, you'll need to load that too

lilactown19:01:53

and reference that in a way that it can be accessed on the page

lilactown19:01:22

honestly I would ask @dustingetz how he would try and do this 😛 he's building hyperfiddle

Daniel Hines19:01:49

I was just chatting with him.His solution (and this sounds much more practical), is to just host hyperfiddle yourself, and include whatever deps you need at build time. I just wondered why we can't do what JS can do, which ya'll answered very well (thanks!)

thheller19:01:43

we can do everything JS does. its not different really, JS has pretty much the same restrictions

thheller19:01:08

just remember that you are not just using YOUR library. you are also using cljs.core and some of the closure library

thheller19:01:41

to compare with that a JS library assume that your JS lib wants to use immutable-js + lodash or so

thheller19:01:03

so you are paying the extra overhead for loading those too

thheller19:01:25

if you accept that then CLJS is no different

thheller19:01:55

you just can't do jQuery style libs where everything assumes that there is a global jQuery loaded somewhere

thheller19:01:14

stitching things together via some global variables won't work

Daniel Hines19:01:44

Not even with eval?

Daniel Hines19:01:53

Like lilactown was saying?

thheller19:01:43

eval fundamentally needs access to the source which isn't what you wanted AFAICT

thheller19:01:53

and no not even with eval

lilactown19:01:08

so, for my use case. I am building a dev tool that I want to allow people to extend with their own 3rd party plugins

lilactown19:01:40

would I be able to instruct people building those plugins how to build their CLJS code in a way that could interface with my main dev tool application?

lilactown19:01:30

cause right now, the solution I have is to communicate via stringified-EDN or transit and hand them a DOM node to render their plugin to 😕

thheller19:01:39

@lilactown not sure I understand but no you cannot pass CLJS collection instances between different cljs.core versions

thheller19:01:43

they will not be compatible

lilactown19:01:48

right. I've gotten that far

thheller19:01:50

normal JS interops works as normal

lilactown19:01:14

would I be able to instruct people building those plugins how to build their CLJS code in a way that could use my application's version of cljs.core?

thheller19:01:10

if you are fine using no optimizations at all?

thheller19:01:20

but generally no that won't work

lilactown19:01:22

it is a developer tool 😜

thheller19:01:37

you want to build something REBL like right?

lilactown19:01:39

OK, I'll have to think on that. there will be times that this lives on the same page as a user's application at development time, so I also need to make sure I don't clobber their application's cljs.core etc.

thheller19:01:06

thats gonna be much much much harder in CLJS than CLJ

thheller19:01:55

problem is ensuring the library code uses the same version of everything your code uses

thheller19:01:24

but unoptimized code can be combined at will

thheller19:01:36

just need to "trick" the loader a little

Daniel Hines19:01:13

I really appreciate the explanations guys.

lilactown19:01:34

with shadow-cljs, would :output-wrapper be able to ensure I don't clobber a user's application?

thheller19:01:19

not sure what you mean by that

thheller19:01:31

I thought the goal is to incorporate user code into yours?

thheller19:01:57

but honestly don't waste your time trying to do this

thheller20:01:08

protocols and all that stuff won't work properly

thheller20:01:23

its just gonna be annoying as hell to work around all the limitations

lilactown20:01:26

there's 3 types of things that might be living on the page: - user's application - REBL application - 3rd party REBL viewers

thheller20:01:25

ah ok. yeah don't do that

thheller20:01:32

load REBL code in an iframe or so

thheller20:01:40

do not load it in the users application

thheller20:01:09

but yes :output-wrapper would isolate everything

lilactown20:01:17

well. right now, I'm compiling the REBL application and keeping it to one module. so it should be isolated

lilactown20:01:36

the user's application communicates to it via EDN + some JS interop

lilactown20:01:52

what I'm trying to figure out now is how the 3rd party REBL viewers will hook into the REBL application

thheller20:01:07

probably want to use a plain JS API

thheller20:01:12

no CLJS collections

thheller20:01:43

not sure how that would work in a REBL style application given its protocol heavy nature

lilactown20:01:47

atm, the user installs a library in their application code that handles serializing and sending data to the REBL application, and listening for events and calling nav / datafy

thheller20:01:35

but thats gonna be pretty inefficient

lilactown20:01:11

in my testing I haven't noticed any real perf problems. but I've yet to use it in really complex applications

lilactown20:01:39

but the upside, is that it works in Node.js too

lilactown20:01:03

and eventually JVM Clojure

lilactown20:01:20

it just connects over a websocket

thheller20:01:21

still no clue how any of that would actually work though

thheller20:01:32

nav + datafy are protocols and sometimes on metadata

thheller20:01:48

you can't serialize objects that have those in metadata

thheller20:01:05

well you can but you are gonna lose the meta impls

lilactown20:01:20

the way it works right now is that when you want to send something to the REBL app, you tap> it

thheller20:01:52

gotta go, bbl

👋 5
lilactown20:01:19

the library the user installed, adds a tap listener that adds the data as an entry to a global atom and sends it over the wire for viewing with an ID that it can use to reference the entry in the atom

lilactown20:01:43

anytime the REBL app needs to call nav or datafy, it sends an event back to the user's application to call nav or datafy on the entry in the atom and then send the result back to the REBL app