How does shadow handles npm transitive deps? Core has deps.cljs I wonder whether I can rely on it in shadow

What do you mean? npm handles npm transitive deps

@lee.justin.m say you have a cljs dep that includes an npm dep. You want to consume that in shadow

you have to add it to your package.json. e.g., reagent requires react. you have to add react to your package.json and install it via npm

that’s why there is the whole cljsjs shim thing

Uhm, I thought this was a thing in shadow as well, but maybe Thomas was against it and therefore not implementated: https://dev.clojure.org/jira/browse/CLJS-1973

In your scenario a user of many lib should know and include all the npm transitive deps manually

i’m not 100% sure what he’s talking about, but i think npm-deps actually shells out to npm

in shadow, you need to install npm modules yourself, which you can do via npm or yarn

for example: https://shadow-cljs.github.io/docs/UsersGuide.html#npm-install

So yeah, that means that I would need to know what npm deps each lib is using. Example, cuerdas uses xregexp, if I want to use it I need to install it manually?

Yeah well the answer seems yes

It feels a bit backwards

it’s great 🙂

having manually packed npm dependencies is fine when you have one-offs, like in cuerdas. but the moment you start including, say, more than one react library, it’s a nightmare

Say you are using 100 cljs libs including 100 npm deps...I do not think this method scales 😄

on the contrary, it’s the only method that would scale

you would never get 100 cljs libs with 100 npm deps to work if they all had hardcoded crap installed

Maybe I am missing something...

What does hardcoding means?

Each cljs dep declares what it is using

cuerdas, for example, literally includes the javascript in its distribution. so if some other library has xregexp, you’ll get two copies

Yep that is not good ok

in some cases, like react, it just won’t work

Vanilla cljs allows an :npm-deps in deps.cljs in your jar

npm has a rather sophisticated algorithm to flatten the tree

So that you can transitively know which npm dep you will be using

And shell out to npm install

Cuerdas' current way is not optimal if it does the bundling and I agree with you there

yea i don’t know what npm-deps is doing under the covers (I understand that it uses npm, but beyond that, it’s a mystery to me)

i think at this point in time, most libraries (every one that I’ve looked at) that includes npm stuff just bundles the npm in the distribution

Yeah that's bad, but: > ClojureScript libraries that package foreign dependencies can also benefit from these enhancements. Ticket CLJS-1973 adds support for the :npm-deps option in deps.cljs files, allowing library authors to develop and distribute libraries that directly depend on Node.js modules.

https://anmonteiro.com/2017/03/requiring-node-js-modules-from-clojurescript-namespaces/

This is how it should be done if I understand correctly

Not sure though 😉

there are a couple of things to know (that aren’t obvious from that article or the official docs). (1) npm-deps will run the npm module through advanced compilation, which may or may not work, and (2) it seems incredibly brittle and basically impossible to debug. I believe that is why dnolen published the webpack guide

Agree and switched to shadow for exactly those reasons

Still, a way to declare and detect transitive npm deps seems good

it would be interesting if libraries could just ship with their own package.json

then you could just do normal transitive resolution

But that's what the deps.cljs is for I guess

And note "I am guessing"

Will actually try it out tomorrow

i’m not sure deps.cljs will ever be expressive enough, but maybe

actually it should work yea

whether shadow can or should interpret npm-deps is something thomas would have to answer

that makes my head hurt 🙂

Ah ah! I hear you

Cool so I'll stay tuned, thanks for answering 😉

I’m pretty sure shadow-cljs respects deps.cljs

seeing as I installed a cljs dependency just now and a bunch of stuff magically appeared in my package.json when I did git status 😄

@lilactown whoa. oh fascinating. so you installed a cljs dep that had an npm-deps in it, and shadow added that to your package.json?

what dep did you add? I want to test

https://github.com/nubank/workspaces/

deps.cljs defined here: https://github.com/nubank/workspaces/blob/master/src/deps.cljs

yea look at that!

@richiardiandrea there’s your answer. it does respect deps.cljs

Awesome @lilactown @lee.justin.m with this answer in mind my goal is to check if it works with cuerdas and fix it if not. Thank you!

@richiardiandrea deps.cljs works BUT in case of the shadow-cljsjs compatibility packages no deps.cljs is provided. that is because there is no way to selectively install :npm-deps. Since this package contains a lot of cljsjs.* shims adding a deps.cljs would always install too many packages. https://github.com/thheller/shadow-cljsjs

cuerdas is one of those cljsjs/foreign-libs compat things

therefore its xregexp dep is not installed

FWIW the thing I don't like about deps.cljs and :npm-deps is about their interaction with the compiler. using them to declare actual npm dependencies is fine by me

is there a cljs way of reading/setting env vars before starting the dev server?

not sure I understand that question

So I want to set some env vars that my cljs app should read to know e.g which API url to make requests to

also have defaults

assuming your app is :target :browser?

yes

ok what I always recommend is forgetting about env vars

the browser does not have env vars

Okay

take this setup for example

https://github.com/shadow-cljs/quickstart-browser/blob/master/src/starter/browser.cljs#L7-L12

https://github.com/shadow-cljs/quickstart-browser/blob/master/public/index.html#L10

it is absolutely fine and recommended to pass whatever "env" vars you want to the init method

ie. call <script>starter.browser.init({url:""});</script>

and do whatever with that in the code

so the HTML file controls the env

but this var changes from dev to prod

thats exactly the point. create a different html depending on env

the problem with feeding env vars to the compiler

is that you actually have to recompile to get the changes in

you can also use :closure-defines https://shadow-cljs.github.io/docs/UsersGuide.html#closure-defines

you can also use :closure-defines {some.app/var #shadow/env "MY_VAR"} if you are deadset on using environment variables

but I really really do not recommend doing so.

I would expect one sets them only once when starting dev server or while doing advanced compilatiion so recompiling to get the changes in doesn't sound like a problem since as the closure defines describes it they are compile time constants

the list of vars can be large

*grow

well .. if they are actually compile time constants then use :closure-defines

but often they are actual runtime configuration data

and keeping them out of the compiler has many benefits, like being able to easily change them

hmmm yeah they are runtime config data

to me setting things in the HTML is the equivalent to what env vars usually do

since they control the env

doing it in the compiler is just weird

ok here's a situation. I set it in the html and read it in my init fn then from there I use a setter to create a sort of global for the app object that other functions can access

well don't you have a global state atom anyways?

eg. reagent, re-frame or whatever? all of them pretty much do

just put it in there in the init fn

hmm ya

🙂

otherwise a (my.app.env) (defonce config (atom {:the "defaults"})) also works

ya I'll just swap! the app state

I'm wondering what could go wrong with two index.html files

not two at the same time but...

whats your server side?

I expect to point nginx to get pointed to the index.html file of my choice

but the SRE guys could feel different

well you can just use a build hook to generate a proper index.html

*index.html meaning intial html file

or anything else you do to deploy really

I was wondering whether to use something like a build hook or sed to set the necessary vars in the index.html file

so that I have one

if if you intended to call MY_ENV=foo shadow-cljs release your-build

yeah that would be great

you can call shadow-cljs release your-build && generate-prod-index.sh

aj

*ah

you can totally use env vars if you want to ... I just don't recommend it. up to you what you do

yeah the generate-prod-index.sh would edit the html file to have the necessary values for vars

makes sense

and then init function would read them

I know a few people use the clj-run option

I do too https://shadow-cljs.github.io/docs/UsersGuide.html#clj-run

(defn release []
  (shadow/release :my-build)
  (generate-prod-index))

shadow-cljs run my.build/release

if you want to stay in clojure and not resort to other shell scripts

ah seems perfect for my case

actually this is really good. It answers a lot of questions that I had

you can also split the shadow/release call into calling (shadow/get-config :my-build) updating that config (its just a map) and then calling shadow/release* with it

so you could adjust the config pretty much any way you like

is there a way to get closure to chill out about This code cannot be converted from ES6. extending native class: Array

I’m trying to mess with the compiler options and adjust what it expects as input/output, but it’s not working quite yet

gonna need more info. never seen this message.

a library I’m trying to do use does something akin to:

class Foo extends Array {
  ...
}

their advice is basically that they only support ES6

which library?

https://github.com/williamngan/pts/issues/22

npm install pts if you want to test it

meh

😂

the package doesn't contain the sources. only a pre-bundled webpack npm package

wish people would start putting proper ES6 sources in their packages 😞

it’s hard to get people to understand what to bundle/transpile vs not

its easy: transpile anything that isn't standard es6 but not more

I had to tell a bunch of contractors at work that no, you cannot deliver a bunch of internal npm packages that have JSX in them 🙃

well I guess for maximum compatibility you also want a commonjs package

so es6 in node_modules usually gets transpiled by babel for reasons such as yours

but the file is not properly identifed as es6 since it doesn't have any import/export statements

thats my mistake probably, closure probably identifies it properly when pre-parsing

hm. this looks like a case where both babel and closure fail

I have not yet figured out how to stop closure from transpiling

for some reason :language-out :ecmascript6 still transpiles most of the stuff out

when there isn't actually a reason to do so

are you sure that babel fails?

> Yes, Pt extends Float32Array so that we can keep all the native functions while adding new ones.

it’s not babel AFAIK

[:workspaces] Compiling ...
Closure compilation failed with 1 errors
--- node_modules/pts/dist/pts.js:288
This code cannot be converted from ES6. extending native class: Array

ah the good old OOP mentality 😉

hahaha

@lilactown yes its not babel. I mean did you try babel manually since you said "babel and closure fail"

shadow-cljs currently does not try to use babel for that file

oh I see what you mean. there might be a babel plugin we can use, but usually babel also cannot handle this

he mentions it in the issue

babel seems to work fine

well not sure if the package actually becomes usable but you can try it

:thinking_face: so should I vendor the lib and run babel on it, since shadow isn’t detecting that it’s ES6?

no. I'll fix the detection part but first I want to verify that it actually works

since I have no idea what pts actually does I could use some help with the verify part

what you want to do is: in your project dir run npx babel --presets env node_modules/pts/dist/pts.js -o node_modules/pts/foo.js

and then (:require ["pts/foo" ...]) instead of just pts

hmm ok doesn't actually work

TypeError: Constructor Float32Array requires 'new'
    at Pt.Float32Array (<anonymous>)

@lilactown seems to work if you set :js-options {:language-out :ecmascript6}

seems like it just can't be compiled down properly

:face_palm: I was trying to set that in :compiler-options

oh right. I don't think its properly documented for :js-options

:compiler-options is only for the final :advanced optimization

but since the JS sources don't go through that they have their own setting

ahhh

oof yeah this library is painfully OO 😞

and docs sounds kind nice, could be cool to have a functional cljs wrapper around it

yeah I’m going to get something working and then maybe start hacking on a CLJS interface

the library just looks freakin’ cool

@thheller Do I need to do anything to turn this JS object into a clj(s) obj <script>starter.browser.init({url:""});</script>

because in my case it's undefined in the cljs

nvm I wasn't using json

@urbanslug dont need to use json. js->clj it fine or just normal js interop

has anyone used shadow-cljs to develop on a raspberry pi?

why would you do that to yourself? thats just gonna be way too slow 😉

lol nice

I mean running code on it is fine

but developing on it?

well the approach i was thinking was running shadow-cljs + emacs on my laptop and pushing the compiled js to the raspberry pi

yeah thats fine

is there a hook somewhere in shadow-cljs to do this sort of thing?

ideally i’d have a dev environment with hot code reloading

well what do you want to build? assuming a node-script?

the only hooks i see are inside JS

yes a node script

how is the pi connected?

can you mount a network drive on it?

ssh at the moment 😅

sshfs or something like that would work too

yeah i could do that

ah damn that would probably suffer the same issue I fixed yesterday

didn't release the fix yet though

well what I would probably do a) mount a sshfs drive on the pi that points to your local project

b) run node out/your-thing.js on the pi

that will connect to your local machine for the hot-reload/repl stuff

cool, what’s the issue you fixed?

and thank you!

well :node-script currently has some hard coded absolute paths

so they basically only run on the machine they were compiled on

during development, release is fine.

but yeah the pi won't find the absolute path

so that doesn't work

i see, so the above approach won’t work yet right?

right now no

I should have a release ready soon

ok no worries

also congrats on the clojurists together grant, well deserved!

thx although I'm beginning to think I scammed myself

spending way more time with this than I planned initially 😉

but its good to work on the stuff I wanted to work on for a while

it’s definitely good for the community

More of a cljs question. I set a var as you said @thheller but when it comes to retrieving it from the app-state does something change? all my attemps to retrieve give me null/undefined

how are you setting it?

(-> @app-state :config :url) however it's there when I console log the (:config @app-state)

(swap! app-state merge {:config (js->clj config)})

where config is the var from js

I'm confused. is it there or not?

its there if you do (:config @app-state) but not when (-> @app-state :config :url)?

yes

or even (:url :config @app-state)

the threading doesn't matter

thats not valid

hmm

(get-in @app-state [:config :url]) doesn't work?

same as threading so it shouldn't be there either then

one sec

ah wait ...

the (:config @app-state) probably shows {"url" "..."} right?

thats a string key and you are trying to get a keyword

try either (get-in @app-state [:config "url"]) or (js->clj config :keywordize-keys true) when doing the initial swap!

> thats a string key and you are trying to get a keyword

ya