Fork me on GitHub
#shadow-cljs
<
2018-06-13
>
kurt-o-sys07:06:52

Another question @thheller: if I have a npm package which embeds css, it seems to fail to import/require the css. For example: draft.js, in ES:

import 'draft-js-static-toolbar-plugin/lib/plugin.css'
import 'draft-js/dist/Draft.css'
If I require both in cljs, the css doesn't seem to be added. Is this the normal behaviour? If so, the best way would be, I assume, to add the css file to index.html?

kurt-o-sys07:06:34

@mynomoto I only got 10x dev working on shadow-cljs, never on lein (well, not if I wanted to add other npm-libraries, which was pretty much a mess)

thheller09:06:02

@kurt-o-sys .css stuff is currently not supported but might come at a later date. until then you need another way to bundle your css. unfortunately some JS libs are designed with the expectation of webpack being used and adding all the weird conventions that entails is a bit time consuming

kurt-o-sys09:06:54

ok... makes sense, thx.

kurt-o-sys09:06:01

so, adding to index.html will do

thheller09:06:58

@danielcompton deps are done via https://github.com/cemerick/pomegranate so the exact same lib that lein or boot use. it is done via a standalone uberjar via shadow-cljs-jar launched before the actual JVM. its is basically only a very tiny wrapper using pomegranate, see https://github.com/thheller/shadow-cljs/blob/master/packages/shadow-cljs-jar/src/shadow/cljs/npm/deps.clj

thheller09:06:08

@kurt-o-sys index.html probably won't have access to node_modules/draft-js/dist/Draft.css so probably need to copy it manually

kurt-o-sys09:06:07

no need to copy.

thheller09:06:50

yeah that works too

kurt-o-sys11:06:26

will script it to fetch css from downloaded npms

kurt-o-sys13:06:45

ok, well, if you ever need external npms and you use http://blob.tomerweller.com/reagent-import-react-components-from-npm , it won't work to add 10x. The reason being: you'll take out reactjs to webpack, and 10x will not find react on startup (or someting like that). In shadow-cljs, it's no issue, since shadow-cljs inverts the logic (compile cljs to js and add it to a normal npm project - lein and boot try to embed js in their own compilation, which makes the google closure compiler cry for many npm packages). That's, at least, how I think it is πŸ˜›.

mynomoto13:06:10

@kurt-o-sys I mean using lein to call shadow https://shadow-cljs.github.io/docs/UsersGuide.html#Leiningen I was hoping almost nothing would change except who creates the classpath and manage dependencies.

thheller13:06:29

@mynomoto yes nothing else changes

thheller13:06:06

:source-paths and :dependencies move to project.clj basically. that is it.

kurt-o-sys13:06:34

right, that's it. That's what I do as well πŸ™‚

kurt-o-sys13:06:46

and add :lein true to shadow-cljs πŸ˜›

mynomoto13:06:30

@kurt-o-sys is that setup that you were refering to about not getting re-frame-10x to work?

mynomoto14:06:04

Looks like shadow can check the artifact on a private s3 maven but cannot fetch it whan adding :repository. That way I can fetch the dependency using lein and run using only shadow. Adding a :plugins seems to make no difference.

thheller14:06:33

maybe its enough if I add the s3-wagon-private stuff to the classpath? no idea how this works

thheller14:06:44

(classpath of the deps tool that is)

thheller14:06:10

best to stick with lein for now

mynomoto16:06:57

I think I got a good tradeoff, using lein to only fetch the dependency and shadow can go from there. I think it's interesting that it can check the dependency but not download it, but I may be misunderstanding what's going on.

thheller16:06:00

maven works by always looking at ~/.m2 first so if all the deps are there there is no need to ever talk to s3. so yeah this works but using the lein interop is probably best since you otherwise need to manually sync the deps in shadow-cljs.edn and project.clj

mynomoto16:06:47

It's only one or two deps, and calling from lein is breaking re-frame-10x and I'm more willing to keep things in sync than debugging that. The odd thing is that without adding the :repositories key it doesn't work even with the dependency cached on ~/.m2 It complains that cannot find metadata.

thheller16:06:01

breaking re-frame-10x does not make sense. it should still work if your dependencies are identical

thheller16:06:18

lein literally just constructs your classpath, nothing more.

thheller16:06:32

it may be a dependency conflict however

richiardiandrea19:06:17

@afhammad I don't think that is possible, how can it run in the browser then?

richiardiandrea19:06:34

I mean, you are use fs, which is node specific

afhammad19:06:50

using js/require seems to work for the same package, i thought shadow would be smart to use an appropriate import depending on the target

richiardiandrea19:06:39

oh well, I am not 100% that works, let's wait for better answers

justinlee19:06:57

I believe it would have to do static analysis on the clojurescript code to make that work. presumably, that’s why you do this in the namespace forms instead of in the middle of arbitrary clojurescript code: the ns form is designed to be analyzed statically

justinlee19:06:12

on node targets, it can just load the javascript dynamically whenever it encounters the require

justinlee19:06:19

this is why you end up with the immense complexity of webpack: it’s trying to figure out what to do given that you can stick a require anywhere

thheller19:06:16

@afhammad js/require is just a regular function call and not indexed by shadow-cljs. fs is a node only package, you will get a shim if you (:require ["fs" :as fs]) in a browser build but that does not support using fs.readFileSync

thheller19:06:37

node-only functionality is not available in the browser no matter how you package it

justinlee20:06:03

well, unless you do what webpack does and simulate the functionality. πŸ™‚

thheller20:06:52

that is what I do

thheller20:06:06

doesn't change the fact that fs.readFileSync cannot possibly work in the browser

thheller20:06:26

webpack uses these https://github.com/webpack/node-libs-browser and so does shadow-cljs

justinlee20:06:50

but you’re not going in an looking for requires and then re-writing them, are you? webpack does that. (I wouldn’t think that would be something you’d want to do)

thheller20:06:43

I'm confused. what are we talking about?

thheller20:06:03

js/require is never processed no but webpack does

thheller20:06:12

fs.readFileSync doesn't work in either case

richiardiandrea20:06:04

@thheller a question about a shadow resource: does this look right?

{:cache-key [1528920986368],
   :output-name "index.js",
   :resource-id
   [:shadow.build.targets.azure-fn/fn-index "query-events$index.js"],
   :resource-name "query-events$index.js",
   :type :js,
   :source "module.exports = blax",
   :last-modified 1528920986368,
   :requires #{ep-cloud.query-events},
   :provides #{}}

justinlee20:06:12

sorry ignore me. i’m making a stupid point.

richiardiandrea20:06:26

it is not outputed in the filesystem and maybe I am doing something wrong

thheller20:06:11

@richiardiandrea uhm looks good but what are you trying to do?

richiardiandrea20:06:25

I would like to write to fn-name/index.js

thheller20:06:29

resource creation is only useful if you need something to be compiled

richiardiandrea20:06:47

does not need to be compiled a guess

richiardiandrea20:06:05

I saw :source there and though I could paste my own JS πŸ˜„

thheller20:06:35

you can but the resource itself does nothing. it needs to be required somewhere to become part of the compilation and output

thheller20:06:52

I was going to do some azure-fn stuff now

richiardiandrea20:06:14

I can push what I have if you want to have a look

richiardiandrea20:06:34

I was implementing what you were describing

thheller20:06:46

yeah I can take a look if you are on the correct path

richiardiandrea20:06:01

feel free to discard everything if not πŸ˜„

spieden20:06:15

i’ve been having great success translating ES6 into shadow-cljs require forms, but how about something like this:

var FileSaver = require('file-saver');

thheller20:06:18

the funcpack stuff that was mentioned looks like its just webpack. could be useful as a hook though

spieden20:06:35

ES6 imports that is

thheller20:06:45

thats not es6? (:require ["file-saver" :as FileSaver]) should do it

richiardiandrea20:06:00

yeah but I am not sure we need it, in any case it is good that you have a look and you validate

richiardiandrea20:06:25

@spieden sorry crossed wires πŸ˜„

spieden20:06:41

i’m blissfully ignorant of js packaging =)

thheller20:06:55

@richiardiandrea do you think a namespace per fn makes sense or would you want to put multiple fns into the same ns? (referring to :azure-fn or course)

richiardiandrea20:06:52

I think it makes sense...but I am passing :fn map as is to :exports so probably what you say is still supported?

richiardiandrea20:06:10

(using :node-library target at the moment - maybe not the right one)

thheller20:06:30

unrelated to that. if we only allow one fn per ns we can abuse :modules better

thheller20:06:52

but I guess thats not useful if everything is loaded all at once anyways

thheller20:06:20

:azure-fn doesn't really fit anymore when using multiple fns

thheller20:06:35

:azure-fns or :azure-app or something else maybe better?

richiardiandrea20:06:00

I meant to change that as well after getting it up and running

richiardiandrea20:06:28

azure-app feels better

thheller20:06:31

@richiardiandrea yep picked that. master has the updated version. your stuff went kind of the wrong direction making things more compilcated then needed

richiardiandrea20:06:54

@thheller can you print the structure of your app-dir ?

richiardiandrea20:06:29

I mean do you have one subfolder per function?

thheller20:06:51

.
β”œβ”€β”€ cljs
β”‚   └── shared.js
β”œβ”€β”€ host.json
β”œβ”€β”€ local.settings.json
└── simple
    β”œβ”€β”€ function.json
    └── index.js

richiardiandrea20:06:30

Uhm, i just don't see where you map over the fn-map

thheller20:06:42

(defn do-flush [{::keys [fn-map] :as state}]
  (reduce-kv flush-fn-file state fn-map))

richiardiandrea20:06:56

Oh that is where, cool

thheller20:06:38

pushed another update that adds a check if the fn has :azure/* metadata and fails if missing

thheller20:06:45

could probably generate a better error

thheller20:06:58

also fixed the caching thingo

thheller20:06:19

total nonsense to optimize that but oh well

richiardiandrea20:06:32

Awesome, will validate the work on master today and report back problems in case

thheller20:06:51

you are using the azure fn v2 stuff right?

thheller20:06:55

I was testing with v1

richiardiandrea20:06:57

The cross validation of function params would be cool at some point, but it needs some fiddling with bindings...not a priority for me now

richiardiandrea20:06:33

Wanted to get it working in func host by today though so thanks for you help

richiardiandrea20:06:46

Will try after lunch

thheller20:06:59

yeah func host doesn't exist in my version

thheller20:06:09

func run simple works for the thing I built

thheller20:06:24

will try to test with v2 instead

richiardiandrea20:06:28

Will try soon too

thheller20:06:39

just saw the beta tag and decided to stick with "stable"

richiardiandrea21:06:31

Learn a bunch of shadow today btw

richiardiandrea21:06:42

Very clean approach to targets

richiardiandrea21:06:12

Everything is spec-ed data, so like it

thheller21:06:09

you could actually put the :azure-app target into a library. no reason for it to be in shadow-cljs itself

thheller21:06:34

but the entire :target stuff is completely undocumented so I don't expect anyone to write targets just yet

thheller21:06:14

its easy to take the wrong path. your resource approach could eventually be done if things get more complex but for now its overkill.

πŸ‘ 1
wilkerlucio21:06:18

@thheller I've been getting a lot of those lately:

wilkerlucio21:06:20

[:workspaces] Build failure:
FileNotFoundException: resources/public/js/workspaces/main.js (No such file or directory)
	java.io.FileOutputStream.open0 (FileOutputStream.java:-2)
	java.io.FileOutputStream.open (FileOutputStream.java:270)
	java.io.FileOutputStream.<init> (FileOutputStream.java:213)
	 (io.clj:230)
	 (io.clj:230)
	 (io.clj:69)
	 (io.clj:166)
	 (io.clj:166)

wilkerlucio21:06:37

I get when I'm starting from a clean compilation, and then when I try again it works

thheller21:06:17

which target is that? probably just a missing (io/make-parents ...) call

thheller21:06:38

ie. trying to write a file in a directory that wasn't created yet

thheller21:06:57

might be due to moving stuff into async recently

wilkerlucio21:06:29

its a :browser

thheller21:06:37

yep found it

thheller21:06:47

the :modules output can be written before the sources are flushed. only sources create the parent dirs, the modules did not

thheller21:06:02

explains that the second pass works since the directory was created then

wilkerlucio21:06:15

yeah, makes sense

thheller21:06:06

@wilkerlucio pushed 2.4.3 which should fix the issue (async sure is fun, brought way more issues with it than I would have expected)

wilkerlucio21:06:17

async is good, and hard...

wilkerlucio21:06:35

seems to be working now πŸ‘

justinlee22:06:45

something got a lot faster in 2.4. one of my files caused a nearly 6 second recompile now causes a 3 second recompile

thheller08:06:31

probably the new async stuff for cache writes. They can be pretty expensive and are now done in a thread pool in parallel.

richiardiandrea22:06:28

@thheller I see one thing that is odd, if I change the meta on the function, the data map from the compiler does not change

richiardiandrea22:06:50

it looks like it has to do with caching

richiardiandrea22:06:04

(keeping the server running)

richiardiandrea22:06:00

wait now it changed...weird..

richiardiandrea22:06:08

opened a PR for scriptFile