Fork me on GitHub
#shadow-cljs
<
2022-03-02
>
folcon14:03:16

I'm trying to port an existing cljsbuild to shadow-cljs, the reason I used cljsbuild in the first place was that I couldn't work out how to get shadow to use my externs and local javascript files, I'm trying to require my files and so far I'm not having any luck in getting shadow to detect them... Is there a specific file layout that I should be using? Also is it possible for all of the functions in the javascript files to included verbatim, jsdoc's and all? Basically I'd like to compile the clojurescript code with :advanced optimisations, but include some js in the source that's not been modified at all... By include I mean inline. ===== EDIT ===== Ok, I've managed to get it to build, but not sure how to get the js code to inline in correctly? (I'm also wondering is the appropriate thing to do is to just cat <path to js files> >> shadow-generated.js or is there something better? I'd like to do everything cljs land, but at the moment generating the correct structures is not going well...

thheller17:03:05

@folcon what kind of code are you trying to append? does it not make more sense to keep as a separate file?

folcon17:03:09

I'm building a google app script project. I have clojurescript that wraps the google externs and then some javascript for writing the functions that can be read by the google environment. Google's engine only accepts certain forms: https://developers.google.com/apps-script/guides/v8-runtime And if you want to use custom functions (which we do) then we also have to emit jsdoc's of the correct format, which we've not worked out how to do yet... So for the moment, we hand write that and append it on...

folcon17:03:30

I've gotten this working with cljsbuild But I'd like to use shadow so I can start pulling in npm modules. It would be useful if I can just output a single file which we can then trigger an upload on. Though failing that I can do some bash magic to get the output

thheller17:03:18

you can do this fine in just CLJS?

folcon17:03:33

Yes, with cljsbuild

thheller17:03:44

no, just plain CLJS. no .js file

thheller17:03:49

or I'm missing something. do you have an example of the .js file you are appending?

folcon17:03:13

The environment doesn't understand cljs itself, so not sure what you mean there? They have a format called .gs which is broadly v8 compatible which is what the target is

folcon17:03:20

Sure, let me grab an example

folcon17:03:38

Lambda island did a short writeup if that's helpful as well: https://lambdaisland.com/blog/2016-10-01-clojurescript-and-google-apps-script

thheller17:03:39

ah. I remember something like this

thheller17:03:25

I'd just (js* "onOpen(e) { attendomat.core.create_menu(); }") in my .cljs ns instead of appending it 😛

folcon17:03:34

/**
 * Returns the domain for a given name
 * @param {"Apple"} name The value or range of cells
 *     to lookup the domain for.
 * @return The domain of the website
 * @customfunction
 */
function WEBSITE_LOOKUP(name) {
 Array.isArray(name) ?
        name.map(row => row.map(cell => fetch_sitedata(cell, 'domain'))) :
        fetch_sitedata(name, 'domain');
}

function test_WEBSITE_LOOKUP() {
    Logger.log(String(WEBSITE_LOOKUP("Google")));
}

folcon17:03:58

Does that work for stuff like the above?

folcon17:03:12

If the jsdoc comment is not correctly written out it barfs

thheller17:03:02

hmm yeah for jsdoc it is best to just append manually

thheller17:03:11

:advanced will otherwise kill it

thheller17:03:26

so yeah appending might be best

thheller17:03:36

you could do that in a build hook I guess

folcon17:03:46

I'm trying to work out how to structure this

folcon17:03:07

{:source-paths
 ["src/dev"
  "src/main"
  "src/js"
  "src/test"]

 :dependencies
 [[applied-science/js-interop "0.3.1"]]

 :builds
 {:appscript {:output-to "Code.js"
              :output-dir "target"
              :compiler-options {:externs ["resources/gas.ext.js"]}
              :exports-fn humble-app-script.core/generate-exports
              :js-options {:js-provider :shadow}
              :release {:compiler-options {;:optimizations :whitespace
              																													:pretty-print true}}
														:target :npm-module
														:entries [example.core]}}}

folcon17:03:53

is it :npm-module or :node-library or :browser?

thheller17:03:26

neither 😛

folcon17:03:50

So what should my target be 😃

thheller17:03:02

technically it would be :target :google-appscript or something

thheller17:03:07

that just doesn't exist yet 😛

thheller17:03:01

but that what :target is for. this is neither a browser nor node so none of the above fit.

folcon17:03:05

Ah, so it doesn't call any special functionality based on the target name

folcon17:03:43

Target ":appscript" for build :appscript was not found. The built-in targets are:
  - :browser
  - :browser-test
  - :node-script
  - :node-library
  - :npm-module
  - :karma
  - :bootstrap

thheller17:03:46

targets assume the presence of certain functionality.

thheller17:03:12

well yes. as I said this doesn't exist. someone would need to implement it

folcon17:03:27

Right, what would that require? Or should I just use an existing target?

thheller17:03:21

you can use an existing target but as I you have encountered that will always be a hack and require certain manual tweaks

thheller17:03:46

I don't expect you to implement a custom target but thats what I would do if I had time and actual use for it 😛

folcon17:03:08

Ok, well at the moment I'm trying to generate functions that fulfil this format:

function normalFunction() {}
      async function asyncFunction() {}
      function* generatorFunction() {}

      var varFunction = function() {}
      let letFunction = function() {}
      const constFunction = function() {}

      var namedVarFunction = function alternateNameVarFunction() {}
      let namedLetFunction = function alternateNameLetFunction() {}
      const namedConstFunction = function alternateNameConstFunction() {}

      var varAsyncFunction = async function() {}
      let letAsyncFunction = async function() {}
      const constAsyncFunction = async function() {}

      var namedVarAsyncFunction = async function alternateNameVarAsyncFunction() {}
      let namedLetAsyncFunction = async function alternateNameLetAsyncFunction() {}
      const namedConstAsyncFunction = async function alternateNameConstAsyncFunction() {}

      var varGeneratorFunction = function*() {}
      let letGeneratorFunction = function*() {}
      const constGeneratorFunction = function*() {}

      var namedVarGeneratorFunction = function* alternateNameVarGeneratorFunction() {}
      let namedLetGeneratorFunction = function* alternateNameLetGeneratorFunction() {}
      const namedConstGeneratorFunction = function* alternateNameConstGeneratorFunction() {}

      var varLambda = () => {}
      let letLambda = () => {}
      const constLambda = () => {}

      var varAsyncLambda = async () => {}
      let letAsyncLambda = async () => {}
      const constAsyncLambda = async () => {}
Or that are a custom function in the form I gave earlier

thheller17:03:39

yeah I do remember that this was a problem

thheller17:03:53

given that the CLJS compiler can't emit that

thheller17:03:10

and (defn ^{:export "onOpen"} my-on-open [e] ...)

folcon17:03:35

What's the best way to proceed, I'd just want to sort of get started, I've got a bunch of js code and a bit of cljs code and I'm slowly porting the js bits over which can be ported...

thheller17:03:39

ends up as a goog.exportSymbol("onOpen", function(e) { ... }) which is not recognized

thheller17:03:01

just append the file seems best to me

folcon17:03:50

Ok, well is there a way to do that internally? I'm currently just using bash to cat and >>

thheller17:03:32

I'd be using cat and >> too

folcon17:03:37

I also need to make sure the output doesn't produce calls to global

thheller17:03:14

(defn hook
  {:shadow.build/stage :flush}
  [build-state & args]
  (spit
    (io/file "regular" "out.js")
    (slurp (io/file "file-to-append.js"))
    :append true)
  build-state)

folcon17:03:43

I'm trying to use :target :node-library and it doesn't like this:

'use strict';
var b, a = global;
error:
ReferenceError: global is not defined
(anonymous)	@ 
(anonymous)	@ 
(anonymous)	@ 

folcon17:03:00

looking at the hooks docs you linked

thheller17:03:03

as I said .. :node-library the target EXPECTS a node runtime

thheller17:03:19

you cannot make that go away. this is hardcoded into the target as the name implies

thheller17:03:01

you best bet might be :browser but then disable everything browser related

thheller17:03:16

so :devtools {:enabled false} in the build config

folcon17:03:13

Is that going to have an effect when I'm trying to get a repl working for dev?

thheller17:03:07

yes. no REPL support then

thheller17:03:25

meaning to REPL into the google appscript

thheller17:03:33

but doesn't sound like that would work anyways?

thheller17:03:49

you can open a browser-repl or node-repl anytime. that is not affected in any way

folcon17:03:32

No, but it would be useful to be able to have a local repl

folcon17:03:27

I can then mock some of the environment and then do repl based development, but still compile out a release that goes to the google appscript

thheller17:03:48

yes, that is what I described

folcon17:03:11

Ok, that's good 😃

thheller17:03:21

you can npx shadow-cljs browser-repl anytime. requires no build config or so

folcon17:03:24

Just testing this out

folcon17:03:37

Btw, does the args in the hook you gave contain the final build path?

folcon17:03:52

I'd prefer not to hardcode it like:

(io/file "regular" "out.js")

folcon17:03:54

Or should I check out the build/targets directory on github for how to navigate that 😃

folcon17:03:23

Hmm, not sure why, but it's not generating the export. I've got an included js script file that I'm importing using your example:

(:require ["./entry_points" :as entry-points])

(defn generate-exports []
  #js {:fetch_sitedata entry-points/fetch_sitedata})
I've got this in shadow-cljs.edn:
:output-to "Code.js"
:exports-fn example/generate-exports
So when I look inside the target/main.js there is no fetch_sitedata function defined, I've been grepping for the name and it's not there... Also not sure why it's ignoring my :output-to instruction and is just generating the file main.js inside target dir...

thheller18:03:22

please consult the actual shadow-cljs documentation for the chosen :target

thheller18:03:39

ie. :target :browser does not have a :output-to option and neither does it have a :exports-fn option

folcon18:03:46

Ah cool, thanks

folcon18:03:48

I'll do that

thheller18:03:25

and be requiring the entry_points.js you are adding it to your build

thheller18:03:32

which means it'll go through :advanced. which I thought you were trying to avoid?

folcon18:03:51

There's two js files, entry_points.js which has some wrapper code which I'm slowly going to be getting rid of, but probably should go through the pipeline and custom_functions.js which I'm appending

folcon18:03:37

I'll stick js in the appropriate file, but some functions in the entry_points.js need to be called in my clojurescript and for some reason though they're in my :exports-fn example/generate-exports in shadow-cljs.edn, it's not appearing inside the final main.js

thheller18:03:43

I don't know what you are expecting to appear

thheller18:03:33

but :target :node-library recognizes :exports-fn. no other target does, so it will do nothing for those. and the only thing that will appear for :node-library is module.exports = your.ns.generate_exports();

thheller18:03:55

whereas the your.ns.generate_exports() part may be renamed and shortened by :advanced

folcon18:03:59

Ah, sorry, yep that makes sense. I'm mixing up target instructions. Basically my goal is to take some of the functions defined inside entry_points.js and export them so they're available in the final output. Should I just create something like this:

(ns example
  (:require [clojure.string :as str]
            [applied-science.js-interop :as j]
            ["./entry_points" :as entry-points]))

(def ^:export fetch_sitedata entry-points/fetch_sitedata)

;; OR
(defn ^:export fetch_sitedata [] (entry-points/fetch_sitedata))

thheller18:03:11

this is not how this works at all

thheller18:03:28

and not solving your problem in any way either

thheller18:03:57

by including entry_points like above you are including it as part of the build

thheller18:03:18

meaning it will be fully processed, minimized and everything

thheller18:03:31

so whatever you did there to be app-script specific will be gone

thheller18:03:04

or I'm missing something here

thheller18:03:19

it would make this really much much easier to talk about if you create a repo with some code

thheller18:03:40

standalone partial snippets are just not enough

folcon18:03:16

Yep, just saw this:

n("example.fetch_sitedata", vf.fetch_sitedata);

}).call(this);
The definition of fetch_sitedata is missing... vf is probably some import, but one that doesn't seem to contain any of the original fetch_sitedata definition

folcon18:03:29

Hmm ok let me put together an example case after dinner and poke you in this thread? Would that work?

thheller18:03:34

yes that is the export problem I was referring to above

folcon19:03:18

How does this look? I tried to make a minimal example https://github.com/Folcon/example-appscript

thheller19:03:04

instead of (println (keys build-state)) in the build hook

thheller19:03:21

try (tap> build-state) and then look at it in

folcon19:03:49

Yea, still not used to tap> in cljs projects

thheller19:03:16

this is clojure code 😉

folcon19:03:24

This is why I want to get shadow working 😃

thheller19:03:15

what are you using to edit code? you should really run a formatter. the shadow-cljs.edn is a complete mess :P

thheller19:03:58

:output-to "Code.js" has no effect for :browser

folcon19:03:14

Normally cursive, but currently sublime as I'm throwing a minimal example together 😃

folcon19:03:37

Yep, I need to generate a single file output, similar to webpack

folcon19:03:18

Which contains a mixture of advanced compilation code + appended js code

thheller19:03:36

this build config will generate this as target/main.js

thheller19:03:50

:output-dir + the keys in the :modules map

folcon19:03:42

Right, so I should use a Makefile or another hook, to copy it to the correct location and append the other file?

thheller19:03:47

{:source-paths
 ["src/dev"
  "src/main"
  "src/js"
  "src/test"]

 :dependencies
 [[applied-science/js-interop "0.3.1"]]

 :builds
 {:appscript
  {:target :browser
   :output-dir "target"
   :modules {:main {:entries [app-script.core]}}
   :devtools {:enabled false}
   :build-hooks [(app-script.hooks/build-hook)]
   :compiler-options {:externs ["resources/gas.ext.js"]}}}

thheller19:03:28

the hook you have now does nothing since it doesn't use the target/main.js file?

folcon19:03:33

But yea, not sure why the function inside the entry_points.js isn't getting exported

thheller19:03:52

btw npx shadow-cljs pom then import pom.xml in cursive as a project

thheller19:03:38

as I keep trying to tell you ... you are importing it into the build. IT WILL BE PROCESSED!

thheller19:03:52

that means that the function it has will be DELETED since it is not referenced anywhere

thheller19:03:06

you can actually export it by writing proper ESM code

thheller19:03:23

so instead of

function mult_vals(a, b) {
  return a * b;
}

thheller19:03:31

export function mult_vals(a, b) {
  return a * b;
}

folcon19:03:56

So just add an export directive in the js will work?

thheller19:03:01

then mult_vals is accessible via entry-points/mult_vals

thheller19:03:12

BUT that still doesn't make :exports do what you want and it never will

thheller19:03:35

this is not how the code is constructed

thheller19:03:15

(def ^:export mult_vals [a b]
  (* a b))

thheller19:03:40

is identical as far as the generated code is concerned. the entire entry_points.js indirection does absolutely nothing

thheller19:03:19

can you explain what exactly you are trying to achieve via that entry_points stuff? like what does the final generated JS need to look like

folcon19:03:58

Sure give me a sec to write it up

folcon19:03:24

So the entry_points.js contains effectively working code that is broadly legacy. When I was porting it to cljs I realised that because app-script has it's own weird formats and rules I'd have to feed it javascript to be able to work around some behaviours. So I split it up into custom_functions.js and the original entry_points.js. The idea was that anything we couldn't just port over that was externally facing could live inside custom_functions.js. Any legacy stuff that was still in the process of being ported over that was internal behaviour could live in entry_points.js which could be called and wrapped by clojurescript code. So we have functions like mult_vals which need to be called by functions in the clojurescript code or by functions in custom_functions.js. But are not by themselves public. I'd prefer to run as much as possible through the closure compiler to minimise / remove stuff we don't need. But in the worst case we can just use simple optimisations for now.

thheller19:03:33

ok so then prepend or append BOTH entry_points.js and custom_functions.js to the output

thheller19:03:54

instead of requiring it just use js/mult_vals or whatever in the CLJS code

thheller19:03:04

since they are global functions that is fine

thheller19:03:42

if you want to include it in the build then you can require it like you did

thheller19:03:09

but it will require adding export in the .js and re-exporting it so app-script can find it WILL NOT WORK!

thheller19:03:43

this is just a limitation in the way the closure compiler treats exported things and there is no way to change that

thheller19:03:47

so you can only :export them and refer to them by the full name via the JS that is not going through the closure compiler

folcon19:03:07

Ok, fair enough does that also work if I'm using them at the repl?

folcon19:03:31

As in if I append / prepend with the hook?

thheller19:03:32

they'll not be available in the REPL in any way

thheller19:03:15

well you can sort of make them available I guess

thheller19:03:53

create a public folder and put the entry_points.js there

thheller19:03:31

then create a index.html that loads them via <script src="/entry_points.js"> and <script src="/js/main.js">

thheller19:03:46

then in the shadow-cljs.edn add :dev-http {3000 "public"} and set :output-dir "public/js"

folcon19:03:06

Ok, that makes sense

thheller19:03:10

and will access to the functions via js/mult_vals or whatever

thheller19:03:14

just like the appscript would

folcon19:03:32

Thanks a lot, I'll see if I can use that to fix my project

thheller17:03:29

there are :prepend and :append options in :modules but they only take strings. so doesn't make sense if its too long

thheller17:03:59

:modules {:main {:init-fn :append "\n// yo\n"}}}

socksy18:03:21

I'm having a weird issue where I can't seem to run all the tests in my project with a node-test target. Using shadow 2.17.5, and with a basic target like this:

:test {:target :node-test
                 :output-to "target/test.js"
                 :ns-regexp "-test$"}
I can get it to work if I pass in the NS directly with like node target/test.js --test=foo.foo-test. It successfully finds 4 -test$ NS's, but no others... I tried using ".*" as the regex also to no avail. I can also confirm that the namespaces are correctly built and are loaded (by chucking a console.log into the built namespace in .shadow-cljs/builds/tests/dev/out/cljs-runtime/foo.foo_test.js). Any ideas?

thheller18:03:49

how are the tests written? should be fine without any --test option

socksy18:03:23

tests are written the same way as the working namespaces – using cljs.test , deftest, and the async macro

thheller18:03:16

and what happens if you just run node target/test.js?

thheller18:03:28

I mean all --test does is filter out tests that don't match

thheller18:03:35

the default is to run all. so something must be off? --test does not affect compilation in any way.

socksy18:03:51

it runs the tests in the 4 namespaces I mentioned

socksy18:03:12

compilation is not seemingly affected in anyway, no – it's all there

socksy18:03:18

is there some way to step through and see the logic?

socksy18:03:26

ah! I think I solved it! One of the tests in the last NS found used (async done blah) but didn't call (done) and so stopped the rest of the tests running. But weird that the test completed at all

thheller18:03:23

its node so it'll exit when there are no more pending callbacks or whatever. it just assumes the program ended properly.

Alexis Schad19:03:27

Hi there! I'm having a dependencies issue that I can't reproduce in pure JS, so I think it is shadow-cljs related. I want to use a library that uses multiple versions of the same dependency, it looks like this folder tree: node_modules -- dep A ---- node_modules ------ dep B v2 -- dep B v1 => It is always the dep B v1 that is used, even in the dep A code. The dep A code uses common JS API (`require`). I can't find in the guide how I can make this work as expected. (The library I try to use is @uiw/react-textarea-code-editor )

thheller19:03:00

which shadow-cljs version do you use? this was not supported until rather recently

Alexis Schad19:03:39

The last one: 2.17.5 (I double checked with shadow-cljs info)

thheller19:03:30

do you use project.clj or deps.edn? if so the version listed by info does not apply

Alexis Schad19:03:37

hmm, I don't know. It is installed by npm

thheller19:03:48

that version should support the nested packages. unless you disabled that in the build config

Alexis Schad20:03:05

I don't think so

{:asset-path "/js"
        :modules {:main {:init-fn app.core/main}}
        :output-dir "public/js"
        :target :browser}
I can create a minimal projet if you want

thheller20:03:59

that always helps

Alexis Schad21:03:15

Here it is: https://github.com/schadocalex/shadow-cljs-deps-issue Actually it uses the wrong version of a sub sub dependency, not directly a sub dependency

Alexis Schad21:03:31

it works for the first sub dependency

Michael Mossinsohn21:03:50

My boss would like my JS output to make life as easy as possible for JS programmers to both interact with and possibly even to 'take over' in case I leave the project. I use Reagent to create a React based app and use Shadow-cljs for compilation. How can I get my code and/or my config to be as JS friendly as possible (when compiled for development)?

Alexis Schad22:03:31

You can't, you can only set up your project to be used as a black box library

Michael Mossinsohn22:03:48

Ohh, I see. Thanks for clarifying this up for me.