scittle

Andrea 2023-08-21T15:17:13.517019Z

Hi, would it be possible to configure scittle to resolve external libraries imported via <script> tags, like so?

<script src="" type="application/javascript"></script>
    <script src="" type="application/javascript"></script>
    <script type="application/x-scittle">
      (ns my-ns
        (:require [clojure.string :as str]
                  ["@codemirror/state" :refer [EditorState]]))

      ...
    </script> 

Andrea 2023-08-22T12:15:37.929999Z

would that require a custom build of scittle, like you do with plugins? I mean to tell scittle to use the :async-load-fn option...

borkdude 2023-08-22T12:22:40.477129Z

yeah, it would require some (breaking) changes, but I wonder if I can accomodate both ways in one build, both have their trade-offs. Perhaps it's possible to include scittle in a second way that does it the ESM + async way

Andrea 2023-08-22T13:27:53.968529Z

amazing this already works:

(ns scittle-build
  (:require [sci.core :as sci]
            [sci.async :as scia]
            [scittle.core :as scit]
            [shadow.esm :refer [dynamic-import]]
            [scittle.impl.error :as error]))

(swap! scit/!sci-ctx
       sci/merge-opts {:async-load-fn (fn [{:keys [ctx libname]}]
                                        (.then (dynamic-import libname)
                                               (fn [mod]
                                                 (js/console.log :require libname
                                                                 :mod mod)))
                                        {:handled true})})

(set! scit/eval-string (fn [txt]
                         (try
                           (scia/eval-string* @scit/!sci-ctx txt)
                           (catch :default e
                             (error/error-handler e (:src @scit/!sci-ctx))
                             (throw e)))))

Andrea 2023-08-22T13:28:44.221269Z

with the import mapping eg. “@codemirror/state”: “https://cdn.jsdelivr.net/npm/@codemirror/state/dist/index.js”, then it can eval

(ns foo
  (:require ["@codemirror/state"])

borkdude 2023-08-22T13:28:53.246239Z

nice!

borkdude 2023-08-22T13:29:29.605219Z

note that scia/eval-string* returns a promise, so you should handle the promise error probably

👍 2
Andrea 2023-08-22T13:29:45.422159Z

now I guess I’ll go find the 1000 implementation of the async-load-fn we did already 🙂

Andrea 2023-08-22T13:30:05.448499Z

well not 1000 probably...

borkdude 2023-08-22T13:30:34.394799Z

1000 + 1 Promise pending

Andrea 2023-08-22T13:30:52.291149Z

exactly!

borkdude 2023-08-22T13:30:54.346199Z

what are you working on?

Andrea 2023-08-22T13:31:56.365659Z

extracting with Philippa the stuff behind the split-view editor from Clerk

Andrea 2023-08-22T13:32:46.291049Z

trying to write code once but target more build envs like shadow/scittle/squint possibly 🤞

borkdude 2023-08-22T13:33:17.644849Z

nice :)

Andrea 2023-08-22T14:23:39.293819Z

I’m a bit confused now, it seems forms in the namespace gets evaluated before the requires are resolved (via async-load-fn), this kind of makes sense in an async setting, but how can forms refer to the required names?

borkdude 2023-08-22T14:24:54.570069Z

you should return handled true from the promise

borkdude 2023-08-22T14:25:05.498829Z

then the code will run after the promise, not parallel

Andrea 2023-08-22T14:25:26.519869Z

ah, ok, I was returning it from the the load-fn body

borkdude 2023-08-22T14:25:37.171439Z

right

Andrea 2023-08-22T15:33:57.695219Z

I now have two tags:

<script type="application/x-scittle" src="folio.cljs"></script>
    <script type="application/x-scittle" src="demo.cljs"></script>
first loads just fine, the second however doesn’t seem able to require the namespace defined by the first

borkdude 2023-08-22T15:35:00.374039Z

because async

Andrea 2023-08-22T15:35:07.639989Z

yeah

borkdude 2023-08-22T15:35:29.275619Z

the code that loads the script tags must handle the promises in order

Andrea 2023-08-22T15:36:15.938189Z

ok, then I also need to change eval-script-tags*

borkdude 2023-08-22T15:36:20.734059Z

yup

Andrea 2023-08-22T15:38:29.169009Z

or I also copy the ns defined in the first into the sci ctx

Andrea 2023-08-22T15:38:42.733459Z

it’s not clear at this point what goes where

borkdude 2023-08-22T15:39:16.773369Z

it's not clear to me what you mean with "or I also copy the ns defined in the first into the sci ctx"

borkdude 2023-08-22T15:39:55.254829Z

if you change scittle.core/eval_string to be async, you also need to account for that in other places

Andrea 2023-08-22T15:40:07.999269Z

I need to swap a bunch of nss into scittle.core/!sci-ctx in order for the example above to work at all

borkdude 2023-08-22T15:40:18.369899Z

btw I wouldn't merge this since that is a breaking change, but for experimentation it's ok

borkdude 2023-08-22T15:40:43.402339Z

the order will work once you fix the eval-script-tags to work with promises

Andrea 2023-08-22T15:40:51.874369Z

sure, I can experiment with our build so far

Andrea 2023-08-22T15:40:54.269739Z

thank you!

borkdude 2023-08-22T15:43:35.231439Z

once you're finished, please let me know, perhaps we can find a way to make scittle1 work with scittle async so we don't have breakages and we can have both behaviors at once

Andrea 2023-08-22T15:44:00.383219Z

ok, will do!

borkdude 2023-08-22T15:44:15.477519Z

I think this will mean a new var scittle.core/eval-string-async and perhaps a new attribute on the script tags to indicate that it's ES6 code

borkdude 2023-08-22T15:44:27.779969Z

but we can figure that out later

Andrea 2023-08-22T15:44:40.073549Z

cool, yes

borkdude 2023-08-22T15:55:04.196589Z

I think we can make eval-script-tags work with both sync and async code by just using Promise.resolve on the return value and always work in a promise-like fashion

borkdude 2023-08-22T15:55:32.995979Z

then we'll just leave eval-string synchronous for people who already used it and introduce a new var for async evaluation

Andrea 2023-08-23T09:53:21.276399Z

Got the sequential eval working 🙂

Andrea 2023-08-23T09:56:09.866119Z

but still cannot demo the editor because of a react mismatch error when using hooks

borkdude 2023-08-23T09:57:06.669509Z

oh yeah, that again

borkdude 2023-08-23T09:57:33.959139Z

perhaps you can pin the react version using import-maps

Andrea 2023-08-23T09:57:45.487919Z

could it be react version shipped with reagent the problem

Andrea 2023-08-23T09:57:54.406129Z

I do pin them to 18.2.0

borkdude 2023-08-23T09:58:02.376479Z

no, react isn't shipped with reagent, with scittle you provide your own react version

borkdude 2023-08-23T09:58:04.288889Z

in the script tag

Andrea 2023-08-23T09:58:49.685249Z

I’m adding reagent to the sci context

Andrea 2023-08-23T09:59:16.843649Z

and reagent requires react from cljcljs

borkdude 2023-08-23T09:59:18.537779Z

scittle already has a reagent module

borkdude 2023-08-23T10:00:45.587759Z

this means that react isn't packaged along with scittle, you need to provide your own (global) version

Andrea 2023-08-23T10:00:46.507459Z

hmm, but I’m using my own build

borkdude 2023-08-23T10:00:58.354429Z

are you using target esm?

Andrea 2023-08-23T10:01:02.598149Z

yes

Andrea 2023-08-23T10:01:32.196969Z

in our code I expect requires ["react" :as react] to be picked by the import map now

borkdude 2023-08-23T10:01:45.149259Z

ok, you can check nbb which also packages reagent without react with esm

borkdude 2023-08-23T10:01:56.419969Z

I've made a specific version of reagent there to work around some of the issues

Andrea 2023-08-23T10:02:12.315669Z

yeah, I was thinking of forking it with the string requires

borkdude 2023-08-23T10:02:24.111899Z

I already did that :)

Andrea 2023-08-23T10:02:34.289179Z

cool, under which repo?

borkdude 2023-08-23T10:02:35.089739Z

https://github.com/borkdude/nbb-reagent

Andrea 2023-08-23T10:03:32.697929Z

but it’s based on reagent 1.1.1 which expects react 17 I think?

borkdude 2023-08-23T10:03:50.634979Z

I didn't even use string requires there, I think I ran into specific issues so I just made it depend on a global being defined

borkdude 2023-08-23T10:04:00.848379Z

could be, it's been a while. I don't follow the react trends

Andrea 2023-08-23T10:07:01.908979Z

yeah, I’m seeing that, I’ll try with the string require I’m not super keen on this global stuff

Andrea 2023-08-23T10:10:31.981309Z

Is there any specific reason to prefer sci.configs for reagent than to use sci.core/copy-ns? I see e.g. set-default-compiler! is missing in sci.configs ?

borkdude 2023-08-23T10:11:52.904969Z

just try it and you will find out

Andrea 2023-08-23T10:24:28.240059Z

the string require didn’t fix my issue, and also if I’m providing reagent through :namespaces in the sci context, it will pick the one compiled by shadow right? so string requires will be sourced from npm_modules right?

borkdude 2023-08-23T10:25:11.487019Z

yes, but just look at nbb for how to configure so that it won't pull in react from your project

Andrea 2023-08-23T10:25:41.769549Z

then I have to setup the goog/global thing

borkdude 2023-08-23T10:25:59.570209Z

no, I mean the shadow config

Andrea 2023-08-23T10:26:17.484109Z

ok I see

Andrea 2023-08-23T10:29:49.129939Z

or wait, :keep-as-import as we did before

borkdude 2023-08-23T10:32:12.928569Z

you need both the string require + keep-as-import

Andrea 2023-08-23T10:32:23.244239Z

yeah, trying that

borkdude 2023-08-21T15:18:22.174549Z

I don't think this is possible since "@codemirror/state" refers to something external which is not known by scittle or the JS environment, right?

borkdude 2023-08-21T15:19:44.224589Z

should work with cherry though if you stick the complete CDN url in there: https://dev.livecodes.io/?template=clojurescript

borkdude 2023-08-21T15:20:05.198689Z

but usually when you use a script tag in this way, the script defines a global which you can then use

borkdude 2023-08-21T15:20:20.930579Z

An example of codemirror with scittle: https://babashka.org/scittle/codemirror.html

Andrea 2023-08-21T15:25:29.863459Z

oh, would squint-js + an import map, more appropriate?

Andrea 2023-08-21T15:26:19.758799Z

oh, your example has na import map

borkdude 2023-08-21T15:27:47.717539Z

scittle will work, but you need to access the codemirror stuff via the globals it defines (see example)

borkdude 2023-08-21T15:28:28.814589Z

or if it only defines modules, you even need to define globals yourself:

<script type="importmap">
      {
        "imports": {
          "codemirror": "",
          "@codemirror/commands": "",
          "@codemirror/search": "",
          "@codemirror/autocomplete": "",
          "@codemirror/lint": "",
          "crelt": "",
          "@nextjournal/lang-clojure": "",
          "@nextjournal/lezer-clojure": "",
          "@lezer/highlight": "",
          "@lezer/lr": "",
          "@lezer/common": "",
          "@codemirror/language": "",
          "@codemirror/state": "",
          "@codemirror/view": "",
          "style-mod": "",
          "w3c-keyname": ""
        }
      }
    </script>
    <script type="module" type="application/javascript">
      import * as cm from 'codemirror';
      import * as lc from '@nextjournal/lang-clojure'
      import * as cv from '@codemirror/view';
      import * as cs from '@codemirror/state';
       = cm;
       = lc;
       = cv;
      globalThis.cs = cs;
      scittle.core.eval_script_tags();
    </script>

Andrea 2023-08-21T15:30:12.251299Z

yeah I saw that... I was thinking more of keeping the cljs code as is to suit different targets (e.g. shadow)

borkdude 2023-08-21T15:30:35.569119Z

squint or cherry should work I think

borkdude 2023-08-21T15:30:51.308029Z

this is a minimal example of how to compile and run: https://squint-cljs.github.io/squint/

Andrea 2023-08-21T15:31:29.586269Z

nice thank you!

Andrea 2023-08-21T15:48:18.928879Z

I see, the eval part is done via an import of url encoded data

borkdude 2023-08-21T16:00:03.838259Z

indeed

borkdude 2023-08-21T16:00:42.822789Z

I think you could theoretically use eval if you would wrap it in a self-executing async function

borkdude 2023-08-21T16:01:03.737699Z

hmm, but then import doesn't work as is

borkdude 2023-08-21T16:01:10.652489Z

so yeah, that's why it's done this way

Andrea 2023-08-21T16:11:41.035889Z

I’ll check it out

Andrea 2023-08-21T16:11:44.712349Z

thank you!

borkdude 2023-08-21T16:13:03.430329Z

I'm contemplating an output that is more suited for hot-reloading / nREPL, then I'd have to emit import() instead and some global stuff where (defn foo []) becomes the_namespace.foo = function and then you could likely also also use eval

mkvlr 2023-08-21T16:50:36.710759Z

could sci async also support require of js modules using import maps or does it already maybe?

borkdude 2023-08-21T17:30:00.321269Z

yes, sci async can

borkdude 2023-08-21T17:53:02.283439Z

remember this: https://twitter.com/borkdude/status/1532281699018629121

borkdude 2023-08-21T17:53:45.422479Z

if you insert the right import maps in that page, dynamic import using those aliases will automatically work, this is just browser behavior

mkvlr 2023-08-21T18:06:09.144159Z

👌