Fork me on GitHub
#clojurescript
<
2023-10-14
>
adi04:10:34

I've been trying (failing) to port this code to ClojureScript: https://github.com/phiresky/sql.js-httpvfs/blob/master/example/src/index.ts I think my problem is: 1. I am mistaken about how to do the import(s), including the wasm import. (I am aware import.meta.url will not work under ClojureScript / google closure compiler). 2. If I get that to work, I don't (yet) quite understand how to write the JS and/or wasm interop to actually use the library code. Would appreciate some pointers or reading reference material. (nb. The original example expects webpack build. I am using shadow-cljs. And so I am using shadow with the "https://shadow-cljs.github.io/docs/UsersGuide.html#js-provider-external" option for webpack.)

thheller07:10:14

you don't have to use import.meta.url at all. all that gives you is the url of the script file used. you can also maybe bypass webpack entirely, but I haven't looked at the package and its requirements exactly

thheller07:10:10

basically all it wants is a url to the wasm/worker files

thheller07:10:21

you can just construct those manually. and just copy the files manually from the package dir

thheller07:10:13

this is not supported by shadow-cljs, so webpack it is

adi15:10:01

Yes, like I mentioned in point #1, there is a wasm import. URL construction is solved. I could even hard-code it. I considered manually copying files, but I don't understand how JS packages work, so I could very well break things badly. I was hoping the webpack build would "just work". However it appears to work differently between the TS project build and my Cljs project's emitted JS. I am unable to reconcile that difference, nor have I figured out how to use whatever the Cljs webpack build is producing. In the TS webpack build, the use of import.meta.url in the TS code works like an import. It copies over the sql worker dependency and wasm worker dependency into separate files, with hashed names. I haven't inspected to see if it is a straight copy or some pre-processed bundle. My best bet seems to be to try and get the Cljs-emitted JS to result in the same kind of build output. Right now it seems to be bundling the sql worker js into a bundle file with my code. And the wasm binary seems to be copied over. Maybe today will be my lucky day. Trial and error error error continues 😅 Thanks for looking into it.

adi03:10:08

Tried and failed again. I can't figure this out. I give up... using JS and getting on with life.

thheller07:10:43

as far as I can tell you can maybe just let webpack build the worker

thheller07:10:54

and let shadow-cljs build the rest

adi19:10:22

Code to repro my problem (details in the README, including error trace) https://github.com/adityaathalye/cljs-webpack-build-woes

adi19:10:22

The specific permutation of #{appropriate import statements in Cljs} X #{shadow config settings} X #{webpack config settings} X #{JS interop to actually use the imports in my Cljs} has eluded me.

thheller08:10:11

so I got this working, but I have no clue how to test it is actually working

thheller08:10:36

(ns core
  (:require
    ["sql.js-httpvfs" :as vfs]
    ))

(def db-worker
  (vfs/createDbWorker
    (clj->js [{:from "inline"
               :config {:server-mode "full"
                        :url "/example.sqlite3"
                        :requestChunkSize 4096}}])
    "/js/worker.js"
    "/js/sql-wasm.wasm"))

thheller08:10:50

;; shadow-cljs configuration
{:source-paths
 ["src"]

 :dependencies
 []

 :dev-http
 {8001 "target/public"}

 :builds
 {:app {:target :browser
        :output-dir "target/public/js"
        :modules {:main {:entries [core]}}}}}

thheller08:10:16

const path = require('path');

module.exports = {
    entry: './js/index.js',
    resolve: {
        // copied over from the reference typescript example
        extensions: [".tsx", ".ts", ".js"],
    },
    output: {
        path: path.resolve(__dirname, './target/public/js'),
        filename: 'worker.js',
        wasmLoading: 'fetch',
    },
    experiments: {
        asyncWebAssembly: true
    },
    devServer: {
        publicPath: './target/public',
    },
};

thheller08:10:42

this is js/index.js file

import "sql.js-httpvfs/dist/sqlite.worker.js";

thheller08:10:02

so, as I suggested. webpack is ONLY building the worker. shadow-cljs is building the rest

thheller08:10:18

I manually copied the wasm file cp node_modules/sql.js-httpvfs/dist/sql-wasm.wasm target/public/js/

thheller08:10:39

oh, and changed the index.html to only include the main.js as there no longer is a bundle.js

thheller08:10:11

actually, this works just fine without webpack at all

thheller08:10:50

(ns core
  (:require
    [shadow.cljs.modern :refer (js-await)]
    ["sql.js-httpvfs" :as vfs]
    ))

(defn init []
  (js-await [worker (vfs/createDbWorker
                      (clj->js [{:from "inline"
                                 :config {:server-mode "full"
                                          :url "/example.sqlite3"
                                          :requestChunkSize 4096}}])
                      "/js/worker.js"
                      "/js/sql-wasm.wasm")]
    (js-await [result (-> worker .-db (.exec "select * from mytable"))]
      (js/console.log "result" result))))

thheller08:10:02

(ns worker
  (:require ["sql.js-httpvfs/dist/sqlite.worker.js"]))

;; nothing to do, worker.js file does all the things when loaded
(defn init [])

thheller08:10:21

;; shadow-cljs configuration
{:source-paths
 ["src"]

 :dependencies
 []

 :dev-http
 {8001 "target/public"}

 :builds
 {:app {:target :browser
        :output-dir "target/public/js"
        :modules {:shared {:entries []}
                  :main {:init-fn core/init
                         :depends-on #{:shared}}
                  :worker {:init-fn worker/init
                           :web-worker true
                           :depends-on #{:shared}}
                  }}}}

thheller08:10:44

then change the html to inlucude the shared.js and main.js files, delete everything webpack related

thheller08:10:02

the worker in the npm package is already sort of pre-bundled, so there isn't any wasm processing shadow-cljs would need to do

thheller08:10:15

only thing you need to manually do is copy the .wasm file

adi04:10:43

A new woe arose with a more real-world data set. The TS code "just works", but the cljs code errors out.

Uncaught (in promise) TypeError: e is undefined
    SplitFileHttpDatabase 
It seems to originate in the sqlDOT_js-httpvfs/dist/index.js_ file. I've updated the sample repo with sqlite data, and comparable Typescript code (README explains more): https://github.com/adityaathalye/cljs-webpack-build-woes/#readme

adi04:10:12

(thheller, if you happen to take a look at this --- no expectation at all --- you can just do an --ff-only merge of my branch into your fork.)

git remote add adi [email protected]:adityaathalye/cljs-webpack-build-woes.git
git fetch adi
git merge --ff-only adi/master

thheller06:10:57

it seems to be :from "jsonconfig" and not :from "jsconfig", otherwise seems to be fine

Stef Coetzee21:10:11

👋 When using https://github.com/luciodale/fork, @lucio's forms library for Reagent and re-frame, how does one use the reset handler to clear form state upon form submission? The docs include an example of form submission and separately mention the reset handler (https://github.com/luciodale/fork#quick-overview; https://github.com/luciodale/fork/blob/ddc799eecddf9f7e0a5923ec582f6bf4db83ac24/src/fork/reagent.cljs#L72) available to form definitions. It's usage is unclear to me. The following works (edit: only once), but does not make use of said handler:

(def form-init-vals {:input nil})

(defonce form-state (r/atom form-init-vals))

(defn submit-form []
  [fork/form
   {:keywordize-keys true
    :form-id "form-id"
    :prevent-default? true
    :state form-state
    :on-submit (fn [input]
                 (.log js/console (-> input (get-in [:state]) (deref) :values))
                 ;; form state is provided and reset outside of form definition
                 (reset! form-state form-init-vals))} 
   (fn [{:keys [values form-id handle-change handle-blur submitting?
                handle-submit reset]}] ; note unused `reset` handler
     (let [input-name :input
           input-value (input-name values)]
       [:form
        {:id form-id
         :on-submit handle-submit}
        [:input
         {:name input-name
          :value input-value
          :placeholder "Provide test input"
          :on-change handle-change
          :on-blur handle-blur}]
        [:button
         {:type "submit"
          :disabled submitting?}
         "🚀"]]))])
When including reset in the value of :on-submit, a full page reload is triggered. For example:
:on-submit (fn [& args] (reset) handle-submit)

1
p-himik22:10:52

There shouldn't be a full page reload as reset only reset!s the state atom, nothing else. But in that last :on-submit, you aren't calling handle-submit, you're just using it as a value.

Stef Coetzee22:10:33

Whoops! Thanks for your assistance, @U2FRKM4TW. Altered version works as expected, specifically:

:on-submit (fn [e] (handle-submit e) (reset))

👍 1
Stef Coetzee05:10:30

(Thanks also to @lucio for the library!)

👏 1