The :practicalli/service project template create REPL driven web app projects and provides a working API server (reitit ring), optionally using donut system or integrant to manage app components. https://practical.li/clojure/clojure-cli/projects/templates/practicalli/

If you just want unique named "things" then :token/left-paren, :token/right-paren, :token/left-brace are probably good.

so just (keyword "token" "left-paren") etc etc?

would this be a good macro usecase? or would having some macro that wraps around a bunch of keyword calls like (keywords-in-ns {ns} {tokentype} {tokentype} ... ) not be autocomplete/ide-friendly?

Just type :token/left-paren

You don't need a macro or to call the keyword function

So I shouldn't be defining all the kinds of token in one specific place in the first place?

You don't need to in Clojure.

I see. I suppose that falls under what I've read about clojure's opinions on data structures etc

Yeah, it's an extremely different language to things like Java... you have to let go of a lot of your type-based and class-based thinking...

No encapsulation -- just plain data, since it is all immutable. No "types" in the traditional sense (Clojure is typed -- at runtime).

There are HTTP client libraries like hato you can use, which will handle redirects etc.

https://github.com/gnarroway/hato

i was already thinking that i might need a http lib than

if i call the map directly it works but if i do a do and add a step afterwards it doesn't work is that somehow related to the lazyness

because map is lazy. try doall or doseq instead

mapv should work as well but as long as you don't need the result it is probably better to use do... variants

ok doall it was thanks again !!!

hashmaps have value semantics, so as long as you're using objects with value semantics (normal clojure objects, not functions or atoms etc), that will be fine.

maps are harder to use in general as keys (you have to say (get m {:i-am-using "hash-map"}) instead of (:i-am-using-a-keyword m)) but not impossible or destructive

But don't use data structures with infinite seqs, because hash will be called on the value

@U06DF0BFQTZ Go right ahead and use maps & vectors as map keys, when you would otherwise compose a string and use it as a key. It's a great feature of Clojure!

(additional context: I am running on Node via shadow-cljs)

xhrio has a sync mode that can still be called in workers I believe. Not sure about the main thread on node

node might have a native sync... call too

luckily this is for a library rather than "in production"

You mean for the build or something?

https://github.com/JulianBirch/cljs-ajax/blob/master/src/ajax/xhrio.cljs this is what you mean by XHRIO, correct? I'm pretty inexperienced with CLJS - my first foray into it here is writing a CLJC-compatible library

https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest_API/Synchronous_and_Asynchronous_Requests

It's going to give you the nasty gram though "Synchronous XHR is now deprecated and should be avoided in favor of asynchronous requests."

It's generally recommended "don't do that"

I have a library for simulated clojure blocking semantics in workers here: https://github.com/johnmn3/cljs-thread

But couldn't yet recommend it for libraries, as it imposes build complexities on your users

I suppose I understand all the reasons for it in the abstract, but I am literally just loading a file from a URL one time and binding it to a var, in a test namespace it seems like "simple things should be simple" isn't the case here

That they should be able to choose to opt into

most folks recommend this for doing that kind of thing on node: https://github.com/funcool/promesa

(and my cljs-thread lib isn't ported to node yet)

That'll be a pure SharedArrayBuffer solution, without the ServiceWorker solution, so much different impl

this looks closer to what I want, but I don't know exactly how to use it: https://clojureverse.org/t/promise-handling-in-cljs-using-js-await/8998

Yeah if that fixes it for you, for sure. Does that work in vanilla CLJS? Prolly doesn't matter if it's just a test ns, but I'd be cautious of that for a lib

I don't think it solves my problem. I may be more confused now than when I asked this question. It is hard for me to reconcile my expectations of being able to program using plain values with the way promises work. I keep seeing await in JS examples but there's not an exact equivalent in ordinary CLJS. It would be nice to just be able to dereference a Promise to obtain its result value, (just like a future in JVM Clojure), but it seems that's not possible?

Async testing has always been a pain in cljs. Probably js as well

Not as much with the await stuff perhaps

Depending on the kind of test, you might want to just save the result of the fetch in a fixture and test against that static data

If you're actually trying to exercise that fetch machinery though then yeah it's gonna be a pita

the most expedient thing is going to be just saving a copy of this file locally and using the filesystem API

Once cljs-thread is ported to node and I figure the testing story out, it should be way easier for clojurists to write tests on cljs. It's as easy as ... @(fetch ...

and your project's tests don't impose build requirements on your library's users, so no big deal

that is exactly the DX I am hoping for - thanks for helping me puzzle through this!

I once built a working version on SABs in the browser but my last attempt on node was leading to deadlocks. Not sure if it's a node thing or if I just forgot how I did it - might be in an old repo somewhere. Also note that there are a number of https://clojurescript.org/tools/testing#async-testing

Node is asynchronous, you cannot do synchronous (i.e. blocking) operations.

You have to rewrite your logic in an asynchronous style.

The entire thing is single threaded, so any sync operation would block the entire app/server. Which is why it's forcing asynchronicity everywhere.

Well, there's workers, so you can simulate most of that stuff, but that's the general case

No, workers are also non-blocking. So the interaction with them is still asynchronous.

js/Atomics has a blocking operation. After those came out, JS was no longer the same language. The only remaining constraint is that there's no interuptability. You can block with js/Atomics.wait() though

also you can block with an xhr hack, which is what I'm doing with cljs-thread in the browser

and there's other blocking hacks too

But the js/Atomics blocking semantic is the blessed way

.wait only works in workers in the browser though

it doesn't block on the main thread in the browser

not sure about node

Main thread can't use that I believe. Both in browsers and in Node it's not allowed

Yeah that's what I thought

Which is why you cannot block in JS 😉. You can deadlock your worker threads. But you can never remove the asynchronous style of programming from your main application. So interaction with worker threads is same as with any other Node API (some of them already spawned thread behind the scenes).

That's just not true, you can do everything in workers

So you can have your blocking semantics back

Read https://github.com/johnmn3/cljs-thread

I'm not say you should block in all the same situations you would on the jvm

But it'd make stuff like testing way easier

I mean, that seems pretty anemic. You'll still be asynchronous between the worker and all Node APIs, unless you wrap all Node API back into a worker thread, and massage the output into a SharedArrayBuffer and wrap some atomics around that. But I don't know, at this point it seems a stretch to still say you can write synchronous blocking code.

Either you have it or you don't

I'm also not sure what's going to happen with your error stack and error handling.

EIther you have a blocking semantic or you don't. The performance of that semantic may vary from platform to platform, causing you to make different decisions. cljs-thread serializes everything, so yeah, there's a lot you wouldn't want to do with it. But for simple, mechanical blocking semantics? Yes, it definitely has that

Like for a test

I don't care that it takes 10 extra milliseconds to serialize the data between workers. It's not that big of a deal

A tight loop? Not so much, don't do that kind of pattern that you'd do on the jvm

Technically, underneath it all, everything is async

sync is just an illusion we maintain

Sync is a "developer convenience"

And I'm not just going to give it up because of some sense in which async can be more performant

Ok, but you don't have full blocking semantics, it's very restricted. So I'm not sure it's fair to say you do. The nuance is huge here. First off, the language itself tries to steer you away from it, by not allowing it on the main thread. Then it restricts it to only SharedArrayBuffer. All APIs for I/O are non-blocking. So you need a ton of custom wrappers if you want to recreate more of the APIs to exist in a blocking state. And then, you still can only use them in a blocking style if you've made your main thread a shim to a worker thread that you then use as-if it was the main thread.

It's a way of communicating between developers

Blocking semantics are not a qualitative thing

it's either on or off

Like, anybody you.told, ya come use JS, you'll love it, it totally lets you write synchronous blocking or asynchronicity non-blocking.code, after they try it, will be mad that you scammed them lol. Even if on some technicality you were not totally lying

Well, we'll have persistent datastructures built directly on SABs one day anyway, but yeah, I hear you

But no, people just need to realize there are different performance characteristics

There are tons of use-cases for blocking semantics besides performance related things

where you don't care that a block might add a few milliseconds

Like having a blocking repl, for instance

All kinds of mechanical things that depend on a block

ok, so concretely, if I have tests that I need to make work in CLJS by making them async, I need to refactor from something like:

(def my-test-data (read ""))

(t/deftest my-test
  (do-something my-test-data)
  ...)

(defn execute-test [input-data]
  (do-something input-data))

(t/deftest my-test
  (.then (js/fetch "")
         (fn [resp] 
         (.then (.text resp)
                (fn [resp-text] (execute-test resp-text))))))

Did you see the (async done ... idiom in the docs?

(async done
        (go
          (is (= (<! res) :awesome))
          (done)))))

Read: https://clojurescript.org/tools/testing it shows how to write a test for async code

(deftest test-async
  (async done
    (.then
      (js/fetch "")
      (fn [resp]
       (is (= resp :awesome))
       (done)))))

It's a little bit more annoying, and you have to make sure (done) is called always. Or your tests are going to be waiting forever. There are some libs that implement a more convenient async test which has timeout baked in.

@U0K064KQV also, there's parallelism scenarios in javascript, like for rendering map tiles in parallel, where you're going to be dealing with serialization for your non-binary data anyway, and you've already decided that the compute cost far outweighs the serialization costs, then no, there's absolutely no reason to not give the developer blocking semantics, which might add 5 milliseconds to a 500 millisecond render operation. But yeah, I agree that there's a lot of parallelism use cases on the jvm that you should not try on cljs-thread. FWIW, for pretty much all the languages being ported to wasm, for their threading implementation, they're all built on js/Atomics. So we can't really say that js doesn't do that anymore. It's only a matter of time before those tools are built out on the JS side as well.

Their shared memories are living in SABs though, so those host guest langs can implement shared memory

And all the features those guest langs use to build up their primitives are also available on the js side, so it's only a matter of time before similar interoperability tools are built out on the JS side

Here's an example of a JS object built out inside a SAB, which can act like a shared object between threads: https://github.com/Bnaya/objectbuffer

But even without shared memory, the cljs-thread repo shows an example with the => thread operator that fans work across a worker pool and turns a 20 second job into an 8 second job. It's all a question of compute vs io costs

A 3X parallelism speed up is a 3X speedup, regardless of whether the blocking semantics are being simulated, you know?

If your job is emberassingly compute bound, the serialization/io costs of cljs-thread are going to completely wash out, and then the semantics will be indistinguishable from CLJ, so I don't see the point of not having them

Well, there's still a lot of differences from the clj semantics wrt binding conveyance and whatnot but that's about as close as we're going to get for now until things start moving to SABs

I wouldn't doubt if one day this SAB interface is used to abstract a whole datacenter into a single shared memory space for fanning work across datacenter nodes, it's awesome

Well, it's not correct to say cljs-thread is simulating blocking semantics - they are blocking semantics. They just have a different io/compute budget than on the jvm, so the usecases are different. But even on the jvm, there are algorithms which get ruined when trying to parallelize them, due to thread communication overhead. So it's not like the jvm doesn't have the same problem in simulating a shared memory space, it just has a different budget

I need to check cljs-thread, but how are you able to avoid coloring?

It's a genuine block, so no coloring happens

The whole worker hangs until released

> In workers, spawn returns a derefable, which returns the body's return value. In the main/screen thread, it returns a promise > Right, I get it, from worker to worker you do a real block, but from the main thread it's still async.

So you're using SAB ? I got confused by your serialization? Is there a way to block from one worker to another that doesn't use SAB?

Yeah. And you'll generally move your headspace into the worker, where the main thread is just the screen you flush the colored bits to

cljs-thread is actually using sync xhr and intercepting the call by the service worker, then sending the answer in the response after it's computed by the other worker

SAB works too, but you have to enable COOP security settings in headers, etc, so the current service worker version is a lower common denoninator

> Synchronous requests block the execution of code which causes "freezing" on the screen and an unresponsive user experience > Oh, I didn't know XMLHttpRequest existed in a blocking form.

I think it's been disabled on the main thread in the past decade, but not in workers

this thing does the same thing https://partytown.builder.io/how-does-partytown-work

tries to use SABs but then falls back to the xhr/service-worker hack if sabs can't be enabled

It still colors the code a bit at the top, in the main thread. I don't think we disagree on anything really. Maybe semantics a little. There is limited blocking support in more modern JS. But it doesn't yet seem to allow a fully blocking coding style, because they prevent their use from the main thread, and also don't let you block on everything, so you need to wrap all APIs in the few things that allow blocking. You should definitely use that support when it makes sense, that's why they added it.

Yeah, in this model, you wouldn't want to use the main thread for most things. Keep it free for just rendering concerns

Then your main application logic lives in a main worker

Then there's no coloring

Well, there's less of it haha. I mean, you got colors around setting up a virtual main worker threads. Can you not edit the DOM from a worker thread directly? You say the main thread still needs to render, is that why ?

You could https://github.com/mmis1000/DOM-Proxy from a worker. But yeah, you can just leave dom calls to main thread logic and application/business logic in the workers.

here's a "main" that launches in a worker, launching all the business work in re-frames subs and events https://github.com/johnmn3/cljs-thread/blob/master/shadow_dashboard/src/main/dashboard/core.cljs

Ya, seems tricky, you still need to carefully consider which parts are on the main thread, and those will still interact in an async style. For legacy reasons like using existing scripts in workers, I get it. But I'd favor just writing all my app in an async style personally.

With lots of examples in that namespace of doing things involving blocking, with no coloring

I'm not sure what you mean

that core namespace runs in a worker and you don't have to consider which parts are running on the main thread

just none of it runs on the main thread

Once you're done building out the widgets, you don't really have to do too much work on the main screen thread and you can start building out logic in backend workers

Ya but that's an explicit design you put in place.

You need to find a way to have the Dom changes done on the main thread.

Dom changes are like 2% of your app

and in react apps, they don't happen until after a reconcile, on the main thread

For backend it might make more sense. You could probably have a framework that calls your API entry point in a worker. But it kind of defeats the point of Node, for solving the 10k challenge.

What I mean is that, the whole framework needs to be setup for it.

With cljs-thread, you have to set up the build system. Then you just use it as a lib, from the different entry points defined by the build

So when I program, I need to be aware of it all, wanted to write some quick Dom code here, oh wait, I can't, this isn't running on a thread that allows it, etc.

Look at this folder: https://github.com/johnmn3/cljs-thread/tree/master/shadow_dashboard/src It has three folders in it. Anything that touches the dom is in the "main" folder

It's a little bit like how core.async always feels like a big hack, because it can't see inside HOF, and those are commonly used, but now you always have to be aware of it.

But, from a worker, you could do @(in :screen (touch-dom ...)) derefing it in the worker and getting the result of the dom touch

How do you order that?

This specifically frees you from the coloring of await/core.async

what do you mean?

I guess I'm not sure what that does. ":screen" is the main thread ?

Yes, you can refer to nodes by their var or their id, which is a key word. And you can do work on them with in and :screen is one node in the mesh

You can schedule things to run on the main thread? Even though it's not a worker ?

It gets done on the main thread asynchronously, but if the worker making the call wants to wait for the result, they can

And this does not "infect" forms that come after the deref. The actual value is returned from the deref. So things in workers aren't getting colored

In this call, the value returned by the deref is the actual map, not a promise of it:

(->> @(in s1 (-> (js/fetch "")
                 (.then #(.json %))
                 (.then #(yield (js->clj % :keywordize-keys true)))))
     :iss_position
     (println "ISS Position:"))

I think when some people see cljs-thread, some think calls must be magically getting queued up. But they're not. It's actual blocking

Ok, I'm intrigued how you queued up an async Dom change on the main thread with "in". But that's another topic. Let me see if I can explain. Coloring might not be the best word. I think what I mean is more, what's annoying with coloring, is that everything in the code is always wrapped in something else. Maybe I'm missing it, but I feel there's still be a lot of wrapping happening here. And also, depending what ops you want to do, figuring out where it needs to be scheduled and all that. So say I just wanted to do:

(def ele (.getElementById js/document "foo"))

(defn on-click []
 (let [a (.text (fetch "some/url"))
       b (.text (fetch "another/url")]
  (set! (.-innerHTML ele) (str a b))))

Yeah, that's the common usage of the term "colored functions" I think

This is written in blocking style. So how would it look in cljs-thread? I'm assuming changes are needed no?

Okay, so for that, you'd probably implement that in the screen/main folder that I mentioned above

oh, you put the fetch there

You'd put a dispatch in the click that did the fetch in a worker

then a sub in the place in the dom where you want the result put

and your regs and subs pass the data between the screen thread and the backend transparently

well, the worker backend I mean

The idea is you just build the necessary bits you need to operate on the dom

Well, I'm assuming this is some "blocking" fetch that we used cljs-thread to wrap the async fetch with

In that dashboard demo, I'm storing some state in the screen thread, like hover states and ephemeral data, while I'm keeping more long lived app state in the app db shared between all the workers and the front end

Right, so this is what I meant by, it requires considerate design. You don't just go and write blocking code like if it was Python or Java

Right, well if you remember writing awt on java back in the day, it's kinda like that. Where you have this thread-safe dom type thing you can manipulate, which has its own limitations. But the rest of your app isn't about awt and doesn't have the same limitations

again, you could bring in a dom proxy thing, so your workers feel like they're acting on the dom. But those things queue their actions. And we can literally wait, so we don't need to queue actions. We can just issue form to the main thread to execute on the dom and wait, or just define the thing on the thread that runs in the main thread.

And with the way we write cljs apps these days, we decouple most of our business logic from our dom manipulation code anyway

If you follow that dashboard app architecture you'll see the pattern. Future dev on that app would probably be 25% screen thread logic, writing literal hiccup forms, and the other 75% can all be done in worker

Some new fancy frameworks define components fully in data, pre-wrapping click handlers and what not. Those could be defined and controlled fully from the worker, kinda like SSR but from a worker

So you can ship the hiccup from the worker if you're willing to wrap the functiony things in dataish DSLs

I just prefer to have a mini app, dedicated to rendering some things super fast, like the shell of the app and its main components, while offloading the rest of the app to the workers

Ya, it's cool, and it's nice that you can do it.

Update: here is something interesting. If I disable permessage-defale, I no longer get the error.

(defn ws-request [^HttpServerExchange exchange ^IPersistentMap headers ^WebSocketConnectionCallback callback]
  (let [handler (->  (WebSocketProtocolHandshakeHandler. callback)
                     #_(.addExtension (PerMessageDeflateHandshake. true 6)))]
    (when headers
      (set-headers (.getResponseHeaders exchange) headers))
    (.handleRequest handler exchange)))

sounds like a proxy in the middle between the client and server closing connections because it doesn't understand websockets

What's weird is, I am able to get this to repro locally. This means all that is chrome, opening a connection to localhost:8888. I am not sure where the proxy would come from

Wanted to follow up here, for posterity. Here's what caused this: I called [Websockets/sendText](https://github.com/undertow-io/undertow/blob/master/core/src/main/java/io/undertow/websockets/core/WebSockets.java#L74) with a timeout When sending large messages, the timeout would trigger, and close the connection. However, there's no indication that this was due to a timeout, because of how undertow cancels. https://github.com/undertow-io/undertow/blob/master/core/src/main/java/io/undertow/websockets/core/WebSockets.java#L971-L975 I removed the timeout for now

it is the same for java. It’s all the jvm underneath.

❯ clj -Sdeps '{:deps {hiccup/hiccup {:mvn/version "2.0.0-RC3"}}}' -M -e '(System/getProperty "java.class.path")'
"src:/Users/dan/.m2/repository/hiccup/hiccup/2.0.0-RC3/hiccup-2.0.0-RC3.jar:/Users/dan/.m2/repository/org/clojure/clojure/1.11.2/clojure-1.11.2.jar:/Users/dan/.m2/repository/org/clojure/core.specs.alpha/0.2.62/core.specs.alpha-0.2.62.jar:/Users/dan/.m2/repository/org/clojure/spec.alpha/0.3.218/spec.alpha-0.3.218.jar"

Okay this makes sense. I copied paste from calva channel's reply to my post.

if that is how dependencies and classpaths are managed. Since it is only hiccup that causes you these troubles it could be something with how that dependency is specified.

the confusion -> could be something with how that dependency is specified? hmm...

What I meant is that since only hiccup causes those linter warnings (and only hiccup.form from what it looks like) maybe something with how the dependency on hiccup is declared could be funny. If you can create a small project which has the problem and publish to a public repository, we can help in figuring out what’s going wrong.

Thanks @U0ETXRFEW I will test first, see whether can resolve this issue first.

Reproducing the problem in a minimal project could be an effective way to realize what the problem is about. It happens to me very often when I try to minify my reproductions.

I'm assuming there's a transitive dependency mismatch between figwheel-main and the jetty adapter?

Seems like there's a ticket to update ring in figwheel-main to the latest... From two years ago. And there haven't been any changes to the project on that amount of time either. Is figwheel-main dead?

Jetty has major API changes between each major release. 9 -> 10 was seriously breaking. 10 -> 11 not as much, as I recall. 11 -> 12 is another big, breaking release. Ring 1.12.1 uses Jetty 11.0.20, so you won't be able to use that with tooling that relies on Jetty 9.

Figwheel main depends explicitly on Jetty 9 for the websocket stuff.

I like Figwheel but even I'm beginning to accept that I'll probably need to switch to Shadow-cljs the next time I build any ClojureScript stuff...

Yeah, guess I'll try shadow. I liked the idea of not needing npm and package.json

You don't need npm etc. You can use Shadow-cljs as a pure JVM/Clojure dependency.

When I raised that same objection, that's what thheller told me 🙂

Oh, interesting. The quickstart showed npx emitting a package json right off the bat. I'll poke around further. Thanks for the hint!

Right, you don't need to install it via npx if you're trying to avoid npm etc. But that's the default and the "recommended" approach because "who builds frontends without npm?" 🙂

as cljs is a hosted language, eschewing it’s platform is avoiding a key benefit of the language. For cljs, many times people want to make a react app quickly. And the old way of doing this is cljsjs which is a huge project with boot scripts to package npm projects into cljs consumable packages. https://github.com/cljsjs/packages . You had to hope that the package you needed was in there, and then hope that the version you needed was in there. Using npm is so much nicer than the old way.

Yeah, I've changed my stance a bit after working on our frontend app quite a bit over the past year. It's full of npm "goodness" and, while I still think the ecosystem has many flaws, I don't really see it being entirely practical to avoid it completely...

Same. Like to keep it simple but there are some goodies out there.