This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-09-13
Channels
- # announcements (1)
- # babashka (12)
- # beginners (10)
- # biff (9)
- # calva (2)
- # cherry (21)
- # cider (14)
- # clj-commons (76)
- # clj-kondo (8)
- # clj-on-windows (34)
- # cljs-dev (5)
- # clojure (48)
- # clojure-austin (7)
- # clojure-europe (97)
- # clojure-nl (1)
- # clojure-norway (14)
- # clojure-uk (22)
- # clojurescript (137)
- # conjure (33)
- # cursive (4)
- # datalevin (1)
- # deps-new (4)
- # devcards (2)
- # duct (3)
- # events (1)
- # fulcro (12)
- # graphql (9)
- # hyperfiddle (16)
- # jobs (8)
- # kaocha (1)
- # leiningen (6)
- # lsp (39)
- # malli (38)
- # membrane (20)
- # nbb (68)
- # observability (7)
- # off-topic (49)
- # pathom (11)
- # polylith (8)
- # portal (22)
- # re-frame (6)
- # releases (1)
- # remote-jobs (2)
- # shadow-cljs (24)
- # spacemacs (2)
- # squint (6)
- # xtdb (7)
hi everybody! What is the best way of accomplishing something like (System/identityHashCode o)
in ClojureScript being o
any js object?
Neat! Although, a couple of caveats: • It mutates the object • Unsafe to use with function prototypes (for whatever reason - the docstring doesn't go into the detail)
you can just use cljs.core/hash
. it has a default impl to call getUid
. https://github.com/clojure/clojurescript/blob/8306bd72c5d188648256dddbc4d13f61ee3132b0/src/main/cljs/cljs/core.cljs#L1433-L1436
@thheller yeah but cljs.core/hash has the same problems as clojure.core/hash for identity, like (= (hash "Aa") (hash "BB")) => true
@U053XQP4S nice, didn't knew about it, I think it will work for my purpose, thanks!
hmmm but it only works with js objects and not with strings for example
cljs.user=> (def a "hello")
#'cljs.user/a
cljs.user=> (g/getUid a)
38
cljs.user=> (g/getUid a)
39
cljs.user=> (g/getUid a)
40
so I'm looking for something that works on every object not just #js {...}
My very first response to this thread was exactly because of all the associated problems. The thing is, JVM gives you such information but JavaScript does not. As simple as that. To exclude the XY-problem, why do you need a unique identifier for each possible value in the first place?
@U2FRKM4TW when values leave my process via a remote api I'm replacing the values with a generated value-id (kind of a reified pointer) which is stored in a map, so the callers can refer to those values later just by providing the value-id. I don't want to generate random uuids, since the value ids will change for every call even when the same values are returned making it impossible for the clients to cache some calls responses. Since I'm trying to reify references, something like identityHashCode works, but the thing I'm building also needs to work for cljs. Does it make sense?
> I don't want to generate random uuids, since the value ids will change for every call even when the same values are returned But it's exactly the same in Java, no?
jshell> System.identityHashCode(111111111111111L)
$1 ==> 648129364
jshell> System.identityHashCode(111111111111111L)
$2 ==> 1104106489
Or what do you mean?jshell> long x = 111111111111111L
x ==> 111111111111111
jshell> System.identityHashCode(x)
$4 ==> 812265671
jshell> System.identityHashCode(x)
$5 ==> 109961541
but that is ok, since the longs aren't the same object, I want basically to reify references
hmmm that is weird
identityHashCode
does not give you any guarantees, so you should not rely on any implicit assumptions you have about it.
user=> (def x 111111111111111)
#'user/x
user=> (System/identityHashCode x)
814111376
user=> (System/identityHashCode x)
814111376
user=> (System/identityHashCode x)
814111376
oh you used a primitive
hmmm...
so, I have a constraint that all my vals are values of a clojure map, those can't be primitives right?
And, BTW, a JS object is mutable. So caching it is iffy, but how much - depends on the specifics.
Maybe you can create a protocol that uses a SHA1 hash or similar for strings, getUid
for JS objects, and the like?
@U922FGW59 not sure I follow, this values can ve anything that can be put as a value in a clojure map
> so, I have a constraint that all my vals are values of a clojure map, those can't be primitives right? Can you stick to Clojure's immutable data structures? If so, I'd just use a two-way map with UUIDs where instead of generating a random UUID each time, you first check whether there's already an identical object there.
> Can you stick to Clojure's immutable data structures? no, it should be anything that can be put inside a clojure map value
I'm implementing a clojure[script] debugger https://github.com/jpmonettas/flow-storm-debugger/ and trying to use some caches to improve perf. So this values are anything that can be traced by the debugger, so anything that can flow thru a clojure program
You can write your own id
function then. But, as I said, it's iffy, will mutate the passed objects, an the whole function will be a memory leak (which is sometimes acceptable, but still).
I saw something like that here https://stackoverflow.com/questions/1997661/unique-object-identifier-in-javascript
but that is kind of goog/getUid, which I don't think will work for numbers, strings etc, since they aren't js/Object
monkey patching js/Object shouldn't leak I guess, but it will not work with numbers
This is what I meant. Use getUid
for JS-Objects, and other means for other types. Numbers can probably represent themselves, strings can be hashed, etc
yeah, hashing strings is kind of like just using hash
In the end you want to solve two problems: 1. Assign unique IDs to values so that you can look them up later by that ID 2. Efficiently check if a value is already known / assigned and ID The first can be an increasing counter. The second can be a map value -> ID, but since not all values can be keys in maps, you will need to be more creative.
This is where hashes come into play, e.g. use a map hash => vector of objects
?
I need to look by id any way
so that solution will require a {val-id value} and a {value val-id} and a counter to be efficient
> Which ones cannot be? I was assuming that JavaScript objects can't, but maybe I'm wrong?
> so that solution will require a {val-id value} and a {value val-id} and a counter to be efficient I think this is what I was saying 🙂
oh sorry, I understood that I needed only one map
Since it's for a debugger, perhaps you can use a registry. Which, well, will be a memory leak but maybe you can clean it up every now and again if you find that to be a problem. JS objects can be keys in a CLJ map. But objects of the same value will be different keys.
cljs.user=> (assoc {} #js {} 1 #js {} 2)
{#js {} 1, #js {} 2}
But seems like that's exactly what is wanted, so @U0739PUFQ can probably just find a cross-platform implementation of a bidirectional map (or write something) and use that....or a cache that will automatically expire old values
You can even assign the very same IDs locally and remotely without explicitly sending the IDs - by just using an ever increasing counter that will be increased in the same manner both locally and remotely (assuming the means by which you communicate preserves message order).
yeah, maybe that is the solution, create a bidirectional map
thanks for all the ideas
I'll give it a little more thought but at least I have that solution
I'm not sure if this is the right forum for this conversation, but I am really struggling with code/module splitting in ClojureScript for browser applications. This sounds like a complaint but the actual question at the end is, "am I missing something?" Splitting up the actual ClojureScript code is easy enough using the guides, but if you have a significant number of NPM dependencies, the size of your cljs_base.js
file is still going have the lower bound of the NPM dependencies index.bundle.js
. It's starting to seem like a mistake to bundle NPM deps with ClojureScript code, and instead go the extern route. Because then at least there is the hope of chunking the index.bundle.js
into smaller files. Annoyingly, neither the cljs.loader
framework nor the :modules
directive in foo.cljs.edn
seem to have any interesting in working with js/npm code, so it seems I'd have to handle that part manually.
Lastly and perhaps tangentially, it seems if code/module splitting were take to its logical extreme, every namespace would look like this:
(ns some-ns
(:require [cljs.loader :as loader]))
(defn bar [a b]
"blah")
(loader/load :some-other-ns
(fn [e]
((resolve 'some-other-ns/foo))))
(loader/set-loaded! :some-ns)
This is a bit cumbersome, in my opinion, as far as maintenance and API goes.
I haven't seen any more advanced guides about it on the net besides the official docs and the figwheel one, so again I'm just wondering am I missing some strategic/tactical point?As far as alternatives go, I'm considering dumping the NPM deps to externs, chunking them, then bootstrapping the initial page load with vanilla JS and then asynchronously loading the clojurescript code. Thoughts?
@goomba we cannot solve the splitting wrt. bundlers, if you want to use many NPM modules - code size is nearly always going to be an issue anyway - I think shadow does it’s own thing here? It should be possible to chunk etc. but not ClojureScript’s problem
I don’t know what you mean by “logical extreme” - but maybe it’s not clear that such things are not necessary w/ Google Closure Compiler code splitting
"logical extreme" meaning, in my mind (I may be incorrect) all dependencies (every namespace) would be loaded asyncronously using loader/load
in order to make each network hop as small as possible. That would therefore imply that there would be few if any dependencies listed in the :require
s. Again this is an extreme -- given unlimited time and maintenance was not an issue -- but in practice I would assume you would try to find a compromise of implementation speed and maintainability with performance and dynamic loading. Again -- I could be wrong here about the intentions. Seeking information. But I also agree with your point that this really isn't ClojureScript's charter to do the NPM deps, that makes sense to me. just making sure I'm not missing something obvious.
(the reason for this, in case it's not obvious, would be to make the cljs_base.js
as small as possible -- ignoring NPM deps -- so that the page loads as quickly as possible)
what I mean is that what you are suggesting is unlikely to be any better than how Google Closure Compiler works - which as far I know is quite optimal and been measured for over a decade
Pulling back from the specifics, the main problem here is I'm trying to get the initial bundle size as small as possible, and the cljs_base.js
is currently my bottleneck for that. I'm trying to move as much code out of that as possible so I can bootstrap the page as quickly as possible. And what it sounds like, and it is a fair point, that this has nothing to do with the ClojureScript compiler or the GCC and so other techniques must be used.
i.e. if you don’t use any CLJS data structures then of course the advanced output will be gzipped into bytes
it is possible to setup things so that one of your namespaces (i.e. a login page) doesn’t use anything but JS primitives
Would that then be compiled with a separate build entirely?
I should also add that this in the case of a large/complex SPA
Agreed, that's what I'm trying for. So I'm thinking the approach here is probably going to be to get rid of the NPM deps as part of the advanced compilation and side-load it, then setup my externs appropriately. And I can defer the loading of the index.bundle.js
using some other techniques. That will keep the cljs_base.js
extremely small and allow for fast page load. Worst case scenario can put the bare minimum on some vanilla js bootstrapping code and then pull in the cljs code. Does that sound practical or does it sound like I'm using the tools incorrectly?
it’s some work to setup for sure it will be annoying - but it is kind of one time job for a project - what you’re suggesting sounds like one way to do it
Also maybe one other thing is, it doesn't seem to me like GCC advanced compilation does any dead code elimination on NPM deps. Because I'm am 100% certain I am not touching all 5 or so MBs of the NPM deps but my cljs_base.js
still seems to be lower bounded by the size of the index.bundle.js
. Am I correct in that a webpacked index.bundle.js
does not benefit from GCC advanced compilation dead code elimination and I need to be more careful about pulling in only the required things?
FWIW the problem likelr is that the regular CLJS tools put all npm dependencies into the cljs_base module and webpack then adds them all to base. in shadow-cljs however npm packages end up in the modules that actually use them. closer to the edges basically. by doing so it basically does what you are looking for I guess. assuming those chunky npm deps you use aren't used you can make tiny other modules. there isn't even any hardcoded assumption that cljs.core
is in the base module, like there is in the regular tooling
you also get https://shadow-cljs.github.io/docs/UsersGuide.html#build-report which makes it much easier to tell what ends up where and giving you opportunities to move stuff if needed
I also outlined a couple techniques for better splits here https://code.thheller.com/blog/shadow-cljs/2019/03/03/code-splitting-clojurescript.html
Great guide, one of the best resources available so far. There are still a ton of issues existing literature such as it is barely touches on, though. For instance, in some senses module splitting discourages modularity. The more modular/common your code is, the more things depend on it -- so the larger your (in your example) :main
code will be. Additionally, the path for sharing data between asynchronously loaded modules is not entirely clear -- given that the pathway looks something like
(cljs.loader/load :main
(fn [e]
((resolve 'some-other-ns/foo))))
this is a bit cumbersome, if, say, foo
is a config file and what you really wanted was the value of debug-mode?
. It's very overkill. Because you'd have to further do something like
(ns some-ns.core
(:require [cljs.loader :as loader]
[cljs.core.async :as a]))
(defn is-debug-mode!? []
(let [done? (a/promise)]
(cljs.loader/load :config
(fn [e]
(a/offer done? ((resolve 'some-ns.config/get-debug-mode-async?))))))
done?))
(defn -main []
(a/go
(let [debug-mode? (a/<! (is-debug-mode!?))]
(if debug-mode?
...
...))))
(ns some-ns.config)
(def debug-mode? (some-ns.hand-wave/read-from-html-or-something :debug-mode))
(def get-debug-mode-async? []
debug-mode?)
that's a lot of work compared to
(ns some-ns.core
(:require [some-ns.config :as cfg]))
(defn -main []
(if cfg/debug-mode?
...
...))
Which may be the price that needs to be paid for production applications with small code sizes, as there is no silver bullet
Additionally I've noticed that the callback in cljs.loader/load
is not very reliable. Sometimes the event needs to be fired twice for the callback to occur, still haven't gotten to the bottom of this yet.
yes, figuring out the "best" split points can be tricky and there is no general solution. it varies greatly between apps
I made an abstraction over cljs.loader, makes it a little more convenient to use https://clojureverse.org/t/shadow-lazy-convenience-wrapper-for-shadow-loader-cljs-loader/3841
oh that is nice!
Well while we're at it with you solving most of my problems for me, do you happen to have a good answer for runtime dependency injection in clojurescript? 😁
Cool, I'll ask on the main thread anyway as this has been another tricky issue
makes sense. Thanks again for your hard (and mostly thankless) work, I appreciate it! Won't take up any more of your time.
what I would recommend is toy around w/ code splitting separate from what you are doing w/ NPM
will do. I'll do a write-up if I identify some good patterns, I think we could use some more info out there on the subject.
try what I said, i.e. minimal login ns w/ JS primitives only - and then another ns w/ core.async or something and you’ll get a sense of how it’s supposed to work
another possibility is tsickle
I haven’t time to really use it - but I think this is in heavy use at Google to use random libs - it converts ES6/TypeScript to Closure w/ generated externs
whoaaa, interesting (regarding tsickle
).
yes, a bit short on time, but it’s on my list of things to assess - it could drop the need for multiple bundlers which is a real pain
https://github.com/angular/tsickle - super light on documentation on how to use it
It also seems like there's a good possibility to use goog.events
to "communicate" between modules without explicit ((resolve 'some-ns/bar))
, so there are certainly opportunities there
I think the point where this is becoming a real pain is for things like reagent
which require react
and react-dom
as external deps, so some form of bundling (whether advanced compilation or cljsjs/react
) seem kind of necessary and the "best" way to handle it is unclear. But nothing that can't be solved with some roll-up-the-sleeves engineering effort.
But despite that, cljs is still an island in the middle of the js insanity
I’ve been in React Native land for a few years where the code size thing is significantly less of a concern and the existing approach is satisfactory
fwiw, I think for people who really care about bundle size NPM modules are serious dead end - I would just do everything in Closure
Yeah I was actually thinking about going to React/Typescript for the components, might make runtime dependency injections easier too.
again not to discourage React stuff, but some times you just want more control over the bundle, and that could be a good starting point
as an alternative to React? :thinking_face:
the thing is for “bottom” stuff writing in JS has some benefits, it can be easier to eliminate depending
the problem with writing ClojureScript is that implies the data structures which compose like 50% of the source of the standard library
but anyways, if there was a functional DOM thing in JS that you could use w/ ClojureScript that just got the job done - you could do fairly light weight development easily idiomatically
this is a separate thing then big / complex SPAs - where likely you’ve already chosen heavy solutions
but currently there’s not many super light weight things for idiomatic ClojureScript UI augmentation for web stuff
interesting. I guess hypothetically you would have to split out cljs.core to allow for runtime selection of the underlying primitives, defaulting to the cljs data structures. I'm assuming only an edge case of users would even bother with that implementation unless it were made the default.
like cljs-a-la-carte haha
At least with cljs.core it would be possible -- I was pretty shocked when I found out there were no protocols for things like assoc
and deref
in clojure.core.
In the case of deref
I stand corrected, but I'm not sure how you would polymorphically extend the behavior of assoc
in clojure.core
since it delegates to clojure.lang.RT/assoc
?
ahh I see. Good eye. Much appreciated!
I am very new to clojure(script) ecosystem and I am trying to set up cider with shadow-cljs. I can get REPL all right, but the source file is not seen as the part of the same session.
hey. indeed I am. I do not use cider, so I can't really answer any questions related to that
are you aware of any bug around it? the emacs buffer with gui.cljs has cider[not connected] at the bottom and I cannot M-. (go to source) or C-z (go to repl) from it
Ok does anyone have any insight into how cljsjs works? As in, why is it if you provide your own, for instance, cljsjs.foo
, I can (:require foo)
at the top level namespace form in a cljs project?
is that clojurescript compiler thing?
It saved my a** monkey patching reagent
did the default output of npx create-cljs-app
change recently? I just ran this for a different project last week and it did not include devcards or reagent iirc