Fork me on GitHub
#clojurescript
<
2022-09-13
>
jpmonettas12:09:42

hi everybody! What is the best way of accomplishing something like (System/identityHashCode o) in ClojureScript being o any js object?

p-himik13:09:46

A leading question - what would be a way to achieve that in JavaScript?

leonoel13:09:28

(goog/getUid o)

👍 1
p-himik13:09:23

Neat! Although, a couple of caveats: • It mutates the object • Unsafe to use with function prototypes (for whatever reason - the docstring doesn't go into the detail)

thheller15:09:39

same as clojure does for System/identityHashCode

jpmonettas16:09:14

@thheller yeah but cljs.core/hash has the same problems as clojure.core/hash for identity, like (= (hash "Aa") (hash "BB")) => true

😱 2
jpmonettas16:09:05

@U053XQP4S nice, didn't knew about it, I think it will work for my purpose, thanks!

jpmonettas16:09:47

hmmm but it only works with js objects and not with strings for example

cljs.user=> (def a "hello")
#'cljs.user/a
cljs.user=> (g/getUid a)
38
cljs.user=> (g/getUid a)
39
cljs.user=> (g/getUid a)
40

jpmonettas16:09:03

so I'm looking for something that works on every object not just #js {...}

p-himik16:09:13

My very first response to this thread was exactly because of all the associated problems. The thing is, JVM gives you such information but JavaScript does not. As simple as that. To exclude the XY-problem, why do you need a unique identifier for each possible value in the first place?

jpmonettas16:09:16

@U2FRKM4TW when values leave my process via a remote api I'm replacing the values with a generated value-id (kind of a reified pointer) which is stored in a map, so the callers can refer to those values later just by providing the value-id. I don't want to generate random uuids, since the value ids will change for every call even when the same values are returned making it impossible for the clients to cache some calls responses. Since I'm trying to reify references, something like identityHashCode works, but the thing I'm building also needs to work for cljs. Does it make sense?

p-himik16:09:46

> I don't want to generate random uuids, since the value ids will change for every call even when the same values are returned But it's exactly the same in Java, no?

jshell> System.identityHashCode(111111111111111L)
$1 ==> 648129364

jshell> System.identityHashCode(111111111111111L)
$2 ==> 1104106489
Or what do you mean?

p-himik16:09:19

jshell> long x = 111111111111111L
x ==> 111111111111111

jshell> System.identityHashCode(x)
$4 ==> 812265671

jshell> System.identityHashCode(x)
$5 ==> 109961541

jpmonettas16:09:20

but that is ok, since the longs aren't the same object, I want basically to reify references

jpmonettas16:09:44

hmmm that is weird

p-himik16:09:15

identityHashCode does not give you any guarantees, so you should not rely on any implicit assumptions you have about it.

jpmonettas16:09:16

user=> (def x 111111111111111)
#'user/x
user=> (System/identityHashCode x)
814111376
user=> (System/identityHashCode x)
814111376
user=> (System/identityHashCode x)
814111376

jpmonettas16:09:29

oh you used a primitive

p-himik16:09:51

I did - same way you used a primitive in JS. ;)

jpmonettas16:09:17

so, I have a constraint that all my vals are values of a clojure map, those can't be primitives right?

p-himik16:09:21

And, BTW, a JS object is mutable. So caching it is iffy, but how much - depends on the specifics.

Ferdinand Beyer16:09:31

Maybe you can create a protocol that uses a SHA1 hash or similar for strings, getUid for JS objects, and the like?

jpmonettas16:09:56

@U922FGW59 not sure I follow, this values can ve anything that can be put as a value in a clojure map

p-himik16:09:58

> so, I have a constraint that all my vals are values of a clojure map, those can't be primitives right? Can you stick to Clojure's immutable data structures? If so, I'd just use a two-way map with UUIDs where instead of generating a random UUID each time, you first check whether there's already an identical object there.

jpmonettas16:09:46

> Can you stick to Clojure's immutable data structures? no, it should be anything that can be put inside a clojure map value

jpmonettas16:09:05

I'm implementing a clojure[script] debugger https://github.com/jpmonettas/flow-storm-debugger/ and trying to use some caches to improve perf. So this values are anything that can be traced by the debugger, so anything that can flow thru a clojure program

p-himik16:09:12

You can write your own id function then. But, as I said, it's iffy, will mutate the passed objects, an the whole function will be a memory leak (which is sometimes acceptable, but still).

jpmonettas16:09:12

but that is kind of goog/getUid, which I don't think will work for numbers, strings etc, since they aren't js/Object

jpmonettas16:09:00

monkey patching js/Object shouldn't leak I guess, but it will not work with numbers

Ferdinand Beyer16:09:34

This is what I meant. Use getUid for JS-Objects, and other means for other types. Numbers can probably represent themselves, strings can be hashed, etc

p-himik16:09:59

But gotta keep in mind that hashes aren't unique.

jpmonettas16:09:36

yeah, hashing strings is kind of like just using hash

Ferdinand Beyer17:09:47

In the end you want to solve two problems: 1. Assign unique IDs to values so that you can look them up later by that ID 2. Efficiently check if a value is already known / assigned and ID The first can be an increasing counter. The second can be a map value -> ID, but since not all values can be keys in maps, you will need to be more creative.

Ferdinand Beyer17:09:42

This is where hashes come into play, e.g. use a map hash => vector of objects?

p-himik17:09:45

> not all values can be keys in maps Which ones cannot be?

jpmonettas17:09:54

I need to look by id any way

jpmonettas17:09:33

so that solution will require a {val-id value} and a {value val-id} and a counter to be efficient

Ferdinand Beyer17:09:37

> Which ones cannot be? I was assuming that JavaScript objects can't, but maybe I'm wrong?

Ferdinand Beyer17:09:09

> so that solution will require a {val-id value} and a {value val-id} and a counter to be efficient I think this is what I was saying 🙂

jpmonettas17:09:38

oh sorry, I understood that I needed only one map

p-himik17:09:35

Since it's for a debugger, perhaps you can use a registry. Which, well, will be a memory leak but maybe you can clean it up every now and again if you find that to be a problem. JS objects can be keys in a CLJ map. But objects of the same value will be different keys.

cljs.user=> (assoc {} #js {} 1 #js {} 2)
{#js {} 1, #js {} 2}
But seems like that's exactly what is wanted, so @U0739PUFQ can probably just find a cross-platform implementation of a bidirectional map (or write something) and use that.

Ferdinand Beyer17:09:14

...or a cache that will automatically expire old values

p-himik17:09:58

You can even assign the very same IDs locally and remotely without explicitly sending the IDs - by just using an ever increasing counter that will be increased in the same manner both locally and remotely (assuming the means by which you communicate preserves message order).

jpmonettas17:09:32

yeah, maybe that is the solution, create a bidirectional map

jpmonettas17:09:40

thanks for all the ideas

jpmonettas17:09:12

I'll give it a little more thought but at least I have that solution

Lone Ranger16:09:59

I'm not sure if this is the right forum for this conversation, but I am really struggling with code/module splitting in ClojureScript for browser applications. This sounds like a complaint but the actual question at the end is, "am I missing something?" Splitting up the actual ClojureScript code is easy enough using the guides, but if you have a significant number of NPM dependencies, the size of your cljs_base.js file is still going have the lower bound of the NPM dependencies index.bundle.js. It's starting to seem like a mistake to bundle NPM deps with ClojureScript code, and instead go the extern route. Because then at least there is the hope of chunking the index.bundle.js into smaller files. Annoyingly, neither the cljs.loader framework nor the :modules directive in foo.cljs.edn seem to have any interesting in working with js/npm code, so it seems I'd have to handle that part manually. Lastly and perhaps tangentially, it seems if code/module splitting were take to its logical extreme, every namespace would look like this:

(ns some-ns
  (:require [cljs.loader :as loader]))

(defn bar [a b]
  "blah")

(loader/load :some-other-ns
               (fn [e]
                 ((resolve 'some-other-ns/foo))))

(loader/set-loaded! :some-ns)
This is a bit cumbersome, in my opinion, as far as maintenance and API goes. I haven't seen any more advanced guides about it on the net besides the official docs and the figwheel one, so again I'm just wondering am I missing some strategic/tactical point?

Lone Ranger16:09:32

As far as alternatives go, I'm considering dumping the NPM deps to externs, chunking them, then bootstrapping the initial page load with vanilla JS and then asynchronously loading the clojurescript code. Thoughts?

dnolen16:09:40

@goomba we cannot solve the splitting wrt. bundlers, if you want to use many NPM modules - code size is nearly always going to be an issue anyway - I think shadow does it’s own thing here? It should be possible to chunk etc. but not ClojureScript’s problem

dnolen16:09:42

I don’t know what you mean by “logical extreme” - but maybe it’s not clear that such things are not necessary w/ Google Closure Compiler code splitting

dnolen17:09:05

it is nothing like “chunking”

dnolen17:09:29

Closure will move functions, properties, object methods etc.

Lone Ranger17:09:50

"logical extreme" meaning, in my mind (I may be incorrect) all dependencies (every namespace) would be loaded asyncronously using loader/load in order to make each network hop as small as possible. That would therefore imply that there would be few if any dependencies listed in the :requires. Again this is an extreme -- given unlimited time and maintenance was not an issue -- but in practice I would assume you would try to find a compromise of implementation speed and maintainability with performance and dynamic loading. Again -- I could be wrong here about the intentions. Seeking information. But I also agree with your point that this really isn't ClojureScript's charter to do the NPM deps, that makes sense to me. just making sure I'm not missing something obvious.

Lone Ranger17:09:25

(the reason for this, in case it's not obvious, would be to make the cljs_base.js as small as possible -- ignoring NPM deps -- so that the page loads as quickly as possible)

dnolen17:09:47

what I mean is that what you are suggesting is unlikely to be any better than how Google Closure Compiler works - which as far I know is quite optimal and been measured for over a decade

dnolen17:09:12

there are no chunks in Google Closure

dnolen17:09:34

what you are really is saying this is an entry-point for my application

dnolen17:09:44

and it is irrelevant what is in that file or in the dependences

dnolen17:09:56

Google will move and eliminate

dnolen17:09:40

more chunks doesn’t necessarily make anything faster

dnolen17:09:04

but getting only what a chunk needs down to individual functions probably does matter

Lone Ranger17:09:18

Pulling back from the specifics, the main problem here is I'm trying to get the initial bundle size as small as possible, and the cljs_base.js is currently my bottleneck for that. I'm trying to move as much code out of that as possible so I can bootstrap the page as quickly as possible. And what it sounds like, and it is a fair point, that this has nothing to do with the ClojureScript compiler or the GCC and so other techniques must be used.

dnolen17:09:49

there are few other things which are not obvious that I verified a long time ago

dnolen17:09:41

i.e. if you don’t use any CLJS data structures then of course the advanced output will be gzipped into bytes

dnolen17:09:22

it is possible to setup things so that one of your namespaces (i.e. a login page) doesn’t use anything but JS primitives

Lone Ranger17:09:42

Would that then be compiled with a separate build entirely?

Lone Ranger17:09:19

I should also add that this in the case of a large/complex SPA

dnolen17:09:54

it is not magic, but you can design the initial load to avoid all the other stuff

dnolen17:09:12

really the only thing that should be in base is ClojureScript data structures / fns

dnolen17:09:31

so 20-25K gzipped base

Lone Ranger17:09:10

Agreed, that's what I'm trying for. So I'm thinking the approach here is probably going to be to get rid of the NPM deps as part of the advanced compilation and side-load it, then setup my externs appropriately. And I can defer the loading of the index.bundle.js using some other techniques. That will keep the cljs_base.js extremely small and allow for fast page load. Worst case scenario can put the bare minimum on some vanilla js bootstrapping code and then pull in the cljs code. Does that sound practical or does it sound like I'm using the tools incorrectly?

dnolen17:09:37

it’s some work to setup for sure it will be annoying - but it is kind of one time job for a project - what you’re suggesting sounds like one way to do it

Lone Ranger17:09:18

Also maybe one other thing is, it doesn't seem to me like GCC advanced compilation does any dead code elimination on NPM deps. Because I'm am 100% certain I am not touching all 5 or so MBs of the NPM deps but my cljs_base.js still seems to be lower bounded by the size of the index.bundle.js. Am I correct in that a webpacked index.bundle.js does not benefit from GCC advanced compilation dead code elimination and I need to be more careful about pulling in only the required things?

thheller18:09:54

FWIW the problem likelr is that the regular CLJS tools put all npm dependencies into the cljs_base module and webpack then adds them all to base. in shadow-cljs however npm packages end up in the modules that actually use them. closer to the edges basically. by doing so it basically does what you are looking for I guess. assuming those chunky npm deps you use aren't used you can make tiny other modules. there isn't even any hardcoded assumption that cljs.core is in the base module, like there is in the regular tooling

thheller18:09:33

you also get https://shadow-cljs.github.io/docs/UsersGuide.html#build-report which makes it much easier to tell what ends up where and giving you opportunities to move stuff if needed

Lone Ranger17:09:31

Great guide, one of the best resources available so far. There are still a ton of issues existing literature such as it is barely touches on, though. For instance, in some senses module splitting discourages modularity. The more modular/common your code is, the more things depend on it -- so the larger your (in your example) :main code will be. Additionally, the path for sharing data between asynchronously loaded modules is not entirely clear -- given that the pathway looks something like

(cljs.loader/load :main 
   (fn [e]
     ((resolve 'some-other-ns/foo))))
this is a bit cumbersome, if, say, foo is a config file and what you really wanted was the value of debug-mode? . It's very overkill. Because you'd have to further do something like
(ns some-ns.core
  (:require [cljs.loader :as loader]
            [cljs.core.async :as a]))

(defn is-debug-mode!? []
  (let [done? (a/promise)]
     (cljs.loader/load :config 
        (fn [e]
          (a/offer done? ((resolve 'some-ns.config/get-debug-mode-async?))))))
     done?))

(defn -main []
   (a/go 
     (let [debug-mode? (a/<! (is-debug-mode!?))]
        (if debug-mode?
             ...
             ...))))
(ns some-ns.config)

(def debug-mode? (some-ns.hand-wave/read-from-html-or-something :debug-mode))

(def get-debug-mode-async? []
   debug-mode?)
that's a lot of work compared to
(ns some-ns.core
  (:require [some-ns.config :as cfg]))

(defn -main []
  (if cfg/debug-mode?
     ...
     ...))

Lone Ranger17:09:00

Which may be the price that needs to be paid for production applications with small code sizes, as there is no silver bullet

Lone Ranger17:09:04

Additionally I've noticed that the callback in cljs.loader/load is not very reliable. Sometimes the event needs to be fired twice for the callback to occur, still haven't gotten to the bottom of this yet.

thheller17:09:23

yes, figuring out the "best" split points can be tricky and there is no general solution. it varies greatly between apps

thheller17:09:38

I made an abstraction over cljs.loader, makes it a little more convenient to use https://clojureverse.org/t/shadow-lazy-convenience-wrapper-for-shadow-loader-cljs-loader/3841

thheller17:09:47

not perfect but works

Lone Ranger17:09:22

oh that is nice!

Lone Ranger17:09:09

Well while we're at it with you solving most of my problems for me, do you happen to have a good answer for runtime dependency injection in clojurescript? 😁

Lone Ranger17:09:52

Cool, I'll ask on the main thread anyway as this has been another tricky issue

dnolen17:09:48

advanced compilation cannot work on random JS libraries

dnolen17:09:14

you have to try different packers if you want to get that smaller

Lone Ranger17:09:13

makes sense. Thanks again for your hard (and mostly thankless) work, I appreciate it! Won't take up any more of your time.

dnolen17:09:40

what I would recommend is toy around w/ code splitting separate from what you are doing w/ NPM

Lone Ranger17:09:22

will do. I'll do a write-up if I identify some good patterns, I think we could use some more info out there on the subject.

dnolen17:09:24

try what I said, i.e. minimal login ns w/ JS primitives only - and then another ns w/ core.async or something and you’ll get a sense of how it’s supposed to work

dnolen17:09:56

another possibility is tsickle I haven’t time to really use it - but I think this is in heavy use at Google to use random libs - it converts ES6/TypeScript to Closure w/ generated externs

dnolen17:09:13

then you could avoid trying to combine two bundler (JS thing, Closure)

Lone Ranger17:09:40

whoaaa, interesting (regarding tsickle).

dnolen17:09:22

yes, a bit short on time, but it’s on my list of things to assess - it could drop the need for multiple bundlers which is a real pain

dnolen17:09:29

https://github.com/angular/tsickle - super light on documentation on how to use it

Lone Ranger17:09:37

It also seems like there's a good possibility to use goog.events to "communicate" between modules without explicit ((resolve 'some-ns/bar)) , so there are certainly opportunities there

dnolen17:09:37

but I did succeed at converting random libs to Closure style

Lone Ranger17:09:01

I think the point where this is becoming a real pain is for things like reagent which require react and react-dom as external deps, so some form of bundling (whether advanced compilation or cljsjs/react) seem kind of necessary and the "best" way to handle it is unclear. But nothing that can't be solved with some roll-up-the-sleeves engineering effort.

Lone Ranger17:09:46

But despite that, cljs is still an island in the middle of the js insanity

dnolen17:09:18

I’ve been in React Native land for a few years where the code size thing is significantly less of a concern and the existing approach is satisfactory

dnolen17:09:47

fwiw, I think for people who really care about bundle size NPM modules are serious dead end - I would just do everything in Closure

Lone Ranger17:09:07

Yeah I was actually thinking about going to React/Typescript for the components, might make runtime dependency injections easier too.

dnolen17:09:35

a Google Closure incremental DOM thing in pure JS a la Transit would be nice

dnolen17:09:57

again not to discourage React stuff, but some times you just want more control over the bundle, and that could be a good starting point

Lone Ranger17:09:24

as an alternative to React? :thinking_face:

dnolen17:09:58

the thing is for “bottom” stuff writing in JS has some benefits, it can be easier to eliminate depending

dnolen17:09:20

the problem with writing ClojureScript is that implies the data structures which compose like 50% of the source of the standard library

dnolen17:09:18

but anyways, if there was a functional DOM thing in JS that you could use w/ ClojureScript that just got the job done - you could do fairly light weight development easily idiomatically

dnolen17:09:49

this is a separate thing then big / complex SPAs - where likely you’ve already chosen heavy solutions

dnolen17:09:10

but currently there’s not many super light weight things for idiomatic ClojureScript UI augmentation for web stuff

Lone Ranger17:09:14

interesting. I guess hypothetically you would have to split out cljs.core to allow for runtime selection of the underlying primitives, defaulting to the cljs data structures. I'm assuming only an edge case of users would even bother with that implementation unless it were made the default.

Lone Ranger17:09:16

like cljs-a-la-carte haha

Lone Ranger17:09:05

At least with cljs.core it would be possible -- I was pretty shocked when I found out there were no protocols for things like assoc and deref in clojure.core.

thheller18:09:52

they are both protocols in cljs.core and interfaces in clojure.core

Lone Ranger18:09:46

In the case of deref I stand corrected, but I'm not sure how you would polymorphically extend the behavior of assoc in clojure.core since it delegates to clojure.lang.RT/assoc?

thheller18:09:44

RT.assoc just casts to that interface

thheller18:09:33

(which IPersistentMap extends)

Lone Ranger19:09:08

ahh I see. Good eye. Much appreciated!

puchacz20:09:14

hi all, hi @thheller - I presume you are the author of shadow cljs 🙂

puchacz20:09:17

I am very new to clojure(script) ecosystem and I am trying to set up cider with shadow-cljs. I can get REPL all right, but the source file is not seen as the part of the same session.

skylize03:09:10

The folks in #cider will likely be able to help you.

thheller20:09:02

hey. indeed I am. I do not use cider, so I can't really answer any questions related to that

puchacz20:09:02

are you aware of any bug around it? the emacs buffer with gui.cljs has cider[not connected] at the bottom and I cannot M-. (go to source) or C-z (go to repl) from it

puchacz20:09:51

thanks, no worries

Lone Ranger21:09:08

Ok does anyone have any insight into how cljsjs works? As in, why is it if you provide your own, for instance, cljsjs.foo , I can (:require foo) at the top level namespace form in a cljs project?

Lone Ranger21:09:01

is that clojurescript compiler thing?

Lone Ranger21:09:38

It saved my a** monkey patching reagent

Ben Lieberman21:09:16

did the default output of npx create-cljs-app change recently? I just ran this for a different project last week and it did not include devcards or reagent iirc