Fork me on GitHub
#clojurescript
<
2023-07-24
>
joshcho07:07:32

I am building a docs tool for ClojureScript. I want to get the list of symbols in a namespace. I have it mostly working with ns-publics, but the (ns-publics 'clojure.core) is returning an empty map. I was under the impression that (ns-publics 'clojure.core) would give me a list of functions like ns-publics, etc.

thheller07:07:01

its cljs.core for CLJS

👍 2
joshcho08:07:06

Thanks. So in CLJ, (ns-publics 'clojure.core) includes defn, defmacro, but in CLJS (ns-publics 'cljs.core) does not include them (and others like ns-publics itself). Where are these symbols? Are they treated differently?

thheller08:07:36

what do you intend to do exactly? var based stuff is kinda tricky in CLJS since it doesn't really exist in runtime. it is all constructed from the analzyer data on the compiler side

thheller08:07:06

so macros live in the clojure side, which is not all reflected in that analyzer data

thheller08:07:03

if you are building a tool it might make sense to start with the compiler side analyzer data, rather than the slim layer on top provided by things like ns-publics which are all rather limited because of this separation

joshcho09:07:02

I have been trying something like (cljs.analyzer.api/ns-publics ns), but from what I can gather, you are suggesting I run this in CLJ? I want to pass the following things to Emacs: 1. list of all available namespaces (done through cider ns-list) 2. for each namespace, available symbols

thheller09:07:38

this is a CLJ function? so I'm a little confused now 😛

thheller09:07:58

where do you want it is the question? is the tool written in CLJS or CLJ? is the tool just running as part of the CLJS runtime? as in self-inspecting?

joshcho09:07:42

I want it to eventually support both, but currently cljs. I want it to be a docs-explorer for clj/s in emacs.

joshcho09:07:26

I am using nrepl for connecting to CLJS runtime I believe

thheller09:07:28

thats confusing again. so emacs is the UI, aka client loading the info somehow and displaying it?

thheller09:07:46

ok but this all happens over a REPL connection?

👍 2
joshcho09:07:15

(defun ch-eval (str namespace)
  (assert (memq major-mode '(clojurescript-mode
                             clojurec-mode
                             clojure-mode)))
  (let* ((res (nrepl-sync-request:eval
               str
               (cider-current-connection)
               namespace))
         (status (nrepl-dict-get res "status"))
         (res (cond ((member "eval-error" status)
                     (error "Eval error"))
                    (t
                     res)))
         (val (nrepl-dict-get res "value"))
         ;; (out (nrepl-dict-get res "out"))
         )
    (read (string-trim val))))
This is an emacs lisp function that evaluates some clj/s code. I use this to run (ns-publics ns).

joshcho09:07:06

Something like this:

(ch-eval (format
                "(->> (ns-publics '%s)
                (map (fn [[k v]]
                       (list (str k) (cond
                                 (:macro (meta @v)) 'macro
                                 (fn? @v) 'function
                                 :else 'variable)))))"
                namespace)
               ch-user-ns)

thheller09:07:09

ok, yeah that should be ok. there are limits though. if you look at other emacs/cider related things they'll often look at the analyzer data directly

thheller09:07:23

so cljs.env/*compiler* in particular

joshcho09:07:36

Ah, okay. So I shouldn't look in cljs.analyzer.api.

joshcho09:07:46

Which namespaces should I look at?

thheller09:07:58

well, it is tricky

thheller09:07:19

you need to realize that CLJS does not have reified vars at runtime, so everything var related is "fake"

thheller09:07:10

I'd suggesting looking at the cljs.analyzer.api ns directly

joshcho09:07:03

Right. I tried something like (cljs.analyzer.api/ns-publics 'cljs.core) , and getting errors.

Execution error (Error) at (<cljs repl>:1).
No protocol method IDeref.-deref defined for type null: 
:repl/exception!

thheller09:07:04

it is basically only meant for either usage from CLJ directly, or self-hosted CLJS

👍 2
thheller09:07:25

yes, exactly because of what I just said. it'll not work in the regular non-self-hosted CLJS build

joshcho09:07:54

hmm. tricky.

thheller09:07:41

you can use cljs.core/ns-publics instead, but that then has all the limitations macros have. i.e. no dynamic arguments

joshcho09:07:46

i guess one workaround is (at least for personal use) always connect full-stack, and just query clj

thheller09:07:28

but it should be fine for the snippet you linked above

joshcho09:07:44

i have been doing that, and there are two issues: 1. some definitions don't show up (I can live with this as these are fairly basic like defn) 2. it seems... unreliable? the program sometimes breaks. i can't trace the issue rn

thheller09:07:41

shouldn't be unreliable, but yes it may be incomplete with regards to macros

joshcho09:07:05

Yeah, it may be something I am doing in the wrapper/reading that is causing issues. I just need to get down and look at it.

joshcho09:07:49

Oh, I think I was restarting so often for debugging that I forgot to launch/refresh the CLJS runtime. That was causing issues.

vemv19:07:26

by any chance there's a nice regex/spec around for valid property identifiers? For instance .-42 is not a valid identifier, .-foo is one

hifumi12319:07:53

I dont think one exists, but you can make a regex spec like so

(def dot-property-re #"^\.-[a-zA-Z_$][0-9a-zA-Z_$]*$")

hifumi12319:07:17

(map (partial re-matches valid-re) 
     [".-foo" ".-42" ".-snake_case"])

;; => (".-foo" nil ".-snake_case")

hifumi12319:07:40

unfortunately this will fail against unicode property names… in javascript you can still use

let o = {定数: 2.71828}
const e = o.定数;

vemv19:07:13

Thanks! Yes, ideally I would get something very generic, it's for tooling so I need to keep an eye for unforeseen cases

hifumi12320:07:05

well, i just took a look at CLJS compiler and I do not see any validation done

(defn emit-dot
  [{:keys [target field method args env]}]
  (emit-wrap env
    (if field
      (emits target "." (munge field #{}))
      (emits target "." (munge method #{}) "("
        (comma-sep args)
        ")"))))

(defmethod emit* :host-field [ast] (emit-dot ast))
(defmethod emit* :host-call [ast] (emit-dot ast))
Taken from cljs.compiler lines 1422 - 1432

hifumi12320:07:21

so if you write (.-42 obj) the compiler will try to emit obj.42 and that results in an JS error

hifumi12320:07:46

so yeah… the best thing you can do IMO is extend the regex I provided to supporting unicode characters

hifumi12320:07:41

but the regex I wrote should at least mark everything cljs.core emits as valid… problems come when you run into the 0.1% of code actually using non-ASCII characters for object fields

vemv20:07:39

> well, i just took a look at CLJS compiler and I do not see any validation done nice dive, kudos! maybe for the time being I will banlist instead of allowlist. For example I have the practical problem of would-be symbols starting with a digit. It's easy to hardcode a few rules like that. If you're curious about the context, it's: https://github.com/clojure-emacs/clj-suitable/issues/39

hifumi12320:07:03

Huh, interesting. A few months ago I was playing around with some JavaScript code to compute all fields and methods of an object by looking at its prototype chain. The goal was to obtain autocomplete of javascript code from npm modules, but I never go that far

vemv20:07:46

we also have an issue for npm modules. maybe we don't need much special stuff for it to work, I'd just have to dig into the code

hifumi12320:07:18

I came up with something like this to compute fields

const fields = (obj) => {
  let result = new Set();
  let current = obj;

  do {
    Object.getOwnPropertyNames(current).map(key => result.add(key));
  } while ((current = Object.getPrototypeOf(current)));

  return [...result.keys()].filter(key => typeof obj[key] !== "function");
}
not sure how robust it is, but it works for toy examples I tried months ago

hifumi12320:07:56

e.g. given

let example = {
    foo: 1,
    bar: () => 1,
    baz: 1
};
if we run
console.log(fields(example));
then we get printed
[ "foo", "baz", "__proto__" ]

hifumi12320:07:11

the hard part is determining what object to inspect when the user simply supplies an alias… e.g. if they have

(ns example
  (:require ["react" :as react]))
and they type react/use in their buffer… then we want the autocomplete to compute the fields of module.exports since react is a CJS module… if we have an ESM module, things become complicated (e.g. if the ESM module uses default exports, then the imported default is the object we want to inspect)

hifumi12320:07:13

since shadow-cljs already has a way to convert all of these kinds of exports into an object in the global namespace (e.g. module$node_modules$react$dist$index… or something like that), then I think it would be easy to support autocomplete with just 2 functions: • computing fields and methods like I described above • using shadow-cljs api to resolve a required node module to the global object in one’s JS environment

vemv21:07:11

Thanks! Will study this carefully and link it to in our github issue

DeepReef1120:07:23

I have a function that is in javascript and take an object containing 2 function (show() and hide()). How to translate that to cljs? Here's what I got that doesn't seem to work:

(js/logseq.provideModel (clj->js
                           {
                            :show (fn []
                                    (js/console.log "show ui")
                                    (render-app)
                                    (js/logseq.showMainUI))
                            :hide (fn []
                                    (js/console.log "hide ui")
                                    (js/logseq.hideMainUI))
                                    }))
Here's a typescript example:
logseq.provideModel({
      show() {
        console.log("PROVIDE MODEL SHOW")
        renderApp()
        logseq.showMainUI()
      },
      hide() {
        console.log("PROVIDE MODEL HIDE")
        logseq.hideMainUI()
      },

p-himik21:07:08

That should work. A better way to write that would be to use #js {...} instead of (clj->js {...}) - works for static keys, does the processing at compile time and is shallow, so overall a better fit.

DeepReef1121:07:19

For some reason, it doesn't work with #js:

(#js {
                            show (fn []
                                    (js/console.log "show ui")
                                    (render-app)
                                    (js/logseq.showMainUI))
                            hide (fn []
                                    (js/console.log "hide ui")
                                    (js/logseq.hideMainUI))
                                    })

DeepReef1121:07:22

I get an error

p-himik21:07:36

Drop the parens around #js.

p-himik21:07:53

And add : in front of the fields, #js {:show ..., :hide ...}.

DeepReef1121:07:30

Awesome it works! thanks a lot

👍 2