Fork me on GitHub
#cljs-dev
<
2020-09-02
>
cfleming00:09:13

Following on from my earlier analyser problem, I’ve now updated my code to use some code cribbed from cljdoc, which does something very similar to what I was doing. It works fine for simple cases, but I can’t get it to work for a test case I’m working on. I’m trying to use a cljc file to define a macro which will generate some vars in cljs which Cursive can’t see via static analysis. However when I run the analyser over it, what I get back looks more like the clj side of things:

{:name stub-test,
 :publics ({:name normal-def, :type :var}
           {:name normal-defn, :arglists ([]), :type :var}
           {:name hidden-defmacro2, :arglists ([]), :doc "Doc", :type :macro}
           {:name gen-clj-stuff, :arglists ([]), :type :macro}
           {:name NormalProtocol, :type :protocol, :members ({:name normal-method, :arglists ([this]), :type :var})}
           {:name hidden-defmacro5, :arglists ([foo__2740__auto__]), :type :macro}
           {:name gen-stuff, :arglists ([]), :type :macro}
           {:name hidden-defmacro6, :arglists ([foo__2740__auto__] [foo__2740__auto__ bar__2741__auto__]), :type :macro}
           {:name normal-multimethod, :type :multimethod}
           {:name hidden-defmacro4, :arglists ([]), :doc "Moar doc", :type :macro}
           {:name normal-method, :arglists ([this]), :type :var}
           {:name hidden-defmacro3, :arglists ([]), :type :macro}
           {:name hidden-defmacro, :arglists ([]), :type :macro}
           {:name normal-macro, :arglists ([]), :type :macro})}
To me, it looks like the analyser is taking the :clj branches in the reader conditionals since the hidden-defmacro* forms are found but e.g. the hidden-defn* ones are not. Am I missing something? The code I’m using to analyse is straightforward:
(defn- analyze-file [js-dependencies file]
  (let [state (cljs.env/default-compiler-env)
        faked-js-deps (fake-js-deps js-dependencies)]
    (swap! state update :js-dependency-index #(merge faked-js-deps %))
    (ana/no-warn
      ;; The 'with-core-cljs' wrapping function ensures the namespace 'cljs.core'
      ;; is available under the sub-call to 'analyze-file'.
      ;; 
      (comp/with-core-cljs state nil #(ana/analyze-file state file nil)))
    state))

(defn- read-file [ns-name resource js-dependencies]
  (let [state (analyze-file js-dependencies resource)]
    (if-let [ns (ana/find-ns state ns-name)]
      (-> ns
          (select-keys [:name :doc])
          (merge (-> ns-name meta (select-keys [:author :deprecated :added])))
          (utils/remove-empties)
          (assoc :publics (read-publics state ns-name))))))
*Edit:* Similarly, when analysing helix.dom (https://github.com/lilactown/helix/blob/master/src/helix/dom.cljc), the analyser sees the defmacro forms in the cljs, which I did not expect.

dnolen13:09:03

@cfleming a bit busy today will try take a look later if someone else doesn't get to it first

mfikes13:09:49

@cfleming I took a look yesterday and didn’t make any traction. But with your last edit, it is worth pointing out that defmacro creates a function var when compiled directly by the ClojureScript compiler (as opposed to when they are compiled by the Clojure compiler). This is how the self hosted compiler can compile and make use of macros.

mfikes13:09:23

IIRC that function has meta on it or something else that marks it as a macro implementation

dnolen14:09:12

I can't remember there's a probably a reason for this?

dnolen14:09:17

(do
  (def x (transient {}))
  (dotimes [n 32]
    (conj! x [n :foo]))
  (println (persistent! x)))

dnolen14:09:30

@alexmiller this only prints out 8 kv pairs

mfikes14:09:36

You discarding the result of conj! right; that's the problem

cfleming21:09:25

@mfikes Oh, interesting, thanks - I didn’t know that (in fact, my knowledge of cljs macros in general is pretty limited).

cfleming21:09:11

However I would have expected the defmacros not to have even been seen by cljs at all since they’re behind a reader conditional.

mfikes21:09:22

The fact that compiling a macro in ClojureScript makes a function var could be thought of as bit of an implementation detail.

cfleming23:09:08

@dnolen If you do get a chance to look at this at some point, I have a repro repo here: https://github.com/cursive-ide/cljs-analyzer-repro

dnolen23:09:35

@cfleming the backlog is really too disjointed for me to understand the issue here

cfleming23:09:06

@dnolen Ok, I’ll update the README on that repo

dnolen23:09:52

there's also just way too many thing in that repro

dnolen23:09:25

and it's not clear what the problem or goal is?

dnolen23:09:43

@cfleming before looking at anything let's just backtrack and talk about it for a second

dnolen23:09:55

so I understand the bigger picture

dnolen23:09:45

so you want to analyze a file and macros are appearing in the analysis when you don't expect it?

cfleming23:09:48

So this is related to the stub generation in Cursive, where I generate function stubs for vars which are dynamically generated at runtime. Cursive can’t see these via static analysis.

dnolen23:09:12

sure I get that, but if we can remove Cursive the discussion we can just focus on the fundamental issue

dnolen23:09:06

so is my previous statement correct?

dnolen23:09:20

trying to analyze some file (irrelevant how it was created, generated or not)

dnolen23:09:29

and it has macros and those are appearing in analysis?

cfleming23:09:26

The issue is that when analysing two example files (helix.dom and my test cljc ns), I’m not getting back the results that I would expect. When I analyse this file: https://github.com/cursive-ide/cljs-analyzer-repro/blob/master/src/stub_test.cljc, the ns publics contain macro forms which I did not expect (although mfikes explained why this might be happening, to support self-hosting). It also does not contain vars which I did expect, and I don’t understand why.

dnolen23:09:48

so let's break apart this first thing that you've said

dnolen23:09:02

what should and should not appear because I don't understand that at all from looking at the file

dnolen23:09:16

line 4 the comment says the follows defs shouldn't appear

cfleming23:09:17

I understand that the analysis might contain macro vars even for cljs, as mfikes explained. But as far as I can tell they should have been completely hidden from cljs by the reader conditionals.

dnolen23:09:31

but that I don't understand at all

dnolen23:09:36

you're still trying to get ahead of me 🙂

dnolen23:09:04

line 6 will definitely appear via analysis

cfleming23:09:06

Sorry, that is my mistake - that is a comment for what I would expect to see in Cursive which I left in for the repro. It’s not relevant here. I’ll remove that.

dnolen23:09:28

great - I just want to see something that make sense wrt. ClojureScript

dnolen23:09:43

then hopefully you can sort out the Cursive part once we have an understanding

cfleming23:09:48

Absolutely.

dnolen23:09:41

ok this is what I think will happen

dnolen23:09:59

4-8, only 6 will not appear