Fork me on GitHub

I’ve run into a bug (quite some months back) with macros in self-host where syntax-quoted symbols aren’t always expanded. Found some time today… I came across this bit here where we build up an alias-map and bind that to reader/**alias-map*,* but then override `reader/resolve-symbol` to ana/resolve-symbol which doesn’t appear to have access to those aliases directly (though the analyzer is building up aliases in its own way, i presume)


the symbol that doesn’t expand is sub/wrap


now, weirdly, this only happens if there is both a macros.cljs and macros.cljc file. if I delete the .cljs file and use only .cljc with the self-require there, the symbol expands as expected.


maybe it’s because with the cljs file present, in the first “pass” there are no :require’s found by the analyzer


whereas with a self-requiring cljc file, it finds the sub-macros namespace on the first (non-macros) pass


so I guess that is my theory, that the read-loop in shadow’s compiler builds up an alias map which is not visible to the analyzer namespace, but is required for syntax-quote expansion. however sometimes it “works anyway” if the analyzer has already “seen” the same namespace in a prior pass, and the namespace-alias was also visible to that pass.


isn't this a known problem? macro namespaces can't have dependencies?


because it is undefined which namespace it should include?


does this include the .cljs file or the .cljc file? it is technically speaking in a :cljs branch of the .cljc file but loading in CLJ mode since it is a macro


If it's not behind a reader conditional then I'd expect it to load / be a available in all passes


This is a macro using another macro


Something related to this behaviour changed several months ago (or more) and I haven't been able to update’s dependencies since then because it breaks a lot of macros


Eg Reagent.core/with-let used to work but doesn't anymore because it uses an ns alias in a macro to delegate to a macro in another namespace


In this case I know that shadows compile loop is aware of the alias from logging at that point


The code is compiled and available, it's just the syntax quote expansion that doesn't work


If I replace the use of the alias with the fully qualified symbol it works


as far as I can tell you are in undefined territory and macros cannot be included this way


this might be fine in .clj files and just .cljc being the problem?


so if I diagnose this correctly the problem is that requires in CLJ files are not followed


(that includes requires in the CLJ path of the CLJC files)


so as far as the compiler is concerned nothing ever requires the sub-macros


if you instead however properly expose that require it all works out


ie. add

(ns bootstrap-test.macros
  (:require-macros bootstrap-test.macros)
  (:require [bootstrap-test.sub-macros]))


self-host is mind bending enough already, if you add CLJC to the mix it all becomes even worse


dunno if that actually fixes anything though. just a theoretical observation


But isn't it the case that these requires are being followed? Because it works if I change the symbol to be fully qualified. It is only the read phase where the syntax quote doesn't work


And it seems weird to be building up a map of ns-aliases but then overriding resolve-sym so that it can't see them


what makes you say that it can't see them?


ana/resolve-symbol is the only thing that can see them?


They are bound in the reader ns, not ana?Ana is only seeing stuff from previous passes? If I log the ns-aliases I see one for sub but that's not being seen or used by ana/resolve-symbol


When I'm back at my computer I can point to the line


ana/resolve-symbol sees things in the analyzer data. any ns data is added to that


Line 311 of


I can log there that we encounter a syntax quote sub/wrap symbol, sub is found in ns-aliases but resolve-symbol doesn't find it


bootstrap-test.macros does a self :require-macros but in a cljs file separate from the cljc file


but isn't this because it is self-host compiling with this reference context.

(ns bootstrap-test.macros
  (:require-macros bootstrap-test.macros))
so there is no sub alias?


shouldn't it work if you add it here?


I'm kinda lost with self-hosted and cljc to be honest


never makes sense to me


But shouldn't aliases work when read in the same pass?


This is also broken (ra alias not resolving)


Though I haven’t had time to verify it's exactly the same cause


IIRC I thought these issues were shadow specific, couldn't repro in the cljs test suite


But I checked that a few months ago


But that's what also makes me think it could be a quirk in


so it’s the same reason why i can’t compile reagent’s macros


the macro pass’s syntax-quote can’t read its ns-aliases


I mean it is absolutely possible this is a problem in shadow-cljs code somewhere


but to me this all looks like it is working as intended? this issue with using fully qualified names for macros has been there since forever (and even exists in non-self-hosted code)


something definitely changed with this several months ago though because this broke a bunch of things with Maria


that is possible since I added support for :as-alias a couple months ago. maybe that messed something up?


it was earlier than that


it was already broken in november 2020


maria is still on shadow-cljs 2.8.36


i git bisected it a long time ago but can’t find the result


i’m just trying to “unfreeze” maria now so i can work on it again


hmm yeah would help to know the version number it breaks


there were 2 bugs in ClojureScript that had to be fixed first


but those are all taken care of now


i know there are many cases where one needs to use fully qualified symbols in macros but this I think is different - these are symbols for macros, from macro namespaces that are required by a macro namespace. eg this works as expected


(but is broken for me)


of course one has always had to use fully qualified namespaces for stuff that is calling at runtime


but this makes me think, maybe im wrong about the main issue being syntax quote expansion


(i wouldn’t think so)


as far as I can tell the same problem exists in 2.8.36?


what do you mean supposed to extend macros?


sorry, gotta go. bbl


ach now i remember how hard it was to bisect this, i can’t even run the old version anymore because of java incompatibilities


[:failed-to-compare "^1.2.0" "1.2.2" #error {
 :cause "Cannot invoke \"Object.getClass()\" because \"target\" is null"
 [{:type java.lang.NullPointerException
   :message "Cannot invoke \"Object.getClass()\" because \"target\" is null"
   :at [clojure.lang.Reflector invokeInstanceMethod "" 97]}]


well, looks like reagent has other reasons why its not selfhost-compatible so that’s not a good goal. I will see again how many macros broke that we were actually using in maria


those errors you can ignore. they don't affect compilation


so as far as I can tell your repro also breaks in 2.8.36, so this issue isn't the one affecting maria I guess?


well I apologize for this rabbit trail. When I made those original repro’s I think I assumed that since shadow and cljs appeared to diverge here it was also a bug that was probably contributing. However, I’ve finally fixed all the other things that had broken in the meantime and all our curriculum runs fine without any changes needed to this macro-syntax-quote stuff.


I think I have still run into this issue elsewhere when trying to write selfhost-compatible macros. But I’ve mostly given up on that because it’s so hard.


I did verify that a :refer in a .cljs file makes an alias available during the macro pass of a .cljc file - as you described above

Jacob Emcken06:02:44

I found requiring Hero Icons like this also includes the Hero icons not referred:

(:require ["@heroicons/react/solid" :refer [CheckIcon ChatIcon]])
While the following does not:
(:require ["@heroicons/react/solid/CheckIcon" :as CheckIcon]
          ["@heroicons/react/solid/ChatIcon" :as ChatIcon])
Is there any way to require without repeating "@heroicons/react/solid" over and over? I tried with:
(:require ["@heroicons/react/solid" ["CheckIcon" :as CheckIcon]
                                    ["ChatIcon" :as ChatIcon]])
Which causes: Invalid namespace declaration


no, that is not supported

👍 1
Jacob Emcken06:02:45

Is it something that is impossible to do with how such things work or is it just something not important enough to get any attention?


changing :require to allow the nested stuff is not an option since that would break compatibility with standard CLJS


changing that :refer only actually imports what was referred is a substantial amount of work, which I simply don't have time for currently. it is also non-standard behavior and basically something webpack invented.

Jacob Emcken07:02:25

Thanks for the insight. I'm just curious 😄

Jacob Emcken07:02:28

JS tooling is something I've successfully managed to dodge for a very long time... exactly because examples like "non-standard behavior and basically something webpack invented".


yeah its annoying


Hey thheller - general question for you. Is it possible to read the metadata for clojurescript functions from build hooks (which are in clojure)? Or is it generally possible to parse the namespace and get publics ~ trying to figure out the best way to inspect a clojurescript file from the build and determine some general information about it


sure, all analyzer data is available


[:compiler-env :cljs.analyzer/namespaces 'your.ns :defs 'foo :meta] in the build state


this is amazing. you’re the best.

Quentin Le Guennec10:02:21

Hello, I'm getting an error in a release build (works in dev mode): $.clearTimeout is not a function the corresponding call is here:

goog.functions.debounce = function(f, interval, opt_scope) {
  'use strict';
  let timeout = 0;
  return /** @type {function(...?)} */ (function(var_args) {
    'use strict';;
    const args = arguments;
    timeout = {
      'use strict';
      f.apply(opt_scope, args);
    }, interval);


which :target is this and what is it actually running in?

Quentin Le Guennec11:02:57

I'm using npx shadow-cljs release main --debug for :target = :browser


and the output is running in an actual brower?

Quentin Le Guennec11:02:49

my deps.edn is:

{:paths ["src/main" "resources"]
 :deps  {org.clojure/clojure    {:mvn/version "1.10.3"}
         com.fulcrologic/fulcro {:mvn/version "3.5.9"}}

 :aliases {:dev {:extra-paths ["src/dev"]
                 :extra-deps  {org.clojure/clojurescript {:mvn/version "1.10.914"}
                               thheller/shadow-cljs      {:mvn/version "2.16.9"}
                               binaryage/devtools        {:mvn/version "1.0.4"}
                               cider/cider-nrepl         {:mvn/version "0.27.4"}}}}}

Quentin Le Guennec11:02:58

I pulled it from the fulcro guide


with that being the only JS on the page?

Quentin Le Guennec11:02:10

it's webpack-ed though


well ... that is the problem then

Quentin Le Guennec11:02:39

I'll try to release a version without webpack


the output of :browser is not supposed to be bundled by webpack again?


and why would you do that?

Quentin Le Guennec11:02:53

isn't it a good practive?


I don't know what you are trying to accomplish by doing so?


I mean does it have a reason you are doing it? is there something missing otherwise?

Quentin Le Guennec11:02:41

I thought it would reduce the loading times


no, it will in fact make it worse

Quentin Le Guennec11:02:22

I need the bundle at least for css files though, is that fine?


I mean you can run webpack separately. that is fine as long as it doesn't try to process the shadow-cljs output

Quentin Le Guennec11:02:21

yeah, it should be fine just for css then

Quentin Le Guennec11:02:54

I'd like to avoid having to manually add copy css files from node_modules/ to my public directory]


Hi there! I'm trying to import a npm dependency, and I'm getting the error IllegalArgumentException No matching field found: getAbsolutePath for class clojure.lang.PersistentHashMap. I know it's probably an error with the way I'm importing the lib, but is there something else I can do to debug this issue?


@mauricio.szabo which npm dependency would this be? you can also provide the full stacktrace and shadow-cljs version please. that would already telling me about 95% of what I need to know


It's the monaco-editor, the full require is ["monaco-editor/esm/vs/editor/editor.api" :as monaco]. This is the stacktrace on the shadow-cljs console:

IllegalArgumentException No matching field found: getAbsolutePath for class clojure.lang.PersistentHashMap
        clojure.lang.Reflector.getInstanceField (
        clojure.lang.Reflector.invokeNoArgInstanceMember (
        shadow.cljs.devtools.errors/fn--13467 (errors.clj:228)
        shadow.cljs.devtools.errors/fn--13467 (errors.clj:222)
        clojure.lang.MultiFn.invoke (
        shadow.cljs.devtools.errors/error-format (errors.clj:449)
        shadow.cljs.devtools.errors/error-format (errors.clj:440)
        shadow.cljs.devtools.errors/error-format (errors.clj:443)
        shadow.cljs.devtools.errors/error-format (errors.clj:440)
        shadow.cljs.devtools.server.worker.impl/build-failure/fn--14276 (impl.clj:133)
        shadow.cljs.devtools.server.worker.impl/build-failure (impl.clj:132)
        shadow.cljs.devtools.server.worker.impl/build-failure (impl.clj:128)


thanks. yeah thats a bug, fix incoming


my guess its trying to include .css files


fixed in 2.17.4


Is an import from a css file even valid? Why does Monaco is trying to import it? facepalm


@U05224H0W is there a way to ignore these CSS files when I compile my code with Shadow-CLJS?


:js-options {:ignore-asset-requires true}

👍 1

but usually packages that have these expect to be bundled by webpack


so other stuff might break too

Quentin Le Guennec21:02:30

what's the best way to import css files from node_modules into the shadow-cljs build in the browser? I tried webpack but it's not working


have you looked into postcss? I'm currently using tailwind with postcss, and importing tailiwind CSS modules directly from node_modules.

Quentin Le Guennec09:02:04

Thanks for your answer. Does it also import other assets (like fonts) like react does natively?


It's been a while since I touched anything front-end related, and I've never before worked with Tailwind. Also, it's been a long time since I used postcss. I'm sorry I can't give you a detailed answer. afaik tailwind has its own way of dealing with fonts. and postcss has plugins