sci

Sam Ritchie 2022-11-11T15:30:09.992159Z

has anyone seen this before?

[:clerk] Build failure:
------ ERROR -------------------------------------------------------------------
 File: jar:file:/Users/sritchie/.m2/repository/org/babashka/sci/0.4.33/sci-0.4.33.jar!/sci/impl/namespaces.cljc:1477:20
--------------------------------------------------------------------------------
1474 |    'unchecked-subtract (copy-core-var unchecked-subtract)
1475 |    'unchecked-negate-int (copy-core-var unchecked-negate-int)
1476 |    'unchecked-inc (copy-core-var unchecked-inc)
1477 |    'unchecked-char (copy-core-var unchecked-char)
--------------------------^-----------------------------------------------------
Encountered error when macroexpanding sci.impl.namespaces/copy-var.
NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object, Object)" because the return value of "clojure.lang.Var.getRawRoot()" is null
	sci.impl.namespaces/copy-var (namespaces.cljc:91)
	sci.impl.namespaces/copy-var (namespaces.cljc:73)
	clojure.core/apply (core.clj:671)
	clojure.core/apply (core.clj:662)
I am getting this when I try and upgrade clerk, and sci => 0.4.33 from 0.3.5.

Sam Ritchie 2022-11-11T15:30:49.040349Z

@borkdude I can push a repro of course. very odd since this is the 4th project I’m upgrading… obviously I’ll need to find the difference here

borkdude 2022-11-11T15:32:00.215129Z

check your clojurescript version

Sam Ritchie 2022-11-11T15:33:21.975529Z

org.clojure/clojurescript {:mvn/version "1.11.4"}

Sam Ritchie 2022-11-11T15:33:27.975539Z

I’m making sure it’s getting picked up…

borkdude 2022-11-11T15:33:41.406349Z

and your shadow-cljs version?

borkdude 2022-11-11T15:33:46.345369Z

also removing .shadow-cljs might help

Sam Ritchie 2022-11-11T15:33:49.718729Z

thheller/shadow-cljs {:mvn/version "2.17.4"}

borkdude 2022-11-11T15:33:59.249659Z

that's pretty old, maybe that's the issue

Sam Ritchie 2022-11-11T15:34:24.440909Z

same as on the other projects where it works

borkdude 2022-11-11T15:34:27.731969Z

well, not that old, so not sure

Sam Ritchie 2022-11-11T15:34:28.510859Z

i’m killing all caches

Sam Ritchie 2022-11-11T15:35:03.132719Z

ah, killing .cpcache got me to a new error

Sam Ritchie 2022-11-11T15:35:41.923409Z

maybe… trying again

Sam Ritchie 2022-11-11T15:36:29.718559Z

nope, same problem. let me try and shrink down the project

borkdude 2022-11-11T15:36:50.571199Z

feel free to push a repro

borkdude 2022-11-11T15:36:59.536709Z

smaller repro is better indeed

Sam Ritchie 2022-11-11T15:46:13.879499Z

------ ERROR -------------------------------------------------------------------
 File: /Users/sritchie/code/clj/repro/src/demo/viewers.cljs:33:10
--------------------------------------------------------------------------------
  30 |          #_#_'demo.mathbox (sci/copy-ns demo.mathbox (sci/create-ns 'demo.mathbox))
  31 |
  32 |          'demo.jsxgraph
  33 |          (sci/copy-ns demo.jsxgraph (sci/create-ns 'demo.jsxgraph))
----------------^---------------------------------------------------------------
Encountered error when macroexpanding sci.core/copy-ns.
NullPointerException: Cannot invoke "clojure.lang.IFn.invoke(Object)" because the return value of "clojure.lang.Var.getRawRoot()" is null
	sci.core/copy-ns (core.cljc:409)
	sci.core/copy-ns (core.cljc:362)
	clojure.core/apply (core.clj:671)

Sam Ritchie 2022-11-11T15:46:42.704819Z

the error tightened up when i re-checked out the project. compilation is fine until i use copy-ns on any namespace

Sam Ritchie 2022-11-11T15:46:52.321939Z

(working on a repro, feeding a baby too, sorry for livestream)

Sam Ritchie 2022-11-11T16:05:53.676979Z

it’s a flaky bug, doesn’t happen every time, trying to pin it down

Sam Ritchie 2022-11-11T18:06:36.015359Z

@borkdude okay I slimmed the repro WAY down: https://github.com/sritchie/sci-repro

Sam Ritchie 2022-11-12T15:04:08.282239Z

@borkdude there might be an issue, seeing this now:

Encountered error when macroexpanding sci.core/copy-ns.
IllegalStateException: Attempting to call unbound fn: #'sci.impl.cljs/cljs-ns-publics
	clojure.lang.Var$Unbound.throwArity (Var.java:45)
	sci.core/copy-ns (core.cljc:411)

borkdude 2022-11-12T15:05:36.007059Z

Can you update your repro? I'll clone it locally

Sam Ritchie 2022-11-12T15:05:55.982269Z

yup, will do, going to pin it down now

Sam Ritchie 2022-11-12T15:18:55.954799Z

yikes, deleting .shadow-cljs fixes it

borkdude 2022-11-12T15:22:14.103099Z

🤷

Sam Ritchie 2022-11-12T15:24:14.894239Z

I’m going to keep digging, it is back again

Sam Ritchie 2022-11-11T18:08:26.011849Z

no clerk, sicmutils etc, this is basically just SCI. It seems that if I require either sci.core or anything that transitively requires sci.core in my user namespace, then I get this big error later calling copy-ns in cljs code.

borkdude 2022-11-11T18:11:58.714359Z

ok, will try

Sam Ritchie 2022-11-11T18:12:13.504819Z

issue: https://github.com/babashka/sci/issues/832

Sam Ritchie 2022-11-11T18:12:18.069209Z

thanks @borkdude, no rush!

borkdude 2022-11-11T18:12:35.469789Z

This weekend I don't have to give a talk, hehe

Sam Ritchie 2022-11-11T18:14:07.595269Z

FAIR, but still I can work around this now that I know what’s causing it

Sam Ritchie 2022-11-11T18:14:36.577259Z

you are the most productive dude of all time, of course I’m just making it clear that I’m not trying to toss more on your extremely full plate!

borkdude 2022-11-11T18:17:12.259629Z

Thanks 😅

Sam Ritchie 2022-11-11T18:18:17.841709Z

@borkdude interesting, doesn’t happen with 0.3.5

Sam Ritchie 2022-11-11T18:18:25.001379Z

only with 0.4.33 and 0.5.34

borkdude 2022-11-11T18:29:30.403969Z

@sritchie09 You can work around this with:

(ns user
  (:require [cljs.analyzer.api]
            [sci.core]))

borkdude 2022-11-11T18:31:01.847219Z

The issue is that when you load sci.core without loading CLJS cljs.analyzer.api resolves to nil. But then later you are loading SCI again, but as a ClojureScript dependency and the logic that needs the cljs analyzer doesn't have the right var

Sam Ritchie 2022-11-11T18:36:47.007619Z

is there some way to use something like requiring-resolve here if it’s nil?

borkdude 2022-11-11T18:37:03.399949Z

not without screwing up graalvm builds

Sam Ritchie 2022-11-11T18:37:52.979129Z

what I was doing in user.clj is rebinding some dynamic variables that I use for rendering options, so that when my cljs loads and creates my SCI namespaces I have the proper values set inside of Clerk

borkdude 2022-11-11T18:38:40.857519Z

I'll take another look at this now

borkdude 2022-11-11T18:38:47.538989Z

perhaps there is something to improve, perhaps not

Sam Ritchie 2022-11-11T18:39:04.452619Z

it’s odd, presumably cljs is on the classpath at this point. but I am probably misunderstanding the load order

borkdude 2022-11-11T18:39:32.382739Z

So the issue is in cljs.cljc:

(macros/deftime
  #?(:clj (do (def cljs-ns-publics (resolve 'cljs.analyzer.api/ns-publics))
              (prn :yes cljs-ns-publics)))
  #_:clj-kondo/ignore
  (defmacro ^:private require-cljs-analyzer-api []
    (macros/? :clj
              ;; noop, macro executed from JVM Clojure, not within CLJS compiler
              nil
              :cljs #?(;; macro executed from JVM Clojure, within CLJS compiler
                       :clj
                       (do (require '[cljs.analyzer.api])
                           (def cljs-ns-publics (resolve 'cljs.analyzer.api/ns-publics)))
                       ;; self-hosted CLJS, no require supported but also not necessary
                       :cljs nil))))

borkdude 2022-11-11T18:39:48.913329Z

The first time you load this, then cljs-ns-publics becomes nil if you have not yet loaded clojurescript

borkdude 2022-11-11T18:40:58.423979Z

it doesn't require clojurescript since that's an optional dependency. I guess I could use requiring-resolve there (but this requires clojure 1.10) but the downside there is that maybe you do have CLJS on the classpath, but you're not really using it, so then it loads it for nothing

Sam Ritchie 2022-11-11T18:42:21.879449Z

I would argue that the fix for the user there is easy, to exclude cljs

Sam Ritchie 2022-11-11T18:42:25.194689Z

But yeah I hear you

Sam Ritchie 2022-11-11T18:51:09.757899Z

Side effects making life complicated, as usual!

borkdude 2022-11-11T18:52:51.660159Z

I'm still thinking about an improvement

👍 1
borkdude 2022-11-11T19:05:53.440589Z

I think I found a solution

borkdude 2022-11-11T19:08:41.232149Z

for JVM Clojure... now satisfying self-hosted...

borkdude 2022-11-11T19:32:01.609599Z

always self-hosted :(

borkdude 2022-11-11T19:54:26.158519Z

ok fix coming up ...

🎉 1
borkdude 2022-11-11T20:11:27.860059Z

I can do a proper release on monday, so it can be bumped in clerk as well

borkdude 2022-11-11T20:11:32.973219Z

(cc @mkvlr)

Sam Ritchie 2022-11-11T20:25:47.429319Z

Excellent trying in 10-15m

Sam Ritchie 2022-11-11T20:37:29.476289Z

@borkdude works great!!

🎉 1
Sam Ritchie 2022-11-11T20:38:39.275949Z

amazing, thank you @borkdude

mkvlr 2022-11-11T20:47:32.861659Z

@borkdude thank you, already on a git sha for sci so I’ll just bump it, no need for a release

👍 1
borkdude 2022-11-14T19:44:05.878439Z

@sritchie09 I reverted the fix I did to solve the problem since it caused more issues than it solved. Here is a workaround for the user.clj situation https://github.com/babashka/sci#shadow-cljs--userclj

Sam Ritchie 2022-11-14T20:09:47.116369Z

thank you!

👍 1
2022-11-11T06:27:51.561959Z

@borkdude I made this tiny branch to fix some type hint warnings. Not sure if you would go with ^clj or ^js or js/Promise there, I figured ^clj since it should be ok for these to be renamed https://github.com/babashka/sci/compare/master...mhuebert:sci:sci-await-type-hints?expand=1