This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-04-25
Channels
- # announcements (4)
- # babashka (3)
- # beginners (79)
- # biff (4)
- # calva (17)
- # cider (18)
- # clj-kondo (21)
- # cljdoc (45)
- # cljs-dev (14)
- # cljsrn (9)
- # clojure (90)
- # clojure-europe (86)
- # clojure-italy (3)
- # clojure-nl (3)
- # clojure-portugal (1)
- # clojure-uk (9)
- # clojurescript (20)
- # code-reviews (23)
- # conjure (14)
- # cursive (12)
- # datascript (12)
- # emacs (5)
- # events (2)
- # fulcro (13)
- # gratitude (1)
- # holy-lambda (9)
- # lambdaisland (2)
- # malli (6)
- # nbb (1)
- # nextjournal (2)
- # nrepl (30)
- # off-topic (63)
- # pathom (1)
- # portal (24)
- # reagent (5)
- # reitit (13)
- # releases (2)
- # remote-jobs (1)
- # sci (90)
- # shadow-cljs (49)
- # spacemacs (5)
- # sql (13)
- # testing (20)
- # tools-build (17)
- # xtdb (27)
Hello everyone! 👋 shadow-cljs outputs source-maps in “sections” format but Sentry (error tracking SaaS) fails to parse it. The “sections” format looks like:
{
version : 3,
file: "app.js",
sections: [
{ offset: {line:0, column:0}, url: "url_for_part1.map" }
{ offset: {line:100, column:10}, map:
{
version : 3,
file: "section.js",
sources: ["foo.js", "bar.js"],
names: ["src", "maps", "are", "fun"],
mappings: "AAAA,E;;ABCDE;"
}
}
],
}
While a more traditional “section-less” format, that works with Sentry, looks like:
{
"version" : 3,
"file": "out.js",
"sourceRoot": "",
"sources": ["foo.js", "bar.js"],
"sourcesContent": [null, null],
"names": ["src", "maps", "are", "fun"],
"mappings": "A,AAAB;;ABCDE;"
}
(some info on source-maps formats I found https://sourcemaps.info/spec.html)
I went through the shadow-cljs code (https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/build/output.clj#L355-L375) but I didn’t find any option to output source-maps in “section-less” format so I tried to convert it in {:shadow.build/stage :flush}
build hook.
However, the naive merging sections back to the top-level map didn’t work - it went past initial sentry’s parsing error "Sourcemap was invalid or not parseable"
but then it hit errors like "Invalid location in sourcemap: (37101, 117)"
.
my naive (not working) attempt:
(defn flatten-source-map [path]
(let [source-map (-> (io/file path)
slurp
ch/decode)
sections (get source-map "sections")]
(->> sections
(reduce (fn [acc section]
(-> acc
(update-in ["names"] (fnil into []) (get-in section ["map" "names"]))
(update-in ["mappings"] #(if (not-empty %)
(str % ";" (get-in section ["map" "mappings"]))
(get-in section ["map" "mappings"])))
(update-in ["sources"] (fnil into []) (get-in section ["map" "sources"]))
(update-in ["sourcesContent"] (fnil into []) (get-in section ["map" "sourcesContent"]))))
(dissoc source-map "sections"))
ch/encode
(spit path))))
(defn shadow-cljs-after-build-hook
{:shadow.build/stage :flush}
[build-state]
(let [{:shadow.build/keys [config mode]
:shadow.build.modules/keys [module-order]} build-state
{:keys [output-dir]} config
filename (-> module-order
first
name
(str ".js.map"))
path (str output-dir "/" filename)]
(when (= :release mode)
(println "shadow-cljs-after-build-hook running (flatten-source-map " path ")")
(flatten-source-map path)))
build-state)
I was wondering whether I should try to facilitate some functions from https://github.com/clojure/clojurescript/blob/master/src/main/clojure/cljs/source_map.clj or try to tweak shadow-cljs config but that appears to be a deep hole to dig in, and I feel kinda out of my depth.
I would much appreciate any help/hints on what direction to pursue so shadow-cljs’s source-maps can work on Sentry. Thank you! 
@pt.roterski the source map mappings
encode in them line number information. so basically each section acts as an offset for that line information. as such without the offset and just concatenating the mappings
does not work. you could rewrite source maps by adjusting those mappings accordingly in theory but I have never attempted to do so
what issue exactly does sentry have with this? I mean sections aren't exactly new and as far as I can tell all browsers support them? even common JS libs like source-map
do?
Thanks for the swift response @thheller!
I understand that sections are widely supported (and they work for this code in my browser) but not by Sentry apparently and their error output is really poor (literally just those two strings "Sourcemap was invalid or not parseable"
and then "Invalid location in sourcemap: (37101, 117)"
) which makes debugging it difficult. I’ve also tried their validation tool https://docs.sentry.io/platforms/javascript/sourcemaps/validating/ but it also just fails with no descriptive stacktrace so my guess is that “sections” format is just not supported by them and I need to convert the source-maps to “section-less” format (because the same codebase’s source-maps used to work there before the transition from lein+figwheel to shadow-cljs build).
Thank you for the hint, I’ll try to improve my flatten-source-map
function based on that.
just write a function that takes an already generated .map file and spits out a new one
I guess that's the thing I already try to do in this hook, and the :flush
hook seemed just like a convenient place to do it but I guess it could be called from different place as well
if you get to a working solution and wish to share it we can talk about having shadow-cljs spit this out directly
hmm yeah that website for validating is useless. just threw the shadow-cljs UI map at it and just gave me
Errors 1
InvalidSourceMapFormatError
Invalid SourceMap format: "." is not in the SourceMap.
The previous lein+figwheel build had "section-less" source-map format and it worked, the shadow-cljs's source-maps use sections
format so I assumed that's the cause but of course I might be wrong about it and it could be something else :thinking_face:
the lein+figwheel likely also had much smaller source map due to all cljsjs packages being unmapped
I mean you can just trim down the source map by removing all except the last sections entry
the early sections are all for npm packages so maybe the error also goes away if you remove them?
I can try doing that
just checked https://github.com/getsentry/sourcemaps.io/blob/master/server/package.json
and it is indeed just using the source-map
npm package for the validation. source-map
definitely supports sections
just fine, so must be something else
https://sourcemaps.io/report/1650885603858_https%3A%2F%2Fcdn.ravenjs.com%2F3.17.0%2Fraven.min.js
yeah, it’s a bit of a hit-or-miss debugging game 🐛 🔨 more than anything
Hi. Is it possible to specify the autobuild
part in shadowcljs.edn file
(shadow.cljs.devtools.api/watch :app {:autobuild false})
actually I forgot. you can set :devtools {:autobuild false}
in your build config and that would be same as above
Hi. Is it possible to run shadow-cljs compilation using a shadow-cljs.edn
from another directory? I tried passing it to --config-merge
but the builds aren't recognized and I get an error like
{:build-id :modules, :known-builds #{}}
ExceptionInfo: no build with id: :modules
I guess the answer is no? can give a better answer if you provide more details about what you are trying to do specifically
Sure. I'm building a bb lib which will allow users to build custom versions of https://github.com/babashka/nbb. The users of this lib will be running shadow-cljs
from their own project but will want to pull in https://github.com/babashka/nbb/blob/main/shadow-cljs.edn as their initial config
I can workaround this by copying the shadow-cljs.edn before compilation and deleting after but it would be nice if there was a way to just point shadow to use the config from the lib directory
but you can manually call all the API build functions with a build config map you created yourself by any means
not sure what problem this is supposed to solve though. what is the point of a custom nbb version?
Ship versions of nbb with X number of features/libraries enabled on the classpath. There are a large number of cljs libraries that aren't sci compatible yet
Will be pretty neat when nbb has this ability as users will be able to put preconfigured features/libraries on the classpath like https://github.com/babashka/nbb/tree/main/features and build a version of nbb that just works with those libraries
yeah, sorry I still don't get why anyone wouldn't just do a regular build (eg. with shadow-cljs) and then use the generated output
which will no longer depend on the build tool used, or incur any cost regarding to that on startup and such
> yeah, sorry I still don't get why anyone wouldn't just do a regular build (eg. with shadow-cljs) and then use the generated output This probably comes down to the type of users you're thinking of. In my case I'm largely focused on supporting cljs beginners who will only write a couple small ad-hoc scripts and nothing else
A lot of their scripts will use datascript which has a much better cljs api than js one so just giving them a batteries-included approach to write their scripts saves them time
I mean you have a plan so go build it. I'm by no means the target audience so I don't have to get it 😉
Hehe. Happy to explain, especially to someone like you who's been so key in making cljs and nodejs work so smoothly together. We're a cljs editor, https://github.com/logseq/logseq and almost all our users are only familiar with nodejs. Distributing a nbb-like lib will allow them to not only run adhoc scripts with a larger part of the datascript API but also let them do useful things like run CI jobs, without having to think of compilation