This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2023-06-05
Channels
- # announcements (11)
- # architecture (22)
- # babashka (33)
- # beginners (15)
- # biff (8)
- # calva (7)
- # clj-otel (1)
- # cljs-dev (3)
- # cljsrn (5)
- # clojure (76)
- # clojure-art (1)
- # clojure-europe (36)
- # clojure-hamburg (3)
- # clojure-nl (1)
- # clojure-norway (7)
- # clojure-poland (12)
- # clojure-spec (2)
- # clojure-uk (7)
- # clojurescript (9)
- # cursive (22)
- # data-science (6)
- # datomic (7)
- # fulcro (9)
- # hoplon (14)
- # instaparse (2)
- # jobs-discuss (14)
- # london-clojurians (1)
- # matrix (32)
- # music (1)
- # nbb (8)
- # off-topic (18)
- # pathom (29)
- # pedestal (6)
- # portal (34)
- # reagent (2)
- # reitit (4)
- # releases (1)
- # sci (10)
- # shadow-cljs (7)
- # tools-deps (4)
- # vim (6)
@djblue here’s how I install any npm deps from deps.cljs
entries on the classpath:
https://github.com/mentat-collective/Clerk-Utils/blob/main/src/mentat/clerk_utils/build/shadow.clj#L70-L90
using this code from shadow:
https://github.com/thheller/shadow-cljs/blob/master/src/main/shadow/cljs/devtools/server/npm_deps.clj
I think it would be nice to extract this out into some prep function in portal.api
… wdyt?
(ns emmy.portal.deps
"Borrowed from shadow-cljs and clerk-utils."
(:require [clojure.data.json :as json]
[clojure.edn :as edn]
[clojure.java.shell :refer [sh]]
[clojure.string]
[ :as io]))
(def ^:private windows?
(clojure.string/starts-with?
(System/getProperty "os.name")
"Windows"))
(def npm-cmd
"System-specific NPM command, tuned for Windows or non-Windows."
(if windows? "npm.cmd" "npm"))
(defn dep->str [dep-id]
(cond (keyword? dep-id)
;; :some/foo? :react :@some/scoped?
(subs (str dep-id) 1)
(symbol? dep-id)
(str dep-id)
(string? dep-id)
dep-id
:else
(throw (ex-info (format "invalid dependency id %s" dep-id) {}))))
(defn get-deps-from-classpath []
(let [deps
(-> (Thread/currentThread)
(.getContextClassLoader)
(.getResources "deps.cljs")
(enumeration-seq)
(->> (map (fn [url]
(-> (slurp url)
(edn/read-string)
(select-keys [:npm-deps])
(assoc :url url))))
(into [])))]
(vec (for [{:keys [url npm-deps]} deps
[dep-id dep-version] npm-deps]
{:id (dep->str dep-id)
:version dep-version
:url url}))))
(defn resolve-conflicts [deps]
(vals
(reduce
(fn [acc {:keys [id] :as dep}]
(update acc id #(or % dep)))
{}
deps)))
(defn read-package-json [install-dir]
(let [package-json-file (io/file install-dir "package.json")]
(if-not (.exists package-json-file)
{}
(-> (slurp package-json-file)
(json/read-str)))))
(defn is-installed? [{:keys [id]} package-json]
(or (get-in package-json ["dependencies" id])
(get-in package-json ["devDependencies" id])
(get-in package-json ["peerDependencies" id])))
(defn install-deps [deps]
(when (seq deps)
(let [args (for [{:keys [id version]} deps]
(str id "@" version))
install-cmd ["npm" "install" "--no-save"]
full-cmd (into install-cmd args)
full-cmd (if windows?
(into ["cmd" "/C"] full-cmd)
full-cmd)]
(println (str "running: " (clojure.string/join " " full-cmd)))
(println
(:out
(apply sh full-cmd))))))
(defn install-npm-deps!
"This command:
- Triggers an `npm install`
- Installs any dependency referenced by a `deps.cljs` file on the classpath
- Installs these dependencies into the calling project's `package.json` file.
Unlike `shadow-cljs`'s native `npm-deps/main`, this command also
installs [[shadow-npm-dep]], negating any need to tell the user to have
versions match or to remember to do this install themselves."
[]
(let [package-json (read-package-json ".")
deps (->> (get-deps-from-classpath)
(resolve-conflicts)
(remove #(is-installed? % package-json)))]
(when (seq package-json)
(println "Running npm install...")
(println
(:out
(sh npm-cmd "install"))))
(install-deps deps)))
@djblue this is a slimmed down version of what’s in shadow
I'll take a look later today. I think my main reservation is automatically updating a users package.json without asking first :thinking_face:
yeah for sure
I guess we can install and remove shadow’s --save
or explicitly flag NOT to save
the reason I ran npm install
and THEN did the shadow thing was that if someone had, say, mafs
in their package.json already, shadow marks it as “already installed” and skips the phase
not for me
maybe search both?
the two cases are:
1. I already have a dep installed, in which case I would expect portal to Just Work
2. I don’t know anything about npm or dependencies, and I want to run a (prepare!)
command that will install all dependencies (without me having to know what they are, what versions etc)
Okay, I need to think about it a little more but I definitely want to make this easier for everyone 👌
sg, let me tighten up this code
edited the above to make it use --no-save
and do nothing if there are no deps on the classpath
(ns emmy.portal.deps
"Functions for resolving npm dependencies declared in `deps.cljs` files on the
classpath.
This code is a simplified version of code in
thheller's [shadow-cljs]()
library and
the [clerk-utils]()
library."
(:require [clojure.data.json :as json]
[clojure.edn :as edn]
[clojure.java.shell :refer [sh]]
[clojure.string]
[ :as io]))
(def ^:private windows?
(clojure.string/starts-with?
(System/getProperty "os.name")
"Windows"))
(def npm-cmd
"System-specific NPM command, tuned for Windows or non-Windows."
(if windows? "npm.cmd" "npm"))
(defn dep->str
"Given a dependency name (specified as a key in the `:npm-deps` map of
`deps.cljs`), returns an npm-compatible (string) name for the dependency."
[dep-id]
(cond (keyword? dep-id)
;; :some/foo? :react :@some/scoped?
(subs (str dep-id) 1)
(symbol? dep-id)
(str dep-id)
(string? dep-id)
dep-id
:else
(throw
(ex-info (format "invalid dependency id %s" dep-id) {}))))
(defn get-deps-from-classpath
"Returns a sequence of maps of the form {:id <str> :version <str> :url
<url-of-this-deps.cljs>} for every dependency specified via the `:npm-deps`
key of each `deps.cljs` file on the classpath."
[]
(let [xform (map
(fn [url]
(-> (slurp url)
(edn/read-string)
(select-keys [:npm-deps])
(assoc :url url))))
files (-> (Thread/currentThread)
(.getContextClassLoader)
(.getResources "deps.cljs")
(enumeration-seq))
deps (into [] xform files)]
(for [{:keys [url npm-deps]} deps
[dep-id dep-version] npm-deps]
{:id (dep->str dep-id)
:version dep-version
:url url})))
(defn resolve-conflicts
"Given a sequence of dependencies, returns a sequence of dependencies distinct
by ID generated by picking the first one on the classpath.
Not ideal, but this is how shadow currently does it!"
[deps]
(vals
(reduce
(fn [acc {:keys [id] :as dep}]
(update acc id #(or % dep)))
{}
deps)))
(defn read-package-json
"Returns a map with the contents of package.json if it exists in
`install-dir` (default \".\"), {} otherwise."
([] (read-package-json "."))
([install-dir]
(let [package-json-file (io/file install-dir "package.json")]
(if-not (.exists package-json-file)
{}
(-> (slurp package-json-file)
(json/read-str))))))
(defn is-installed?
"Returns true if the supplied dependency is already present in some entry in the
parsed `package-json`, false otherwise."
[{:keys [id]} package-json]
(or (get-in package-json ["dependencies" id])
(get-in package-json ["devDependencies" id])
(get-in package-json ["peerDependencies" id])))
(defn sh-print
"Takes a vector of command pieces, prints the command and prints the results."
[command]
(println (str "running: " (clojure.string/join " " command)))
(println
(:out
(apply sh command))))
(defn ^:no-doc install-deps
"Given a sequence of dependencies, runs `npm install --no-save` for each
id/version combination."
[deps]
(when (seq deps)
(sh-print
(into [npm-cmd "install" "--no-save"]
(for [{:keys [id version]} deps]
(str id "@" version))))))
(defn npm-install!
"This command:
- Triggers an `npm install` (if ./package.json is present)
- Installs any dependency referenced by a `deps.cljs` file on the classpath
The `npm install` makes sure that if some dependency specified in a
`deps.cljs` is already present in `package.json` that it actually ends up in
`node_modules`."
[]
(let [package-json (read-package-json)
deps (->> (get-deps-from-classpath)
(resolve-conflicts)
(remove #(is-installed? % package-json)))]
(when (seq package-json)
(sh-print [npm-cmd "install"]))
(install-deps deps)))
@djblue here is a documented, tidied up version
all of the options code works great @djblue. Any chance we can get a version cut with that code added to SCI?
If you mean the reagent sci code, it should be released in 0.41.0, I think you might still be on 0.40.0?
I meant the options code
For use-options inside of sci
I added that in a local copy of portal and it works great
perfect, I merged the portal work yesterday, I’ve got free time this afternoon to get the rest documented and merged… then a big docs page…
@djblue I want to get calva set up and try out portal integration, and maybe try out github codespaces
final code: • https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal.clj • https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal/css.cljs • https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal/deps.clj • https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal/mafs.cljs • https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal/reagent.cljs • https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal/tex.cljs
I’d love a skim if you get some time @djblue, just to make sure I didn’t do anything silly
the tex viewer is the most obvious one, with support for emmy expressions stripped out of course
I didn’t make this change in time: https://github.com/mentat-collective/emmy-viewers/blob/main/src/emmy/portal.clj#L33-L36 the shift from explicit slurp to requires
For notebooks to work correctly, make sure to use the portal.nrepl/wrap-notebook
nrepl middleware. I'll take a look through the code when I get a chance 👍
okay sounds good