Fork me on GitHub
#announcements
<
2021-09-30
>
vemv08:09:37

nvd-clojure + lein-nvd - https://github.com/rm-hull/nvd-clojure/blob/1.7.0/CHANGELOG.md - security checker The Github repo is now named nvd-clojure to emphasize that it's compatible with Lein and deps.edn alike. For both tool chains, the recommended (but not enforced, yet) way of running it is described in https://github.com/rm-hull/nvd-clojure/tree/1.7.0#avoiding-classpath-interference. This cleaner approach has allowed us to get the issue count down to zero. Let us know how it works for you - enjoy!

❤️ 5
👍 3
🎉 8
dharrigan08:09:55

Great stuff! Been waiting for this! You may want to consider adjusting the project description for it reads National Vulnerability Database dependency-checker plugin for Leiningen, without also mentioning deps 🙂

🍻 1
👀 1
steveb8n12:09:54

Brilliant. I've been running this in CI but it wasn't loading the deps. I'll try this new version and will report back. Thanks!

✌️ 1
seancorfield20:09:04

Thank you for this @U45T93RA6 A note for the readme: in that section about avoiding classpath interference, it suggests running the tool outside the project folder but if you do that, it complains about directory paths that are on the classpath that it can't find. Maybe filter the classpath to just the .jar files?

vemv23:09:30

How does the complaining look like? Never seen that In case it helps, that passage is suggesting something like this:

#!/bin/bash
set -Eeuxo pipefail

NVD_DIR="$PWD/.circleci/nvd"
# calculate a production classpath within the main project:
CLASSPATH="$(clojure -Spath)"

DEPENDENCY_CHECK_COMMAND="clojure -M:nvd $NVD_DIR/nvd-config.json $CLASSPATH"

# cd into a different project defining its own, minimal deps.edn:
cd "$NVD_DIR" || exit 1

# Try a few times in face of flakiness from dependency-check-core:
eval "$DEPENDENCY_CHECK_COMMAND" || eval "$DEPENDENCY_CHECK_COMMAND" || eval "$DEPENDENCY_CHECK_COMMAND"

seancorfield23:09:25

(! 891)-> clojure -M:nvd "" "$(cd /Developer/workspace/wsmain/clojure && clojure -Sforce -Spath -A:dev:everything)"
Downloading: nvd-clojure/nvd-clojure/maven-metadata.xml from clojars
Execution error (AssertionError) at nvd.task.check/-main (check.clj:134).
Assert failed: The classpath variable should be a vector of simple strings denoting existing files. development/resources
(.exists f)
Because the classpath has relative paths on it and those are relative to the target project directory, not the directory where NVD is run.

seancorfield23:09:06

Per your README:

For extra isolation, it is recommended that you invoke nvd.task.check from outside your project - e.g. from an empty project, a git clone of this very repo, or from $HOME

👍 1
seancorfield23:09:53

☝️:skin-tone-2: That won't work because it checks .exists on the classpath entries and fails if they don't.

seancorfield23:09:50

(it's a minor issue -- running it in the project folder, with a separate classpath works great for me since I do not have anything in :deps in my project's deps.edn -- it's all in aliases)

vemv23:09:12

> That won't work because it checks `.exists` on the classpath entries and fails if they don't. That makes sense but I wonder how I've managed to never experience this. I have a CI setup where the inner project will receive entries such as test-resources which certainly don't exist within it Either way I'll fix it

vemv23:09:59

> (it's a minor issue -- running it in the project folder, with a separate classpath works great for me It's not necessarily the same. If your production classpath defines something that happens to overlap with these https://github.com/rm-hull/nvd-clojure/blob/370428eb6b947136d7c39aefd2161a642cf0801e/project.clj#L6-L29 , your project's versions may win over nvd's, creating opaque issues. I've certainly observed that under Lein. Not so sure about deps.edn but either way I wouldn't write a 'blank cheque' trusting the build tool's dep/alias resolution to do the right thing. Those are plausibly subject to change.

vemv00:10:09

ok so I see what was wrong with the check https://github.com/rm-hull/nvd-clojure/blob/370428eb6b947136d7c39aefd2161a642cf0801e/src/nvd/task/check.clj#L133-L140 - it only checks the first and last items from the classpath. This is not so deterministic. (The intent of that check was focused on format, not comprehensive file existence, so checking a subset made sense. clojure.spec does this for large collections)

seancorfield01:10:59

Ah, yeah, t.d.a puts paths at the front of the classpath so you're pretty much bound to get a directory that won't exist if you run the command in a different directory.

seancorfield01:10:58

Re: "blank cheque" above -- I can trust my context at work because the only thing pulled in by default is

:deps ; used by all projects
 {org.clojure/clojure {:mvn/version "1.11.0-alpha2"}}
So when I run clojure -M:nvd ... the only things on the classpath is your nvd-clojure via the :nvd alias in my dot-clojure file and Clojure 1.11 Alpha 2.

seancorfield01:10:31

Oh, and I added :nvd to my public dot-clojure file and documented it in the README to encourage more people to use it!

🙌 9
vemv01:10:04

ah, got it, so your production classpath is "opt-in" :) interesting.

seancorfield01:10:00

Polylith. Without :dev, there is nothing 🙂

👍 5
vemv01:10:41

btw, from your example -A:dev:everything appears to check more than needed (`dev` in particular). (Might have been simply a one-off experiment?) Checking vulns for dev tooling can plausibly frustrate people Either way I'll fix this thing

vemv01:10:53

Re polylith, it might have some value to check nvd for each module (whatever it's called, sorry). If you check only for the final app to be assembled, you'll be testing the final computed classpath. Maybe a module has a vuln that is later overriden.

seancorfield02:10:17

@U45T93RA6 You don't know anything about Polylith 🙂 :dev is the basic set of components (with all their transitive dependencies). It's essentially the "default" in non-Polylith (perhaps non-monorepo) projects.

seancorfield02:10:38

And :everything is simply our equivalent for the non-Polylith code in our repo.

seancorfield02:10:19

clojure -Spath -A:dev:everything is the production code (with one additional :extra-paths entry -- which doesn't impact NVD CVE checking).

seancorfield02:10:30

If I had added :test (Polylith's view of testing), :runner (our legacy equivalent), or :build, then yes, it would have "check[ed] more than needed" and would have included "dev tooling". I'm not stupid -- I know how our repo is setup 🙂

seancorfield02:10:22

(and it actually was quite interesting to see the delta of CVEs in production code vs the stuff that shows up in the dev/test/build tooling -- several were transitive dependencies of HtmlUnit, even with the most recent versions of everything pinned)

vemv02:10:38

:dev has a pretty well-established meaning in the vast majority of clj projects. Do you have an alias called :really-dev or something? (I'm assuming there's a "dev" and "test" separation e.g. no IDE-like tooling for running test suites)

seancorfield02:10:13

Yeah, Polylith's :dev isn't really like most other project's :dev aliases.

👍 4
seancorfield02:10:18

For us, there's no difference between the union of the individual projects and the :dev "project" view, except one one of the projects has to pin an older version of Jackson due to breaking changes they made that we haven't worked around yet.

👀 4
seancorfield02:10:33

All the other projects have a later version of Jackson with no CVEs.

seancorfield02:10:17

Polylith's way of running tests is based on the :test alias in each of the projects or the :test alias in the top-level (workspace) deps.edn file.

seancorfield02:10:23

And I use zero IDE tooling inside the project: plain socket REPLs, with some aliases adding stuff from my dot-clojure file (`revealportal:dev/repl`) so none of that is accessible to nvd-clojure.

seancorfield02:10:50

I guess another option with nvd-clojure would be to use :replace-deps so the :nvd alias will then ignore the project deps altogether. Or install nvd-clojure as a "tool" (in clojure -T terms).

👍 4
vemv02:10:51

Replace thingies make me a bit nervous (especially in Leiningen) but clojure -T seems inherently decoupled from a project. (A "constraint" for the readme is that I should keep it minimalistic and generic for both build tools, and not invite newcomers to do the wrong thing. I have pending related work for the readme in this direction) If you have the chance to contribute the right clojure -T incantation that PR would be most welcome :)

seancorfield02:10:06

Leiningen is not necessarily a good bar to judge anything against 🙂 For -T, you need a -X compatible entry point. Happy to take a look and send a PR.

😁 4
seancorfield06:10:49

OK, PR up for tool install/usage.

👍 1
borkdude10:09:50

SCI v0.2.7: a Clojure/Script interpreter, used in #babashka, #nbb and many other projects. This release focuses on: - Improvements regarding CLJS compatibility (interop, js literals, printing via *print-fn*) - Adding more dynamic vars related to printing - Exposing sci/stacktrace and sci/format-stacktrace for getting a stacktrace from a SCI exception. This allows you to build an error report like babashka and similar tools built with SCI. https://github.com/borkdude/sci/blob/master/CHANGELOG.md#v027

🚀 26
sci 6
sheepy 5
🎉 8
🙌 1
ericdallo16:09:59

clojure-lsp Released https://clojure-lsp.io/ 2021.09.30-15.28.01 with a lot of fixes and improvements, one of them is better feedback during server initialization loading 🚀 For more info check #lsp

🎉 33
👍 4
clojure-lsp 5
just.sultanov20:09:05

The first release of Secret Keeper has been released. It’s a Clojure(Script) library for keeping your secrets under control. https://github.com/sultanov-team/secret-keeper/tree/0.3.61

metal 10
🔒 14
4
🥷 1
metasoarous20:09:24

This is great! Thanks for the release! Any idea when this might be considered non-alpha?

just.sultanov20:09:03

The basic API is stable and no changes are expected. I want to add Secret’s schema, validator, and ring middleware. It doesn’t affect the current API

👍 1
metasoarous20:09:28

Cool; Thanks!

🙏 1