This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2021-08-24
Channels
- # adventofcode (2)
- # anglican (1)
- # announcements (4)
- # aws (2)
- # babashka (28)
- # beginners (18)
- # brompton (3)
- # calva (22)
- # clj-kondo (2)
- # cljdoc (29)
- # clojure (41)
- # clojure-europe (28)
- # clojure-gamedev (14)
- # clojure-nl (2)
- # clojure-spec (2)
- # clojure-sweden (1)
- # clojure-uk (6)
- # clojurescript (53)
- # css (3)
- # cursive (6)
- # duct (3)
- # emacs (5)
- # fulcro (29)
- # introduce-yourself (1)
- # liberator (5)
- # lsp (1)
- # malli (11)
- # meander (4)
- # nbb (11)
- # off-topic (17)
- # pathom (2)
- # polylith (8)
- # practicalli (1)
- # react (6)
- # reagent (11)
- # releases (2)
- # rewrite-clj (11)
- # shadow-cljs (59)
- # tools-deps (21)
- # vim (11)
Random cljdoc dev team code observation: we named the user config file cljdoc.edn
and the API analysis results files cljdoc.edn
.
This can make things confusing when rooting through the code and figuring out which is which.
Yup 😅 I think the analysis one was the first one to exist and then for the “user facing api” in people’s repos any other name than cljdoc.edn
felt wrong
Huh, I don’t think we do any validation on the user’s cljdoc.edn
. Well… other than reading it as edn. Probably not a great idea to add in strict validation now for existing stuff? (Or maybe we could but only apply on new builds and not rebuilds).
I will likely validate :cljdoc/languages
and fail a build if there are unexpected values for that.
Oh yeah, that sounds like a great idea!
Yeah, I’ve also been using malli lately, seems nice 🙂
Yeah, it does! But currently scratching my head on a malli schema that will work for user cljdoc.edn! Probably just my newbieness.
Yeah, also still finding my way around. At some point I tried to merge schemas and that seemed non trivial. In the end I just ended up appending to the [:map ]
vector using into
— it’s just data 🙂
Interesting! My goal is to validate cljdoc.edn but also provide sensible error messages to the library author. If that is not easy with malli or spec I just might end up rolling my own wee validate fn.
yeah that makes sense
The error messages malli provides look really great though
What is it you’re stuck with right now?
Not sure how to express our doc tree structure. Seems like a recursive schema? But not sure yet. Playing around....
I should read this carefully too: https://quanttype.net/posts/2021-03-06-clojure-spec-and-untrusted-input.html
Ah yeah, that is recursive. This looks interesting https://github.com/metosin/malli/pull/209
Yeah, been looking at that and the recursive example in the readme but am not grokking yet!
It might not be that hard after all, after some playing around, here’s user cljdoc.edn malli schema so far:
[:schema
{:registry {"docs" [:catn
[:title string?]
[:opts [:map {:closed true}
[:file {:optional true} string?]]]
[:children [:* [:schema [:ref "docs"]]]]]}}
[:map {:closed true}
[:cljdoc.doc/tree {:optional true}
[:vector {:min 1} [:+ "docs"]]]
[:cljdoc/include-namespaces-from-dependencies {:optional true}
[:vector {:min 1} symbol?]]
[:cljdoc/languages {:optional true}
[:set {:min 1} [:enum "clj" "cljs"]]]]]
I’m finding that adding reasonable constraints helps (especially for recursive bit when generating samples, I otherwise suffer stack overflow exceptions), so now have:
[:schema
{:registry {"docs" [:catn
[:title [:string {:min 1 :error/path ["doc-title"]}]]
[:opts [:map {:closed true :error/path ["doc-opts"]}
[:file {:optional true} [:string {:min 1}]]]]
[:children [:repeat {:min 0 :max 20} [:schema [:ref "docs"]]]]]}}
[:map {:closed true}
[:cljdoc.doc/tree {:optional true}
[:vector {:min 1} "docs"]]
[:cljdoc/include-namespaces-from-dependencies {:optional true}
[:vector {:min 1} [symbol? {:error/path ["namespace-name"]}]]]
[:cljdoc/languages {:optional true}
[:set {:min 1} [:enum {:error/path ["language"]} :clj :cljs]]]]]
Don’t know if I should be checking string content so am only checking for min length of 1 for now to help with typos.
I can’t remember if there are any db limitations for max sizes of these strings when they get stored… I am remembering blobs… but will check.
Could check cljdoc.edn file size itself to see if it seems reasonable-ish.
Will continue to explore what error message reporting gives, but for our structure it seems to help to define some :error/path
s.
More tweaks to come as I learn…I’m also worked on some Malli stuff today, thanks for sharing this I think I might just steal some of it 🙂
Awesome! Don't forget I'm a newbie though (have a question over on #malli right now). I'll post progress here as I learn, which might just end up being me talking to myself, but it won't be the first time that has happened!
I want to do a twitch programming stream some time, I expect that to be a similar experience 😄