Fork me on GitHub

Random cljdoc dev team code observation: we named the user config file cljdoc.edn and the API analysis results files cljdoc.edn. This can make things confusing when rooting through the code and figuring out which is which.


Yup 😅 I think the analysis one was the first one to exist and then for the “user facing api” in people’s repos any other name than cljdoc.edn felt wrong


Yeah, makes sense!


Huh, I don’t think we do any validation on the user’s cljdoc.edn. Well… other than reading it as edn. Probably not a great idea to add in strict validation now for existing stuff? (Or maybe we could but only apply on new builds and not rebuilds). I will likely validate :cljdoc/languages and fail a build if there are unexpected values for that.


Oh yeah, that sounds like a great idea!


Spec or malli… hmmm… probably spec because we are already using spec…


But I might start with malli cause I like it and know it better….


Yeah, I’ve also been using malli lately, seems nice 🙂


Yeah, it does! But currently scratching my head on a malli schema that will work for user cljdoc.edn! Probably just my newbieness.


Yeah, also still finding my way around. At some point I tried to merge schemas and that seemed non trivial. In the end I just ended up appending to the [:map ] vector using into — it’s just data 🙂


Interesting! My goal is to validate cljdoc.edn but also provide sensible error messages to the library author. If that is not easy with malli or spec I just might end up rolling my own wee validate fn.


yeah that makes sense


The error messages malli provides look really great though


What is it you’re stuck with right now?


I've tried humanize support and found it pretty nice!


Not sure how to express our doc tree structure. Seems like a recursive schema? But not sure yet. Playing around....


Ah yeah, that is recursive. This looks interesting


Yeah, been looking at that and the recursive example in the readme but am not grokking yet!


It might not be that hard after all, after some playing around, here’s user cljdoc.edn malli schema so far:

 {:registry {"docs" [:catn
                     [:title string?]
                     [:opts [:map {:closed true}
                             [:file {:optional true} string?]]]
                     [:children [:* [:schema [:ref "docs"]]]]]}}

 [:map {:closed true}
  [:cljdoc.doc/tree {:optional true}
   [:vector {:min 1} [:+ "docs"]]]
  [:cljdoc/include-namespaces-from-dependencies {:optional true}
   [:vector {:min 1} symbol?]]
  [:cljdoc/languages {:optional true}
   [:set {:min 1} [:enum "clj" "cljs"]]]]]


(with the new languages support)


I’m finding that adding reasonable constraints helps (especially for recursive bit when generating samples, I otherwise suffer stack overflow exceptions), so now have:

 {:registry {"docs" [:catn
                     [:title [:string {:min 1 :error/path ["doc-title"]}]]
                     [:opts [:map {:closed true :error/path ["doc-opts"]}
                             [:file {:optional true} [:string {:min 1}]]]]
                     [:children [:repeat {:min 0 :max 20} [:schema [:ref "docs"]]]]]}}

 [:map {:closed true}
  [:cljdoc.doc/tree {:optional true}
   [:vector {:min 1} "docs"]]
  [:cljdoc/include-namespaces-from-dependencies {:optional true}
   [:vector {:min 1} [symbol? {:error/path ["namespace-name"]}]]]
  [:cljdoc/languages {:optional true}
   [:set {:min 1} [:enum {:error/path ["language"]} :clj :cljs]]]]]
Don’t know if I should be checking string content so am only checking for min length of 1 for now to help with typos. I can’t remember if there are any db limitations for max sizes of these strings when they get stored… I am remembering blobs… but will check. Could check cljdoc.edn file size itself to see if it seems reasonable-ish. Will continue to explore what error message reporting gives, but for our structure it seems to help to define some :error/paths. More tweaks to come as I learn…

👍 3

I’m also worked on some Malli stuff today, thanks for sharing this I think I might just steal some of it 🙂


Awesome! Don't forget I'm a newbie though (have a question over on #malli right now). I'll post progress here as I learn, which might just end up being me talking to myself, but it won't be the first time that has happened!


I want to do a twitch programming stream some time, I expect that to be a similar experience 😄


Ha! I always reassure myself that at least I am good company!


I won’t rename the physical filenames, but will likely clarify the code by renaming some vars/fns.


And not griping, just observing, could have been me who did this! simple_smile