This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2024-03-14
Channels
- # aleph (4)
- # announcements (10)
- # babashka (21)
- # beginners (67)
- # biff (7)
- # calva (4)
- # clojure (40)
- # clojure-europe (11)
- # clojure-gamedev (17)
- # clojure-losangeles (3)
- # clojure-madison (1)
- # clojure-nl (1)
- # clojure-norway (78)
- # clojure-uk (3)
- # clojurescript (83)
- # core-typed (18)
- # cursive (1)
- # datalevin (2)
- # datomic (2)
- # gratitude (2)
- # hyperfiddle (56)
- # introduce-yourself (1)
- # london-clojurians (1)
- # matcher-combinators (10)
- # membrane (161)
- # polylith (16)
- # portal (4)
- # reitit (4)
- # releases (3)
- # ring (2)
- # shadow-cljs (9)
- # squint (2)
- # timbre (10)
- # xtdb (14)
- # yamlscript (1)
At my company we have been building a relatively large website and recently have discovered typed Clojure. After typing a couple of our files it seems like we will get much more robust code since it is verifying the inputs and outputs we are defining in the type system along with any applications of typed functions. To us this gives our codebase much more robustness to be able to catch errors statically. We noticed the need for static checks in our frontend pages (we are using rum for writing our html in Clojure). Since we can't easily write automated tests for our frontend pages if there is an error in one of those pages like typos, or incorrect function application we would have to rely on manually testing the site to catch those errors at runtime. Without typed Clojure the only ways I can think of to catch these errors in an automated way are to write a test for each frontend page that checks all of the functionality of the page like buttons, forms, expected data on the page etc using web scraping or since we are using server side rendering we can render the rum pages to an html string and check the exact html. Both sound very expensive and not like a good idea generally. My question is how would Clojure devs handle this kind of thing without using typed Clojure? I have seen the other projects like Clojure Spec, Schema, and Malli, but as far as I can tell these all rely on runtime checks. It seems to me those would only cover everything if our test suite tests every possible code path including our frontend pages which doesn't seem feasible. That leaves us with manual testing. In any large website you cannot possibly rely on devs to manually test the frontend perfectly every time. You can't enforce that in anyway in CI obviously so to me that is flaky and we have had frontend bugs slip through due to this. We actually have a couple recurring bugs right now due to null pointer exceptions that we think could've been avoided with static type checks. We also practice Continuous Delivery so there is no code review or QA team that holds releases back only our CI/CD pipeline that runs tests and various other checks before deploying but it does not check all the functionality of frontend pages besides simple smoke tests making sure the site is accessible. We also use various other analyzers like clj-kondo, and eastwood. These will catch typos in functions and other issues but not typos in keywords to access data from maps for instance, not handling functions that may throw an exception, or incorrect function application. Also, we have been using lein-typed in CI is that not fully supported? Being able to check the types like you would run any other static analyzer is crucial for us to be able to run it in CI. lein typed is what we have used to type our first few files as well. Sorry for the wall of text and hopefully it makes sense but I have just been thinking about this a lot lately trying to figure out the best way to make our Clojure codebases more robust. Thanks for reading and any replies and Ambrose thanks for all your work on this project. TL;DR: My company has been testing out typed Clojure lately to make our codebases more robust and I have a few questions. • How would Clojure devs handle making sure untested/untestable files (like frontend pages written using the rum library you can't exactly write tests for this in a good way imo) don't have any common errors statically without using Typed Clojure? • Is lein typed officially supported and if not what should be used for type checking namespaces from the command line and in CI? • What is the best source of documentation for this project? • What are the future goals of this project and is it going to be maintained for the foreseeable future? • What is the best way to sponsor/support this project? (My company may be interested in this)
I think the answer to ensuring that frontend pages/forms "...don't have any common errors statically without using Typed Clojure" might be generative testing (e.g. test.check
). Malli schemas are pretty capable for this purpose (in part due to @U055XFK8V's work adding recursion to them). If you define your specification correctly you can generate arbitrarily nested HTML and then run your correctness checks over a wide variety of pages.
I have https://github.com/fabricate-site/fabricate/blob/1c91fdfb9900f095776514543c623f9454b241d8/test/site/fabricate/prototype/page_test.clj#L380-L384 to verify that a paragraph detection algorithm always returns valid HTML (Hiccup rather than Rum). The challenge is really figuring out how to specify the property you want to verify and the functions that generate "realistic" data for your tests.
At our org, and in the general js/web dev ecosystem, we use a combo of unit tests for non html logic, react-testing-library
(or dom-testing-library
if not react) for actual ui, and finally cypress
for e2e tests, all running in cd+ci.
Coming from typescript
, I sorely, dearly miss static typing, even if I love clojure overall more. We certainly have more trivial/preventable bugs than in typed projects I’ve worked on.
There are approaches to using guardrails
, or local-dev-only instrumented functions via spec
/`malli`/`schema`, which may help. I get the impression you might be using server side rendered templates, so your UI components then would not be functions per se like they are in react, so not sure how applicable these are to you.
At my company we would be hesitant to adopt typed-clojure (even tho I love the project and its ambition) due to the question of support.
But in your case, if your company is willing to sponsor this project, you might make it better for the whole community, let alone yourselves and Ambrose.
@U037TPXKBGS Cypress is a very powerful tool that I had not heard about previously. We already started exploring with it and adding it to our CI pipeline. I'm in the same boat about typing after using Rust for a few projects. Although, I love Clojure too the lack of static typing can let many trivial bugs creep though especially in untested UI code that could be caught at compile time. I have seen those runtime libraries for Clojure i'm just not that interested in runtime checks where we'd have to test every code path to make sure its correct and we are using server side rendered templates so i'm unsure how we can even use that for those. I think typed clojure can allow you to write more robust code with certain guarantees before your code even runs. I'm still not sure if we are going to fully adopt it, but we have been feeling the need for something like it in our Clojure codebases and exploring our options.
@UFTRLDZEW I don't have any experience with that form of testing, so sorry if these are obvious questions. Are you saying we can define spec validators for our hiccup and then run those checkers over our rum templates? Would that handle/verify function calls or would the templates have to just contain hiccup? I'm more interesting in making sure there are no type errors, incorrect function application, or possible exceptions that would cause unexpected/incorrect output from our templates.
If typed-clojure works for your case, and your project/team are large/complex enough, that would be a wonderful endorsement and proof of its maturity. In that case do let me know haha!
Otherwise, the point of something like guardrails
plus dev-only testing, is that you get something akin to compile time checking, and is probably the closest you'll get if the typed project doesn't meet your needs tbh
We are still in the exploration phase testing out which method will work best for us, so I will definitely checkout guardrails and spec. Those do seem to be more the traditional way in Clojure as well. I just personally like the idea of a static type checker opposed to runtime checks but perhaps in Clojure we don't have such a luxury. I have noticed that typed Clojure kind of forces you to stray away from some of the conciseness Clojure offers which is not ideal but seems like just a tradeoff for the guarantees of the type checker. I will definitely mention here what we end up deciding on for our projects.
Also, a more technical question say we have two HMaps defined as type Map1 and Map2 and then a third type that represents both HMaps merged with (t/Merge) as Map3. How come functions expecting a Map1 or Map2 cannot accept a Map3 when Map3 has all the same fields as a Map1 and Map2 since it is those two Map types merged together.
A union of maps is more specific then merging them. A function taking (U '{:a Int} '{:b Int})
cannot accept their merged type {:a Int :b Int}
. The left type is not allowed a :b
and the right not allowed an :a
.
Ok, that makes sense but what is the best way to handle passing in a {:a Int :b Int}
to a function that expects just a {:a Int}
. I assumed it would work kinda like inheritance in oop languages where if a Dog inherits from Animal you can pass a Dog into any function that accepts an Animal since Dog has everything an Animal would have. The function in question isn't accepting a union it just accepts {:a Int}
In my types I am using (t/HMap) is that the issue?
Like it has to match that map structure exactly no more or no less if its all :mandatory keys with :complete? true
Ok so perhaps all I have to do is change that to false in my 'Animal' type
Ok I think I misunderstood how complete? worked. That fixed my issue thank you. And when you have a chance I would appreciate if you could checkout the tl;dr at least of my other message. I understand if you are busy though no rush on anything of course. Thank you again for the help and the great project.
Thanks @U034ELFSHPS will do.