James, thank you very much for your work updating Duct. I am especially grateful for your attention to detail, which is evidenced by the quality and depth of the new documentation. 🙂 I recently read through all of it and completed the tutorials, step-by-step. I would like to customize the configuration of the Hikari connection pool which is added by the Duct SQL module. Assuming that I'm following along with the TODO list example, how would I go about doing that? Where is the seam in Integrant and the duct.edn file in which I could specify those customizations (e.g. :maximum-pool-size, :connection-timeout, etc.)?
Thanks for the positive feedback. Though it looks like I'll need to explain the 'seam' a little better in the documentation!
The :system key contains an Integrant configuration. It's as straightforward as that. Modules expand out into additional keys using the expand-key multimethod, but any configuration you add overrides that expansion.
So to give an example, suppose you have a :system configuration like:
{:duct.module/sql {}}
This expands out to:
{:duct.database.sql/hikaricp
{:jdbcUrl #ig/var jdbc-url, :logger #ig/refset :duct/logger}
:duct.migrator/ragtime
{:database #ig/ref :duct.database/sql,
:logger #ig/refset :duct/logger,
:migrations nil,
:strategy :rebase}
:duct.repl/refers
{db duct.repl.sql/db, sql duct.repl.sql/sql}}
(You can use duct --show to see exactly what the config looks like, or check the config map at the REPL)
This expansion is deep-merged into the configuration, with any existing keys taking priority. So you can write:
{:duct.module/sql {}
:duct.database.sql/hikaricp
{:connection-timeout 10000
:maximum-pool-size 32}}
The options you set will override those generated by the expansion of the module.
Does that make sense?Thank you for the quick reply, James. This makes perfect sense and is as simple and straightforward as I hoped it would be. 🙂
James, I have another question for you. I am attempting to incorporate Ring "route data validation" into the new Duct TODO list example. Specifically, I am attempting to reproduce the following example which is part of the reitit documentation: https://github.com/metosin/reitit/blob/master/doc/ring/route_data_validation.md I have added the explicit "spec via middleware" as the example suggests, but I cannot seem to trigger the error which is expected when I do not specify the :zone in the route. I suspect that I am trying to blend aspects of ring middleware, reitit, spec, and Duct that need to be combined in a specific manner with a specific syntax. I am sorry to ping you with such an open-ended question, but if you could provide some hints about how to approach route data validation and spec/malli integration in the duct.edn file, it would be much appreciated.
Hi, don't worry about pinging me - answering questions like this is what this channel is for. First, it's worth mentioning that the Duct Reitit router has inbuilt support for coercing and validating via Malli schema. Is that sufficient, or do you need more complex validation? When you say you've added the middleware map, how exactly are you doing it? Perhaps there's something that you've missed or gotten slightly wrong, so being able to see your configuration (or at least the relevant parts of it) would really help.
James, thank you for your reply. I will make another attempt to understand Reitit and its interaction with Malli schemas for coercion and validation next. I apologize for the open-ended nature of my question. Reading your reply, I think the root cause of my issue is not Duct, per se, but a lack of helpful, complete, easily accessible documentation about how to leverage Reitit and Malli. And I assure you, my inability to find it is not from any lack of trying. 😞
That's probably somewhere we can improve the docs then. In a nutshell though, you can add Malli schema to parameters via the :parameters key on a route to validate and coerce data from the request map.
So in general the format goes:
{:duct.module/web
{:routes
[[ROUTE {:parameters {PARAM-LOCATION MALLI-SCHEMA}}
...]]}}
Where ROUTE is your route, PARAM-LOCATION is either :query, :body, :form, :header or :path.
For example:
{:duct.module/web
{:routes
[["/example/:id"
{:parameters
{:path {:id [:int {:min 0}]}}}]]}}
This uses the Malli 'Lite' syntax. It could also be written:
{:duct.modules/web
{:routes
[["example/:id"
{:parameters
{:path [:map [:id [:int {:min 0}]]]}}]]}}
If you want more sophisticated validation, like for instance if a ID exists in your database, then you'll need to break out the middleware (or validate in your handler).
(This is all Reitit, btw. Duct isn't adding anything special here, aside from setting up Malli schema as the default, as its the only spec system that has an all-data option.)Thanks for the clarification and the example. I am reading through the Reitit Malli coercion page again -- https://cljdoc.org/d/metosin/reitit/0.9.2/doc/coercion/malli -- and the Malli GitHub README, and they align with what you've written above. Thanks!
I think it's probably worth me adding a section on coercion/validation to the docs, as the otherwise the documentation is spread across Malli, Reitit and (to a small extent) Duct itself.
And I see now that the ever-helpful Kari Marttila published a useful blog post on this also -- https://www.karimarttila.fi/clojure/2020/12/07/malli-clojure-library.html Thanks again!
BTW, James, I just did some testing to see if I could induce data validation errors by violating the Malli schema, and it worked as expected. 🙂