This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-02-08
Channels
- # announcements (14)
- # babashka (12)
- # beginners (140)
- # calva (2)
- # cider (22)
- # clj-commons (14)
- # clj-kondo (49)
- # cljdoc (34)
- # clojure (92)
- # clojure-europe (41)
- # clojure-france (2)
- # clojure-new-zealand (2)
- # clojure-nl (2)
- # clojure-norway (60)
- # clojure-uk (17)
- # clojured (2)
- # clojurescript (7)
- # community-development (3)
- # conjure (2)
- # cryogen (13)
- # cursive (4)
- # data-oriented-programming (2)
- # datahike (5)
- # datomic (12)
- # defnpodcast (10)
- # events (2)
- # fulcro (20)
- # gratitude (3)
- # honeysql (4)
- # introduce-yourself (3)
- # jobs (10)
- # lsp (58)
- # malli (12)
- # missionary (19)
- # off-topic (8)
- # pathom (18)
- # podcasts-discuss (1)
- # polylith (41)
- # releases (1)
- # remote-jobs (3)
- # shadow-cljs (52)
- # spacemacs (1)
- # sql (37)
- # xtdb (19)
How do you folks use multiple bases in one project? All examples I’ve seen (poly, usermanager, realworld) define a -main
function in the only base. If I’d combine bases in a project, I would have to choose one, right?
Specifically I am playing around with designing a web service that exposes both a REST API and a HTML-based web UI on the same server. From the definition of base, both rest-api
and web-ui
should be bases (external entrypoints). However, I don’t want them to compete on who “owns” the web server.
So I am thinking of going with a base that includes “routes” (reitit, compojure, whatever) from components rest-api
and web-ui
, again ending up with a single base.
How would you design this with Polylith’s bases, components, and projects?
Hi! You should be able to have more than one base if you have e.g. a REST API by splitting it up in several bases. This could maybe be useful if you have a huge REST API where different teams are responsible for different parts of the API (this was not your use case, but wanted to mention it anyway). I have no experience of this, but it should work I think. A command line tool needs one single -main
function so in that case you need to use a single base.
If the rest-api
and web-ui
code is only used in this service, then I would put everything in that single base, and create two “top” namespaces, e.g. top-ns.my-base.rest-api
and top-ns.my-base.web-ui
. It feels natural to me to put the “baseish” code into bases, and not into component, but if you want to reuse any of the functionality, then creating components of them should be fine.
I wouldn't consider the API and UI to be different entry points, if the entry point really is the same HTTP server process listening on the same port.
I like the idea of splitting web-ui
and rest-api
into separate bricks. This makes it easy to manage library dependencies, e.g. hiccup
for web-ui
(or some templating engine) and json/transit middleware for rest-api
.
When splitting them into bases, I still have the challenge how/where to integrate them. E.g. I have a web-server
component that can spin up a server and could take routes from both rest-api
and web-ui
.
I could use a third base to have a -main
and glue things together, but then this one would need to depend on peer bases.
They would be different entrypoints if the architecture splits them into different servers, perhaps a port or a different domain. Or if there's an external load balancer / proxy that is the real entry point
Depends on how you define entrypoint I guess. For me, I am most interested in where commands or queries come in. These could be web requests, or a events consumed from a message queue, etc.
While I haven't used polylith much, it doesn't sound wrong to put the routes on components, and have the single base then combine what routes it wants on the actual external http interface (jetty or whatever)
Yeah, this is what I ended up doing.
Then it would be also trivial to have three bases in three projects: one combined monolith, and two that serve only one of the routes
Exactly, this was my thinking as well.
I created one base that defines -main
and uses integrant to manage the “system” state. I keep all the integrant knowledge in there, and use components’ interfaces to start/stop parts of the system.
Then I have components for rest-api
and web-ui
, that just implement the Web part of things, translating requests to calls to other components.
So far I like the approach, but I wanted to make sure I am not missing something as this does not seem to be 100% compliant to Polylith’s definition.
And to me the original question still stands: How would one combine multiple bases into one project?
The only way I can imagine is having some kind of spring-like magic, e.g. scanning the classpath and let bases register themselves as service providers. 😞
I would say no to spring-like magic. In my opinion, black box magic frameworks always create big problems in the long run. Polylith’s functional interface approach is both simple and easy to understand. Apart from that, the approach you have is the right approach. We are doing the same with some of our test/development services. In production, we have several services that are responsible of their own endpoints. In test/dev services, we combine those endpoints into one, to save on resources. We have multiple components for different sets of endpoints and the bases are just gluing them together. We keep 1-1 relationship between bases and projects, each project is associated with a unique base. However, bases can combine any endpoint by delegating to one or more components which define the endpoints. This gives us more flexibility if we want to move endpoints around services, or especially, on test/dev services.
Thanks, sounds reasonable. I am wondering, we keep telling folks in the docs that you can have projects with multiple bases — is that really the case? Maybe the “one project, one base” guidelines should make it in the docs?
There is no restriction about having more than one bases in one project. I find 1-1 relationship easier to understand personally, however, folks might have different takes on that. What do you think about changing the docs or maybe adding a FAQ about this topic @U1G0HH87L? The important thing for me is that no base should require another base, since bases only have external interfaces, not internal ones.
Yes, we can update the doc. And yes, we have the restriction today that a base (or a component) can only access interfaces, not bases. When I think about it, it should be possible today to add an interface
namespace to a base, and then access it, but that was never the idea (haven’t even tried it). If someone has that need, put the code in a component instead. I’m not sure it will work with the poly
tool either, this little “hack” (probably a bad idea even to mention it!).
I think it makes sense to not allow bricks depend on bases, this is what distinguishes them from components after all, right?
I also don’t see an immediate need to not allow multiple bases… but practically, I fail to see how this could be done.
Even if each base would spin up an independent server on different ports, we would still require a path from -main
to each base, right?
The main responsibility of a base
is to exposes a public API, and only delegate to components
via their interfaces
. In that sense, bases and components have different roles. If there is a use case that could be solved by allowing bases to access other bases via their interfaces (if we allow them to have interfaces) then I think that could maybe be okay. I’m not sure if this can be misused or not, but as long as we don’t introduce circular dependencies I think we should be fine. I think this is allowed already today by accident, but not fully supported by the poly
tool (e.g., the info
command). Letting a component depend on a base will sooner or later result in circular dependencies, so that will probably solve itself. It could still be a good idea to explicitly disallow this by adding a validation.
We should continue to allow multiple bases in a project as today (this is very important for the development
project). What I talked about was if we should allow bases to access other bases or not.
> The main responsibility of a `base` is to exposes a public API I do understand the concept and I agree that it makes sense for public APIs to only translate to other components. However, guidelines like these can also apply to components, e.g. I could decide to borrow from DDD and define “application-level” components that will delegate to “domain-level” components. Giving bases interfaces and just defining a base as something that has a public API seems a bit arbitrary limiting to me, as I would violate this rule with the setup described above. Another definition of base, something that actually “starts up” the system and interacts with its environment depending on the deployment, makes more sense to me. E.g. you need a different “deployment” for web services, command line tools, GUI apps, Lambda functions, etc. Having said that, let me be clear that I am not complaining and I am totally fine with the ways things are at the moment, as I could solve my problem in an elegant way. Thank you for your work on Polylith!
Okay, cool that you can solve your problems in a good way. It’s always interesting to hear what problems people have, and we are always open for improvements in the Polylith team!
This is a really interesting discussion. It hadn't occurred to me that a project could even have multiple bases (except development
since it has "all code") but I guess you could have a small -main
ns in a project's src
folder if your bases were web apps where the entry point was route mappings -- so the project would just, literally, compose those bases' entry points and hand off to a web server (as a component).
You'd have to be pretty disciplined to avoid base code creeping into your project code tho'...
I was thinking of this as well, but rejected the idea since in my case it would require moving “setup” code into the project that I’d rather avoid: Start a web server and pull in routes from two bases. Plus it requires the project to “reach into” the bases, which suggests that bases should have “interfaces” as well, further blurring the line between components and bases.
Yes, this is a tricky one. We could allow bases to access each other if we at the same time disallow components from accessing bases. I think this would open up the possibility to create small building blocks that could only be used by bases, similar to components. I agree that this will blur the line between components and bases but they will also signal that they live in the “land of bases”. I’m not sure if this is a good change or not, but it’s an interesting idea, worth thinking about.
My two cents: I would not change this. Bases should be bases, e.g. roots. Other pieces should not depend on them. If you want to expose an interface, use components.
As stated before, I see value in ignoring the advise that “public/external interfaces” should be bases. You could get more reuse that way, e.g. by writing a base that provides -main
and spins up web components using interfaces. This can be used for multiple microservice projects, each picking different web components that adhere to the same interface using the project’s dependencies.
Other examples for reusable “external interfaces”: Health endpoints, metric endpoints, message consumers, webhook integrations. To me, these should be components.
I just had a lunch walk and thought a bit more about this and I came to the same conclusion that it’s good as it is.
> would require moving “setup” code into the project that I’d rather avoid: Start a web server and pull in routes from two bases Not much more than:
(ns my.web.project
(:require [my.rest-api.web :as api]
[my.app-ui.web :as ui]
[my.web-server.interface :as server]))
(defn -main [& args]
(server/start (comp api/routes ui/routes)))
(possibly passing args
in through something to parse them and pass them to the server?)(but, yes, in general I would expect to see 1:1 bases:projects and no code in projects)
The multiple bases in a project I could see using, are for example wanting an artifact that has both a webserver and a CLI built in. Not that you could use the CLI to start the web base, but you would have to put the class in to the java jar call
But it would have to be pretty specific need to sometimes have the webserver only (starting automatically through its main), the other CLI tools only, and sometimes combined. If we wanted more to have a separate webserver, and CLI that always includes the webserver too, I would just make the web part a component, and let the CLI start it. That way we wouldn't need to remember the magical class paths, but could rely on the CLI's --help
It might make sense to deploy a web service JAR in production, but having the tools ready for debugging so we could just ssh in and run some commands easily, opposed to the option of running a REPL in production all the time and connecting to that. But this is just speculation
@U8ZQ1J1RR Given any Clojure app JAR, you can always run java -cp path/to/the.jar clojure.main
and get a REPL so nothing special is needed there (in terms of Polylith projects
or bases
).
(even when java -jar path/to/the.jar
starts a web server)
True, that is a better comparison to having a cli tool jar on the server. I got a bit mixed up there and was thinking about having a repl to the same process that is up and running the webserver, but of course the CLI tools couldn't be used to introspect that
I think sometimes it does make sense to have multiple bases in a project, that reference each other. For example a base serverA, a base serverB, and a base all-in-one that has a main
that basically calls (serverA/start)
and (serverB/start)
. This all feels very much as “base-like responsibility” and avoids (a) a lot of duplication in deps.edn if you create an additional project for this so that the bases don’t have to depend on each other, or (b) the need to create additional components that contain (what I consider) “base-like responsibilities”. So while this is not a big issue to me, the ability to do this would make sense to me.
Today the components are the composable building blocks. Bases are not composable, they only expose a public API. I think this separation has a big value in itself. The common case is to have one base per service, and I think it makes sense to only support that use case to keep things as simple as possible. If the community disagree, then we can have a discussion and consider a change.
I think it would also be good to expound upon what kinds of information will be handled at these various conceptual levels. That can help clarify the true roles of these various concepts.
In case anyone reads this thread and missed the high level documentation, https://polylith.gitbook.io/polylith/. There are detailed explanations of different concepts and the reasoning behind them.
Quick question. If i have some shared code that I want to leverage in tests for different components. Where would that live? just in another component?
Yes, put that code in a separate component. The https://github.com/polyfy/polylith/tree/master/components/test-helper component in the Polylith repo is a good example.
:thumbsup: