Fork me on GitHub
#beginners
<
2022-01-01
>
Aviv Kotek16:01:05

hey, what is the equivalent of Layered architecture (n-tiers architecture) on enterprise clojure projects? our code base (web app) grew over time from a simple functional-related-namespaces and now needs some different approach, mostly to increase maintainability, flexibility and be better organised. one idea I have experimented with and currently thinking on is the Polylith, are there any other options? "recommended" / common functional way to arrange code bases? thx : )

Ben Sless17:01:53

I think a layered architecture is the equivalent of going all-in on Component

Aviv Kotek17:01:13

We use component for managing state but not to structure / organize code, what is going all-in on component?

Drew Verlee18:01:16

The layered metaphor is an interesting one, keep in mind that physically that's not the case. The reason we like layers is because it gives us a sense of delineation, protection ... really of order from least to most important. The heart is what we tend to think of as our true business value, sometimes people say domain data, typically stored in databases, which themselves require communication and life-cycle manage (they start and stop/live and die) hence the need for "Component" to manage communication with this continuous sense of "them". Because in reality, those components are like a fast flickering light, so fast we abstract it as "continuous". Or rather, we want our users/customers to see it that way, so create phoenixes that burn out and rise again (one breed of this bird is called kubernetes pods). Meanwhile, on the edges of our systems, in the layered metaphor, are the so called "side effects". But wait, we had to do side effects to talk to our hearts (sorry domain model) as well. So the whole layered abstraction seems to fall apart. So the layering isn't about side effects, but trust. We trust ourselves thats where we have the most control, and trust is fundamentally about being able to predict behavior. The more you trust, the more harmony.

Drew Verlee18:01:05

@USPQF75AS The growth of a codebase isn't by itself something to fear as long as that growth is worth the cost. What specifically do you feel you aren't getting a good ROI on?

Ben Sless18:01:05

By going all in on component I mean you program your application to business logic, you build that layer on top of a behavior layer, and that on top of an implementation layer, where state will actually reside But why?

Aviv Kotek21:01:56

@U0DJ4T5U1 @UK0810AQ2 my need is find some "pattern" or "idea" to organize code better / re use it components / maintain it. as code grows, without some "technique" or "pattern", it becomes difficult to maintain it and mostly for new team members -- difficult to get in quickly. with DDD & OO -- the Layers architecture is some kind of common approach which helps on those cases.

Aviv Kotek21:01:18

@UK0810AQ2 but why -- you mean that's an overkill? do you have some example of that?

Drew Verlee22:01:15

I would argue patterns tend to emerge from shared purpose and goal. What is this code trying to do?

Aviv Kotek22:01:57

Have you looked on Polylith?

Drew Verlee22:01:06

I have seen the Polylith, i have no idea what problem it's trying to solve. That being said, people i trust... trust it. So i might just not have have had time to see the value. https://corfield.org/blog/2021/10/13/deps-edn-monorepo-7/

Ben Sless06:01:51

organizing Clojure code is not a solved problem 🙂

Ben Sless06:01:39

But if you think about designing for composition you can end up with plenty of architectures

Ben Sless06:01:36

Just a contrived example:

(defn wrap-async
  [f success fail]
  (fn [in]
    (f in
       (fn [result] (success result))
       (fn [error] (fail result)))))

(def request+metrics
  (wrap-async
   request
   (fn [result] (mark! http-success) result)
   (fn [error] (mark! http-fail) error)))

Aviv Kotek07:01:41

so right now @U0DJ4T5U1 I think Polylith tries to solve that, just with overhead of migrating from poly-repo to mono-repo

Drew Verlee16:01:14

I'll give a concert example from my codebase that i'm actively working on both to get past the current hurdle and deliver business value but also thinking on the meta problem so that we can maintain abstract it in order to maintain velocity and so focus on the important parts of our business. Our browser's react components currently update based on the query path but not the query params. There are some easy to reach for functions to parse ?a=1&b=hi into the ideal structure {:a 1 'hi'}. but they require the user to search around to find them. Worse off, the logic to serialize and deseralize the params would naturally be spread out, even though those two things need to be synced and so it pays to have them close together. If you think on this problem for a bit of time you end up with this idea from reitit, in which we combine coercion and validation: https://github.com/metosin/reitit/blob/master/doc/ring/coercion.md

(require '[reitit.coercion.schema])
(require '[schema.core :as s])

(def PositiveInt (s/constrained s/Int pos? 'PositiveInt))

(def plus-endpoint
  {:coercion reitit.coercion.schema/coercion
   :parameters {:query {:x s/Int}
                :body {:y s/Int}
                :path {:z s/Int}}
   :responses {200 {:body {:total PositiveInt}}}
   :handler (fn [{:keys [parameters]}]
              (let [total (+ (-> parameters :query :x)
                             (-> parameters :body :y)
                             (-> parameters :path :z))]
                {:status 200
                 :body {:total total}}))})
That's it then? were done? Not really, in our situation we need the query params to convey two different sets of relationships/filters. e.g I want all men who ware a blue shirt AND woman who ware green shirts. So gender=men&gender=woman&shirt=blue&shirt=green won't work here, because this fails to join the gender to the shirt preference. That's because query params are just a key value relationship. In fact, the grammar doesn't even have any room for nested relationships like {1 {:gender :men :shirt :blue}} which would address this problem in some way. This complexity is just the tip of the iceberg in terms of what some systems try to convey in the URL. Take this specification for healthcare https://cloud.google.com/healthcare-api/docs/how-tos/fhir-advanced-search So what should we do? The true path lies not in solving for every use case, but for solving the one we have, and doing so in a way that has as little friction as possible. Giving the system the smallest pieces it needs to build the application at hand. In this case, were using pathom and so our frontend communicates pathom queries to our backend. we could, for example, put pathom structures (joins and queries) in the url and just parse them as edn. But we also don't want to have breaking changes, so instead we just add a special query param key that is parsed as pathom. something like pathom=<valid pahtom>. I'm not promoting this as universal way of doing things, it's not a panacea, it seems to be working in our case and were close enough to monitor the trade offs it presents. In your case, i can't suggest anything meaningful other then to have a more refined problem statement before you make changes.

rmxm22:01:45

Funny question, how do you cover for "get" word being a function name, say you have namespace 'computer, surely it would be intuitive to have "get" function obtaining "computer". Just wondering how do you overcome this linguitically 🙂 or do you just place get at the bottom

practicalli-johnny10:01:08

Sounds like you are on a path that will end up rewriting a good part of Clojure.core rather than using it (it has over 600 relatively generically named functions) . I suggest spending time learning Clojure.core as it is full of these functions. The comment also suggests thinking in object oriented style, where objects are define and their data is accessed. Data (values) are held in Clojure data structures such as vectors and maps, which the Clojure.core/get and many other functions were designed to retrieve values If a function is written to wrap get or other functions, consider the purpose you are trying to achieve and name the function after that purpose (I.e. why are you getting the value)

rmxm12:01:15

To be honest, data models typically are expressed in semi "object-oriented style" (I wouldnt call it that tho). In RDBMS you typically persist them this way, you have a user, post, comments etc. How else would you model say a blog? Further this is often how restful apis are organized. Now I have one particular entity that wouldn't fit wrapping it into generic (get/update/delete/create-entity :entity opts) and decided to drill down providing its seperate "actions".

practicalli-johnny14:01:05

Whilst there are a great many examples of what I would see as OO design (typically hierarchical in nature) that is rarely what I have used for the last decade, even when working in an OO language. I prefer flat data structures in the main, unless I want to specifically categories data into a hierarchy and access it via the hierachty path (eg using get in hash-map [key key ,,,]) Even when working with a relational persistent storage I tend to use a flat structure, preferring more data in tables that joins across tables. This is how some of the biggest companies scale their data across vast numbers of customers I don't perceive data models as being specifically OO in nature. They are commonalities in the way that data is expressed in OO and functional approach, but that does not mean the same thinking is applied.

rmxm17:01:10

Just to add the "computer" in my exmple is more or less exactly what you describe. An entitty that is assembled from joins that combines all data related to this particular entity.

emccue22:01:33

can you give an example of what you mean?

rmxm22:01:22

you have namespace "computer", this namespace has function "get"... core func "get" gets overshadowed

emccue22:01:26

you are allowed to have a function named get - you just need to put (:refer-clojure :exclude [get]) in your namespace declaration to avoid warnings

emccue22:01:40

and then you refer to the core get as clojure.core/get within that file

Drew Verlee22:01:57

rmxm, you don't get computers you get values (in a key value relationship). That value could be describing a computer. The common linguistic nature betrays the operation the computer takes and the simpler path to modeling your program.

rmxm22:01:58

I am aware, I am just wondering how linguistically/semantically you deal with it 🙂

rmxm22:01:37

it would be typical to have entity tied to namespace and have "get" "delete/remove" etc. ?

pavlosmelissinos09:01:33

If the function does not cross scope boundaries I wouldn't use get (or get*) as a function name or any other verb. Elements of Clojure has a great chapter about naming: https://leanpub.com/elementsofclojure/read_sample

pavlosmelissinos09:01:08

> shadowing Clojure functions like get is safe and useful, but we should take care to specify this at the top of our namespace: > > (ns application.data.payload > (:refer-clojure :exclude [get])) > > This signals to our readers that get means something else in this namespace. > but: > If a function only transforms data, we should avoid verbs wherever possible. A function that calculates an MD5 hash, defined in our payload namespace, should be called md5. A function that returns the timestamp of the payload’s last modification can be called timestamp, or last-modified if there are other timestamps. A function that converts the payload to a Base64 encoding should be called —>base64. In a less narrow namespace, these functions should be named payload-md5 and payload—>base64

pavlosmelissinos09:01:17

I've found that following this advice forces you to think more about abstractions, which does make code quite readable. It puts you in the shoes of the reader/user in a sense.

emccue22:01:45

so

(ns some.computer
  (:refer-clojure :exclude [get]))

(defn get [o]
  (clojure.core/get o :thing))

rmxm22:01:00

sure, sure its not a technical question

emccue22:01:14

or

(ns some.computer
  (:require [clojure.core :as core])
  (:refer-clojure :exclude [get]))

(defn get [o]
  (core/get o :thing))

rmxm22:01:27

more of, do you use a different function name, or do you choose another word etc.

emccue22:01:30

I do this with update in some contexts

emccue22:01:58

no i just use get if i want to - but you have to be more specific about the use case for me to give a better name

emccue22:01:30

you can usually call it get-thing or thing

rmxm22:01:34

If you mention both get and update in this case I will assume this is ok practice

emccue22:01:47

its fine, depends on your domain

rmxm22:01:58

sure, sure, thanks 🙂

rmxm22:01:49

I could also get* or something like that for, "c/grud" stuff, get*, update*, delete*, create* seems, ok I think

emccue22:01:05

if the intended usage of the namespace is :require [a.b.thing :as thing], thing/func and the ns is small and focused (enough that you wont forget you shadowed a core fn) then don’t worry about it

rmxm23:01:49

thanks, going the get* route, this less bother down the road i think