Fork me on GitHub
#clojure
<
2023-09-22
>
Noah Bogart20:09:55

idiom check: we use Component and write protocols that are implemented alongside each component to act like interfaces/public API for the component. (the client data database service has access to the client data database and the ClientData protocol has the classic ORM queries defined for that database.) Our protocols are placed in foo.services.bar.proto namespaces and then all of the implementation logic is put into foo.services.bar.{core, data, etc} namespaces. Is this normal? Should we be defining our protocols commingled with the implementations?

Noah Bogart20:09:10

The current problems are 1) it feels weird to have a namespaces with a single protocol that has 3 methods on it (proliferation of namespaces and files), and 2) because we use :extend-via-metadata, it's hard to find the actual implementations (have to use clojure-lsp/cursive's "find usage" and then scroll the list until you find the with-meta call and then jump to the actual impl there)

ghadi20:09:16

totally normal to do that

👍 1
ghadi20:09:28

commingling has so many issues

Noah Bogart20:09:44

i guess I wrote commingling but I mean, "better to keep the protocol in its own file or better to put it in the file that has the base impls?"

Noah Bogart20:09:19

or should we even use protocols? it sometimes feels like all we're getting is a public description of the API which is helpful but also a lot of boilerplate and hurts repl-driven development

ghadi20:09:35

put the impls away from the protocol, unless they're only one impl

seancorfield21:09:40

@UEENNMX0T Do you actually have multiple implementations? I don't think I'd bother with protocols at all based on the description you provided... perhaps I'm not understanding your use case?

1
Noah Bogart21:09:45

Haha I also don’t fully understand our use case as the original author has long since left the company. Of the roughly 30 protocols we have, only 2-3 have multiple implementations. I agree with you that it feels like overkill, like we don’t need protocols in most instances

seancorfield21:09:24

Maybe they were an OOP type? 🙂

Noah Bogart22:09:09

Other devs in the company have said it’s nice to have all of the “intended as public api” functions grouped and documented together, which I appreciate. I feel like there’s gotta be a way to have both “nicely grouped set of domain functions” and “simplicity of plain functions”

seancorfield22:09:14

Polylith! :rolling_on_the_floor_laughing:

Noah Bogart23:09:43

It’s been in discussion! First we gotta move off leiningen lol

seancorfield23:09:08

Oh man, that's a BIG ball of yarn you've got to unwind... 🙂

Noah Bogart23:09:38

I’m glad to know we’re currently doing one version of best practices lol

vemv07:09:03

I'd keep writing a protocol for each component, for testability. It's a big point of using Component - you swap out components a la carte. While it's possible not to write a protocol until you feel you need it, in practice we programmers tend to be lazy, so when the time comes to write a test, we'll reach for with-redefs , which prevents parallel test running.

vemv07:09:20

> it's hard to find the actual implementations (have to use clojure-lsp/cursive's "find usage" and then scroll the list until you find the with-meta call and then jump to the actual impl there) You could always ask for a more streamlined UX. clojure-lsp might already have it: https://github.com/clj-kondo/clj-kondo/issues/1484

Hendrik20:09:06

I have a clojure app. I deploy it by building an uberjar, which I put into a docker container. Is there any benefit for using an uberjar in docker compared to just copying the source files and running clojure instead of java command? I am asking because I want to reduce docker build times

p-himik20:09:52

I wasn't able to see any tangible benefit so I stuck to running my apps from sources with clojure. One of the build steps also uses clojure -P with the right aliases to make sure that ~/.m2 is populated.

thumbnail20:09:28

Faster initial startup may be an advantage if you ship AOT compiled files. How much of a benefit that is depends on your requirements

thumbnail20:09:09

We use a jib to create a docker image which ships the source directly, so far were pretty with the results. Build time was a big factor for us

hiredman21:09:10

why are your docker build times so large?

hiredman21:09:01

like, an uberjar is a zip file(usually with compression turned off) of the contents, so why would copying the contents individually result in a faster docker build time?

hiredman21:09:14

I know timestamps in the jar can do weird things

hiredman21:09:20

I guess maybe if copying in the jar isn't the last thing you do, that might invalidate all the following layers, but then why isn't it the last thing you do?

hiredman21:09:56

I would do something like use clj -Spath to spit the classpath to a file as part of the build, and then use something like java -cp cat thefile clojure.main .... to start, that ensures even if some depednecy is using snapshots or something you are not trying to download dependencies in production

hiredman21:09:22

(that might already be the case with clj, I forget how it handles things like snapshots with its caching model, but I don't want to run build tools in production)

cjohansen21:09:23

A docker file is already an archive, no need to put uberjars inside them. Compile sources, copy classes and libraries to the docker image and run with java. This also allows you to better utilize layer caches in docker

cjohansen21:09:21

I've used badigeon to do this in the past. I think maybe tools build can do it now as well.

hiredman21:09:07

it is complicated because clojure libraries are distributed as source and clojure compilation is non-deterministic

hiredman21:09:21

and compilation is transitive

hiredman21:09:14

so if you are aot compiling a large clojure project, even with small changes between builds you still end up with a lot of classfiles that don't hash the same

practicalli-johnny21:09:17

Copying a pre-built uberjar is arguably the simplest approach if the deployed environment already has Java installed (ideally a consistent version with development environment) and access is provided to copy the uberjar file (without security concerns To manage the JVM enrollment and any other requirements to run a Clojure app, then a multi-stage Dockerfile is a relatively straightforward approach Building a Clojure uberjar can be very quick in docker when making use of caches layers, especially when dependencies are cached https://practical.li/blog/posts/build-and-run-clojure-with-multistage-dockerfile/ It depends how much access is available (and control desirable) to the deployment environment

1
hiredman21:09:14

if you are not aot compiling (which is the best, I encourage you not to) then you are just hashing source files and things will cache much better

seancorfield21:09:34

The only reason we are AOT-compiling at work is faster app startup. If we switched to a Docker-based build and did blue-green deployment (like we do for the frontend app), then startup time would be a non-issue and we'd switch back to a source-based approach, I expect.

Hendrik06:09:07

Thanks for all you replies. I was already using a multi stage build. In my case startup times are neglectable. So uberjar and docker is unnecessary. I already suspected this. So thanks for your confirmation.

pesterhazy09:09:55

You don't want to download new dependencies at startup. I'd argue the simplest way to make sure dependencies are baked into the image is an uberjar. Also this does the compilation early, at build time rather than in prod, so errors are caught more quickly

p-himik09:09:07

> You don't want to download new dependencies at startup. That's what clj -P is for. > Also this does the compilation early Compilation and uberjaring are orthogonal. You can have one without the other.

pesterhazy09:09:43

Do you also include your .m2 folder in the docker image?

p-himik09:09:50

Not "mine" but the one that was created in that image with that clj -P command. It's not that different from creating an uberjar.