Fork me on GitHub
#polylith
<
2023-10-24
>
namenu04:10:20

How do you manage library dependencies? Almost every domain components we have depends on next.jdbc and/or honeysql, and keep their versions consistent feels meaningless effort. Like bricks don't specify their brick dependencies, I am thinking of moving component's maven dependencies up to base (or project). What can go wrong with this approach?

seancorfield05:10:29

As someone with over 150 bricks, I agree that it seems a bit repetitive. You probably could move them to the projects (including development) but poly libs isn't going to be able to help you identify what libraries are used by what bricks if you do that -- which may be fine for common dependencies: we specify the MySQL driver and log4j2 dependencies in our projects and I have to admit it has gotten tempting to move some of our very common libs to projects (since we only have 23 projects, including development) vs 66 components that depend on next.jdbc for example.

seancorfield05:10:01

One problem is if you are likely to be creating new projects -- you'd have to copy the block of common dependencies into each new project, whereas with dependencies in components that is handled automatically.

seancorfield05:10:56

Even with such common dependencies, we still have one project that doesn't do any database operations and so it doesn't depend on any of those 66 components and therefore doesn't need next.jdbc -- I don't have to figure that out tho', Polylith does that for me.

seancorfield05:10:17

I'll be interested to hear what the Polylith team say about this -- it's an interesting question.

1
furkan3ayraktar05:10:28

We have a component for some common external libraries, like next.jdbc, that wraps the library and exposes the functions we need. The library dependency is defined at that component. Then, we use that component instead of the library from other components.

namenu05:10:51

Yes, that is one way to solve this problem. But you can't wrap every core libraries like core.async and core.match. And you can't prohibit implicit use of next.jdbc from another bricks. Then why do we distinguish library/brick? One of monorepo's goals is to use the same 3rd party libraries as much as possible. we can archive the goal better by specifying them lesser.

seancorfield05:10:19

I don't think it's a "problem" per se. poly libs can tell you if any brick is referencing an outdated lib and can show you if you have multiple versions declared in different bricks. Changing a version globally is a pretty straightforward search'n'replace. The first time you reference a library in an impl file, add it to the brick's deps.edn and pretty much forget about it.

👍 2
seancorfield05:10:59

As I noted above, while we have 66 components that depend on next.jdbc, we have nearly 100 that don't.

john-shaffer05:10:07

antq helps with changing all the versions at once. I have a postgres component that depends on honeysql and next.jdbc, a file-util that depends on babashka.fs, aws-client that depends on aws-api, etc. There are still some annoyances like stuartsierra component, but I've been switching to donut.system which lets you define using only (require '[donut.system :as-alias ds]) so your components don't actually need it as a dep

seancorfield06:10:29

You don't need Component as a dep -- the protocols support metadata and you can provide the lifecycle that way (just to clarify that you can avoid that "annoyance").

john-shaffer06:10:44

oh, interesting

seancorfield06:10:00

next.jdbc has Component lifecycle support without depending on it at all 🙂

seancorfield06:10:28

See https://github.com/seancorfield/next-jdbc/blob/develop/src/next/jdbc/connection.clj#L292 if you're curious -- note that starting this component returns a function that still satisfies the stop lifecycle portion of the protocol, not a hash map 🙂

😮 1
tengstrand06:10:02

One idea is to add a sync command, that finds the newest version in use of each library, and that updates all the libraries in the deps.edn files, to use these library versions. This can for example be implemented by using the https://github.com/clj-commons/rewrite-clj library, as a way to keep the files as close as possible to the original versions of them. We have said that the tool shouldn't edit the code for you because we don't want to add magic, but this could maybe be an exception (and you don't have to use it) because it would be quite convenient for large projects. We probably also need to make it possible to configure exceptions which bricks that shouldn't be affected by the sync command. If the community thinks this is a good idea, I can create an issue.

seancorfield06:10:11

@U1G0HH87L The antq tool can already do that to a Polylith repo -- isn't poly already using antq for the libs :outdated command?

tengstrand06:10:18

Yes, but it doesn't update the deps.edn files for you.

tengstrand06:10:50

Aha, so antq has support for that already? Then that could be used of course.

seancorfield06:10:03

Yes, built-in to antq.

tengstrand06:10:30

Okay, didn't know that. What do you think of creating a sync command?

seancorfield06:10:43

I resisted letting tools update my dependencies for a long time, but antq can also update your GitHub workflows and so on 🙂

seancorfield06:10:28

I'm not sure sync is a good name for it. poly libs :update perhaps, to go with poly libs :outdated?

seancorfield06:10:40

Make it clear that it will update things.

seancorfield06:10:28

(for us, poly libs :outdated is a bit overkill in terms of output, as I recall, so we have our own build.clj task that runs clojure -T:antq outdated with the appropriate paths etc)

seancorfield06:10:28

But having poly libs :update built-in would be nice.

tengstrand06:10:10

> But having poly libs :update built-in would be nice. Yes, that sounds like the way to go. Another option is to have a sync command that can update more than one thing, e.g. add missing components to projects, and maybe more things in the future. Then you only have to run the command once, and we could pass in arguments like :libs to narrow down what we want to sync. But I like the poly libs :update idea more. I can create an issue if more people agree on that it's a good idea.

👍 2
1
namenu06:10:15

Sounds good. I'm more appeal to the idea of being able to remove unused components from a project than adding them.

seancorfield06:10:19

poly check already tells you that component deps are not needed (as well as missing ones).

tengstrand06:10:45

> poly check already tells you that component deps are not needed At least if you work from the master branch (`0.2.18-SNAPSHOT` ).

namenu06:10:05

oh, thanks for letting me know 🙂

👍 1
seancorfield06:10:42

I've been tracking master for so long (and sometimes branches too) that I forget how much work has been done since the last official release!! gratitude

👍 1
tengstrand06:10:37

After the 0.2.18 release, we will try to release more often.

tengstrand03:10:55

I created https://github.com/polyfy/polylith/issues/376. I'm not sure if there is a better name for the :keep-lib-versions key.

seancorfield04:10:29

Sounds good and I like that name, to be honest. I will say that in keeping with deps.edn itself, the lib names should be symbols not strings!

tengstrand04:10:28

I changed to symbols in the issue. I had an idea to use strings everywhere when I started with the poly tool, but I think it's time to reconsider that idea! There are a few more places where we use strings instead of symbols. We have https://github.com/polyfy/polylith/issues/315 that will migrate the project and brick configs into separate files. We could let this migration change some of the data from strings to symbols, when it makes sense.

❤️ 1
🔥 1
Stefan14:10:48

For those who are interested: I created a #joyride script to toggle between a poly interface and implementation file in VSCode. It does not work for all cases and is very basic, but it works for me. Let me know if you want to know more 🙂

👀 1
joyride 2
catjam 1
Stefan14:10:37

Basically going from implementation to interface works normally, but the other way around only when you use the convention of having an interface folder. So it will jump from .../components/foo/my/ns/interface/something.clj to .../components/foo/my/ns/something.clj . (Because how do you know where to go if the filename is interface.clj?)

pez14:10:42

Very cool! I ran into similar constraints when implementing the Go to/from Fiddle file in Calva. I want to know more. When are you adding this script to the examples on the Joyride repo? 😃

furkan3ayraktar15:10:21

It is nice and interesting that doing such a thing with Joyride is possible. Thanks for sharing 🙌 Would it be possible to implement something for VSCode/Calva so that when I CMD+Click a symbol, it skips the interface and jumps to the implementation if it’s a pass-through interface?

pez15:10:07

I think it should be possible. It may get to be a race for looking up the definition between Calva and the Joyride script. If someone gives it a try I’ll be eager to help figure it out.

pez15:10:05

The answer for “Is it possible with Joyride” has a general answer of “Yes”. A more defensive answer is “If an extension can do it, Joyride can do it”. (VS Code’s API is a bit less permissive than one would sometimes wish for.)

Stefan15:10:33

I got to go now, I’ll post more details Soon. (Thursday likely)

Stefan07:10:58

> Would it be possible to implement something for VSCode/Calva so that when I CMD+Click a symbol, it skips the interface and jumps to the implementation if it’s a pass-through interface? I would even like to take that one step futher: the symbol table should maybe hide the interfaces as well. I recently discovered "go to symbol in workspace" (CMD-T on mac), which is super convenient, but it shows lots of things "duplicate" by showing both the symbol in the interface and the one in the implementation. And something similar applies to searching for files: also there lots of "duplicates" show up, and the names are very long so it is not always easy or even possible to see which is the interface and which is the implementation.

pez07:10:46

Thanks for the PR!

pez07:10:42

The VS Code Search allows for filtering. If you find some setting that is good for Polylith, please share!

Stefan07:10:02

Oooohhhh nice 🙂 I didn't know about that. Added **/interface.clj and **/interface/*.clj for now. Thanks for that!!

🙏 1
pez07:10:24

That second pattern looks funny? Does it really work?

Stefan07:10:03

Oh sorry slack messed that up, fixed it.

metal 1
pez07:10:11

Are search filters, something to consider for the RealWorld project settings, @U2BDZ9JG3?

Stefan07:10:15

For others, this is the relevant setting:

"search.exclude": {
    "**/interface.clj": true,
    "**/interface/*.clj": true
  }

pez07:10:18

Enlighten me, please. Why wouldn’t you want interface hits when searching?

Stefan07:10:45

I'm not sure that I'm going to keep this, but most of the time I want the implementation and the vscode "search files by name" popup has limited width, so the part at the end that does or does not contain "interface" falls off and is not visible, so then I don't know which one I need to select.

🙏 1
Stefan07:10:05

It's probably also because we're using namespaces with eu.<companyname>.something... which makes them maybe longer than others are used to?

furkan3ayraktar08:10:19

@U0ETXRFEW I guess I would skip it for RealWorld to keep it as simple as possible for newcomers but I could consider it for our work projects.

pez08:10:58

It could be argued that it simplifies it for newcomers. But yes, it also complicates if the search result is filtered. I lean towards agreeing with you.

Stefan08:10:51

+1. Files that exist but somehow don't show up can be very confusing if you don't know exactly why.

pez08:10:36

Indeed. Better throw in some tips there somewhere. Possibly a vscode-tips.md file which can point at that Joyride example as well as ways to configure search and what not for sweet Polylith ergonomics.