Fork me on GitHub
#clojure
<
2018-06-10
>
rplevy18:06:55

With the activity this year on how we deal with library dependencies in Clojure it seems like there could be some interest in solving the "dependency hell" problem. Is anyone working on that, maybe a tool already exists? Naively, two thoughts come to mind. 1. Every library packages all of its dependencies uberjar-style. 2. When requiring/referring/importing code from a library, something akin to namespacing dustinguishes code from one lib from another. Easier said than done, but worth thinking about how it could conceivable be accomplished.

dominicm19:06:03

@rplevy I think within the clojure community the accepted answer is to changing apis.

dominicm19:06:12

If you make a breaking change, create a new namespace, function, etc.

rplevy19:06:54

I get that, Rich's rallying of the community around accretive changes to library APIs. However, unless everything is written and changed going forward that way, it doesn't solve the problem that you have 100 deps and fucked if you want to upgrade anything

rplevy19:06:27

I agree with Rich, and it will help a lot to do things that way

rplevy19:06:07

Especially if it can be more systemically guided rather than "best practice" only

johnj19:06:02

wasn't modules in Java 9 design to help with this?

👍 4
rplevy19:06:28

Awesome! Looking into this, it does appear to address this exact problem.

danielcompton21:06:08

@rplevy unfortunately, Java 9's modules don't really address the issue for libraries, you can still only have one of each artifact on the class/modulepath AFAICT

danielcompton21:06:34

Things like OSGI and it's brethren do address the issue of conflicting transitive dependencies, but introduce their own costs which many people find to be higher than addressing the issue

danielcompton21:06:02

https://github.com/benedekfazekas/mranderson lets you create source dependencies ala NPM/node_modules

rplevy21:06:39

Damn, that's unfortunate. But using NPM-style modules could be a good option.

danielcompton21:06:39

Yeah in statically typed languages you would run into trouble with that approach if you had different versions of the same types leaking out into 'your' app because you would get different behaviour depending on which version you use

👍 8
danielcompton21:06:57

Most Clojure libraries don't use custom types, so this would be less of an issue

👍 4
danielcompton21:06:34

With a sufficiently clever type system, it can determine which types 'leak' out of the API and make version resolution decisions based on that

sophiago22:06:35

Does anyone have advice for working with tools.analyzer ASTs? At lot of what seem like obvious transformations on them, e.g. filtering for certain keys and all their nested values while preserving the structure of the entire AST, turn into non-trivial tree traversal problems. I've been using Specter so far, which has its own learning curve, but the ability to specify a path for traversal doesn't alone enable transformations like the example I just mentioned. Given the number of projects that use these libraries I'm thinking there must be a better way than what I've been trying, either through settings when generating the ASTs or just from better understanding their common structure. Any tips?

sophiago22:06:36

I'm thinking of things like how in the tools.analyzer.jvm README is suggests one "interactively explore the AST structure, inspecting the :children and :op fields of a node and the keys function rather than printing it to see its content," but am not sure how to apply this in the context of using jvm/analyze in macros to do source transformation on entire functions rather than inspecting smaller forms at the REPL as in the examples.

sophiago22:06:52

Ah, part of my problem may be having tried the functions from clojure.walk rather than the same from tools.analyzer.ast that use update-children and allow for short-circuiting.

dpsutton23:06:41

At the bottom of tools analyzer there are some transforms. You could read over them to see how they are done

dpsutton23:06:22

One uniqueifies all the variables. Can't remember the other ones. But there are some limited transformation to follow

tristefigure23:06:08

You could give a try to a semi-released library I've been working on: https://github.com/TristeFigure/dance/blob/master/src/dance/core.clj#L336 It's basically prewalk/postwalk on steroids. Immediate benefits is that it allows to compose smaller walks into bigger walks and supports passing a context. There is a walk that keeps track of the depth or the path in the tree that you can use right away with a more specific walk.

sophiago00:06:14

I did actually look into this after I saw lexikon in this week's "The REPL" 🙂

sophiago00:06:16

Keeping track of the depth is definitely useful. I'll have to look more into how it handles paths. The root, so to speak, of the traversal problems I've encountered is generally that I can't decouple predicates. So in the example I gave, pre-order traversal is fine...but what I care about is essentially do certain keys appear anywhere in a given path. That's sort of a combination of checking for whether the keys match and whether the values are colls, i.e. whether the path has terminated, but those two predicates can't just be relationally composed nor ran separately. I need to be able to say, "Stop a given depth-first search when you see this key and if you never see it then delete every k-v pair in the path." Does that make sense?

sophiago00:06:32

Thanks! I somehow missed this one because it's in its own namespace. I'm not sure it's what I want, though. I was asking more generally whereas it seems you have an idea of what I may be trying to accomplish immediately 🙂

dpsutton00:06:52

i'm not sure what you're trying to accomplish but this function explicitly walks over the ast and modifies it

sophiago00:06:57

It does one particular thing. I guess you're suggesting it as an example of how to write my own transformations?