Fork me on GitHub
#cursive
<
2023-03-05
>
Jeff Evans19:03:59

Does anyone have a Clojure project using deps.edn with sub-projects that works well with Cursive? I'm struggling mightily to get things working and have to resort to unspeakable hacks and duplication to get things working. Starting to think I'm organizing stuff very wrong.

caleb.macdonaldblack20:03:53

I ended up using one deps.edn file with aliases that group the sub-projects together. Only issue I’ve had is with Datomic ions that can’t use aliases to deploy code. I needed to generate a deps.edn file ad-hoc for that

Jeff Evans20:03:29

So basically you just have deps.edn at your overall root level and use aliases for "activating" the sub projects or combinations thereof?

Jeff Evans20:03:18

Ack, thanks for the data point

👍 2
caleb.macdonaldblack20:03:52

Keep us updated on what you end up with if you can. It would be interesting to see what some other approaches look like

Jeff Evans20:03:09

Sure, absolutely. I think part of my problem is I can't understand deps no matter how many times I try. It's too simple to comprehend I think.

caleb.macdonaldblack20:03:06

What are you having trouble with?

Jeff Evans20:03:10

Mostly with dependency resolution really. Like with GitHub coordinates... Should I expect that to bring in main classes? What about test classes? I tried for several hours to get a Java based project on GitHub to resolve from a SHA coordinate but couldn't. If I used clojure cmd line with -S it works, though. So I ended up just copying the jar into my source.

caleb.macdonaldblack20:03:30

Which Github project? How are you starting the REPL in cursive?

caleb.macdonaldblack20:03:54

Also what did the clojure cmd that worked for you look like?

Jeff Evans20:03:45

What works:

clojure -Sdeps '{:deps {io.github.findinpath/testcontainers-kafka-avro {:git/sha "943ae540e8db36a7515e5b454b71c7fdd54ea4b0"}}}'
(this is the library I want to use classes from)

Jeff Evans20:03:16

My project (in which I switched to manually including the jar): https://github.com/jeff303/kc-repl/blob/support-avro/deps.edn#L31

Jeff Evans20:03:57

How I am starting Cursive nREPL

caleb.macdonaldblack21:03:01

{:paths ["src"]
 :mvn/repos {"confluent" {:url ""}}
 :deps {io.github.findinpath/testcontainers-kafka-avro {:git/sha "943ae540e8db36a7515e5b454b71c7fdd54ea4b0"}}}

caleb.macdonaldblack21:03:13

Something like that works for me to pull in that project

caleb.macdonaldblack21:03:16

org.clojure/clojure 1.11.1
  . org.clojure/spec.alpha 0.3.218
  . org.clojure/core.specs.alpha 0.2.62
io.github.findinpath/testcontainers-kafka-avro 943ae54
  . io.confluent/kafka-avro-serializer 5.5.0
    X org.apache.avro/avro 1.9.2 :older-version
    . io.confluent/kafka-schema-serializer 5.5.0
      . io.confluent/kafka-schema-registry-client 5.5.0
      . io.confluent/common-config 5.5.0
      . io.confluent/common-utils 5.5.0
    . io.confluent/kafka-schema-registry-client 5.5.0
      . org.apache.kafka/kafka-clients 5.5.0-ccs
        . com.github.luben/zstd-jni 1.4.4-7
        . org.lz4/lz4-java 1.7.1
        . org.xerial.snappy/snappy-java 1.1.7.3
        . org.slf4j/slf4j-api 1.7.30
      . io.confluent/common-config 5.5.0
      X org.apache.avro/avro 1.9.2 :older-version
      X com.fasterxml.jackson.core/jackson-databind 2.10.2 :older-version
      . org.glassfish.jersey.ext/jersey-bean-validation 2.30
        . org.glassfish.hk2.external/jakarta.inject 2.6.1
        . org.glassfish.jersey.core/jersey-common 2.30
          .  2.1.6
          . jakarta.annotation/jakarta.annotation-api 1.3.5
          . org.glassfish.hk2.external/jakarta.inject 2.6.1
          . org.glassfish.hk2/osgi-resource-locator 1.0.3
          . com.sun.activation/jakarta.activation 1.2.1
        . org.glassfish.jersey.core/jersey-server 2.30
          . org.glassfish.jersey.core/jersey-common 2.30
          . org.glassfish.jersey.core/jersey-client 2.30
            .  2.1.6
            . org.glassfish.jersey.core/jersey-common 2.30
            . org.glassfish.hk2.external/jakarta.inject 2.6.1
          .  2.1.6
          . org.glassfish.jersey.media/jersey-media-jaxb 2.30
            . org.glassfish.jersey.core/jersey-common 2.30
            . org.glassfish.hk2.external/jakarta.inject 2.6.1
            . org.glassfish.hk2/osgi-resource-locator 1.0.3
          . jakarta.annotation/jakarta.annotation-api 1.3.5
          . org.glassfish.hk2.external/jakarta.inject 2.6.1
          . jakarta.validation/jakarta.validation-api 2.0.2
          . jakarta.xml.bind/jakarta.xml.bind-api 2.3.2
            . jakarta.activation/jakarta.activation-api 1.2.1
        . jakarta.validation/jakarta.validation-api 2.0.2
        . org.hibernate.validator/hibernate-validator 6.0.17.Final
          . org.jboss.logging/jboss-logging 3.3.2.Final
          . com.fasterxml/classmate 1.3.4
        . jakarta.el/jakarta.el-api 3.0.3
        . org.glassfish/jakarta.el 3.0.2
        .  2.1.6
      . io.swagger/swagger-annotations 1.5.22
      . io.swagger/swagger-core 1.5.3
        . org.apache.commons/commons-lang3 3.2.1
        X org.slf4j/slf4j-api 1.6.3 :older-version
        X com.fasterxml.jackson.core/jackson-annotations 2.4.5 :older-version
        X com.fasterxml.jackson.core/jackson-databind 2.4.5 :older-version
        . com.fasterxml.jackson.datatype/jackson-datatype-joda 2.4.5
          X com.fasterxml.jackson.core/jackson-core 2.4.5 :older-version
          . joda-time/joda-time 2.2
        . com.fasterxml.jackson.dataformat/jackson-dataformat-yaml 2.4.5
          X com.fasterxml.jackson.core/jackson-core 2.4.5 :older-version
          . org.yaml/snakeyaml 1.12
        . io.swagger/swagger-models 1.5.3
          X com.fasterxml.jackson.core/jackson-annotations 2.4.5 :older-version
          X org.slf4j/slf4j-api 1.6.3 :older-version
          X io.swagger/swagger-annotations 1.5.3 :older-version
        . com.google.guava/guava 18.0
      . io.confluent/common-utils 5.5.0
    . io.confluent/common-config 5.5.0
      . io.confluent/common-utils 5.5.0
      X org.slf4j/slf4j-api 1.7.26 :older-version
    . io.confluent/common-utils 5.5.0
      X org.slf4j/slf4j-api 1.7.26 :older-version
  . org.apache.avro/avro 1.10.0
    . com.fasterxml.jackson.core/jackson-core 2.11.0
    . com.fasterxml.jackson.core/jackson-databind 2.11.0
      . com.fasterxml.jackson.core/jackson-annotations 2.11.0
      . com.fasterxml.jackson.core/jackson-core 2.11.0
    . org.apache.commons/commons-compress 1.20
    . org.slf4j/slf4j-api 1.7.30

caleb.macdonaldblack21:03:24

It seems to pull it in too.

caleb.macdonaldblack21:03:40

Afaik it will only pull src though. Not test

Jeff Evans21:03:03

Ah maybe that's it then. I was trying to find some page that explains all those nitty gritty details

caleb.macdonaldblack21:03:24

If you’re having issues on the REPL, sometimes a missing alias is the issue

caleb.macdonaldblack21:03:35

But I would agree it is lacking in many ways. Often I’ve not been able to find out what I need to know from it

Jeff Evans21:03:37

Yeah I've read it a hundred times

caleb.macdonaldblack21:03:47

It’s not the best

Jeff Evans21:03:24

But thanks for double checking!

caleb.macdonaldblack21:03:35

No problem. Good luck with it

cfleming21:03:32

@U0183EZCD0D I’ll take a look at those two projects later and see if I can see any problems.

cfleming21:03:09

If there’s anything specific you’re having problems resolving let me know and I can see if I can figure it out.

Jeff Evans21:03:26

Nah that's ok. Let me try on my other machine first just to make sure I didn't do something obviously dumb

cfleming21:03:47

Ok, if you’re still having problems let me know.

💯 2
Jeff Evans21:03:13

My own issues aside, it would definitely be useful to see a nested deps project that is known to work well in Cursive

cfleming21:03:48

What are the sort of problems you’re seeing? They pretty much always work for me, but perhaps I’m assuming everyone knows how to set the projects up correctly.

Jeff Evans21:03:05

I should step back and make it clear I'm not claiming there are any issues with Cursive. But I struggle to, for instance, get an nREPL started under the submodule to be able to resolve the parent module symbols. Could very well be my lack of understanding of deps. I've read the official docs and it's still not clear how it's supposed to look to set up the dependencies (child to parent in this case).

caleb.macdonaldblack21:03:27

There are many nuances in Clojure’s tooling that just require experience or perseverance to resolve. If you have some specific examples, the community can usually help with it.

cfleming21:03:56

Sure, I get that. Part of what I’m trying to figure out is if it’s a deps problem (which I also find extremely confusing), or with how Cursive presents it.

caleb.macdonaldblack21:03:59

I would start a repl with the parent module that uses the submodule though. Not the other way around

cfleming21:03:35

Yes, I think that’s right. I wouldn’t normally expect a submodule to be able to see symbols from its parent module, in any build system.

cfleming21:03:07

If you start the REPL from the parent, it should see symbols from both.

caleb.macdonaldblack21:03:30

Also by “submodule” i’m referring to nested projects, not anything related to GIT. Just to clarify

cfleming21:03:54

So something you’d use :local/root for, right?

cfleming21:03:38

Right. One Cursive-specific caveat is that I implemented those with a horrible hack, which I’m actually implementing a much better solution for as we speak. So there’s sometimes some IntelliJ-related funkiness which should go away in the next release.

cfleming21:03:03

Due to the weird way deps propagates dependencies (specifically, aliases not being transitive).

caleb.macdonaldblack21:03:10

:local/root would be the preferred way. You can also include the nested source paths in :paths in the top-level deps.edn as well. Or :extra-paths for aliases.

cfleming21:03:04

I agree that :local/root is a better solution than the others, which might also give you problems in IntelliJ (which doesn’t like multiple modules sharing the same source root).

caleb.macdonaldblack21:03:06

Yeah there is definitely a deps.edn, intellij & cursive love-triangle that is not fun to navigate

cfleming21:03:28

Yeah, I think that will be much better after the changes I’m making.

cfleming21:03:48

I’m hoping that I can get to the point where the only problem is general deps confusion 🙂

Jeff Evans21:03:58

Right, ok. Now that I think about it more, that's closer to what I did with a previous repo that I'm much more familiar with (repl from top, aliases for extra submodules). Then if each submodule also needs to produce its own build artifacts and whatnot, that's handled in their deps.edn file

cfleming21:03:34

I don’t think I’ll ever understand all the different alias types.

Jeff Evans21:03:53

Thanks, folks. Appreciate the sanity check

caleb.macdonaldblack21:03:18

@U0567Q30W What do you mean by alias types?

Jeff Evans21:03:44

And relieved to know I'm not the only one that still doesn't fully grok deps. LOL. Almost easier to deal with Maven

cfleming21:03:20

-R -X -M -T etc

caleb.macdonaldblack22:03:36

I really like deps.edn. I wouldn’t want to use anything else. For example leiningen. But there are definitely trade-offs, learning curves and lacking documentation that make it challenging. It is also a little opinionated, which I think is a good thing, but it does mean you can hit a brick wall when trying to do things that aren’t recommended. As apposed to other tooling that might let you do whatever you want, getting you by in the short term but screwing you later on

cfleming22:03:08

Yes, I should also say that deps is my default choice for everything too. And it’s the only thing that has been flexible enough to allow me to build Cursive without a huge amount of ceremony (although there’s still some awful hacks to do so).

cfleming22:03:40

But I still have to read the reference doc any time I want to do anything non-trivial. It’s kind of like git in that respect.

👍 2
caleb.macdonaldblack22:03:19

An example of opinionated decisions with tools.deps would be not supporting inheriting between aliases. Another would be the fact that it’s edn and not clj, which means less DRY. I agree with the decisions around those however. But understanding why it doesn’t do those things isn’t always easy. And other tooling might just let you do it anyway.

cfleming22:03:07

I must admit, I still don’t understand why aliases aren’t transitive. I’m definitely a fan of using EDN rather than executable code, though.

caleb.macdonaldblack22:03:09

I must admit namespacing probably solves that issue though

cfleming22:03:30

> What would you expect to happen if your current project deps.edn file defined a :test alias and one of your transitive dependencies also defined it? I’d expect my test code to be able to access test code from the dependency 🙂

caleb.macdonaldblack22:03:43

I haven’t spent much time on this issue tbh. I mainly recall seeing it and figured it’s worth mentioning regarding my point about opinionated aspects of deps.tools. Maybe it’s fine to do. Or maybe it’s overcomplicated. Idk either way I guess.

cfleming22:03:20

Yeah, I’m not really sure either. I’m willing to believe that there are a lot of edge cases with making them transitive, but I’ve not seen a clearly articulated argument about it.

👍 2
Jeff Evans22:03:23

So am I understanding correctly that Cursive is written in Clojure?

cfleming22:03:59

Partly, yes - it’s a mix of Clojure, Java and Kotlin. The Java code is mostly legacy these days, new code is either Kotlin or Clojure.

nice 2
onetom09:03:48

we have a monorepo, too. i found, that some complications can be avoided, if the root of the monorepo DOES NOT have a deps.edn, only modules in sub-directories have a deps.edn

Jeff Evans00:03:11

Interesting. That seems to be the opposite of what many others are doing

cfleming21:03:00

Just to reiterate, you shouldn’t have to do that, and Cursive should support pretty much any deps structure. If you have a case you’re having difficulty with, I’d like to understand it better to try to figure out whether it’s a code problem or a doc problem (or just a deps problem).