Fork me on GitHub
#tools-deps
<
2019-09-17
>
avi19:09:27

👋 are there any known gotchyas with using gitlibs via t.d.a in CircleCI? I just switched a lib from Maven coords to git coords; it worked fine locally for me but over in CircleCI the deps were downloaded but then the files couldn’t be found on the classpath during the test run.

ghadi19:09:22

@aviflax hard to say without seeing what bin/tests-with-coverage is doing

ghadi19:09:40

in general if you have private ssh repos, you'll need to use an ssh-agent

avi19:09:44

clojure -J-Xmx2g -A:test:test/coverage

avi19:09:49

It’s not a private repo

avi19:09:17

I was able to reproduce the problem just now locally via Docker so I should 😅 be able to figure it out

avi19:09:10

It looks like the problem is that when the classpath is generated, there’s an extra segment main at the end; that dir doesn’t exist in the repo hosting this particular library

avi19:09:29

Here’s the cp entry: /home/circleci/.gitlibs/libs/clj-chrome-devtools/clj-chrome-devtools/58a396f54f469e98b5aae92b9357d65434914228/src/main

avi19:09:26

Anyone have any suggestions on why that might be the case? I’ve been looking through https://clojure.org/reference/deps_and_cli and so far haven’t found any clues

ghadi19:09:38

Seems like that lib has an invalid paths inside its deps.edn?

avi19:09:59

ah, maybe, I’ll take a look

ghadi19:09:34

It needs to specify paths explicitly

avi19:09:17

Ohh wait I remember this

ghadi19:09:25

:paths ["src"]. I think there is a bug when it isn't set

avi19:09:29

the dep is inheriting the source path that was specified in my project root

avi19:09:41

I mean, my project’s deps.edn

avi19:09:46

IIRC this is a known bug

avi19:09:10

I’ll see if I can find the JIRA issue

dominicm19:09:24

Make sure you vote!

avi19:09:23

will do, if/when I find it

avi19:09:27

so far no dice

vlaaad19:09:53

wasn't it fixed?

avi19:09:02

It must be that the Docker image I’m using hasn’t picked up the fix yet

avi19:09:19

I’m using circleci/clojure:openjdk-11-tools-deps-browsers

avi19:09:50

@ghadi you reported that bug!

ghadi19:09:38

It's the circle of life

avi20:09:48

Thanks for the help!

avi20:09:54

Alex commented on the issue: > Released in clj 1.10.1.458 And I’ve confirmed that the CircleCI image I’m using has 1.10.0.442 installed. Interestingly, even the official Docker images for Clojure don’t appear to have any variant with a version of Clojure more recent than 1.10.0.442. This is surprising…

Alex Miller (Clojure team)20:09:11

there are no official Docker images for Clojure

gerred20:09:43

i don't even think there should be official Docker images for Clojure in a world of uberjars, to get edgy

avi20:09:27

ah, well that explains it

avi20:09:36

who maintains the images here? https://hub.docker.com/_/clojure

gerred20:09:54

that's Docker owned, like the company.

avi20:09:21

So if I think they should be updated, I guess I pester Docker, the company 😬

avi20:09:24

OK thanks all!

gerred20:09:50

yup! if you're interested in custom images and interested in sponsoring some sort of support model, I'd love to chat though.

gerred20:09:00

we made a #cloud-native channel lately.

gerred20:09:17

so I'm in a process of taking in problems/opinions and trying to distill something

👍 4
avi21:09:35

Next question: • I’ve been using Cambada to AOT my code and build an uberjar containing my code and all my dependencies. • Now that I’ve switched one of my dependencies to a gitlib, Cambada is having trouble building the uberjar. So: Does anyone happen to know whether there’s an existing solution to this somewhere? (I mean either some way to continue using Cambada, maybe a fork, etc; or some alternative to Cambada (whether a single tool or a combination of tools) that I might want to consider switching to.)

mike_ananev13:09:45

@aviflax try this fork of Cambada. There are several issues have been fixed. may be your problem is already fixed too. https://github.com/mikeananev/cambada

avi13:09:52

Will do — thanks!

avi15:09:19

@mike1452 it doesn’t seem to work; it seems to fail with the same error as the root cambada. I see you have GitHub’s issues feature disabled in your fork but the feature is enabled in the root repo — would you be interested in a bug report opened as an issue in that repo, which we could use to discuss and debug the problem, and to potentially craft a fix to be merged into your fork?

avi16:09:57

@mike1452 never mind, I spoke too soon — I took a closer look at the error output and it was not the same error — it was a different error that I’ve seen before and already have a way to work around. And once I applied that workaround, the uberjar process completed successfully. So your fork does in fact resolve my issue with gitlibs. This is great — thank you!

avi18:09:32

For anyone who might have a similar question: @mike1452 suggested I try this fork of Cambada: https://github.com/mikeananev/cambada — and it does indeed seem to have fixed the problem I was encountering with gitlibs, AOT, and uberjarring. Thanks Mike!

seancorfield21:09:18

@aviflax Is AOT a requirement?

avi21:09:24

I guess it’s not a 100% hard requirement but I’d prefer to preserve it. The project is a CLI tool so it needs to start quickly.

avi21:09:17

Thanks, I was just reviewing that list. I was just wondering if anyone had an answer off the top of their head. The descriptions of those tools don’t always include lots of details.

seancorfield21:09:24

I don't know how many of those do AOT although, in theory, you could always do that step separately and then package stuff: -e "(compile 'my.main.ns)"

👍 4
seancorfield21:09:27

I am fairly certain that depstar works with git deps and local deps but it does not do AOT.

avi21:09:40

I believe Cambada is AOT compiling all the Clojure code in my project, including in dependencies. I like that.

avi21:09:04

Would calling compile do the same? I’d guess not...

seancorfield21:09:15

Since AOT is transitive, I suspect just compiling the main ns will get you much the same result...?

avi21:09:44

It’s worth trying out for sure!

avi21:09:59

Gotta get on my bike now, will check back in shortly. Thanks!

seancorfield21:09:16

You'll need to ensure classes exists as a top-level folder in your repo and add it to the :paths vector so it's on the classpath.

thanks2 4