Fork me on GitHub
Slack Archive

This page is not created by, affiliated with, or supported by Slack Technologies, Inc.

2021-01-22

Channels

#tools-deps
<
2021-01-22
>
Eamonn Sullivan06:01:30

No, but I think I figured out what I need to do. It works locally because of the ssh config set up. I will need to set up a docker container configured the same way, save that to an ECR and then specify that docker in the build. 🤞

Jakub Holý (HolyJak)12:01:47

Hi! How to solve > Cloning: https://github.com/borkdude/clj-http-lite > Error building classpath. Destination path "clj-http-lite" already exists and is not an empty directory > org.eclipse.jgit.api.errors.JGitInternalException: Destination path "clj-http-lite" already exists and is not an empty directory > at org.eclipse.jgit.api.CloneCommand.verifyDirectories(CloneCommand.java:257) caused by a borkdude/clj-http-lite {:git/url "" ...} dependency? 🙏 i.e. where does it exist, so that I can delete it?

borkdude12:01:01

@holyjak rm -rf ~/.gitlibs

❤️ 3
Alex Miller (Clojure team)13:01:27

well you don't have to rm the whole thing :)

Alex Miller (Clojure team)13:01:47

but yes, that usually means there is a partial download in the ~/.gitlibs dir

Alex Miller (Clojure team)13:01:07

if you're running into this often, consider using -Sthreads 1 on clj

👍 3
borkdude13:01:47

I usually do remove the whole thing because there are two things inside .gitlibs, repositories and some other things, so just to make sure, I bluntly get rid of it all ;)

Eamonn Sullivan13:01:08

I didn't know it was supposed to work without that error message. :thinking_face: Happens to me 100% of the time, so I do the same: just blow it all away. I didn't think to try single-threading. Also, I just recently discovered why I was getting cryptic error messages about an invalid privatekey, unless I removed my .ssh/config every time. My key format was too old. A regen (`ssh-keygen -p -m pem -f ...`) finally got rid of that for me, making this feature much more useful.

Alex Miller (Clojure team)13:01:26

if that doesn't help, I'd be curious to hear about it. my belief right now is that this is a concurrency issue where parallel downloads happen and one is stomping on the other in the same dir

Eamonn Sullivan13:01:30

I will definitely try this next time. I'm currently trying to get this all to happen in a docker container, so I'll probably have lots of opportunities...

nnichols14:01:40

@UR71VR71S Forwarding the SSH agent for a dockerized build took me a while to grok. It requires the buildkit tools, 

DOCKER_BUILDKIT=1 docker build --ssh default --tag ....
and some of the experimental options: 
# syntax=docker/dockerfile:1.0.0-experimental
### -----------------------------------------------------------------------------
###                 PHASE 1:
### Build the application within a Docker image
FROM clojure:openjdk-8-tools-deps as builder
WORKDIR /home/app
COPY . .

### Install git so we can pull dependencies from GitHub repositories
RUN apt-get update -y
RUN apt-get install git

### Download public RSA keys for 
RUN mkdir -p -m 0600 ~/.ssh && ssh-keyscan  >> ~/.ssh/known_hosts

### Use seancorfield/depstar to create a POM and build the app into an uberjar
### This fetches dependencies AOT, so we don't need to fetch anything when the app starts
### Make sure your app specifies a (:gen-class) directive in the main namespace
### This will AOT compile the main namespace, and anything required in that namespace
RUN --mount=type=ssh clojure -Spom && \
    clojure -Sdeps '{:deps {seancorfield/depstar {:mvn/version "0.5.2"}}}' \
    -m hf.depstar.uberjar my-app.jar -C -m my-app.main

Eamonn Sullivan14:01:02

Thank you very much, @UCZ5ZDJKF! You probably saved me hours...

nnichols14:01:05

Anytime! It wasn’t a fun discovery process lol

Alex Miller (Clojure team)14:01:32

you should write that down somewhere :)

danboykis15:01:17

does anyone know if proxy username/password in ~/.m2/settings.xml are respected when fetching dependencies? From what I can tell the answer is no.

Alex Miller (Clojure team)15:01:35

https://clojure.org/reference/deps_and_cli#_maven_proxies is supported but I'm not familiar with any user/pw thing for that

Alex Miller (Clojure team)15:01:18

let me glance at the code

Alex Miller (Clojure team)15:01:40

there is code there with that intent (can't say I've tested that personally)

borkdude15:01:25

I'm not sure if this is relevant here, but deps.clj forwards proxy environment variables to the tools.deps JVM's java properties: https://github.com/borkdude/deps.clj#proxy-environment-variables

danboykis15:01:43

thanks @alexmiller i'll play with it and see if I come up with something more concrete

Alex Miller (Clojure team)15:01:32

happy to look at fixes if something isn't working there (but I'm somewhat limited in ability to test)