Hello, what is a recommended/nice way of organizing a mono-repo like project of multiple tools.deps backed projects in terms of being able to run all tests and produce JARs/docker images/run tests for each from the root of the project? I have something like this, using Makefiles:

.
├── Makefile
├── docker-compose.yaml
├── consumer
│   ├── Dockerfile
│   ├── Makefile
│   ├── deps.edn
│   ├── src
│   │   └── queue
│   │       └── consumer.clj
│   └── test
│       └── queue
│           └── consumer_test.clj
└── producer
    ├── Dockerfile
    ├── Makefile
    ├── deps.edn
    ├── src
    │   └── queue
    │       └── producer.clj
    └── test
        └── queue
            └── producer_test.clj

@rahul080327 I'd recommend looking at juxt edge which is my implementation of a monorepo. If you are looking to share makefiles, you probably want the include instruction in make (not shown in edge)

In edge I solve the reuse problem with bin scripts at ../bin/eftest, etc

Is it possible to use clj and friends private repositories via a password-protected SSH key?

there's no interactive ssh with the tools @zane

ssh-add the key to your running ssh-agent

then you can run clj

@ghadi Yes, I did that but I’m getting an error (`USERAUTH fail`) when I try it with a password-protected SSH key.

Works when I use a SSH key that does not have a password.

if you run ssh-add -l you see the key you intended to add?

I believe I added it, yes. Let me run through it one more time to be sure.

But it sounds like what you’re telling me is that this ought to work. 🙂

I don't think it should work

I don't think password protected ssh keys are supported in the set of stuff we're running

Ah, okay.

totally should work

it prompts for pw when adding to the ssh-agent

it's non-interactive after that

if you see the key in your agent, make sure it doesn't appear in ~/.ssh/config

JSch, the underlying ssh library, is stupid w.r.t. understanding .ssh/config when there is also an ssh-agent present

(It prefers to read the key directly from the file, if it knows about it)

(Rather than consulting the agent)

So, what's in your ssh agent (`ssh-add -l`) and what's in your .ssh/config?

ok comment out that block in your .ssh/config

temporarily

then run clj again

clj succeeds, but I think it’s pulling from a cache.

i.e. It’s using the copy of the repository I downloaded when I was using a passwordless SSH key.

remove that from ~/.gitlibs to verify

Trying to remember how to clear that cache.

Rad. Thanks.

should be in two places in there

~/.gitlibs/{_repos and libs}

Any harm in just blowing away ~/.gitlibs entirely?

dunno how much you have in there

but anything will be redownloaded if you have creds

no worse than blowing away ~/.m2/repository

Awesome.

so did it work?

Trying.

Started up instantly again.

That’s odd.

i.e. I didn’t see it download the private repository.

clj -Sforce

Worked.

Nice.

@alexmiller

Thanks!

So, what should I do with ~/.ssh/config now?

Leave that stuff commented?

tl;dr: https://clojurians.slack.com/archives/C6QH853H8/p1587737218326100

I confess I don’t understand SSH well enough fully understand the implications of that.

I don't think in your case there is much harm in removing just the IdentityFile portion of that block

Got it. I had hoped that was the case.

because ~/.ssh/id_rsa is one of the default searched keys

But for people with multiple SSH keys that might be an issue, sounds like?

(I’m writing up documentation on how to do this for the rest of my team.)

it depends

do you have 1 SSH key <> per repo?

or 1 SSH key that grants you access to a bunch of repos, but you also have unrelated SSH keys present too?

The former is not supported, the latter is, AFAICT

I personally only have this one key, but I imagine others on the team might have more.

I see.

Also, I just want to confirm one more thing: https://ask.clojure.org/index.php/8725/deps-support-newer-private-file-formats-types-such-ed25519 that clj won’t work with OpenSSH-format keys and that you must use -m PEM when generating them. Is that correct?

My personal recommendation is to not use files at all, only put your ssh keys into the agent

I see. I’ll read up on how to do that.

clj can use fancy newer keys like ed25519 keys as long as JSch is asking the agent to deal with them

If JSch is trying to read them from the key files directly -> no bueno

I see!

all of this is a pain IMHO

github style https auth tokens are a simpler model, and offer finer grained authority

But that’s not supported at the moment?

easier rotation/revocation

right, not supported

Got it.

This is extremely helpful, @ghadi. Thanks again for your time.

I may try to write this up so others can benefit.

a buddy at Github told me <35% of the auth they see is ssh based

Right. I certainly wasn’t using SSH with GitHub until I had to get clj to work with a private repository.

Weird, I always considered https the less-secure approach.

tokens can be ephemeral, ssh identities are long-lived

https://developer.github.com/v3/guides/managing-deploy-keys/

good pros/cons ^