I’ve just cloned a javelin repo which uses boot - what is the fastest way to start a REPL to test modifications which i made to the lib?

@sbondaryev boot repl

hi @sbondaryev you can run the javelin tests

boot watch speak test-javelin

that will run them continuously as you hack on the code

(you'll need to have phantomjs installed) to run the tests

we should probably update javelin to use the cljs test task now

@martinklepsch which tasks do you use for running cljs tests these days?

@micha This has worked pretty well in the past https://github.com/crisptrutski/boot-cljs-test

sweet thanks

but mostly snapshot releases & not so much docs so you'll probably end up reading the code 🙂

haha that's ok with me

oh there is even an example project

@richiardiandrea I wanted to follow up on a discussion from yesterday in #cljs-dev

As you know, Clojars (optionally) lets you sign JARs you push to it's artifact repository - their aught to be a way of querying their API for the registered public key for a particular user. Next, there should be a way of getting a signature from their API, and finally, once you have a public key and signature, checking is straight-forward. Almost everyone just wraps bouncy-castle - here is a wrapper

https://github.com/greglook/clj-pgp

Checking signatures is far more challenging in ClojureScript, where there is a dearth of cryptographic libraries that work well with the Closure Compiler and the zoo of JavaScript platforms.

@xcthulhu boot does show you whether jars you get from maven are signed or not

actually verifying the signatures, though, is another matter

i don't know of any real solution in this area

maybe with letsencrypt though there is a way forward for the future

like currently anyone can sign a jar with a gpg key

but there is no way to know whether you should trust that key or not

we were talking about http://keybase.io a while ago

but then i realized that their whole system is based on npm modules and i just don't trust it

@micha Doesn't maven/clojars provide an API for getting the public key of registered artifact authors?

how would that solve the problem?

You would just use the repository as your authority

but why would you trust it?

You trusted it to download the artifact in the first place

not rally

Oh?

i mean if that's the case then you don't need signatures right?

Well, I guess someone could mess with it on your hard-drive

the signature is to protect against a compromised clojars

like if you already trust clojars to send you the correct jars, you can just use that

which is my current situation

Like I said, a hacker could modify the jars placed in your .m2 directory

one thing i would trust would be jars signed by keys that are themselves signed by a legit CA

where you need to provide personal info to get the cert

and so on

like a SSL cert

http://keybase.io does the next best thing, which is cross reference multiple social media accounts

Eh, CAs can get hacked too. Honestly, a solution that is independent of worrying about various servers getting hacked would be CAS

Storj, IPFS, or whatever Ethereum is inventing

lol then you get neverending 51% attacks

at the end of the day it seems like you need to trust humans to keep everything above board

No

Not IPFS

It's not a blockchain

It's a CAS system

it's it's purely algorithm then it can be exploited

Please sketch in greater detail how you would compromise IPFS

oh i am not the hacker

but look at etherium

i am ok with like verisign handling validation of SSL certs

i feel more comfortable with that than with a blockchain based SSL cert system

because there are people who can be sued or sent to jail

I am not suggesting using a blockchain as a certificate authority

The only reason I mentioned Ethereum is that their "Swarm" system they are building out is a CAS like IPFS

Incidentally, IPFS is already dogfooding using IPFS for versioning their upstream dependencies

Check out the source code - https://github.com/ipfs/go-ipfs/blob/6f1062b4f323037983841585d706bc4d56020552/core/corenet/net.go#L7

I would love it if I could specify my dependencies like that

but how do you know which things to trust though

You might consider reading the IPFS white paper - https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf

i'm talking about transitive dependencies

I need to do some chores around the house, but I would not mind discussing this matter further, especially if I can get devs working on a build-tool excited.

Oh, well either the JAR has dependencies in Clojars/Maven Central, and you are in the same boat you are now

yeah this is a huge problem and we're only seeing the tip of the iceberg

Or, all of its dependencies are specified with IPFS hashes

In which case you just go to the CAS

yeah i meanimmutability is good, i just don't see how it helps you trust the source of the immutable thing

i don't see how you can avoid delagating to some legal entity like a CA authority

something that can be taken to court if they misbehave

It's called "Content Addressable Storage". If the merkel hash of the artifact/directory you have on your hard-drives checks out, then you must have received the right thing.

Please go read the white paper.

i get it, i do

i can be 100% sure that the thing on my hard drive is the correct sequence of bytes

that does not help me know if the sequence of bytes was written by a human i trust or not

specifically transitive dependencies

how do i know if a library is pulling in a trojan horse dependency?

the dependency jar is 100% correct on my harddrive

it just contains malicious code written by a hacker

and it's a transitive dependency, so i'm not likely to inspect its code

Honestly you have such threats now. Anybody can push whatever they want to Clojars, it could potentially be very malicious.

we totally do have this threat now

it's a huge problem i think

i'm really amazed that it hasn't blown up yet

I am discussing a problem of "Well, checking OpenGPG signatures isn't a very good check if a server is compromised" which is true.

And CAS would be a valid alternative.

Rather preferable

if the signature was associated with a legit certificate authority though, you don't care if clojars is compromised

and you also know you can trust the jar, because that's what the CA does

it can revoke, etc

I've never used Let's Encrypt, do they do rigorous KYC?

there would be nothing a hacker could do when they exploit clojars

they're like a free CA you can use to get a SSL cert

That's a "No"

but not every platform includes their keys by default, so it's not really 100% ready

So you can register there, push a Jar with a trojan, and sit easy still anonymous.

how can you be anonymous?

Well, they don't bother with KYC or anything.

Right?

they do the same human verification that verisign does, as far as i know

otherwise you might as well use the mit pgp keyserver right?

man i certainly hope they do, anyway

otherwise i will block them in my browser instanly

Eh, I'm cool with source code being included in my JARs TBH and just using CAS. I'd rather have CAS for versioning anyway, since someone can always push an artifact with the same version tag to a repository and rewrite history.

yes i agree that immutable storage is pretty key

I guess if you wanted to bolt on an elaborate CA thing for legal liability that's cool too. I'm not an anarchist or anything.

but i think it doesn't really solve the whole problem

Nope.

i think we need both

the immutability is more for general dependency hell avoidance than security imo

Yes

Although it does help with validation.

A certified individual would presumably need to sign some hash, might as well be a CAS hash

Right?

totally

i wonder if it would be possible to get some kind of funding for real certs we can use to sign jars

perhaps linked to github user

that seems to be the real solution to the trust issue

because nobody can verify the source of every transitive dependency

that ship has sailed except for maybe the NSA or something

Ethereum has a janky thing using the blockchain as their CA - https://github.com/axic/mango

i think it really needs to be part of the legal system though

Agreed

we're dealing with our customer's money and trust

https://docs.oracle.com/javase/tutorial/deployment/jar/signing.html

this seems pretty sane to me

Okay, step one, get support for checking these in boot, even if it's sort of just for show

there is boot show -v, which checks whether jar is signed or not

does not do verification because that's useless the way things are

it would be false hope 🙂

Like I said, you need to do it just for show

so i think the only impediment is the overhead and cost of maintaining the CA relationship

which makes sense because there are costs to being a legitimate professional

we can build boot tooling to make the mechanics of correctly signing and verifying jars simple

we need some way to help people pay for the certs and manage them

Actually, I have been doing most of my work in ClojureScript lately

A ClojureScript GPG library is a big undertaking for me.

why gpg in cljs?

So you can have a build system that doesn't need the JVM

i would shell out to the jar tooling

like in the tutorial

the final artifact will be a jar

you definitely don't want to rewrite aether in cljs

There's no unified way of shelling out in the various JS CLI platforms.

you need the jvm to manage maven deps anyway

You can tell that to the people who want to build the JS build tool. Honestly, I have no idea when I will have the time / funding to do this sort of work.

why not use boot just for dealing with jars

I do everything with ECC anyway, RSA is terrible.

what js build tool are you referring to?

After google released their JS Closure Compiler, everyone has been excited about eventually getting a JS land build tool, since waiting for the JVM to spin up is lame.

But the mess of course is that there's no tools for dealing with JARs, not even a standard for IO for JS CLI

Nor crypto, but that's like, the last on everybody's list of thing to worry about, but it's been my living lately

yeah i don't have those problems

i write business applications

clojure startup time isn't a concern for me

when i make the UI i'm not playing around with live reaload constantly, because the UIs are for business. i don't need to have a live art performance

what i do need is robust dependency management though lol

It's not a big deal for me either. But, I would like to sort of help the W3C out in putting the right primitives in their cryptographic API, because right now that thing is useless.

Other than that and helping the crypto situation in JS in general out of kindness, I'm ready to admit it's theatre right now when dealing with JARs.

yeah i haven't seen any package manager that beats the reliability of maven really

it has some quirks but they're easy enough to understand because it always does the simplest thing when it's unsure

definitely prefer maven to npm for instance

I prefer maven to NPM as well.

@xcthulhu @micha do you know this dependency story? https://www.kickstarter.com/projects/2066438441/haunts-the-manse-macabre/posts/373466

@onetom This is why the common practice amongst go developers is to "vendor" their upstream developments, and why IPFS dogfoods itself in its source code

sure, it's just a good example of a real story which shows the serious consequences of this dependency problem

Right. It's a bit tougher in go-lang than in Clojure, because you can literally put URLs in your includes, and go will fetch them.

Much as I love the idea of IPFS, there's no JVM port. Who knows when one will land...

Very interesting, I have always wanted to have a quicker build tool, especially for dev, but I agree that a maven port to js would be a big challenge (maybe Aether is enough to port?)

I am checking ipfs ;)

@xcthulhu when u said https://clojurians.slack.com/archives/boot/p1473523449001059 which white paper were you referring to? can u give an URL, pls?

https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf

I enjoy that the URL uses the IPFS hash in the link to the official white paper.

🙂

ah, this one i think i read before

+1 IPFS task

yeah the go dependency troubles are pretty well known now i think

i am really happy with the robustness of maven as a package manager

it's also important to separate package management conceptually from the build process, and even from dependency management

clojars does enforce immutability to the extent possible without something like IPFS

it won't let you change an artifact you've already pushed

and you can't delete anything

I think it would be a good community project to make an IPFS task that works like the old checkout task, that way boot doesnt need to care about it

all we are trying to do is download a file extract the contents and add it to the fileset.

i suppose

personally i am strongly opposed to using anything but versioned dependencies in my projects

especially transitively

if people start publishing artifacts with dependencies on SHAs i think it will be a real shame

i am pretty convinced now that the problem is not a technical one

it's a hygiene problem

if you're going to publish artifacts you need to be thoughtful and move slow

lots of things use sha’s for their build identifiers the are less developer friendly but more computer friendly

but dependencies are not a computer problem

they're a human concern

the halting problem

a computer can't know which version of a dependency you want

right if im leaving it up to the computer i probably want the latest version, it tihnk it works better for rolling systems

and SHA as version is meaningless

the whole point of the version is to communicate information about compatibility

but that information i don't think is computable

a human needs to make a judgement call

i don't see how CAS helps at all

if you're worried about exploits and hackers you need some kind of crypto web of trust anyway

and in that case you can validate the SHA very easily without IPFS

i never want the latest version of transitive dependencies btw

that's why i don't use ruby, basically

maven is great because if it builds and works today it will build and work the same in 100 years

i think it depends on why you want an IPFS file, im not arguing for it’s use with dependencies just it’s availability

but again i dont think it’s boots concern

i specifically don't want to update any dependencies automatically

If you pegged to an IPFS hash, it would not update any dependencies automatically since the thing would be immutable

TBH, you can already do this sort of thing - you can just git hashes with http://jitpack.io

maven is already immutable though

@micha - I think if you email the folks at Clojars they will manually take artifacts down.

Just like over at NPM

Also, if Clojars goes down, you are SOL

Happens from time to time.

i trust toby to make the right decision with respect to deleting things

they might be less likely to do this after padleft incident

the left-pad thing could never happen with clojars

Well, I trust Toby too now, since I trust you, @micha, a total stranger 😛

haha

that's what business is built on though, which i feel very good about really

I thought Clojars was a non-profit? What on earth do they sell?

no i mean business in general

like i trust him to manage clojars and my customers trust me

Oh, okay. Anyway, I think I might have a job opportunity porting IPFS to the JVM.

I'd rather not have to trust Toby or whoever unless they are actually incentivized (with money, not reputation) to not delete stuff.

No idea what it's like over at the other repos like mavencentral or sonatype.

In python, you can just delete stuff on pypi

Somehow there's never been a left-pad type incident

I’d prefer to depend on people who are not incentivized by money, someone who is already incentivized is more likely to switch sides for more incentives… NSA deals with Facebook, Microsoft, Google etc.

well i think there are separate concerns that are being conflated maybe

like the ability to delete things, IPFS is deifnitely good for that

i mean preventing people from doing that

right and prevent them from replacing packages, with say modified or corrupted ones

part of the problem is that a lot of issues are starting to converge

the second point i think is a crypto problem

because SHAs are meaningless

the only meaning they have is as a cache key

but there is no information about what the thing actually is there

right moving from a version to an sha is kinda a regression since, like you said, it’s meant to convey info to the human, a different issue than using something like IPFS

Sounds like it should be optional.

For me, I would vastly prefer to do versioning with hashes rather than numbers

At least for security sensitive things.

i mean you already do that with maven though

the artifact's pom url is the hash

it's true that things can in theory be deleted or changed

but verifying that the thing at the hash url contains the correct contents is trivial with or without IPFS

i mean using crypto

and you need that crypto layer anyway if you have any transitive dependencies

if you get rid of hashes that encode version info in favor of SHAs you lose a lot i think

just in terms of auditing

Why? Can't you still pull down all the source code?

i really enjoy the talks in #boot and #hoplon , very interesting discussions that cover a wide range of issues 🙂

sure, but you end up with kind of the same thing though, no?

each hash needs to be associated with metadata about what the artifact means

like i look at my deps tree and see all hashes

that would not be very useful to me

A human readable view would presumably have the version tags on all of your deps.

right

so the hashes could be an implementation detail then

because you'd need a way to be sure that a hash corresponds to a version

It's just so you can avoid having to worry about Clojars or mavencentral getting hacked or taking a thing down.

Like, because as we already established, the GPG key sigs are rather useless.

indeed

Makes you feel good I suppose

i feel like someone cares when they sign the jar

it's like the thank you note you get in return for the bread machine wedding gift

hahahaha

so if artifacts are hosted in IPFS you'd want some kind of index, right?

the index would need to live in the regular web i imagine

Clojars or GitHub

that's kind of interesting too

the git connection

so the index is also sort of immutable

like you could have your own copy of it

and clone, merge etc

your dependency list could be associated with a specific commit in the index in git

https://github.com/larsks/git-remote-ipfs

because if the index is modified you're still pretty screwed it seems

like for example, suppose i depend on foo/bar version 1.2.3

i look it up in the index and get some hash in IPFS

someone could change that to point somewhere else in the index later

and then it's loading a different thing for the same artifact

oh ipns

this is interesting

IPNS + IPFS seems like something you can build a maven wagon on

that's the part of maven that deals with remote resources

vs. the local cache

Cool. Well, truth be told, I will have to wrangle a contract to port IPFS to the JVM, which will take time.

But I will keep folks posted since it's an exciting development.

is there a fuse filesystem implementation for it?

or something like that?

anyway yeah this looks promising to me

@xcthulhu I would be very interested in tracking a clojure version of IPFS to JVM

I actually know someone who may be interested in helping a project like that, hes waiting for it

Btw y’all @micha @xcthulhu @flyboarder I made an #ipfs channel in here a month ago. 🙂 I’m finding myself at the intersection between Clojure world and Web 3.0 world too, but as a new developer it’ll probably be a while before I can build anything to link up to Ethereum/IPFS/Golem Network/Storj myself.

Also made an #ethereum channel. Not sure if y’all know but the EthereumJ project is a full-out Java implementation of the Ethereum node. So in theory you could run an Ethereum node in Clojure. Nothing for IPFS though from my understanding like you were mentioning. You guys should really look into the Golem Network too for basically decentralized AWS through Docker containers. It’s so cool IMO. That’s written in Python.

https://github.com/ethereum/ethereumj

https://github.com/imapp-pl/golem

interesting!

58477kllllll+,9n6u[h7n6