Fork me on GitHub
#clojars
<
2016-01-13
>
meow13:01:48

how are things going here

meow13:01:08

what do we need to do to provide better support for #C0H28NMAS

tcrawley14:01:43

@meow: howdy! What we need most atm is PRs for issues. we've been given plenty of money, but don't have time to fix everything that needs fixing

meow14:01:43

tell me more

meow14:01:59

so I can get on that and find some help

tcrawley14:01:15

we have a list of issues "ready for work" at https://github.com/clojars/clojars-web/labels/ready

tcrawley14:01:08

there are probably a few more that have come in recently that should be tagged with :ready as well

meow15:01:18

ok, I will review them to see if I can think of how to get volunteers to help with them

tcrawley15:01:50

great, thanks!

tcrawley15:01:13

sorry for the quick responses, I'm just swamped with work stuff atm.

meow15:01:18

I understand

micha15:01:26

@meow: i want to work on a tool that can detect and fix broken maven metadata xml files

micha15:01:52

this is needed before atomic commits can be implemented

meow15:01:57

so what do we need - some kind of parse of xml that can handle misformed xml or something

meow15:01:41

something that validates maven metadata xml or more generic

micha15:01:48

it has to do with what happens on partial commits

micha15:01:08

like when you push a release to the repo there are multiple artifacts that need to be uploaded

micha15:01:35

foo-1.2.3.pom, foo-1.2.3.pom.asc, foo-1.2.3.jar, etc

micha15:01:52

and it's possible that something fails before uploading all of them

micha15:01:46

there is a metadata file that describes all the versions, which can get out of sync with the actual files in the repo

micha16:01:23

so currently you can download jars from repos that don't have those versions listed in the metadata.xml file

micha16:01:46

because the jars made it into the repo but the metadata didn't

micha16:01:31

the idea for atomic deploys, as i understand it, is to use the metadata.xml file to commit a transaction

micha16:01:55

so the release won't be live until all artifacts have been uploaded and verified according to metadata.xml

meow16:01:57

atomic commits is the goal then

meow16:01:24

metadata.xml is needed to commit an atomic transaction

meow16:01:07

clarify for me: isn't a metadata file referencing versions that are out of sync a problem independent of atomic commits?

meow16:01:47

and a missing metadata.xml file is the ultimate in borken

meow16:01:05

@micha: how would you like to proceed? what's the first step? what can I do to help?

tcrawley18:01:37

@meow: correct - bad metadata is its own problem, but needs to be fixed before atomic deploys can be implemented

meow20:01:34

@jonahbenton: any suggestions?

jonahbenton20:01:04

sounds like @micha has the right of it. for problems like this i like approaches like https://github.com/clojars/clojars-web/issues/226#issuecomment-142270596. having the same tool be able to handle an atomic move from a dmz-like upload area to the canonical coordinates, as well as be able to validate/clean/fix artifacts within the canonical tree, probably makes sense. the plan is to continue to keep the repo on a file system?

tcrawley21:01:42

@jonahbenton: the plan is to move the repo off to a cloud file store in the near future

tcrawley21:01:51

but that could be the last step in the verify pipeline

jonahbenton21:01:16

@tcrawley: cool. so a tool that verified a group of file system resources for correctness and also delivered an atomic move of said resources to cloud could be utilized to populate the cloud file store with verified artifacts from the current repo, and also handle new artifact uploads going forward?

tcrawley21:01:10

@jonahbenton: everything in the current repo will end up in the cloud store, correct or not, though we will run a tool across the repo to fix what we can before hand