What causes this limitation @U064X3EF3?

I'm pulling the pom license info, deps projects don't have that in a known place. Can potentially fill that gap.

I would imagine that 90% of projects that have an explicit license have a LICENSE file at the top level? GitHub does that by default when you create a new repo and select a license, and both lein new and deps-new (and clj-new) create LICENSE (EPL, but with a note in the README that you can change licenses). boot-new too, not that anyone uses that these days 🙂

for sure, but what do I do with a blob of text in a LICENSE file?

I have not spent the time yet to devise a strategy to go from that to "license name"

maybe just the first line is often sufficient

Ah, right, I hadn't thought that through very much...

it seemed worth releasing it without that before I figured that part out :)

...looking across a bunch of LICENSE files here, I suspect you could pattern match common ones from the first non-blank line of the file (which looks to be the first line in nearly all cases).

yeah, that's probably a decent hack

(! 647)-> head -1 */LICENSE
==> build-clj/LICENSE <==
                                 Apache License

==> clj-new/LICENSE <==
Eclipse Public License - v 1.0

==> deps-new/LICENSE <==
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC

==> expectations/LICENSE <==
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC

==> next.jdbc/LICENSE <==
THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE PUBLIC

==> polylith/LICENSE <==
Eclipse Public License - v 1.0

==> sean-polylith/LICENSE <==
Eclipse Public License - v 1.0

Looking across my whole machine, a lot of LICENSE files begin with a copyright line. Nearly all of those have this text, whatever license this is:

Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:

MIT.

FYI it would be better to adopt existing best practice in this area i.e. https://spdx.dev/license-list/matching-guidelines/

The folks involved in that initiative have spent a LOT of time considering this problem, including “corner cases” that are a lot more common than you might think (e.g. dual-or-more licensed software where the author has concatenated all of the licenses’ texts into a single ./LICENSE or ./COPYING or whatever file).

I'm not going to do text matching of any license text as I don't want to "pay" for maintaining or including or downloading the text to match against

Right. Just pointing out best practice, based on years of experience (others’, to clarify, not mine).

I linked this site in the prior discussion, I am aware of it

You’ll also hit the issue that many MANY Maven artifacts don’t include license information at all, either in their pom.xml or as a LICENSE or COPYING file. In fact a bunch of Clojure’s own JARs are like this too.

actually, Maven central requires license declaration in the pom and virtually all libs do include it

Clojars does not but in practice seems most I've looked at do include it

so just using pom info yields a pretty high success rate

That may be a recent thing, as there certainly are a large number of popular Maven-hosted artifacts that don’t.

it has always been a requirement on central

or "always" since I started deploying stuff there 10-15 yrs ago

other repos are more lax

Remember: perfect is the enemy of good. So unless you're going to write a patch for tools.deps.alpha, what we have is "good enough" for a lot of people I suspect 🙂

I am happy to get about two standard deviations of "good"

Pretty sure you’re incorrect about that @U064X3EF3 - here’s an example POM published today that does not contain any license information: https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-core/1.12.122/aws-java-sdk-core-1.12.122.pom

if there is another hack or two to cover the deps license files cases, at least for the ~5 most common Clojure libs, I'd take that (and want to consider a way to let people say this more deterministically without a pom)

it's in the parent pom

https://repo1.maven.org/maven2/com/amazonaws/aws-java-sdk-pom/1.12.122/aws-java-sdk-pom-1.12.122.pom

Not that it’s complete (at all), but https://github.com/pmonks/tools-licenses at least attempts to look for license texts inside deps (both Maven and git coord).

I'll get those too

It uses a similar strategy to the one you and @seancorfield were discussing above (simple text matching the header of the license), which SPDX does not recommend.

% clj -X:deps list :extra '{:deps {com.amazonaws/aws-java-sdk-core {:mvn/version "1.12.122"}}}'
com.amazonaws/aws-java-sdk-core 1.12.122  (Apache-2.0)
com.fasterxml.jackson.core/jackson-annotations 2.12.3  (Apache-2.0)
com.fasterxml.jackson.core/jackson-core 2.12.3  (Apache-2.0)
com.fasterxml.jackson.core/jackson-databind 2.12.3  (Apache-2.0)
com.fasterxml.jackson.dataformat/jackson-dataformat-cbor 2.12.3  (Apache-2.0)
commons-codec/commons-codec 1.15  (Apache-2.0)
commons-logging/commons-logging 1.2  (Apache-2.0)
joda-time/joda-time 2.8.1  (Apache 2)
org.apache.httpcomponents/httpclient 4.5.13  (Apache-2.0)
org.apache.httpcomponents/httpcore 4.4.13  (Apache-2.0)
org.clojure/clojure 1.10.3  (EPL-1.0)
org.clojure/core.specs.alpha 0.2.56  (EPL-1.0)
org.clojure/spec.alpha 0.2.194  (EPL-1.0)
software.amazon.ion/ion-java 1.0.2  (The Apache License, Version 2.0)

^^ works just fine on the one you mentioned

https://central.sonatype.org/publish/requirements/#license-information

they have rules that check this ^^

Another example (from 2018): https://repo1.maven.org/maven2/javax/mail/javax.mail-api/1.6.2/javax.mail-api-1.6.2.pom

(and yes, I realise there is license text in an XML comment at the top)

again ,parent poms have the license info

% clj -X:deps list :extra '{:deps {javax.mail/javax.mail-api {:mvn/version "1.6.2"}}}'
javax.mail/javax.mail-api 1.6.2  (CDDL/GPLv2+CE)
org.clojure/clojure 1.10.3  (EPL-1.0)
org.clojure/core.specs.alpha 0.2.56  (EPL-1.0)
org.clojure/spec.alpha 0.2.194  (EPL-1.0)

no license text matching stuff is needed imo for Maven artifacts, it's there for 99.9% of things without that

Right, which leaves other Maven repos (Clojars in particular), and git coords.

Clojars seems to be relatively high even without a requirement, so not worried about that either

mostly trying to look at local/git deps projects

but I'd rather solve this by creating a well-known place to list the equivalent to the pom fields

For that you’d likely have to rely on LICENSE or COPYING files.

Which is why https://github.com/pmonks/tools-pom exists…

Though because it can take its input from anywhere, it’s not helpful for determining licenses.

(also because tools.build task names are not standardised, so there’s no “API” for a license determination task to invoke to generate a POM and then parse that)

sure there is :)

tools.deps and tools.build can both generate and parse poms :)

Sure, but only incomplete ones.

I don't understand what we're even talking about now. to the problem hand, I can just define a well known attribute in the deps.edn to pull this from

There’s no way (as of about a month ago, when I last looked into this in some detail) to tell tools.deps / tools.build about “extra” pom.xml elements that I want included in the poms it generates*. *short of checking in an incomplete “template” pom.xml in my repo, which creates other problems for downstream tooling

you can give it a base pom and sync into it

Which messes with downstream tooling that now thinks my project is buildable with Maven.

and eventually I'll get around to making something broader there

you can call it something that's not pom.xml

How many steps am I away from solving my business problem now? 😉

well like I said, none of this has anything to do with the problem I'm trying to solve

I just wanted to make sure my project isn’t violating my organisation’s open source consumption policies, and now I’m creating thisisnotapom.xml files and learning how to tell my build tooling to consume it. 😉

That’s the thing though - these two areas are intertwined (for better or worse - I’m not advocating on behalf of how Maven pom.xml and repositories work - I’m simply stating how they work).

well I don't know what you're doing, this has nothing to do with providing a license name for a deps.edn project

Sure it does - as you yourself said, putting license information in a pom.xml file makes that use case easy. How do I do that with tools.deps / tools.build?

put it in the deps.edn file, and give it to tools.build from there?

Oh yeah - I don’t. Since that’s not readily supported yet (without checking in pom.xml fragments in my repo).

ok, bye

“Those who cannot remember the past are condemned to repeat it.” ¯\(ツ)/¯

"Which messes with downstream tooling that now thinks my project is buildable with Maven." -- that's just not true. The source file for tools.build to generate pom.xml from does not need to be called pom.xml nor does it need to be in the root of the project.

(I do this in some of my projects -- which is why I jumped on your comment as blatantly false @deactivateduser10790)

Yet that’s the default?

Others here have commented on this in the past, and I don’t have much interest in parroting their (well-founded, IMO) arguments in this regard.

And yet you do.

Not at all. I jumped into this thread to share my experience with license detection, both as someone who had responsibility for it professionally (at an open source Foundation), and as someone who recently scratched an itch because this didn’t previously exist in the tools.build ecosystem.

If you’re not interested in learning from experience, just say so. There’s no need for you or @U064X3EF3 to be dicks about it.

how do you identify licenses? And do you preserve copyright notices when using :full?

oh full vs short is “Eclipse Public License 1.0” vs “EPL-1.0"

"all" published maven deps have a license section so I'm taking from there. :full gives you whatever the project lists (this is license, not copyright) whereas :short is a canonicalized shortened form, trying to use https://spdx.org/licenses/ ids

with :format :edn you get both full name + license url

the "shortener" is literally text matching but I seeded it based on real data (and if not found, just falls back to full)

so it's super dumb, but works about 95% of the time

(doseq [p projects]
      (println "\nRunning: uber on" p)
      (let [project-dir (str (System/getProperty "user.dir") "/projects/" p)
            aliases     (with-dir (io/file project-dir) (get-project-aliases))]
        (binding [b/*project-root* project-dir]
          (bb/clean {})
          (bb/uber (assoc (:uberjar aliases)
                          :compile-opts {:direct-linking true}))
          (bb/clean {}))))

and that alias is just data that specifies the :main namespace so compile-clj and uber know what to do in tools.build

OK thx! Will look at this in my context and see how I can use it.👍

:uberjar
  {:uber-file "../../../build/uberjars/api-1.0.0.jar"
   :main api.main}

(defn- get-project-aliases []
  (let [edn-fn (juxt :root-edn :project-edn)]
    (-> (t/find-edn-maps)
        (edn-fn)
        (t/merge-edns)
        :aliases)))

t is tools.deps.alpha, b is tools.build, bb is build-clj.

And with-dir comes from t.d.a in another ns.

LMK if you have further Qs.

Is that a public project?

No, that's from our Polylith monorepo at work.

OK np - many thanks for your help!

But I think Polylith itself has a similar build.clj if I recall correctly from the PR I sent them...

...yeah, it does: https://github.com/polyfy/polylith/blob/master/build.clj

Ah ok I get it - Polylith is public. Am looking at it..

It has generic jar and uberjar tasks that read alias data from the projects/*/deps.edn files.

I modeled the PR on how we do it at work.

I suspect that may be one of the most "powerful" build.clj scripts out there in OSS land?

I think shadow just reads the aliases from the config file and then calls clojure -Spath -A:alias1:alias2 to get the classpath and then continues its course

Thx @borkdude, will see if Cider can do the same. Thank you both!

I looked into the source. It constructs a command line call to invoke a Clojure main program with the aliases in place.

I wonder if tools.build changes how we build CLJS programs in the future