Fork me on GitHub
#cryogen
<
2021-10-12
>
dorab01:10:14

With the v0.4.0 release of deps-new, (thanks @seancorfield), I have a mostly-working cryogen template using deps-new. Currently, it generates the exact same files as the lein template does. Currently, the git repo is only on my local machine, not on github. Some questions: 1. What should the template be named? Currently I am using io.github.bombaywalla/depscryogen. I think the template should be hosted under cryogen-project, but what should be the name? If we just use cryogen, that will conflict with the lein template of the same name. 2. Eventually, the template should also be on clojars, right? If so, what would be the groupId be? 3. Under the src folder in the cryogen lein template, there is a file called markup.clj. It appears to me that that file is not used anywhere and is not placed in the project generated by the lein template. Would someone verify whether what I said is correct? Discussion under a thread please.

😻 1
seancorfield02:10:26

@U0AT6MBUL deps-new templates cannot conflict with lein (or clj-new) because they work completely differently and would have different coordinates on Clojars.

seancorfield02:10:12

Coordinates for lein have lein-template in their coordinates. Note that this would be a new project on Clojars and therefore cannot be published to the old cryogen group since that cannot be a Verified Group Name.

seancorfield02:10:29

I think the group ID will have to be org.cryogenweb and that will need verifying but it is a domain name the project owns so that should be straightforward.

seancorfield02:10:20

The artifact ID could just be template or deps-template or deps-project perhaps.

seancorfield02:10:53

The coordinates are what you use to get the template on the classpath. The namespace is what deps-new uses to locate the template.edn file. It could be org.cryogenweb.new (and the :template name would be org.cryogenweb/new I believe since that must be qualified to indicate it is external).

Jakub Holý (HolyJak)08:10:07

Awesome job, @U0AT6MBUL! I like the org.cryogenweb.new namespace proposal. Regarding location - would it make sense to include the template's source in a subdirectory of https://github.com/cryogen-project/cryogen, e.g. deps-new-template, published as a separate project (perhaps org.cryogenweb/deps[-new]-template?). I think it would be nice if the lein and deps templates were colocated because typically when we change the one we will also need to change the other. Actually perhaps we would want both templates to share the same https://github.com/cryogen-project/cryogen/tree/master/src/leiningen/new/cryogen/ since this is just copied in the produced project as-is I believe and maintaining 2 duplicated versions of it would be an issue. @U0DJK1VH6 Could you be so kind and talk to #clojars to create the https://github.com/clojars/clojars-web/wiki/Verified-Group-Names org.cryogenweb so that we can use it for new libraries? And, for the sake of simplicity, we should perhaps migrate all libraries to start using it instead of just cruogen ?

dorab18:10:42

Thanks for the explanations and discussion. My takeaway is: • @U0DJK1VH6 will get the org.cryogenweb domain verified at clojars. • The namespace of the template will be org.cryogenweb.new and the template name will be org.cryogenweb/new. • The groupId on clojars will be org.cryogenweb and the artifactId will be deps-new-template. • The sources files will be somewhere under the cryogen-project/cryogen repo on github. Does that sound correct? Have I missed anything?

1
seancorfield18:10:07

If the template is git-accessible, you don't really need to publish it to Clojars since you can run deps-new with a git dep. Clojars just makes it slightly easier.

seancorfield18:10:39

clojure -Sdeps '{:deps {io.github.cryogen-project/cryogen {:git/tag "v1.2.3" :git/sha "456abc0" :deps/root "template"}}}' -Tnew :template org.cryogenweb/new :name myname/myblog
vs
clojure -Sdeps '{:deps {org.cryogen/deps-new-template {:mvn/version "1.2.3"}}}' -Tnew :template org.cryogenweb/new :name myname/myblog

seancorfield18:10:28

(if it went in a separate github repo under cryogen-project you wouldn't need :deps/root in that first invocation -- that's only needed to navigate into the repo for a project that is not at the root)

dorab20:10:00

Thanks @seancorfield. Very helpful.

dorab22:10:44

I have just pushed a deps-template branch to github (and a corresponding draft PR). I have tested this locally using

clojure -Sdeps '{:deps {c/c {:local/root "cryogen/depstemplate"}}}' -Tnew create :template org.cryogenweb/new :name test2/myblog
I had to restructure the files under the lein template. I have verified that lein and deps-new generate the same files in the new project.

🔥 1
dorab22:10:26

However, I noticed that there is a markup.clj file in the lein template that is NOT copied to the new project. Not sure if that is intentional or not. @U0522TWDA would you please take a look? FWIW, the deps-new template does copy it over.

👍 1
dorab22:10:13

I have not put in any instructions on how to deploy this template to clojars.

dorab22:10:40

Any comments / suggestions welcome. I was very pleased by the declarative succinctness offered by deps-new. Thanks @seancorfield.

seancorfield22:10:11

I can confirm that works for me as follows:

(! 1070)-> clojure -Sdeps '{:deps {io.github.cryogen-project/cryogen {:git/sha "66414dd85747e651a05088378d7de07b0c12e848" :deps/root "depstemplate"}}}' -Tnew create :template org.cryogenweb/new :name myname/myblog
Creating project from org.cryogenweb/new in myblog

Wed Oct 13 15:25:26
(sean)-(jobs:0)-(~/clojure/fresh)
(! 1071)-> ls myblog/
content		deps.edn	project.clj	src		themes

Wed Oct 13 15:25:28
(sean)-(jobs:0)-(~/clojure/fresh)
(! 1072)-> tree myblog/
|____
| |____.gitignore
| |____content
| | |____asc
| | | |____pages
| | | | |____adoc-page.asc
| | | |____posts
| | | | |____2014-10-10-adoc-post.asc
| | |____config.edn
| | |____css
| | | |____example.css
| | | |____sassexample.scss
| | |____img
| | | |____cryogen.png
| | |____md
| | | |____pages
| | | | |____about.md
| | | | |____another-page.md
| | | |____posts
| | | | |____2014-03-10-first-post.md
| | | | |____2014-11-04-second-post.md
| | | | |____2020-12-03-docs.md
...

seancorfield22:10:47

(I have not checked any of the substitutions in the files BTW!)

dorab22:10:34

I don't believe the lein template does any either! I first wanted to reproduce what the lein template did. And then, we can try to make it better. Thanks for testing the draft PR.

seancorfield23:10:37

I 💜 that the CLI/`deps.edn` lets you run stuff from branches and pull requests! I have several things in my environment that are running from non-master/non-release git repos 🙂

👍 1
Jakub Holý (HolyJak)09:10:03

@U0AT6MBUL exploring the code, markup.clj is an orphan. It was used years ago here https://github.com/cryogen-project/cryogen/blob/e3b35eeb343fa6cdc61d11f9b7cd000b719b7c26/src/leiningen/new/cryogen/src/cryogen/compiler.clj#L89 but then most of the files were moved to cryogen-core, including a much evolved markup.clj (we now load all markups on the classpath here https://github.com/cryogen-project/cryogen-core/blob/master/src/cryogen_core/plugins.clj#L13 - both markup plugins have a resources/plugin.edn pointing to their init fn, which registers the markup in the global registry). Conclusion: Delete the file!

🙏 1
dorab16:10:45

I have removed the file and pushed the change to github.

⚰️ 1
dorab16:10:32

Given that we have just one git repo for both the lein template and the deps-new template, how / what do we deploy to clojars? I understand that the current repo is published as is to cryogen/lein-template on clojars. Do we also publish the same repo, as is, to org.cryogenweb/new ? Since the deps-new template is under the depstemplate directory, would that be enough?

Jakub Holý (HolyJak)17:10:01

I am not sure what deps-new needs. I assumed we would jar the content of the depstemplate directory (and include the said directory from the lein template, though it does not harm anything being there, only perhaps confusing users)

dorab17:10:36

I think what deps-new needs is just for the template.edn to be on the classpath. The problem I am expecting is that the way I have the :transform set up, it uses relative paths to get at the lein template files. That may not work with the way the files are laid out as of now. I have to think some more about where to place these files so that both lein and deps-new work. If anyone has suggestions please let me know. In the meantime I will try out different options. Thanks.

👍 1
seancorfield17:10:52

Probably at this point, the easiest thing to do would be: 1. move /depstemplate/resources to just /resources and move /depstemplate/deps.edn to /deps.edn 2. adjust the template.edn paths to reflect that 3. modify the lein template build to include /resources into the JAR (might happen automatically?) 4. then the deps-new invocation would just need cryogen/lein-template on the classpath but the :template name would stay the same 5. folks could also depend on the repo via git deps (without needing the :deps/root at that point)

seancorfield17:10:24

Yes, it might be slightly confusing that you specify clojure -Sdeps '{:deps {cryogen/lein-template {:mvn/version "RELEASE"}}}' -Tnew create :template org.cryogen/new :name myname/myblog but deps-new doesn't care how the files get onto the classpath, just that they are present.

dorab18:10:09

Thank you @seancorfield. Makes sense. Let me try that out and push to github.

dorab01:10:27

I made the changes, pushed to github. It seems to work in the local repo. But when I make a jar and then try to use that, deps-new is unable to find the template.edn. Here is a transcript that shows the problem:

Dorabs-iMac:repro dorab$ lein new cryogen orig-lein
Generating fresh 'lein new' Cryogen project.
Dorabs-iMac:repro dorab$ git clone [email protected]:/cryogen-project/cryogen 
Cloning into 'cryogen'...
remote: Enumerating objects: 2824, done.
remote: Counting objects: 100% (255/255), done.
remote: Compressing objects: 100% (146/146), done.
remote: Total 2824 (delta 102), reused 171 (delta 59), pack-reused 2569
Receiving objects: 100% (2824/2824), 938.17 KiB | 1.57 MiB/s, done.
Resolving deltas: 100% (1243/1243), done.
Dorabs-iMac:repro dorab$ cd cryogen
Dorabs-iMac:cryogen dorab$ git fetch
git Dorabs-iMac:cryogen dorab$ git checkout deps-template
Branch 'deps-template' set up to track remote branch 'deps-template' from 'origin'.
Switched to a new branch 'deps-template'
Dorabs-iMac:cryogen dorab$ lein jar
Created /Users/dorab/Projects/repro/cryogen/target/lein-template-0.6.6.jar
Dorabs-iMac:cryogen dorab$ cd ..
Dorabs-iMac:repro dorab$ clojure -Sdeps '{:deps {c/c {:local/root "cryogen"}}}' -Tnew create :template org.cryogenweb/new :name test/new-deps
Creating project from org.cryogenweb/new in new-deps
Dorabs-iMac:repro dorab$ diff -r orig-lein/ new-deps/
Dorabs-iMac:repro dorab$ clojure -Sdeps '{:deps {c/c {:local/root "cryogen/target/lein-template-0.6.6.jar"}}}' -Tnew create :template org.cryogenweb/new :name test/new-deps-jar
Execution error (ExceptionInfo) at org.corfield.new/create (new.clj:52).
Unable to find template.edn for org.cryogenweb/new

Full report at:
/var/folders/lk/mf8f6ghs0zqcw70nmy739khm0000gn/T/clojure-13754515118472280647.edn
Dorabs-iMac:repro dorab$ 
Any ideas where to look?

dorab01:10:46

I looked at the jar, and there seems to be a org/cryogenweb/new/template.edn in the jar.

dorab01:10:02

I also checked to confirm that the jar was in the classpath (using -Spath).

seancorfield01:10:52

Repro'd locally. Interesting. I'll debug it and see what's up.

dorab01:10:13

Thanks. I'll be curious to know what you find.

seancorfield02:10:44

Ah, yes, of course. Since I'm using tools.build it is all geared around actual files on the filesystem -- deps-new can't work with a JAR file. So it actually has to be used via a git dep (since that creates a checked out file tree locally, which tools.build can copy files and directories from).

seancorfield02:10:25

So I wrote the "find template" logic with that in mind and it walks the classpath looking for directories and physical files based on those.

seancorfield02:10:32

I'd forgotten about that (working on too many different projects!) until I just opened the file up and looked at the source code. Sorry I sent you on a wild goose chase with the JAR.

seancorfield02:10:35

But at least you can easily use the cryogen template via git deps (and moving those files around makes both that and :local/root easier since you don't need :deps/root or to worry about a path into the repo).

dorab02:10:43

Thanks for the explanation. I'll update the cryogen template docs. And push to github.

seancorfield02:10:50

(! 1094)-> clojure -Sdeps '{:deps {io.github.cryogen-project/cryogen {:git/sha "f7a0355c7ccc93b7ae21c3c707a2d452fc2de4c2"}}}' -Tnew create :template org.cryogenweb/new :name myname/myblog
Checking out:  at f7a0355c7ccc93b7ae21c3c707a2d452fc2de4c2
Creating project from org.cryogenweb/new in myblog
And that can be replaced with a short SHA and a :git/tag when you make a release on GH.

dorab02:10:31

Hmm.... I guess there is no way to say "latest release" in a git dep? Otherwise, I won't know what SHA to put in the doc before I commit the doc!

seancorfield02:10:07

Correct. You have to update the docs after making the release.

seancorfield02:10:29

That's true for all deps.edn projects that rely on git deps.

seancorfield02:10:11

It's also something the http://cljdoc.org team are aware of and have a proposed solution for building docs from source on GH only, instead of requiring a JAR on Clojars.

dorab02:10:32

@U0522TWDA The PR is now ready to review and merge. After that I can make a release. Thanks to @seancorfield for lots of help guidance and debugging in getting this out.

👍 1
seancorfield02:10:15

I'll give this some thought from a usability p.o.v. but this is intentionally close-to-the-`tools.build`-metal and leverages the CLI tooling as much as possible for as little effort (on deps-new's part) as possible.

seancorfield02:10:54

The coordinates for artifacts on the classpath have zero to do with the template name -- because you can have an arbitrary number of templates in a single artifact. But there probably could be some concept of a "default" template by searching for something specific on the classpath (such as a deps-new.edn file that specifies the template name?) but the theoretical danger is that you might get multiple entries on a classpath and not be able to control which is the default...

dorab17:10:06

Version 0.6.6 of the template is released on Github. I added a PR for a commit to update the short SHA in the README. @U0522TWDA would you please review and merge the PR. Also, if you have the access rights, would you please create the jar file off of the 0.6.6 tag and publish to clojars. Thanks.

🎉 1
seancorfield17:10:56

Confirmed that the following works:

(! 1099)-> clojure -Sdeps '{:deps {io.github.cryogen-project/cryogen {:git/tag "0.6.6" :git/sha "fcb2833"}}}' -Tnew create :template org.cryogenweb/new :name myname/myblog
Checking out:  at fcb28339e642fe038b2b535902111981d7309e27
Creating project from org.cryogenweb/new in myblog
It is more common for tags to have a v prefix on the version (to the point that clj-new, deps-new, and build-clj all assume that to be the case when generating new projects and writing POMs).

seancorfield17:10:32

(just something to bear in mind if you ever try to use build-clj with Cryogen, instead of lein -- you can override the default behavior easily enough)

Jakub Holý (HolyJak)17:10:37

@U0DJK1VH6 or @U050CBXUZ have the access to make the release

dorab20:10:40

Yeah. I kept the "0.6.6" versioning to be consistent with the past. But perhaps we should switch over to "v0.6.7" next time?

dorab20:10:25

@U0522TWDA would you please review and merge PR #246 so that the README reflects the proper short SHA. Thanks.

dorab23:10:25

Thanks for the PR merge. Whenever @U050CBXUZ or @U0DJK1VH6 can push 0.6.6 to clojars, we will be all set.

yogthos16:10:23

nice work and 0.6.6 should be up on Clojars 🎉

gratitude 1
❤️ 1