Fork me on GitHub

hi, Please advice on what I am doing wrong:

- echo "prepare"
    - clojure -Sdeps '{:mvn/local-repo ".m2"}' -Spath -A:prod
    - echo "-----------------------------------------------"
    - ls -lah .m2
    - clojure -Sdeps '{:mvn/local-repo ".m2"}' -X:uberjar :classpath '"'$(clojure -Sdeps '{:mvn/local-repo ".m2"}' -Spath -A:prod)'"'
On second command (when building uberjar) all deps were downloaded again (But I do expect them already be in the local .m2 directory) thanks!


Are you sure it's your :prod deps that are re-downloaded? I would expect the depstar deps to be downloaded in that second command because you had not downloaded them before. You need clojure -Sdeps '{:mvn/local-repo ".m2"}' -Spath -A:prod:uberjar for that.


Actually I think you are right, sorry for bothering


It works locally but still loads all deps on CI

clojure -Sdescribe
clojure -Sdeps '{:mvn/local-repo ".m2"}' -A:prod -Spath
echo "-----------------------------------------------"
clojure -Sdeps '{:mvn/local-repo ".m2"}' -A:uberjar -Spath
echo "-----------------------------------------------"
clojure -Sdeps '{:mvn/local-repo ".m2"}' -X:uberjar :classpath '"'$(clojure -Sdeps '{:mvn/local-repo ".m2"}' -A:prod -Spath)'"'


{:deps {;; Platform and system logic
        aero/aero                           {:mvn/version "1.1.6"}
        buddy/buddy-sign                    {:mvn/version "3.2.0"}
        ch.qos.logback/logback-classic      {:mvn/version "1.2.3"}
        clj-htmltopdf/clj-htmltopdf         {:mvn/version "0.1-alpha7"} {:mvn/version "0.3.2"}
        com.novemberain/pantomime           {:mvn/version "2.11.0"}
        cryptohash-clj/cryptohash-clj       {:mvn/version "0.1.6"}
        expound/expound                     {:mvn/version "0.8.7"}
        haversine/haversine                 {:mvn/version "0.1.1"}
        integrant/repl                      {:mvn/version "0.3.2"}
        integrant/integrant                 {:mvn/version "0.8.0"}
        metosin/spec-tools                  {:mvn/version "0.10.4"}
        org.apache.pdfbox/pdfbox            {:mvn/version "2.0.22"}
        org.apache.pdfbox/pdfbox-tools      {:mvn/version "2.0.22"}
        org.bouncycastle/bcprov-jdk15on     {:mvn/version "1.67"}
        org.clojure/clojure                 {:mvn/version "1.10.1"}
        org.clojure/core.async              {:mvn/version "1.3.610"}
        org.clojure/core.cache              {:mvn/version "1.0.207"}
        org.clojure/data.csv                {:mvn/version "1.0.0"}
        org.clojure/tools.cli               {:mvn/version "1.0.194"}

        ;; Database and SQL
        hikari-cp/hikari-cp       {:mvn/version "2.13.0"}
        honeysql/honeysql         {:mvn/version "1.0.444"}
        lambdaisland/uri          {:mvn/version "1.4.54"}
        migratus/migratus         {:mvn/version "1.3.3"}
        nilenso/honeysql-postgres {:mvn/version "0.4.112"}
        org.clojure/java.jdbc     {:mvn/version "0.7.11"}
        org.postgresql/postgresql {:mvn/version "42.2.18"}

        ;; HTTP and web
        buddy/buddy-auth      {:mvn/version "2.2.0"}
        clj-http/clj-http     {:mvn/version "3.10.3"}
        hiccup/hiccup         {:mvn/version "1.0.5"}
        metosin/reitit        {:mvn/version "0.5.10"}
        org.clojure/data.json {:mvn/version "1.0.0"}
        ring/ring-defaults    {:mvn/version "0.3.2"}
        ring/ring-json        {:mvn/version "0.5.0"}
        ring/ring             {:mvn/version "1.8.2"}

        ;; Integration with 3rd parties
        circleci/rollcage       {:mvn/version "1.0.208"}
        clj-commons/iapetos     {:mvn/version "0.1.11"}
        io.prometheus/simpleclient_hotspot {:mvn/version "0.11.0"}
        clj-commons/clj-ssh     {:mvn/version "0.5.15"}
        com.sun.mail/javax.mail {:mvn/version "1.6.2"}
        ovotech/clj-gcp         {:mvn/version "0.6.11"}}

 :paths ["src/clj" "src/cljc" "resources"]

 {:dev {:extra-paths ["env/dev/clj"]
        :extra-deps  {clj-commons/clj-yaml        {:mvn/version "0.7.0"}
                      criterium/criterium         {:mvn/version "0.4.6"}
                      pogonos/pogonos             {:mvn/version "0.1.1"}
                      ubergraph/ubergraph         {:mvn/version "0.8.2"}
                      vlaaad/reveal               {:mvn/version "1.3.196"}

                      ;; to be deleted?
                      binaryage/devtools {:mvn/version "0.9.10"}}}

  :prod {:extra-paths ["env/prod/clj"]}

  :repl {:extra-paths ["env/dev/resources"]
         :ns-default  cockpit.repl
         :jvm-opts    ["-Dvlaaad.reveal.prefs={:font-size 30 :font-family \"Roboto\"}"]}

  :test {:extra-paths ["test" "test-resources"]
         :extra-deps {clj-http-fake/clj-http-fake {:mvn/version "1.0.3"}
                      nubank/matcher-combinators  {:mvn/version "3.1.4"}
                      cloverage/cloverage           {:mvn/version "1.2.2"}
                      lambdaisland/kaocha {:mvn/version "1.0.887"}
                      lambdaisland/kaocha-cloverage {:mvn/version "1.0.75"}
                      lambdaisland/kaocha-junit-xml {:mvn/version "0.0.76"}}
         :main-opts  ["-m" "kaocha.runner"]}

  :uberjar {:replace-deps {com.github.seancorfield/depstar {:mvn/version "2.1.278"}}
            :exec-fn      hf.depstar/uberjar
            :exec-args    {:jar             "cockpit.jar"
                           :aot             true
                           :main-class      cockpit.server}}

  :frontend {:replace-paths ["src/cljs" "src/cljc"]
             :deps  {thheller/shadow-cljs        {:mvn/version "2.9.10"}
                     camel-snake-kebab           {:mvn/version "0.4.1"}
                     cljs-ajax                   {:mvn/version "0.8.0"}
                     cljs-http                   {:mvn/version "0.1.45"}
                     com.andrewmcveigh/cljs-time {:mvn/version "0.5.2"}
                  {:mvn/version "0.1.6"}
                     metosin/reitit              {:mvn/version "0.3.10"}
                     re-frame                    {:mvn/version "1.2.0"}
                     reagent                     {:mvn/version "1.0.0"}
                     tongue                      {:mvn/version "0.3.0"}
                     fork                        {:mvn/version "2.2.4"}
                     phrase                      {:mvn/version "0.3-alpha4"}
                     binaryage/oops              {:mvn/version "0.7.0"}}
             :main-opts  ["-m" "shadow.cljs.devtools.cli"]}}}


please advice what iI am doing wrong


:version ""


Without access to your systems, I have no idea. This is really a tools.deps issue and I saw you'd posted on and Alex couldn't repro based on the information you have provided so far.


One thing I'll observe about your deps.edn file is that -A:prod does almost nothing: it simply adds one extra path.


Indeed I asked Alex, but thats something different


> One thing I’ll observe about your `deps.edn` file is that `-A:prod` does almost nothing: it simply adds one extra path. indeed


> Without access to your systems, I have no idea. This is really a `tools.deps` issue and I saw you’d posted on and Alex couldn’t repro based on the information you have provided so far. is there a way to debug what is happening? maybe some verbose mode so I can ask Alex with more input?


OK, I ran these commands locally -- deliberately using two different local repos -- and the depstar invocation did not download anything extra:

543  clojure -Sdeps '{:mvn/local-repo "foo"}' -P -A:prod
  544  clojure -Sdeps '{:mvn/local-repo "bar"}' -P -A:uberjar
  545  clojure -Sdeps '{:mvn/local-repo "bar"}' -X:uberjar :classpath '"'$(clojure -Sdeps '{:mvn/local-repo "foo"}' -Spath -A:prod)'"' 2>&1 > /tmp/x
Note that I have -Spath before -A:prod, not after it.


I'm also using -P for "prepare" and, again, before -A:...


thanks for trying


As several people have said, -P / -Spath should be before -A:... so I would suggest making that change to double-check the behavior.


already checking this as we speak


thank you for your advices, but ci still downloads it which tools-deps version you have locally? or


I'm using


(I'm always on the latest version)


maybe it version specific will try on monday Sean, thank you so much for your help! 🙏


I just repeated my tests with (I have lots of versions installed) and it still worked.


That drives me crazy


Then indeed it is my ci setup


That made me go back over the depstar code and I think there is a path through that would create a basis without respecting the :mvn/local-repo.


I realized my tests were "working" because I still have my ~/.m2/repository and I wouldn't notice if it was reaching into that for deps.


I've never noticed this before because people normally have :mvn/local-repo in their deps.edn file rather than on the command-line.


The issue is that it needs a basis for synchronizing the pom.xml file and it has to compute that -- the :classpath alone is not enough. But it preemptively calculates it rather than only calculating it if it needs it -- and it doesn't need it in your case because you are not sync'ing the pom file.


Ah, that's not sufficient. AOT needs a basis as well, so it would still have to calculate for that.


The only way this is going to work is if you have :mvn/local-repo in your deps.edn file -- until I figure out a way to let you tell depstar about it so it can be incorporated into the basis calculations it has to do for pom and AOT...


Ah, i can create a ci alias for this


I don't think that setting can be in an alias -- it has to be at the top-level.


I just realized that t.d.a/create-basis can be passed :extra as a simulation of what -Sdeps provides so that would be the escape hatch.


If I expose that in depstar, you wouldn't need the :classpath stuff -- you could just pass :aliases '[:prod]' :mvn/local-repo '".m2"' and let depstar handle it all.


OK change your :uberjar alias to depend on com.github.seancorfield/depstar {:sha "40f2bd719a026a89d6a37b273ca72efc5d52c93b"} and then the following should work as you expect:

clojure -Sdeps '{:mvn/local-repo ".m2"}' -X:uberjar :mvn/local-repo '".m2"' :aliases '[:prod]'
I'll leave the ticket open until a) you've confirmed it actually works and b) I've properly documented this new option.


(you'll still want the two -P commands to prepare the deps for the cache)


I will! Thanks! But ut is almost 1am now, will try it tomorrow or monday


it worked!!! thank you so much!


Cool. I'll get it fully documented and a new release out, probably early next week.