Fork me on GitHub
#babashka
<
2022-10-20
>
teodorlu09:10:40

Hi! I deleted ~/.m2 because some trouble with a java dependency, and now my babashka script fails with:

$ bb_http_server.clj || cat `which bb_http_server.clj`
----- Error --------------------------------------------------------------------
Type:     java.nio.file.NoSuchFileException
Message:  /home/teodorlu/.m2/repository/org/babashka/http-server/0.1.11/http-server-0.1.11.jar
Location: /home/teodorlu/.local/bin/bb_http_server.clj:7:1
Any advice on what I can do to get back into a working state? Is there a cache babashka uses somewhere that I can delete, perhaps?

teodorlu09:10:49

edit: i'm going to try change the script to a different version of babashka.http-server, and see if that triggers a redownload.

borkdude09:10:48

-Sforce, same as Clojure CLI. This problem has been posted a few messages back as well

👀 1
borkdude09:10:07

Can you maybe shrink the original message a bit, it's flooding the channel

👍 1
teodorlu09:10:49

Posting the original message here in a thread instead:

$ bb_http_server.clj || cat `which bb_http_server.clj`
----- Error --------------------------------------------------------------------
Type:     java.nio.file.NoSuchFileException
Message:  /home/teodorlu/.m2/repository/org/babashka/http-server/0.1.11/http-server-0.1.11.jar
Location: /home/teodorlu/.local/bin/bb_http_server.clj:7:1

----- Context ------------------------------------------------------------------
3: (require '[babashka.deps :as deps])
4: (deps/add-deps
5:  '{:deps {org.babashka/http-server {:mvn/version "0.1.11"}}})
6:
7: (require '[babashka.http-server :as http-server])
   ^--- /home/teodorlu/.m2/repository/org/babashka/http-server/0.1.11/http-server-0.1.11.jar
8:
9: (apply http-server/-main *command-line-args*)

----- Stack trace --------------------------------------------------------------
sun.nio.fs.UnixFileAttributeViews/Basic      - <built-in>
java.util.zip.ZipFile/Source                 - <built-in>
java.util.zip.ZipFile/CleanableResource      - <built-in>
babashka.impl.classpath/path-from-jar        - <built-in>
babashka.impl.classpath.Loader/fn--13316     - <built-in>
clojure.core/some                            - <built-in>
babashka.impl.classpath/source-for-namespace - <built-in>
babashka.main/exec/load-fn--31119            - <built-in>
user                                         - /home/teodorlu/.local/bin/bb_http_server.clj:7:1

#!/usr/bin/env bb

(require '[babashka.deps :as deps])
(deps/add-deps
 '{:deps {org.babashka/http-server {:mvn/version "0.1.11"}}})

(require '[babashka.http-server :as http-server])

(apply http-server/-main *command-line-args*)

borkdude09:10:14

Perhaps deps.clj could be more clever here and check that all directories exist on the classpath, to invalidate the cache. This should be a relatively fast operation and easy to do in Clojure (as opposed to bash?). Pinging @U064X3EF3 to see what he thinks of this.

👀 1
teodorlu09:10:31

Looks like I'm getting the same error with -Sforce:

$ bb -Sforce .local/bin/bb_http_server.clj
----- Error --------------------------------------------------------------------
Type:     java.nio.file.NoSuchFileException
Message:  /home/teodorlu/.m2/repository/org/babashka/http-server/0.1.11/http-server-0.1.11.jar
Location: /home/teodorlu/.local/bin/bb_http_server.clj:7:1

borkdude09:10:46

@U3X7174KS This is because add-deps which probably is used inside that script doesn't "obey" that flag

👀 1
borkdude09:10:10

Just wipe ~/.clojure/.cpcache and ./.cpcache and try again

💯 2
borkdude09:10:26

I'll make an issue for -Sforce + add-deps

💯 1
teodorlu09:10:26

Deleted ~/.clojure/.cpcache - and now everything is working. Thanks a lot @U04V15CAJ 🙌

teodorlu10:10:33

Tank you! 😊

Alex Miller (Clojure team)13:10:45

I’ve considered doing that classpath check but it is valid to have empty source dirs so it would need to be a bit nuanced (just looking at jars probably). I would not expect it to be a perf issue in bash (but that’s something to check)

👍 1
borkdude13:10:14

Hmm, empty source dirs. Good point.

borkdude13:10:25

Or rather, non-existing source dirs, right

borkdude14:10:17

Hah, I noticed that gitlibs are already covered by the manifests checks, so only checking jar files should be sufficient

borkdude14:10:42

That is, removing the gitlibs directory should already work today, without problems

💯 1
Alex Miller (Clojure team)15:10:46

this is kind of a separate question, but is this really a "faithful port" if you add features before the upstream?

borkdude15:10:46

That's a good question, that's why I pinged you in this issue as I expected you would have feedback on this.

borkdude15:10:55

If you think this is a bad idea, I'll remove it

borkdude15:10:18

I think another edge case here is :local/root "foo.jar" which should also not need a new classpath calculation if missing

borkdude15:10:15

so, you're right, I'll wait for the upstream then

Alex Miller (Clojure team)15:10:44

I will take a look at it, maybe today or tomorrow, not sure yet

borkdude15:10:37

ok, reverted and waiting for upstream. https://github.com/borkdude/deps.clj/commit/30bd56472daba5661d4c3f549c2e6572caed57ae at least we got the edge cases more clearly

borkdude09:10:16

@U064X3EF3 I see you implemented the .jar checks so I'll go ahead and re-add the code I reverted https://github.com/clojure/brew-install/commit/f791abf1d93563c1ed8f256830bd0bfc085fdd53 Have you thought about the :local/root "foo.jar" case, which does allow missing jar files? It's a bit of a pathological case though, so maybe worth ignoring

borkdude09:10:55

Another minor issue: can you maybe change the default branch in that repo to 1.11.1? Then we can see the newest commits right away

borkdude12:10:25

As for the non-existing :local/root "foo.jar" case. the tools jar could emit a

.cpcache/01E115B1798E23AB3AB6C356BCC88293.jars
file similar to .manifests with only the existing jar files after the classpath was produced, so only that list would be checked

Alex Miller (Clojure team)12:10:09

I think it’s right to error if any jar is missing

Alex Miller (Clojure team)12:10:15

I’ll get the default branch switched

borkdude12:10:29

@U064X3EF3 It doesn't error, it just marks the classpath cache stale. So if there would be a library on the classpath with a :local/root "non-existing.jar" this would always result in a stale classpath check, while no error would be happening similar to non-existing source dirs on the classpath

Alex Miller (Clojure team)12:10:02

It should only be stale if it’s missing?

borkdude12:10:29

@U064X3EF3 What I'm saying is: • you said it's ok when source dirs are missing • it can also be the case that jar files are missing when people use :local/root "some-jar-file.jar"- what's the difference between src and jar file - that might also be ok? why not? • someone making a mistake in their library could cause end users to always have stale classpath checks with 1177 - it seems wrong that some library's mistake could cause slowness on end users' systems

borkdude12:10:27

I'll make an example library to show the problem

Alex Miller (Clojure team)12:10:16

The “someone” and “end user” here are the same person in the use case for local jars, which is for pulling local jars into your application for something like a database driver jar

Alex Miller (Clojure team)12:10:34

I understand what you’re saying, don’t need an example

Alex Miller (Clojure team)12:10:55

I just don’t think it’s an issue

borkdude12:10:28

OK. I see the program is erroring with a missing local/root jar, so that's covered then:

$ clj
Error building classpath. Manifest type not detected when finding deps for missing-jar/missing-jar in coordinate #:local{:root "/private/tmp/missing-jar/missing-jar.jar"}

Alex Miller (Clojure team)12:10:45

Yeah, that’s what I was expecting

👍 1
borkdude12:10:18

Cool. I think -Sforce will probably be never needed anymore after this change 🤞

Alex Miller (Clojure team)12:10:50

Yeah, should be much less relevant

Alex Miller (Clojure team)12:10:33

Use of RELEASE is maybe the biggest cause to need it after

borkdude12:10:34

Checking another corner case: This isn't supported right? {:paths ["missing-jar.jar"]} But could still trigger the edge case described above

borkdude13:10:37

If this isn't supported, might want to check if all entries in :paths are directories (if they exist) in tools.deps or still improve the stale check for missing jars?

Alex Miller (Clojure team)13:10:09

I’m ok with doing less in the paths check, sometimes there are useful hacks there. The missing jar check error could probably be better

Casey11:10:51

What is the proper way to share some variables among tasks with babashka in a bb.edn? Sort of how one would define some shared vars at the top of a makefile

borkdude11:10:25

@ramblurr

{:tasks {:init (def x 1)}}

👍 1
borkdude11:10:37

Alternatively you could make a bb/tasks.clj file and require this in your tasks and put your constants etc in there

1
borkdude11:10:44

and add :paths ["bb"]

richiardiandrea17:10:38

Hi there, when I build a uberscript I receive a bunch of

[babashka] Ignoring expression while assembling uberscript: (ns lib.deployments.gobo (:require [acuity.core3.models.enum :as enum] [com.cohesic.acuity.core2.models.location :as nil] [com.cohesic.acuity.core2.models.patient-type :as nil] [com.cohesic.acuity.core2.models.priority :as nil] [com.cohesic.acuity.core2.models.study-type :as nil])) near /home/cokap/git/cohesic/acuity/scripting/scripts/bb/corrective-fixes/report-headers.clj:7:77
[babashka] Ignoring read error while assembling uberscript near /home/cokap/git/cohesic/acuity/scripting/scripts/bb/corrective-fixes/report-headers.clj:12:99
...
Do I need to worry? The script seems to work but I am curious about what is happening there

borkdude17:10:39

Those are warnings because those expressions could not successfully be analyzed, perhaps because of :as nil?

borkdude17:10:59

But it's intended that it continues despite those warnings

richiardiandrea17:10:34

ok, I have the following in the namespace referred in the first warning [com.cohesic.acuity.core2.models.location :as-alias model.location]

richiardiandrea17:10:43

(and similar others)

borkdude17:10:14

welcome to put together a repro repository, so I can try it locally

borkdude17:10:20

could be :as-alias related

richiardiandrea18:10:55

I created this one https://github.com/arichiardi/babashka-repros But I actually found the reason of the first issue - there was no namespace backing that alias There are other warnings in there when you run bb build

borkdude18:10:21

how to build the uberscript?

richiardiandrea18:10:47

See above message, bb build

borkdude18:10:03

but there isn't even a bb.edn in this repo

richiardiandrea20:10:16

no? wait a sec let me check

richiardiandrea20:10:02

@U04V15CAJ apologies, for some reason it was not committed

borkdude20:10:12

I suspected as much :) Yes, it definitely has to do with :as-alias, thanks for the repro

Gary Berger19:10:42

i am a little confused how to use babashka.cli with bb.edn tasks framework. I simply want to parse some command options and all my logic is in the bb.edn file . Is that possible?

borkdude19:10:27

Yes. You can call (babashka.cli/parse-opts *command-line-args*) of course, but there is also tasks integration: https://book.babashka.org/#_exec

👍 2