For me too! Thanks 🙂

is it valid to remove that semicolon in other shells?

I don’t think the shell is relevant here. At least for me the issue is present in bash and zsh too.

well what is relevant? that is, this does work now for lots of people, so what determines whether it works or not for different people?

I thought you meant that this “fixes” fish shell, but maybe breaks bash (or others). Maybe I misunderstood. I did some reading, and experimenting. If I understand it correctly, with the semicolon, the IFS variable remains : until printing the map . When removing the semicolon it doesn’t really get changed? And perhaps the classpath check isn’t really working? (not sure what’s happening there) Maybe it should be like this?

# Are .jar files in classpath missing?
  oldIFS="$IFS"
  IFS=':'; read -ra cp_entries <<<"$(cat $cp_file)"
  for cp_entry in "${cp_entries[@]}"; do
    if [[ "$cp_entry" == *.jar && ! -f "$cp_entry" ]]; then
      stale=true
      break
    fi
  done
  IFS="$oldIFS"

the intent here is that using it in the same line means that it is set only for the subcommand in the read and set back after the read

I mean you don't have to figure it out if you don't know already, but this is work I'll have to do

and I'm trying to figure out why it doesn't work on your machine so I can at least verify that I've fixed it after, and I still don't understand that

seems like maybe this has to do with the scope of that IFS setting and the intent not being satisfied that this only is in effect on that one read call

I see. If you echo "$IFS" on the line below I expect you to see it is still :. And that it doesn’t really just apply to that statement.

and then that IFS setting affects printing later on some shells?

I’m not sure why it does, but seems so, yes.

I’m on GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin21) - could it be related to the bash version? As it is bash script after all, and I guess what shell you run it from doesn’t matter.

I created a script to reproduce it, and it works fine in GNU bash, version 5.1.4(1)-release (arm-unknown-linux-gnueabihf)

I mean, the colon doesn’t disappear there, but it does when running it on my mac.

I'm running a newer bash than that and it works fine for me

but yes, I suspect it's likely bash version related

yes, upgrading bash on my mac removes the problem 🎉

I've captured this here: https://clojure.atlassian.net/browse/TDEPS-233

I try not to depend too much on having a recent version of bash though so I will look at it eventually

Cool 🙂

I've just experienced this same symptom for v1.11.1.1182 on GitHub Actions on macOS.

Running clojure -Sdescribe as a run step seems to work fine. But when I spawn the same from a bb script to capture output the colons are missing.

released that in 1.11.1.1189

Thanks @alexmiller! New version works as expected on GitHub Actions macOS.

If deps.edn changes, would it recompute the cached data?

yes

I thought it would but wanted to check.

the stale check hits all the deps.edn files, any local manifest files, and will also become stale now on missing .jar files

Ah, right.

if you catch it again, throw a -Sverbose on there to see the cp_file and grab that set of files, then -Sforce and compare

OK, I'll remove the -Sforce I've added to prevent the problem and I'll keep an eye on it...

seems unlikely you'd hit a collision with no aliases though, so maybe worth thinking about other ways you could see that result

or restating, if that was the collision, seems like you'd see it all the time

for example, if you missing a file system fsync and did not see the just written file at all, you'd have the same result?

mac is well known to cheat fsync for example and not actually flush necessarily

This is Windows/WSL2.

if you retry without force, do you still get the bare path?

if so, I presume the file contents has the shorter path and the problem is in building the classpath (or it's a cksum collision)

After a force, it produces the right classpath for quite a while. I need to figure out what action somewhere in the system is causing the overwrite. Will keep you posted. It happens pretty often.

OK, it happened today. Here's the verbose output trying to run the CLI after it created the wrong path etc:

(/Developer/workspace/wsmain)-(!2092)-> (cd clojure/ && clojure -Sverbose -Spath -A:build:dev:everything:runner:test)
version      = 1.11.1.1182
install_dir  = /var/www/worldsingles/build/clojure
config_dir   = /home/sean/.clojure
config_paths = /var/www/worldsingles/build/clojure/deps.edn /home/sean/.clojure/deps.edn deps.edn
cache_dir    = .cpcache
cp_file      = .cpcache/1422105138.cp

src:/home/sean/.m2/repository/org/clojure/clojure/1.11.1/clojure-1.11.1.jar:/home/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.62/core.specs.alpha-0.2.62.jar:/home/sean/.m2/repository/org/clojure/spec.alpha/0.3.218/spec.alpha-0.3.218.jar

Wed Nov 02 17:33:45
(/Developer/workspace/wsmain)-(!2093)-> (cd clojure/ && tar cvzf /mnt/c/Users/seanc/Downloads/tda_debug.tgz .cpcache/1422105138.*)
.cpcache/1422105138.basis
.cpcache/1422105138.cp
.cpcache/1422105138.libs

LMK if this helps:

Running it with force fixes the problem:

(/Developer/workspace/wsmain)-(!2095)-> (cd clojure/ && clojure -Sverbose -Sforce -Spath -A:build:dev:everything:runner:test)
version      = 1.11.1.1182
install_dir  = /var/www/worldsingles/build/clojure
config_dir   = /home/sean/.clojure
config_paths = /var/www/worldsingles/build/clojure/deps.edn /home/sean/.clojure/deps.edn deps.edn
cache_dir    = .cpcache
cp_file      = .cpcache/1422105138.cp

Refreshing classpath
... giant classpath removed ...

(/Developer/workspace/wsmain)-(!2096)-> ls -l clojure/.cpcache/1422105138.*
-rw-r--r-- 1 sean sean 395258 Nov  2 17:39 clojure/.cpcache/1422105138.basis
-rw-r--r-- 1 sean sean  48481 Nov  2 17:39 clojure/.cpcache/1422105138.cp
-rw-r--r-- 1 sean sean    597 Nov  2 17:39 clojure/.cpcache/1422105138.jvm
-rw-r--r-- 1 sean sean 211646 Nov  2 17:39 clojure/.cpcache/1422105138.libs
-rw-r--r-- 1 sean sean  13604 Nov  2 17:39 clojure/.cpcache/1422105138.manifest

did this start happening with the latest release?

I hadn't seen this happening until recently so I think it's a new failure mode, yes.

This is why I was asking about the hash (crc) thing because the cache file has the "right" number for the aliases etc -- it's always 1422105138 for that invocation (when it works) but sometimes it seems to act as if it ignored the aliases (after calculating the correct hash/crc).

I haven't looked at the shell script, but is it possible that two concurrent runs of the CLI script could interfere with each other? Does it use any fixed temporary file paths, anything like that?

I ask because this seems to coincide with me restarting LSP in VS Code which I have configured to a CLI script to generate the classpath because of the workspace / repo structure I have.

it may be related to the other issue that's being discussed re IFS and I've just released 1.11.1.1189 if you want to bump to that and see if things change

OK, I'll make a note to try that tomorrow at work.

Since it's happening every day or two, I should have a reasonable level of comfort deciding whether that's a fix or not 🙂

really, that's the only change that I can think of that would be related to this