This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2022-11-01
Channels
- # announcements (10)
- # asami (2)
- # babashka (10)
- # beginners (55)
- # biff (37)
- # calva (9)
- # cherry (1)
- # clj-kondo (11)
- # clojure (221)
- # clojure-bay-area (12)
- # clojure-europe (77)
- # clojure-hungary (3)
- # clojure-nl (5)
- # clojure-norway (12)
- # clojurescript (11)
- # cursive (1)
- # data-science (11)
- # emacs (27)
- # figwheel (3)
- # fulcro (11)
- # graphql (5)
- # helix (7)
- # honeysql (3)
- # humbleui (9)
- # interceptors (2)
- # introduce-yourself (2)
- # kaocha (12)
- # lsp (27)
- # malli (6)
- # nbb (70)
- # off-topic (6)
- # re-frame (6)
- # react (3)
- # reitit (9)
- # releases (2)
- # scittle (29)
- # shadow-cljs (26)
- # sql (13)
- # tools-deps (61)
I ran into the same issue on my Mac Seems like the culprit is the semicolon at this line? https://github.com/clojure/brew-install/blob/9c515c07314f9e1458c3425128105a73577c17c3/src/main/resources/clojure/install/clojure#L370
is it valid to remove that semicolon in other shells?
I don’t think the shell is relevant here. At least for me the issue is present in bash and zsh too.
well what is relevant? that is, this does work now for lots of people, so what determines whether it works or not for different people?
I thought you meant that this “fixes” fish shell, but maybe breaks bash (or others). Maybe I misunderstood.
I did some reading, and experimenting. If I understand it correctly,
with the semicolon, the IFS variable remains :
until printing the map . When removing the semicolon it doesn’t really get changed? And perhaps the classpath check isn’t really working? (not sure what’s happening there)
Maybe it should be like this?
# Are .jar files in classpath missing?
oldIFS="$IFS"
IFS=':'; read -ra cp_entries <<<"$(cat $cp_file)"
for cp_entry in "${cp_entries[@]}"; do
if [[ "$cp_entry" == *.jar && ! -f "$cp_entry" ]]; then
stale=true
break
fi
done
IFS="$oldIFS"
Disclaimer: I don’t know what. I’m doing 😅the intent here is that using it in the same line means that it is set only for the subcommand in the read and set back after the read
I mean you don't have to figure it out if you don't know already, but this is work I'll have to do
and I'm trying to figure out why it doesn't work on your machine so I can at least verify that I've fixed it after, and I still don't understand that
seems like maybe this has to do with the scope of that IFS setting and the intent not being satisfied that this only is in effect on that one read call
I see. If you echo "$IFS"
on the line below I expect you to see it is still :
. And that it doesn’t really just apply to that statement.
and then that IFS setting affects printing later on some shells?
I’m on GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin21)
- could it be related to the bash version? As it is bash script after all, and I guess what shell you run it from doesn’t matter.
I created a script to reproduce it, and it works fine in GNU bash, version 5.1.4(1)-release (arm-unknown-linux-gnueabihf)
I'm running a newer bash than that and it works fine for me
but yes, I suspect it's likely bash version related
I've captured this here: https://clojure.atlassian.net/browse/TDEPS-233
I try not to depend too much on having a recent version of bash though so I will look at it eventually
Running clojure -Sdescribe
as a run step seems to work fine. But when I spawn the same from a bb script to capture output the colons are missing.
released that in 1.11.1.1189
Thanks @alexmiller! New version works as expected on GitHub Actions macOS.
Interesting, @athos! I'll try it out as soon as I get back to my mac 🙂 @pithyless that could be, but it works fine with fish on linux 🤷
@alexmiller I'm seeing something very odd with the CLI/deps cache -- I run this command to generate a classpath at work:
clojure -Spath -A:build:dev:everything:runner:test
but every now and then, instead of returning the full classpath as expected (with hundreds of entries), I get back just the bare Clojure deps as if I had run clojure -Spath
(with no aliases).
Is it possible there's a hash collision for the filenames used in .cpcache
? If I blow away .cpcache
(or use -Sforce
), it produces the full classpath again (of course).yes, it's possible
it's actually basically using a crc checksum of the concat'ed alias names (simplifying a little), certainly could be non-unique
on that particular example, I get a string value of val=||:build:dev:everything:runner:test||||||/usr/local/Cellar/[email protected]/1.11.1.1182/deps.edn|/Users/alex.miller/.clojure/deps.edn|deps.edn
which yields cksum of 3992683027
If deps.edn
changes, would it recompute the cached data?
I thought it would but wanted to check.
the stale check hits all the deps.edn files, any local manifest files, and will also become stale now on missing .jar files
Ah, right.
if you catch it again, throw a -Sverbose on there to see the cp_file and grab that set of files, then -Sforce and compare
OK, I'll remove the -Sforce
I've added to prevent the problem and I'll keep an eye on it...
seems unlikely you'd hit a collision with no aliases though, so maybe worth thinking about other ways you could see that result
or restating, if that was the collision, seems like you'd see it all the time
for example, if you missing a file system fsync and did not see the just written file at all, you'd have the same result?
mac is well known to cheat fsync for example and not actually flush necessarily
This is Windows/WSL2.
if you retry without force, do you still get the bare path?
if so, I presume the file contents has the shorter path and the problem is in building the classpath (or it's a cksum collision)
After a force, it produces the right classpath for quite a while. I need to figure out what action somewhere in the system is causing the overwrite. Will keep you posted. It happens pretty often.
OK, it happened today. Here's the verbose output trying to run the CLI after it created the wrong path etc:
(/Developer/workspace/wsmain)-(!2092)-> (cd clojure/ && clojure -Sverbose -Spath -A:build:dev:everything:runner:test)
version = 1.11.1.1182
install_dir = /var/www/worldsingles/build/clojure
config_dir = /home/sean/.clojure
config_paths = /var/www/worldsingles/build/clojure/deps.edn /home/sean/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1422105138.cp
src:/home/sean/.m2/repository/org/clojure/clojure/1.11.1/clojure-1.11.1.jar:/home/sean/.m2/repository/org/clojure/core.specs.alpha/0.2.62/core.specs.alpha-0.2.62.jar:/home/sean/.m2/repository/org/clojure/spec.alpha/0.3.218/spec.alpha-0.3.218.jar
Wed Nov 02 17:33:45
(/Developer/workspace/wsmain)-(!2093)-> (cd clojure/ && tar cvzf /mnt/c/Users/seanc/Downloads/tda_debug.tgz .cpcache/1422105138.*)
.cpcache/1422105138.basis
.cpcache/1422105138.cp
.cpcache/1422105138.libs
I'll attach that tgz file...LMK if this helps:
Running it with force fixes the problem:
(/Developer/workspace/wsmain)-(!2095)-> (cd clojure/ && clojure -Sverbose -Sforce -Spath -A:build:dev:everything:runner:test)
version = 1.11.1.1182
install_dir = /var/www/worldsingles/build/clojure
config_dir = /home/sean/.clojure
config_paths = /var/www/worldsingles/build/clojure/deps.edn /home/sean/.clojure/deps.edn deps.edn
cache_dir = .cpcache
cp_file = .cpcache/1422105138.cp
Refreshing classpath
... giant classpath removed ...
and you can see the sizes of the files now which are expected:
(/Developer/workspace/wsmain)-(!2096)-> ls -l clojure/.cpcache/1422105138.*
-rw-r--r-- 1 sean sean 395258 Nov 2 17:39 clojure/.cpcache/1422105138.basis
-rw-r--r-- 1 sean sean 48481 Nov 2 17:39 clojure/.cpcache/1422105138.cp
-rw-r--r-- 1 sean sean 597 Nov 2 17:39 clojure/.cpcache/1422105138.jvm
-rw-r--r-- 1 sean sean 211646 Nov 2 17:39 clojure/.cpcache/1422105138.libs
-rw-r--r-- 1 sean sean 13604 Nov 2 17:39 clojure/.cpcache/1422105138.manifest
did this start happening with the latest release?
I hadn't seen this happening until recently so I think it's a new failure mode, yes.
This is why I was asking about the hash (crc) thing because the cache file has the "right" number for the aliases etc -- it's always 1422105138 for that invocation (when it works) but sometimes it seems to act as if it ignored the aliases (after calculating the correct hash/crc).
I haven't looked at the shell script, but is it possible that two concurrent runs of the CLI script could interfere with each other? Does it use any fixed temporary file paths, anything like that?
I ask because this seems to coincide with me restarting LSP in VS Code which I have configured to a CLI script to generate the classpath because of the workspace / repo structure I have.
it may be related to the other issue that's being discussed re IFS and I've just released 1.11.1.1189 if you want to bump to that and see if things change
OK, I'll make a note to try that tomorrow at work.
Since it's happening every day or two, I should have a reasonable level of comfort deciding whether that's a fix or not 🙂
really, that's the only change that I can think of that would be related to this
but in a particular project the only thing varying there in this case is the concatenated alias string
Should this be reported in some way?