I think there’s a problem in 1.10 😞

Debugging under JDI seems to be broken because of a change in the behaviour of DynamicClassLoader.

Although I can’t figure it out.

I’ve been working with the debugger guy at JetBrains, and he said: > For some reason java.lang.Object is not reported as visible from the DynamicClassLoader used in evaluation, so jdi method args types checks fail. In 1.09 it does not happen. Definitely some changes in 1.10 trigger that.

I’ve asked him for some clarification on what exactly he means by “not visible”, but he’s inconveniently asleep.

Looking at the change history, there were only two changes to the DCL in 1.10, one making a change for CLJ-1550 and the other reverting it, so AFAICT it shouldn’t have changed. Could there be something with the bytecode changes that affect how it responds?

not to my knowledge

other than the update in min jdk / asm, can’t think of anything else that changed

@cfleming you can test that bytecode theory by testing before and after the change, which was.... beta4?

certainly the classloader structure of java changed in java 9

This is still under Java 8.

k

@ghadi Ok, I’ll try that, hopefully that doesn’t overlap the range where the DCL had actually changed.

the change was between beta4 and beta5

The DCL change, or the bytecode one?

the bytecode one

Ok, so beta4 has the old code?

I’ll test that.

I'm sorry @cfleming , I meant between alpha4-alpha5

not beta

@ghadi any possibility of the custom getCommonSuperClass() being related?

Ok, I can probably do a bisect on 1.10 versions actually, it’s relatively quick to test, but I’ll try alpha4/alpha5 first.

I don't know @alexmiller, but that call is only called during bytecode emission, not runtime

yeah, I can’t construct a theory of how it would be in play, just thinking of the edge stuff

That might affect this - when evaluating expressions, I construct an fn, compile it and invoke it.

The problem is that Egor reproduced the problem, but I don’t have any more information from him than just that message from earlier.

waiting with bated breath

It’ll be in a little bit, I’m still doing morning childsitting and multitasking 🙂

i don't think getCommonSuperClass is related either...

yeah, I can’t see how

Works with alpha4, busted with alpha5

😞

I’ll bisect the changes in alpha5 when I get in to the office.

@ghadi First bad commit is 38705b49fd3dbae11e94c576ef49ff3eb1c47395, which is the getCommonSuperClass change.

Ok hmmm. Let us know what the debugger expert says, too

Looking at it, that also seems to be the commit when opcodes were changed from 1.5 to 1.8, so it’s possible that change affected something else, i.e. that causes JDI to work differently with the newer bytecode version.

@ghadi Was that the main commit which changed the bytecode version, or was the version also changed in other places in other commits?

that was it

one commit for the asm bump, one commit for emitting jdk8+

and asm bump happened in two separate commits due to a bug in the first version of the asm library

1650f9c2546a6fc71c8d74b29e18547d6696fc54 "CLJ-2367 Incorporate ASM merge request 189 and add case tests" right before the alpha6 release

@dpsutton I can’t see anything in the diff for that changing opcode versions.

yeah, alpha 5 broke something for us, as I recall and that was my patch to apply the fix from the ASM project to the ASM code in Clojure... so we went from alpha 4 to alpha 6...

I just brought it up because it changes the asm library which emits bytecode. but it happened after alpha5 which you say is already broken for debugging. so it appears to be after the root cause anyways

Yeah - I can’t see where that commit changes the asm lib, but it’s definitely after the root cause anyway.

Ok, I’ll ask Egor about those two changes, and I’ll get some detail from him to repro the problem myself.

I’m not sure I have the knowledge of the gory bits of JDI to understand it, but I’ll see what he says.

dpsutton you're muddling the issue... the asm bump had a minor emission bug the first time around

shouldn't affect anything @cfleming is looking into

AFAHCT, the salient change is between alpha4 and alpha5

that's already been bisected

I think the thing to look at is how the compiled bytecode differs before/after

This could also easily be a bug in asm

@alexmiller good idea, I’ll check that.

That getCommon... is used for computing stack maps so maybe there is something weird with those. the before doesn’t have stack maps as those were added after the old bytecode version

Yeah, if I had to bet a beer, it would be on something with the stack maps.

Those things are really tricky.

But the verifier is usually good at catching problems AFAIK.

I did actually do a fair amount of bytecode diffing before/after trying to catch anything obvious

@ghadi has a script to revendor asm - could just revendor to latest and see if it works

JDI and classloaders are runtime -- this doesn't sound like a bytecode problem

https://asm.ow2.io/versions.html

Well the class loader loads bytecode

I'm curious what the debug-guru says

yeah but how could you be missing Object?

¯\(ツ)/¯

I don’t have a theory

Other than that the bytecode is different across the change

The issue is that JDI doesn’t load the Object class - I got that far, basically the debugger loops endlessly on expression evaluation since it receives a ClassNotLoadedException. JDI normally takes care of that, so I guess JDI doesn’t like the class for some reason. I don’t understand how the DCL interaction happens though.

does it work on other JVMs? like 11

I don’t know, I can try that now.

https://gist.github.com/cursive-ide/56addd4fd34b12bc7c01e107447d7cdf

That’s the bytecode diff. It looks reasonable to me, but I’m no expert.

Some constant pool differences, the version numbers and the StackMapTable entries.

Still broken on OpenJDK 11