In order to calculate the initial stack map of GC roots for an
exception handler, we do a logical "and" of maps across all the
instructions contained in the try block for that handler. This is
complicated by the presence of jsr/ret instructions, though, because
instructions in a subroutine may have multiple maps associated with
them corresponding to all the paths from which execution might flow to
them.
The bug in this case was that we were using an uninitialized map in
our calculation, resulting in a map with no GC roots at all. By the
time the map was initialized, the damage had already been done. The
solution is to treat an uninitialized map as if it has roots at all
positions so that it has no effect on the calculation until it has
been initialized with real data.
The stack mapping code was broken for cases of stack slots being
reused to hold primitives or addresses within subroutines after
previously being used to hold object references. We now bitwise "and"
the stack map upon return from the subroutine with the map as it
existed prior to calling the subroutine, which has the effect of
clearing map locations previously marked as GC roots where
appropriate.
This test covers the case where a local stack slot is first used to
store an object reference and later to store a subroutine return
address. Unfortunately, this confuses the VM's stack mapping code;
I'll be working on a fix for that next.
The new test requires generating bytecode from scratch, since there's
no reliable way to get javac to generate the code we want. Since we
already had primitive bytecode construction code in Proxy.java, I
factored it out so we can reuse it in Subroutine.java.
Compiling the entire OpenJDK class library into a bootimage revealed
some corner cases which broke the compiler, including synchronization
in a finally block and gotos targeting the first instruction of an
unsynchronized method.
