There's more work to do to derive all the properties of a given class
from its code source (e.g. JAR file), but this at least ensures that
ClassLoader.getPackage will actually return something non-null when
appropriate.
getDeclaredMethods was returning methods which were inherited from
interfaces but not (re)declared in the class itself, due to the VM's
internal use of VMClass.methodTable differing from its role in
reflection. For reflection, we must only include the declared
methods, not the inherited but un-redeclared ones.
Previously, we saved the original method table in
ClassAddendum.methodTable before creating a new one which contains
both declared and inherited methods. That wasted space, so this patch
replaces ClassAddendum.methodTable with
ClassAddendum.declaredMethodCount, which specifies how many of the
methods in VMClass.methodTable were declared in that class.
Alternatively, we could ensure that undeclared methods always have
their VMMethod.class_ field set to the declaring class instead of the
inheriting class. I tried this, but it led to subtle crashes in
interface method lookup. The rest of the VM relies not only on
VMClass.methodTable containing all inherited interface methods but
also that those methods point to the inheriting class, not the
declaring class. Changing those assumptions would be a much bigger
(and more dangerous in terms of regression potential) effort than I
care to take on right now. The solution I chose is a bit ugly, but
it's safe.
An inner class has two sets of modifier flags: one is declared in the
usual place in the class file and the other is part of the
InnerClasses attribute. Not only is that redundant, but they can
contradict, and the VM can't just pick one and roll with it. Instead,
Class.getModifiers must return the InnerClasses version, whereas
reflection must check the top-level version. So even if
Class.getModifiers says the class is protected, it might still be
public for the purpose of reflection depending on what the
InnerClasses attribute says. Crazy? Yes.
When calculating field offsets in the bootimage generator, we failed
to consider alignment at inheritence boundaries (i.e. the last field
inherited by from a superclass should be followed by enough padding to
align the first non-inherited field at a machine word boundary). This
led to a mismatch between native code and Java code in terms of class
layouts, including that of java.lang.reflect.Method.
Most of these regressions were simply due to testing a lot more stuff,
esp. annotations and reflection, revealing holes in the Android
compatibility code. There are still some holes, but at least the
suite is passing (except for a fragile test in Serialize.java which I
will open an issue for).
Sorry this is such a big commit; there was more to address than I
initially expected.
Method.invoke should initialize its class before invoking the method,
throwing an ExceptionInInitializerError if it fails, without wrapping
said error in an InvocationTargetException.
Also, we must initialize ExceptionInInitializerError.exception when
throwing instances from the VM, since OpenJDK's
ExceptionInInitializerError.getCause uses the exception field, not the
cause field.
When creating an object array with more than two dimensions, the
component type was erroneously set to the base type, not the array
type of one less dimension.
This prevented Collection<Class[]>#toArray(Class[][]) from working
properly.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Previously, I used a shell script to extract modification date ranges
from the Git history, but that was complicated and unreliable, so now
every file just gets the same year range in its copyright header. If
someone needs to know when a specific file was modified and by whom,
they can look at the Git history themselves; no need to include it
redundantly in the header.
This mainly moves several sun.misc.Unsafe method implementations from
classpath-openjdk.cpp to builtin.cpp so that the Avian and Android
builds can use them.
It also replaces FinalizerReference.finalizeAllEnqueued with a no-op,
since the real implementations assumes too much about how the VM
handles (or delegates) finalization.
We must be in the Active state, not the Exclusive state when calling
this method since it may execute arbitrary Java code. This fixes an
assertion failure in makeNew.
The OpenJDK library wants to track and run the shutdown hooks itself
rather than let the VM do it, so we need to tell it when we're
exiting.
Also, in machine.cpp we need to use only the modifiers specified in
the InnerClasses attribute for inner classes rather than OR them with
the flags given at the top level of the class file.
This fixes a couple of tests in the Scala test suite
(run/reflection-modulemirror-toplevel-badpath.scala and
run/reflection-constructormirror-nested-good.scala).
Objects which are eligable for finalization must be retained until
after their finalize methods are called. However, the VM must
determine the entire set of such objects before retaining any of them;
otherwise the process of retaining a given object may cause others to
become reachable and thus be considered ineligible for finalization
even though they are only reachable via other finalizable objects.
The end result of this mistake is that only a few of the objects which
are finalizable will be recognized at each GC cycle, so it requires
many such cycles to find them all, and if new objects become
finalizable at a faster rate, the VM will never catch up and
eventually run out of memory.
This patch fixes the above mistake and also includes tuning to
minimize the need for GC in low memory situations.
This is necessary to avoid name conflicts on various platforms. For
example, iOS has its own util.h, and Windows has a process.h. By
including our version as e.g. "avian/util.h", we avoid confusion with
the system version.
Previously, if you forgot to use RUNTIME_ARRAY_BODY to reference an
array declared with (THREAD_)RUNTIME_ARRAY, you wouldn't get a
compiler error until you tried to build on e.g. MSVC, where
runtime-sized stack arrays aren't supported. This change ensures you
find out regardless of what compiler you're using, which ought to
protect us from regressions going forward.
We must use separate va_start/va_end pairs for each call to vsnprintf
on Linux and possibly other platforms in order to avoid a crash.
Also, we need to give it room to null terminate the string at the
right point.
When GetStringCritical or GetPrimitiveArrayCritical are called, the VM
cannot risk new Java heap allocations until the corresponding release
method is called because allocations may result in GC, which cannot
happen while a string or array is pinned in memory. We already have a
check for this latter in the footprint function used during GC, but
it's best to catch the problem as early as possible.
Previously, we would blithely exceed the heap ceiling and force the
next allocation to deal with the problem, including a major GC and
possible OutOfMemoryError. As of this commit, we throw an error
immediately if we find that the allocation will push us over the
ceiling.
Commit c918cbc added this reference to ensure
sun.misc.Unsafe.getLongVolatile could be implemented efficiently on
32-bit platforms. However, I neglected to ensure the reference was
updated to point to the final class instance instead of the temporary
one used in parseClass. This led to extra memory usage and
inconsistent locking behavior, plus broken bootimage builds.
If we don't clear these references, we risk finalizing objects which
can still be reached by one of the special reference types.
It's a bit of a chicken-and-egg problem. We need to visit finalizable
objects before visiting weak references, since some of the weak
references and/or their targets may become reachable once the
finalizable objects are visited. However, that ordering means we have
no efficient way of distinguishing between objects which are reachable
from one or more normal GC roots and those which are only reachable
via the finalization queue. The solution is to clear all weak
references to finalizable objects before visiting them.
The original stub implementation just echoed back its argument, but
that confused URLClassLoader when dealing with sealed JARs --
returning a non-null value for a non-system class from
JVM_GetSystemPackage made URLClassloader think it had already loaded a
class from a package which was supposed to be sealed, resulting in
SecurityExceptions which ultimately triggered NoClassDefFoundErrors.
The solution is to only return non-null values for actual system
classes.
We weren't wrapping exceptions thrown by invoked methods in
InvocationTargetExceptions in JVM_InvokeMethod or
JVM_NewInstanceFromConstructor. Also, JVM_GetCallerClass is supposed
to ignore Method.invoke frames when walking the stack.
My earlier fix (f8e8609) was almost -- but not quite -- sufficient.
It asked the heap to mark the dead fixies too early, so some of them
were marked dead even though they ultimately survived, causing us to
clear weak JNI references when we shouldn't.
The existing code did not handle static field lookups for
synchronization on 32-bit systems, which is necessary because such
systems generally don't support atomic operations on 64-bit values.
resolveClass was correctly respecting throw_ == false if the requested
class was not found, but it still threw an exception if e.g. the
superclass was missing. Now we catch such exceptions and return null
as appropriate.
This led to fixed-position objects being considered unreachable when
they were actually still reachable, causing global weak JNI references
to be cleared prematurely, most notably leading to crashes in AWT
buffered image code.
This commit also fixes a field offset calculation mismatch in
bootimage.cpp relative to machine.cpp.
We were assuming the array element size was always the native word
size, which is not correct in general for primitive arrays, and this
led to wasted space at best and memory corruption at worst.
The first problem was that, on x86, we failed to properly keep track
of whether to expect the return address to be on the stack or not when
unwinding through a frame. We were relying on a "stackLimit" pointer
to tell us whether we were looking at the most recently-called frame
by comparing it with the stack pointer for that frame. That was
inaccurate in the case of a thread executing at the beginning of a
method before a new frame is allocated, in which case the most recent
two frames share a stack pointer, confusing the unwinder. The
solution involves keeping track of how many frames we've looked at
while walking the stack.
The other problem was that compareIpToMethodBounds assumed every
method was followed by at least one byte of padding before the next
method started. That assumption was usually valid because we were
storing the size following method code prior to the code itself.
However, the last method of an AOT-compiled code image is not followed
by any such method header and may instead be followed directly by
native code with no intervening padding. In that case, we risk
interpreting that native code as part of the preceding method, with
potentially bizarre results.
The reason for the compareIpToMethodBounds assumption was that methods
which throw exceptions as their last instruction generate a
non-returning call, which nonetheless push a return address on the
stack which points past the end of the method, and the unwinder needs
to know that return address belongs to that method. A better solution
is to add an extra trap instruction to the end of such methods, which
is what this patch does.
It seems that GCC 4.6.1 gets confused at LTO time when we take the
address of inline functions, so I'm switching them to non-inline
linkage to make it happy.
It seems that GCC 4.6.1 gets confused at LTO time when we take the
address of inline functions, so I'm switching them to non-inline
linkage to make it happy.
Apple's linker tends to remove functions which are never called, which
is not what we want for e.g. vmPrintTrace, since that function is only
intended to be called interactively from within GDB.
My previous attempt wasn't quite sufficient, since it was too late to
call join on a thread which had already exited given the code was
written to aggressively dispose of system handles as soon as the
thread exited. The solution is to delay disposing these handles until
after we're able to join the thread.
The bug here is that when a thread exits and becomes a "zombie", the
OS resources associated with it are not necessarily released until we
actually join and dispose of that thread. Since that only happens
during garbage collection, and collection normally only happens in
response to heap memory pressure, there's no guarantee that we'll GC
frequently enough to clean up zombies promptly and avoid running out
of resources.
The solution is to force a GC whenever we start a new thread and there
are at least N zombies waiting to be disposed, where N=16 for now.
There was a subtle race condition in the VM shutdown process such that
a System::Thread would be disposed after the System instance it was
created under has been disposed, in which case doing a virtual call to
System::free with that instance would potentially cause a crash. The
solution is to just call the C library version of free directly, since
that's all System::free does.
Until now, the bootimage build hasn't supported using the Java
invocation API to create a VM, destroy it, and create another in the
same process. Ideally, we would be able to create multiple VMs
simultaneously without any interference between them. In fact, Avian
is designed to support this for the most part, but there are a few
places we use global, mutable state which prevent this from working.
Most notably, the bootimage is modified in-place at runtime, so the
best we can do without extensive changes is to clean up the bootimage
when the VM is destroyed so it's ready for later instances. Hence
this commit.
Ultimately, we can move towards a fully reentrant VM by making the
bootimage immutable, but this will require some care to avoid
performance regressions. Another challenge is our Posix signal
handlers, which currently rely on a global handle to the VM, since you
can't, to my knowledge, pass a context pointer when registering a
signal handler. Thread local variables won't necessarily help, since
a thread might attatch to more than one VM at a time.
This avoids the requirement of putting the code image in a
section/segment which is both writable and executable, which is good
for security and avoids trouble with systems like iOS which disallow
such things.
The implementation relies on relative addressing such that the offset
of the desired address is fixed as a compile-time constant relative to
the start of the memory area of interest (e.g. the code image, heap
image, or thunk table). At runtime, the base pointer to the memory
area is retrieved from the thread structure and added to the offset to
compute the final address. Using the thread pointer allows us to
generate read-only, position-independent code while avoiding the use
of IP-relative addressing, which is not available on all
architectures.
This monster commit is the first step towards supporting
cross-architecture bootimage builds. The challenge is to build a heap
and code image for the target platform where the word size and
endianess may differ from those of the build architecture. That means
the memory layout of objects may differ due to alignment and size
differences, so we can't just copy objects into the heap image
unchanged; we must copy field by field, resizing values, reversing
endianess and shifting offsets as necessary.
This commit also removes POD (plain old data) type support from the
type generator because it added a lot of complication and little
value.
Internally, the VM augments the method tables for abstract classes
with any inherited abstract methods to make code simpler elsewhere,
but that means we can't use that table to construct the result of
Class.getDeclaredMethods since it would include methods not actually
declared in the class. This commit ensures that we preserve and use
the original, un-augmented table for that purpose.
Previously, we would abort the process if we encountered a truncated
multibyte character in parseUtf8NonAscii (called by the JNI method
NewStringUTF). Now we simply terminate the string at that point.
Also, assume any class which has an ancestor class which has a static
initializer needs initialization even if it doesn't have one itself,
per the Java Language Spec.
