It's amazing to me that ebp and esp have been swapped for over three
years without anybody noticing. It was dumb luck that the Trace test
(which is designed to catch just such a thing) happened to fail when I
ran the whole suite, and further investigation revealed that it was
failing maybe five percent of the times it was run. Now we know why.
Timezone code was broken in the Android class library bootimage build
because the code we use to intercept loading the tzdata file wasn't
working. The reason is have no way of intercepting static methods at
runtime in the bootimage build without telling the bootimage-generator
we're going to do it ahead of time. So now we do tell it so.
This commit also removes the need to intercept Thread methods since we
can update Thread.vmThread in VMThread.create instead.
In order for a thread to enter the "exclusive" state such that no
other threads are active in the VM, it must wait for all active
threads to enter the "idle" state. In order for this to happen in a
timely manner, threads must check frequently to see if a thread is
waiting to enter the exclusive state. These checks happen at every
memory allocation, wait, sleep, native call, etc. However, if a
thread is in a busy loop that does none of those things, it will block
any other thread from entering that state.
The proper way to address this is to detect such loops (or tail
recursion in tail-call-optimized builds) at compile or interpret time
and insert explicit checks. This hasn't been a high priority thus
far, though, since we had yet to encounter such code in the wild.
Now, however, we find that scala.concurrent.forkjoin.ForkJoinPool.scan
(and possibly some versions of java.util.concurrent.ForkJoinPool.scan,
on which we assume the former is based) has just such a loop.
Fortunately, that loop calls Unsafe.getObjectVolatile, which the VM
implements and thus can treat as a checkpoint. That's the workaround
we use in this patch.
Setting this property (e.g. -Davian.trace.port=5555) will cause the VM
to start an extra daemon thread which listens on the specified TCP
port for incoming connections and dumps stack traces for all running
threads to that socket. You can retrieve that dump using e.g. netcat:
nc localhost 5555
Objects which are eligable for finalization must be retained until
after their finalize methods are called. However, the VM must
determine the entire set of such objects before retaining any of them;
otherwise the process of retaining a given object may cause others to
become reachable and thus be considered ineligible for finalization
even though they are only reachable via other finalizable objects.
The end result of this mistake is that only a few of the objects which
are finalizable will be recognized at each GC cycle, so it requires
many such cycles to find them all, and if new objects become
finalizable at a faster rate, the VM will never catch up and
eventually run out of memory.
This patch fixes the above mistake and also includes tuning to
minimize the need for GC in low memory situations.
In the OpenJDK library, ThreadGroup maintains an array of all Threads
in that group, so the VM must explicitly remove threads as they exit
or else neither they nor any objects they reference will be eligable
for GC.
The original goal was to minimize memory usage by garbage collecting
more frequently and more comprehensively as we got closer to the heap
limit. In practice, though, this just slowed the VM to a crawl as
memory pressure increased. If an app really wants to use a lot of
memory, the VM shouldn't penalize it aside from throwing an
OutOfMemoryError if it exceeds the limit.
This is necessary to avoid name conflicts on various platforms. For
example, iOS has its own util.h, and Windows has a process.h. By
including our version as e.g. "avian/util.h", we avoid confusion with
the system version.
In type-generator, we were incorrectly calculating field offsets where
a class inherits from another class whose last field has a natural
alignment which is different from the native word size. Surprisingly,
this only popped up when I built using the Android class library on a
64-bit system.
Previously, if you forgot to use RUNTIME_ARRAY_BODY to reference an
array declared with (THREAD_)RUNTIME_ARRAY, you wouldn't get a
compiler error until you tried to build on e.g. MSVC, where
runtime-sized stack arrays aren't supported. This change ensures you
find out regardless of what compiler you're using, which ought to
protect us from regressions going forward.
It now builds and links, but fails at runtime because
register_libcore_icu_ICU can't find the file it wants. We'll probably need to replace register_libcore_icu_ICU with a better-behaved version.
Stuff compiles, but linking breaks spectacularly. Next step is to
figure out how to build the dependencies without checking out and
building the entire Android platform.
The eventual intent with the lir namespace is to formalize some of
the important bits of Assembler interface, to be tested, debug-printed,
and potentially, serialized.
Also, group arguments to apply(...) in OperandInfos
The primary motivation behind this is to allow all the different Assemblers
to be built at once, on a single machine. This should dramatically reduce
the time required to make sure that a particular change doesn't break
the build for one of the not-so-common architectures (arm, powerpc)
Simply pass "codegen-targets=all" to make to compile all
src/codegen/<arch>/assembler.cpp.
Note that while these architectures are built, they will not be fully-
functional. Certain stuff is assumed to be the same across the entire
build (such as TargetBytesPerWord), but this isn't the case anymore.
If we increment the value while we're still in IdleState, another
thread may try to GC before we are able to enter ActiveState, which
will lead to an assertion failure when the footprint function is
called.
"%*s" means "at least", whereas "%.*s" means at most, and the latter
is what I intended. This only became noticable as of 9f22a70, when I
added another directory to the library path, which caused loadLibrary
to fail to find libraries in either directory.
We must use separate va_start/va_end pairs for each call to vsnprintf
on Linux and possibly other platforms in order to avoid a crash.
Also, we need to give it room to null terminate the string at the
right point.
When GetStringCritical or GetPrimitiveArrayCritical are called, the VM
cannot risk new Java heap allocations until the corresponding release
method is called because allocations may result in GC, which cannot
happen while a string or array is pinned in memory. We already have a
check for this latter in the footprint function used during GC, but
it's best to catch the problem as early as possible.
Previously, we would blithely exceed the heap ceiling and force the
next allocation to deal with the problem, including a major GC and
possible OutOfMemoryError. As of this commit, we throw an error
immediately if we find that the allocation will push us over the
ceiling.
Scala sometimes generates bytecode such that the scope of an exception
handler starts at another exception handler, e.g.:
Exception table:
from to target type
290 372 382 any
382 451 451 any
290 372 451 any
Avian's compiler was incorrectly initializing the stack frame for the
second handler in this case. This commit fixes the problem.
The instruction for 32-bit-to-64-bit sign extension on x86_32 requires
that the input value be placed in EAX and the sign extension in EDX.
However, the compiler can get confused if the input value is in memory
addressed via one of those registers and doesn't know how to move it.
This patch works around that limitation by doing the move explicitly
in MemoryEvent::compile if necessary.
Method.invoke must throw an IllegalArgumentException if it receives
the wrong number or types of arguments, and since this isn't done by
the OpenJDK class library, we must do it in the VM.
This library is placed in the xawt subdirectory of jre/lib/$arch on
POSIX systems, so it isn't found automatically when third-party
libraries which depend on it are loaded. The simplest way to ensure
that it's found seems to be to just load it when the VM starts up.
In order to calculate the initial stack map of GC roots for an
exception handler, we do a logical "and" of maps across all the
instructions contained in the try block for that handler. This is
complicated by the presence of jsr/ret instructions, though, because
instructions in a subroutine may have multiple maps associated with
them corresponding to all the paths from which execution might flow to
them.
The bug in this case was that we were using an uninitialized map in
our calculation, resulting in a map with no GC roots at all. By the
time the map was initialized, the damage had already been done. The
solution is to treat an uninitialized map as if it has roots at all
positions so that it has no effect on the calculation until it has
been initialized with real data.
Hi,
I did some more tests with my x86 QNX Avian port and found one major problem
in Avian VM while trying to run Apache Ivy. The problem manifests as
follows:
1. MySystem::Thread X is created, during its creation pthread mutex and
conditional variable are initialized
2. Program runs for some time
3. MySystem Thread X is disposed, it's memory is freed (during garbage
collection I guess)
4. Program runs for some time
5. MySystem::Thread Y is created in exactly the same memory address as
MySystem::Thread X disposed in step 3 (I suppose that's due to the way
memory allocator works in Avian)
6. During MySystem::Thread Y creation pthread mutex and conditional variable
initialization fail silently with EBUSY. QNX documentation says it means
"The given mutex was previously initialized and hasn't been destroyed."
which is correct, because it's exactly in the same memory address as mutex
and conditional variable of MySystem::Thread X and they haven't been
destroyed during MySystem::Thread X disposal
Fortunately solution for this is easy, see the attached patch. Now Apache
Ivy works without any problems.
Regards,
Stanisław Szymczyk
Some OSes (notably, Windows CE) restrict the size of the call stack
such that recursive compilation of branch instructions can lead to
stack overflow in methods with large numbers of such instructions. In
fact, a worst-case method could even lead to overflow when the stack
size limit is relatively generous.
The solution is to convert this recursion into iteration with an
explicit stack to maintain state about alternate paths through each
branch.
This package name must match the URL protocol we use for loading
embedded resources, but OpenJDK's URL class won't tolerate underscores
in a protocol name. Also, I had not updated the names of the native
methods in avian.avianvmresource.Handler, leading to
UnsatisfiedLinkErrors when they were called.
Commit c918cbc added a reference to ensure
sun.misc.Unsafe.getLongVolatile could be implemented efficiently on
32-bit platforms, but I forgot to update bootimage.cpp to account for
it.
Commit c918cbc added this reference to ensure
sun.misc.Unsafe.getLongVolatile could be implemented efficiently on
32-bit platforms. However, I neglected to ensure the reference was
updated to point to the final class instance instead of the temporary
one used in parseClass. This led to extra memory usage and
inconsistent locking behavior, plus broken bootimage builds.
If we don't clear these references, we risk finalizing objects which
can still be reached by one of the special reference types.
It's a bit of a chicken-and-egg problem. We need to visit finalizable
objects before visiting weak references, since some of the weak
references and/or their targets may become reachable once the
finalizable objects are visited. However, that ordering means we have
no efficient way of distinguishing between objects which are reachable
from one or more normal GC roots and those which are only reachable
via the finalization queue. The solution is to clear all weak
references to finalizable objects before visiting them.
The original stub implementation just echoed back its argument, but
that confused URLClassLoader when dealing with sealed JARs --
returning a non-null value for a non-system class from
JVM_GetSystemPackage made URLClassloader think it had already loaded a
class from a package which was supposed to be sealed, resulting in
SecurityExceptions which ultimately triggered NoClassDefFoundErrors.
The solution is to only return non-null values for actual system
classes.
We weren't wrapping exceptions thrown by invoked methods in
InvocationTargetExceptions in JVM_InvokeMethod or
JVM_NewInstanceFromConstructor. Also, JVM_GetCallerClass is supposed
to ignore Method.invoke frames when walking the stack.
My earlier fix (f8e8609) was almost -- but not quite -- sufficient.
It asked the heap to mark the dead fixies too early, so some of them
were marked dead even though they ultimately survived, causing us to
clear weak JNI references when we shouldn't.
The existing code did not handle static field lookups for
synchronization on 32-bit systems, which is necessary because such
systems generally don't support atomic operations on 64-bit values.
Recent versions of IcedTea will not run unless libjvm.so exports this
symbol. The quick fix is to provide a stub which just always returns
-1 to indicate an error. I'll leave a proper implementation for when
we need to support an app that actually uses this function.
My earlier commit to allow detaching the main thread (1f1c3c4) seems
to have caused subtle stability problems
(e.g. https://groups.google.com/group/avian/msg/d2c797c0dcf925c3), so
for now we'll just ignore that operation, which leaks a bit of memory
but should be harmless otherwise.
set java.vm.version based on makefile version=
in order to display relevant OpenJDK -version information.
Signed-off-by: Matthias Klose <doko@ubuntu.com>
Signed-off-by: Xerxes Rånby <xerxes@zafena.se>
My earlier attempt (fa5d76b) missed an important detail, and somehow I
forgot to test the 32-bit OpenJDK build which made that omission
obvious. Here's the fix.
resolveClass was correctly respecting throw_ == false if the requested
class was not found, but it still threw an exception if e.g. the
superclass was missing. Now we catch such exceptions and return null
as appropriate.
Some apps refuse to run if Runtime.maxMemory returns a value that's
"too small", so our stub implementation returning zero was not
sufficient. Now we return the actual heap size limit in bytes.
sun.misc.Launcher has its own idea about what the application
classloader should be, but we need to override it with the system
classloader created by the VM. This is achieved by running
Launcher.getLauncher (which has the side effect of setting
Thread.contextClassLoader) and then overriding it.
When I originally implemented DetachCurrentThread, I assumed it didn't
make sense for the main thread to detach itself from the VM, and I was
concerned that allowing it might cause problems for any other threads
still attached. However, detaching the main thread is allowed by the
JNI spec as of Java 2, and OpenJDK's java command does this just
before calling DestroyJavaVM. Therefore, this commit ensures that the
VM doesn't abort if the main thread is detached.
We weren't adding entries to the frame map for calls to the instanceof
thunk when compiling methods. However, that thunk may trigger a GC,
in which case we'll need to unwind the stack, which will lead to a
crash if we don't have a frame map entry for that instruction.
Java requires that NaNs be converted to zero and that numbers at or
beyond the limits of integer representation be clamped to the largest
or smallest value that can be represented, respectively.
Our implementation uses Object.wait(long) to implement Thread.sleep,
which had the side effect of interpreting zero as infinity. However,
for Thread.sleep, zero just means zero. I assume that doesn't mean
"don't sleep at all", though, or else the app wouldn't have called
Thread.sleep in the first place, so this patch sleeps for one
millisecond when zero is passed -- just enough to yield the processor
for a bit. Thread.yield might be a better choice in this case, but I
assume the app would have called that directly if that's what it
wanted.
This led to fixed-position objects being considered unreachable when
they were actually still reachable, causing global weak JNI references
to be cleared prematurely, most notably leading to crashes in AWT
buffered image code.
This commit also fixes a field offset calculation mismatch in
bootimage.cpp relative to machine.cpp.
