The problem (which we've only been able to reproduce consistently with
the openjdk-src process=interpret build on Linux virtual machines) was
a race condition during VM shutdown. Thread "A" would exit, see there
were other threads still running and thus enter ZombieState, which
involves acquiring and releasing a lock using RawMonitorResource.
Then the last thread (thread "B") would exit, wait for thread "A" to
release the lock, then shut down the VM, freeing all memory. However,
thread "A" writes to its Thread object one last time after releasing
the lock (in ~Resource, the destructor of the superclass of
RawMonitorResource, which sets Thread::resource). If thread "B" frees
that Thread before ~Resource runs, we end up writing to freed memory.
Thus, we need to update Thread::resource before releasing the lock.
Apparently C++ destructors run in order from most derived to least
derived, which is not what we want here. My solution to split
Resource into two classes, one that has no destructor and another that
extends it (called AutoResource) which does hafe a destructor. Now
all the classes which used to extend Resource extend AutoResource,
except for RawMonitorResource, which extends Resource directly so it
can control the order of operations.
This allows them to be shared between the Avian and Android class
library builds.
This commit also disables the URL test in Misc.java on Android, since
it's known to fail, and we still want to know whether the other tests
pass.
This was causing crashes on 32-bit OS X continuations=true builds.
There were two important differences between vmInvoke and
vmJumpAndInvoke: (1) vmInvoke expects its stack to be aligned on
entry, modulo the return address whereas the stack argument to
vmJumpAndInvoke is aligned without allowing for the return address,
and (2) vmInvoke pushes EBP before doing its frame allocation, whereas
vmJumpAndInvoke did not take that into account. So in order for
vmJumpAndInvoke to allocate the exact same frame size that vmInvoke
would have when calling the same method, it needed to add an extra two
words beyond what it was already allocating.
Aside from alignment concerns, the code is not particularly sensitive
to vmJumpAndInvoke allocating a different frame size than vmInvoke,
since we store the frame pointer in a "thread local" variable:
// remember this stack position, since we won't be able to rely on
// %rbp being restored when the call returns
movl 8(%ebp),%eax
movl %esp,TARGET_THREAD_SCRATCH(%eax)
...
GLOBAL(vmInvoke_returnAddress):
// restore stack pointer
movl TARGET_THREAD_SCRATCH(%ebx),%esp
My original patch makes an equivalent change for the 64-bit changes,
but I'll leave that for after we release 1.0 since we're in
bugfix-only mode right now
This was a bug, wherein upon throwing an exception, we would try to
allocate memory for the message - all while holding a critical
reference to the jbyteArray representing the exception string. This
caused an expect to fail in allocate3.
This ensures that all tests pass when Avian is built with an
openjdk=$path option such that $path points to either OpenJDK 7 or 8.
Note that I have not yet tried using the openjdk-src option with
OpenJDK 8. I'll work on that next.
617bd85 broke the Android build by creating an unresolvable
order-of-operations bug in classpath-android.cpp's
MyClasspath::preBoot method.
The problem is that, while JNIEnv::FindClass is supposed to initialize
the class that it finds, this causes JniConstants::init to indirectly
invoke native methods which are not registered until JNI_OnLoad is
called (which happens after JniConstants::init is called). However,
if we call JNI_OnLoad first, that causes methods to be invoked which
rely on JniConstants::init having already been run.
I haven't checked to see how Dalvik handles this, but I don't see any
way around the problem besides disabling initialization by
JNIEnv::FindClass until the preBoot phase is complete. Moreover, it's
dangerous to allow Java code to be invoked so early anyway, since the
VM is not yet fully initialized.
For some reason, running Avian under the SVN version of Valgrind
caused mmap to fail, which caused tryAllocateExecutable to return a
null pointer, which led to a non-obvious crash later on. Adding an
expect to check the result immediately will at least make it obvious
what went wrong.
bb86500 was a step in the right direction, but there was a bug that
caused Type_pad fields to be inserted between every other field in for
a derived class when type-maps.cpp was generated, and this led to
miscompilation of e.g. Android's
java.lang.reflect.Constructor.getModifiers.
We should define EXPORT to be __declspec(dllexport) on Windows
regardless of architecture, not just non-x86_64 arches. This fixes
errors to to embedded JAVA_HOME files not being found in openjdk-src
builds, e.g. lib/currency.data.
Although the JNI reference documentation does not mention it,
FindClass should initialize the class before it returns it. That's
what HotSpot does, and that's what we have to do too.
In particular, OpenJDK's
Java_java_net_Inet6AddressImpl_lookupAllHostAddr relies on
Inet6Address's static initializer being run when it is resolved using
FindClass, or else it will crash.
* Unsafe.arrayIndexScale was always returning the native word size,
due to a thinko on my part
* Unsafe.getLongVolatile and putLongVolatile did not work for array
elements on 32-bit systems
Turns out Function can do the jobs of both CallbackReceiver and
FunctionReceiver, so I've removed the latter two.
Also, shift and reset should work with a combination of types, not
just a single type, so I've expanded their generic signatures.
There's more work to do to derive all the properties of a given class
from its code source (e.g. JAR file), but this at least ensures that
ClassLoader.getPackage will actually return something non-null when
appropriate.
classpath-common.h's getDeclaringClass was trying to look up
non-existing classes, which led to an abort, and I don't even know
what Class.getDeclaredClasses was trying to do, but it was ugly and
wrong.
The various Architecture::nextFrame implementations were not walking
the stack correctly when a StackOverflowError was thrown. The
throwStackOverflow thunk is called before the frame of the most
recently called method has been fully created, and because tails=true
builds use a different calling convention, we need to treat this
situation carefully when building a stack trace or unwinding.
Otherwise, we will skip past all the java frames to the next native
frame, which is what was happening.
Tail call and dead code optimizations can cause code after a throw to
be eliminated, which confuses findUnwindTarget because it doesn't know
what code is throwing the exception. So we need at least one
instruction to follow the call to the throw_ thunk. Previously, we
only added such an instruction when we knew the throw was the last
instruction in the bytecode, but it turns out there are other cases
where it is needed, including certain try/finally situations.
This is the simplest possible ConcurrentHashMap I could come up with
that works and is actually concurrent in the way one would expect.
It's pretty unconventional, being based on a persistent red-black
tree, and not particularly memory-efficient or cache-friendly. I
think this is a good place to start, though, and it should perform
reasonably well for most workloads. Patches for a more efficient
implementation are welcome!
I also implemented AtomicReferenceArray, since I was using it in my
first, naive attempt to implement ConcurrentHashMap.
I had to do a bit of refactoring, including moving some non-standard
stuff from java.util.Collections to avian.Data so I could make it
available to code outside the java.util package, which is why I had to
modify several unrelated files.
getDeclaredMethods was returning methods which were inherited from
interfaces but not (re)declared in the class itself, due to the VM's
internal use of VMClass.methodTable differing from its role in
reflection. For reflection, we must only include the declared
methods, not the inherited but un-redeclared ones.
Previously, we saved the original method table in
ClassAddendum.methodTable before creating a new one which contains
both declared and inherited methods. That wasted space, so this patch
replaces ClassAddendum.methodTable with
ClassAddendum.declaredMethodCount, which specifies how many of the
methods in VMClass.methodTable were declared in that class.
Alternatively, we could ensure that undeclared methods always have
their VMMethod.class_ field set to the declaring class instead of the
inheriting class. I tried this, but it led to subtle crashes in
interface method lookup. The rest of the VM relies not only on
VMClass.methodTable containing all inherited interface methods but
also that those methods point to the inheriting class, not the
declaring class. Changing those assumptions would be a much bigger
(and more dangerous in terms of regression potential) effort than I
care to take on right now. The solution I chose is a bit ugly, but
it's safe.
An inner class has two sets of modifier flags: one is declared in the
usual place in the class file and the other is part of the
InnerClasses attribute. Not only is that redundant, but they can
contradict, and the VM can't just pick one and roll with it. Instead,
Class.getModifiers must return the InnerClasses version, whereas
reflection must check the top-level version. So even if
Class.getModifiers says the class is protected, it might still be
public for the purpose of reflection depending on what the
InnerClasses attribute says. Crazy? Yes.
This makes them available in all class libraries, not just the OpenJDK
library. Note that I've also removed the unecessary idle statements,
per ab4adef.
Android's class library uses this to find out whether the VM supports
compareAndSwapLong natively. Avian does on all platforms, so we just
return true.
There's a small optimization in compileDirectInvoke which tries to
avoid generating calls to empty methods. However, this causes
problems for code which uses such a call to ensure a class is
initialized -- if we omit that call, the class may not be
initialized and any side effects of that initialization may not
happen when the program expects them to.
This commit ensures that the compiler only omits empty method calls
when the target class does not need initialization. It also removes
commented-out code in classpath-openjdk.cpp which was responsible for
loading libmawt proactively; that was a hack to get JogAmp to work
before we understood what the real problem was.
Previously, we used "lzma:", which worked fine on Windows (where the
path separator is ";") but not on Unix-style OSes (where the path
separator is ":"). In the latter case, the VM would parse
"[lzma:bootJar]" as a path containing two elements, "[lzma" and
"bootJar]", which is not what was intended. So now we use "lzma." as
the prefix, which works on all OSes.
armv7 and later provide weaker cache coherency models than armv6 and
earlier, so we cannot just implement memory barriers as no-ops. This
patch uses the DMB instruction (or the equivalent OS-provided barrier
function) to implement barriers. This should fix concurrency issues
on newer chips such as the Apple A6 and A7.
If you still need to support ARMv6 devices, you should pass
"armv6=true" to make when building Avian. Ideally, the VM would
detect what kind of CPU it was executing on at runtime and direct the
JIT compiler accordingly, but I don't know how to do that on ARM.
Patches are welcome, though!
When calculating field offsets in the bootimage generator, we failed
to consider alignment at inheritence boundaries (i.e. the last field
inherited by from a superclass should be followed by enough padding to
align the first non-inherited field at a machine word boundary). This
led to a mismatch between native code and Java code in terms of class
layouts, including that of java.lang.reflect.Method.
The former just defers to the latter for now, since it provides
strictly weaker guarantees. Thus it's correct to use full
volatile-style barriers, though not as efficient as it could be on
some architectures.
Clang was complaining that newIp might be used uninitialized at the
bottom of our giant, unstructured compile loop, so I initialized it
with a bogus value, which means it will at least fail consistently if
Clang is right and there really is a path by which that code is
reached without otherwise initializing newIp.
It looks like the iOS 7 SDK doesn't have GCC anymore, so we need to
use clang instead. Also, thread_act.h and thread_status.h have moved,
so I updated arm.h accordingly. That might break the build for older
SDKs, but I don't have one available at the moment. If it does break,
I'll fix it.
Unsafe.compareAndSwapLong was moved from classpath-openjdk.cpp to
builtin.cpp, but the fieldForOffset helper function was not, which
only caused problems when I tried to build for ARM. This commit moves
said helper function, along with Unsafe.getVolatileLong, which also
uses it.
Most of these regressions were simply due to testing a lot more stuff,
esp. annotations and reflection, revealing holes in the Android
compatibility code. There are still some holes, but at least the
suite is passing (except for a fragile test in Serialize.java which I
will open an issue for).
Sorry this is such a big commit; there was more to address than I
initially expected.
Method.invoke should initialize its class before invoking the method,
throwing an ExceptionInInitializerError if it fails, without wrapping
said error in an InvocationTargetException.
Also, we must initialize ExceptionInInitializerError.exception when
throwing instances from the VM, since OpenJDK's
ExceptionInInitializerError.getCause uses the exception field, not the
cause field.
This is by no means a complete support for the deserialization compliant
to the Java Language Specification, but it is better to add the support
incrementally, for better readability of the commits.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Previously, we initialized it to the boot class loader, but that is
inconsistent with Java; if compiling against OpenJDK's class library,
the context class loader is therefore initialized to the app class
loader, too.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
The findResources method is supposed to enumerate all the class path
elements' matching paths' URLs, but we used to stop at the first one.
While this is good enough when the system class path contains only a
single .jar file, since b88438d2(sketch of JAR support in Finder)
supports more than a single .jar file in the class path.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When the system class path contains more than one .jar, it is quite
concievable that, say, 'META-INF/MANIFEST.MF' can be found in multiple
class path elements.
This commit teaches the working horse of class path inspection, the
Finder class, how to continue the search at a given state.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
When creating an object array with more than two dimensions, the
component type was erroneously set to the base type, not the array
type of one less dimension.
This prevented Collection<Class[]>#toArray(Class[][]) from working
properly.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This mainly involved reworking the makefile to avoid conflating
Darwin/ARM builds with iOS, since we may also want to build for the
iOS Simulator, which is i386.
Note that I was only able to test this on the Simulator, since I don't
have a real iOS device to test with. Sorry if I broke something; if
so, please fix it :)
GCC now assumes by default that the stack is aligned to a 16-byte boundary in Linux x86, so let's do our part to honour that. :)
Otherwise native code that depends on the stack to be aligned to 16 bytes would seg fault (like SSE).
https://groups.google.com/forum/#!topic/avian/SyCl-Jfw2U8
Previously, I used a shell script to extract modification date ranges
from the Git history, but that was complicated and unreliable, so now
every file just gets the same year range in its copyright header. If
someone needs to know when a specific file was modified and by whom,
they can look at the Git history themselves; no need to include it
redundantly in the header.
There were two issues: the linux->darwin cross compiler is more stringent
about unused variables, and the makefile specified flags for building ON
darwin that were actually applicable whenever we are building FOR darwin.
The original implementation was based on the assumption that the
passed class would be the array element type, whereas it is actually
the array type itself.
It may leak file handles under certain circumstances to do nothing in
JVM_UnloadLibrary, but, for now, an empty implementation is more
useful than one that aborts the process.
At one point, loading libmawt ahead of time was necessary to make AWT
work, but recent versions of OpenJDK seem to take care this from Java
code, in which case loading it ahead of time causes trouble, so we
comment it out for now until we exactly when it's needed.
Previously, we would attempt to initialize a class (e.g. call its
static initializer) whenever a method in that class was called, as
well as in any of the cases listed in
http://docs.oracle.com/javase/specs/jls/se7/html/jls-12.html#jls-12.4.
However, the above approach may lead to deadlock in an app which
relies on being able to call non-static methods in parallel with a
static initializer invocation in the same class. Thus, this commit
ensures that we initialize classes only in the cases defined by the
standard.
Since park does not throw InterruptedException, we must leave the flag
set if we are interrupted while parked so that
e.g. AbstractQueuedSynchronizer can itself throw an exception if
appropriate.
This mainly moves several sun.misc.Unsafe method implementations from
classpath-openjdk.cpp to builtin.cpp so that the Avian and Android
builds can use them.
It also replaces FinalizerReference.finalizeAllEnqueued with a no-op,
since the real implementations assumes too much about how the VM
handles (or delegates) finalization.
We must be in the Active state, not the Exclusive state when calling
this method since it may execute arbitrary Java code. This fixes an
assertion failure in makeNew.
scalac may generate bytecode such that an exception is thrown within
the bounds of a handler for that exception such that the throw is the
last instruction in the method, which we weren't handling properly.
The InnerClasses attribute may have entries for classes declared
inside classes we don't care about, so we must check each entry's
outer class reference and make sure it matches the one we do care
about.
scalac may emit a ldc followed by an i2c, whereas javac does the
conversion (including zero extension if necessary) at compile time.
This commit ensures we handle the i2c case properly.
If sun.java.command or sun.java.launcher are set, then the VM is being
loaded from e.g. libjvm.so, not as a stand-alone executable. This
commit allows libjvm.dylib to be used with OpenJDK's java command on OS
X.
The OpenJDK library wants to track and run the shutdown hooks itself
rather than let the VM do it, so we need to tell it when we're
exiting.
Also, in machine.cpp we need to use only the modifiers specified in
the InnerClasses attribute for inner classes rather than OR them with
the flags given at the top level of the class file.
scalac may generate a ldc followed by an l2i, whereas javac always
seems to condense this into a single ldc_w. The former exposed a bug
in the JIT compiler which we never hit with javac-generated bytecode.
This fixes a couple of tests in the Scala test suite
(run/reflection-modulemirror-toplevel-badpath.scala and
run/reflection-constructormirror-nested-good.scala).
This is necessary to ensure that new threads do not start while we're
shutting down (except for the ones that we start to run the shutdown
hooks), and that the shutdown hook threads can be safely started (it
is not safe to start threads from e.g. an idle state, and an assertion
will fail if we do).
This ensures that, if an exception is thrown later but before the
method has been fully compiled, we will know exactly how much memory
to free. Previously, we would abort when trying to free the wrong
amount due to an assertion failure.
It's amazing to me that ebp and esp have been swapped for over three
years without anybody noticing. It was dumb luck that the Trace test
(which is designed to catch just such a thing) happened to fail when I
ran the whole suite, and further investigation revealed that it was
failing maybe five percent of the times it was run. Now we know why.
Timezone code was broken in the Android class library bootimage build
because the code we use to intercept loading the tzdata file wasn't
working. The reason is have no way of intercepting static methods at
runtime in the bootimage build without telling the bootimage-generator
we're going to do it ahead of time. So now we do tell it so.
This commit also removes the need to intercept Thread methods since we
can update Thread.vmThread in VMThread.create instead.
In order for a thread to enter the "exclusive" state such that no
other threads are active in the VM, it must wait for all active
threads to enter the "idle" state. In order for this to happen in a
timely manner, threads must check frequently to see if a thread is
waiting to enter the exclusive state. These checks happen at every
memory allocation, wait, sleep, native call, etc. However, if a
thread is in a busy loop that does none of those things, it will block
any other thread from entering that state.
The proper way to address this is to detect such loops (or tail
recursion in tail-call-optimized builds) at compile or interpret time
and insert explicit checks. This hasn't been a high priority thus
far, though, since we had yet to encounter such code in the wild.
Now, however, we find that scala.concurrent.forkjoin.ForkJoinPool.scan
(and possibly some versions of java.util.concurrent.ForkJoinPool.scan,
on which we assume the former is based) has just such a loop.
Fortunately, that loop calls Unsafe.getObjectVolatile, which the VM
implements and thus can treat as a checkpoint. That's the workaround
we use in this patch.
Setting this property (e.g. -Davian.trace.port=5555) will cause the VM
to start an extra daemon thread which listens on the specified TCP
port for incoming connections and dumps stack traces for all running
threads to that socket. You can retrieve that dump using e.g. netcat:
nc localhost 5555
Objects which are eligable for finalization must be retained until
after their finalize methods are called. However, the VM must
determine the entire set of such objects before retaining any of them;
otherwise the process of retaining a given object may cause others to
become reachable and thus be considered ineligible for finalization
even though they are only reachable via other finalizable objects.
The end result of this mistake is that only a few of the objects which
are finalizable will be recognized at each GC cycle, so it requires
many such cycles to find them all, and if new objects become
finalizable at a faster rate, the VM will never catch up and
eventually run out of memory.
This patch fixes the above mistake and also includes tuning to
minimize the need for GC in low memory situations.
In the OpenJDK library, ThreadGroup maintains an array of all Threads
in that group, so the VM must explicitly remove threads as they exit
or else neither they nor any objects they reference will be eligable
for GC.
The original goal was to minimize memory usage by garbage collecting
more frequently and more comprehensively as we got closer to the heap
limit. In practice, though, this just slowed the VM to a crawl as
memory pressure increased. If an app really wants to use a lot of
memory, the VM shouldn't penalize it aside from throwing an
OutOfMemoryError if it exceeds the limit.
This is necessary to avoid name conflicts on various platforms. For
example, iOS has its own util.h, and Windows has a process.h. By
including our version as e.g. "avian/util.h", we avoid confusion with
the system version.
In type-generator, we were incorrectly calculating field offsets where
a class inherits from another class whose last field has a natural
alignment which is different from the native word size. Surprisingly,
this only popped up when I built using the Android class library on a
64-bit system.
Previously, if you forgot to use RUNTIME_ARRAY_BODY to reference an
array declared with (THREAD_)RUNTIME_ARRAY, you wouldn't get a
compiler error until you tried to build on e.g. MSVC, where
runtime-sized stack arrays aren't supported. This change ensures you
find out regardless of what compiler you're using, which ought to
protect us from regressions going forward.
It now builds and links, but fails at runtime because
register_libcore_icu_ICU can't find the file it wants. We'll probably need to replace register_libcore_icu_ICU with a better-behaved version.
Stuff compiles, but linking breaks spectacularly. Next step is to
figure out how to build the dependencies without checking out and
building the entire Android platform.
The eventual intent with the lir namespace is to formalize some of
the important bits of Assembler interface, to be tested, debug-printed,
and potentially, serialized.
Also, group arguments to apply(...) in OperandInfos
The primary motivation behind this is to allow all the different Assemblers
to be built at once, on a single machine. This should dramatically reduce
the time required to make sure that a particular change doesn't break
the build for one of the not-so-common architectures (arm, powerpc)
Simply pass "codegen-targets=all" to make to compile all
src/codegen/<arch>/assembler.cpp.
Note that while these architectures are built, they will not be fully-
functional. Certain stuff is assumed to be the same across the entire
build (such as TargetBytesPerWord), but this isn't the case anymore.
If we increment the value while we're still in IdleState, another
thread may try to GC before we are able to enter ActiveState, which
will lead to an assertion failure when the footprint function is
called.
