The main issue was that offsets for GC roots on the stack were being
miscalculated because invokedynamic bootstrap methods are invoked as
static methods but were not being flagged as such.
Also, I forgot to initialize MyThread::dynamicTable in the constructor
(and why the hell doesn't `gcc -Wall` warn me about stuff like that?)
We now detect both metafactory and altMetafactory lambdas when
bootimage-compiling invokedynamic instructions, as well as make
allowance for the same lambda being invoked by multiple invokedynamic
instructions (as is the case for Serializable lambdas).
For lambdas that implement java.io.Serializable, the compiler emits
calls to LambdaMetaFactory.altMetafactory, not
LambdaMetaFactory.metafactory, so I've provided a stub implementation
that ignores that currently ignores the extra parameters it receives.
This also fixes a bug in compiling lambda glue code for lambdas that
take longs and/or doubles.
These expressions are tricky because they rely on invokedynamic, which
normally implies runtime code generation. However, since lambdas
don't actually use the "dynamicness" of invokedynamic, we can convert
them into static calls to synthetic classes at compile time.
Since I had already written code to synthesize such classes in Java
and I didn't want to rewrite it in C++, I needed to add support for
running Java code to the bootimage generator. And since the primary
VM used by the generator is purpose-built to generate AOT-compiled
code for a specific target architecture and is not capable of
generating or running JIT-compiled code for the host architecture, I
added support for loading a second, independent, host-specific VM for
running Java code.
The rest of the patch handles the fact that each method compilation
might cause new, synthetic classes to be created, so we need to make
sure those classes and their methods are included in the final heap
and code images. This required breaking some giant code blocks out of
makeCodeImage into their own methods, which makes the diff look
scarier than it really is.
The two big pieces here are basic invokedynamic support and a working
version of LambdaMetaFactory.metafactory. The latter works by
dynamically building a synthetic class with three methods: a static
factory method, a constructor for the factory method to call, and a
method to satisfy the requested interface which defers to the
specified MethodHandle.
This work relies heavily on Avian's specific MethodType and
MethodHandle implementations, which provide extra, non-standard
features to make code generation easier. That means we'll probably
need to use Avian's versions of java.lang.invoke.* even when building
with the OpenJDK or Android class libraries.
At first, it might look like the atomicIncrement operations here,
since they resolve to OSAtomicCompareAndSwap32Barrier, ought to
provide all the memory barrier guarantees we need; however, it turns
out it's not quite sufficient.
Here's why: Apple's OSAtomic*Barrier operations guarantee that memory
operations *before* the atomic op won't be reordered with memory
operations *after* the atomic op - but makes no guarantee that the
atomic op itself won't be reordered with other operations before or
after it. The key is that the atomic operation is not really atomic,
but rather consists of separate read, check and write steps - in a
loop, no less. Here, presumably, the read of t->m->exclusive is
hoisted by the out-of-order processor to between the read and write
steps of the "atomic" increment of t->m->activeCount.
Longer term, it probably makes sense to replace this with the c11
<stdatomic.h> operations or the c++11 <atomic> types. We ought to
actually use the atomic increment operations provided there. As it
is, our atomicIncrement looks substantially less efficient than those,
since it's actually implemented on arm64 as two nested loops (one in
atomicIncrement and one in the compare-and-swap) instead of one. We
should also evaluate whether the various instances of atomic
operations actually need as strong of barriers as we're giving them.
FWIW, the gcc __sync_* builtins seem to have the same problem, at
least on arm64.
This would theoretically break compatibility with apps using embedded
classpaths, on big-endian architectures - because of the size type
extension. However, we don't currently support any big-endian
architectures, so it shouldn't be a problem.
OpenJDK 8 includes a core class (java.lang.Thread) which so many
fields that it exceeds the class size limit in type-generator dictated
by the logic responsible for calculating each class's GC object mask,
at least on 32-bit systems. There was no fundamental need for this
limit -- it just made the code simpler.
This commit removes the above limit at the cost of slightly more
complicated code. The original motivation for this change is that the
platform=macosx arch=i386 openjdk=$jdk8 build was failing. However,
there doesn't seem to be a prebuild JDK 8 for 32-bit OS X anywhere on
the Internet, nor is there any obvious way to build one on a modern
Mac, so it's safe to say we won't be supporting this combination
anyway. The problem also occurs on Linux and Windows, though,
so it needs to be fixed.
So there I was, planning to just fix one little bug: Thread.holdsLock
and Thread.yield were missing for the Android class library. Easy
enough, right? So, I added a test, got it passing, and figured I'd go
ahead and run ci.sh with all three class libraries. Big mistake.
Here's the stuff I found:
* minor inconsistency in README.md about OpenSSL version
* untested, broken Class.getEnclosingMethod (reported by Josh)
* JNI test failed for tails=true Android build
* Runtime.nativeExit missing for Android build
* obsolete assertion in CallEvent broke tails=true Android build
* obsolete superclass field offset padding broke bootimage=true Android build
* runtime annotation parsing broke bootimage=true Android build
(because we couldn't modify Addendum.annotationTable for classes in
the heap image)
* ci.sh tried building with both android=... and openjdk=..., which
the makefile rightfully balked at
Sorry this is all in a single commit; I didn't expect so many
unrelated issues, and I'm too lazy to break them apart.
If an JNI_OnLoad implementation calls FindClass when using the OpenJDK
class library, the calling method on the Java stack will be
ClassLoader.loadLibrary. However, we must use the class loader of the
class attempting to load the library in this case, not the system
classloader.
Therefore, we now maintain a stack such that the latest class to load
a library in the current thread is at the top, and we use that class
whenever FindClass is called by ClassLoader.loadLibrary (via
JNI_OnLoad).
Note that this patch does not attempt to address the same problem for
the Avian or Android class libraries, but the same strategy should
work for them as well.
This ensures that all tests pass when Avian is built with an
openjdk=$path option such that $path points to either OpenJDK 7 or 8.
Note that I have not yet tried using the openjdk-src option with
OpenJDK 8. I'll work on that next.
