We now detect both metafactory and altMetafactory lambdas when
bootimage-compiling invokedynamic instructions, as well as make
allowance for the same lambda being invoked by multiple invokedynamic
instructions (as is the case for Serializable lambdas).
For lambdas that implement java.io.Serializable, the compiler emits
calls to LambdaMetaFactory.altMetafactory, not
LambdaMetaFactory.metafactory, so I've provided a stub implementation
that ignores that currently ignores the extra parameters it receives.
This also fixes a bug in compiling lambda glue code for lambdas that
take longs and/or doubles.
Per a recent bug report in the hello-ios project, we found that
bootimage-generator would abort with no explanation if it encountered
a lambda invocation and the `-hostvm` option was unspecified. This
commit ensures that a helpful message is printed before exiting.
OpenJDK's java.util.zip.ZipFile.getEntryBytes should return a byte
array that is not null-terminated, but we were giving it one that was
null-terminated, which caused lookups to fail later when
ZipFile.getInputStream was called.
Previously, the following code would throw an IllegalMonitorStateException:
public class Test {
public static synchronized void main(String[] args) {
Test.class.notify();
}
}
The problem stems from the fact that for a long time Avian has had two
representations of a given class: avian.VMClass and java.lang.Class.
It used to be that there was only one, java.lang.Class, but that
didn't play nicely with OpenJDK's class library, so we split it into
two. Unfortunately, we forgot to update the JIT and interpreter
accordingly, so a static synchronized method would acquire the
avian.VMClass instance, whereas Foo.class.notify() would be invoked on
the java.lang.Class instance.
This commit fixes it.
The tower of patches and hacks grows higher. Ideally, we'll just drop
support for JDK 7 soon and clean this mess up a bit, but TravisCI
still hasn't gotten the memo that it's dead, so we muddle onward.
I've tested this on Windows, but not yet Linux or OS X. Wanted to get
a PR before I move on to that.
These expressions are tricky because they rely on invokedynamic, which
normally implies runtime code generation. However, since lambdas
don't actually use the "dynamicness" of invokedynamic, we can convert
them into static calls to synthetic classes at compile time.
Since I had already written code to synthesize such classes in Java
and I didn't want to rewrite it in C++, I needed to add support for
running Java code to the bootimage generator. And since the primary
VM used by the generator is purpose-built to generate AOT-compiled
code for a specific target architecture and is not capable of
generating or running JIT-compiled code for the host architecture, I
added support for loading a second, independent, host-specific VM for
running Java code.
The rest of the patch handles the fact that each method compilation
might cause new, synthetic classes to be created, so we need to make
sure those classes and their methods are included in the final heap
and code images. This required breaking some giant code blocks out of
makeCodeImage into their own methods, which makes the diff look
scarier than it really is.
This allows multiple Avian VMs to share the same process space,
provided they don't try to use functionality that involves global
shared resources (e.g. signal handling).
The two big pieces here are basic invokedynamic support and a working
version of LambdaMetaFactory.metafactory. The latter works by
dynamically building a synthetic class with three methods: a static
factory method, a constructor for the factory method to call, and a
method to satisfy the requested interface which defers to the
specified MethodHandle.
This work relies heavily on Avian's specific MethodType and
MethodHandle implementations, which provide extra, non-standard
features to make code generation easier. That means we'll probably
need to use Avian's versions of java.lang.invoke.* even when building
with the OpenJDK or Android class libraries.
This is a bunch of commits squashed into one per Josh's request.
add dynamicTable field
add invokedynamic instruction
add defaultDynamic bootimage field
add dummy invokedynamic support in bootimage-generator
add defaultDynamic thunk
check dynamicTable offset
comment defaultDynamicThunk to fix unused function
comment defaultDynamicThunk to fix unused function
add dynamicTable / dynamicIndex stuff
comment dynamicIndex and dynamicTable
add invokedynamic instruction impl
stub out addDynamic
unstub addDynamic
don't allow tail calls in invokedynamic
implement stub JVM_GetTemporaryDirectory method
(build broken) begin add InvokeDynamicTest
Revert "(build broken) begin add InvokeDynamicTest"
This reverts commit 77f9c54e32ac66d0803eeab93e4a10d3541987a8.
add InternalError
add URLClassPath.c for openjdk-src builds
implement stub JVM_KnownToNotExist and JVM_GetResourceLookupCache methods
intercept open0 / open for openjdk
add basic java/lang/invoke stubs
remove non-public java/lang/invoke classes
fix invokedynamic example building
<wip debugging>
In a bootimage=true build, we create allocate certain objects as
"immortal fixies", which means they will never been deallocated at
runtime and should only be visited if/when they point to objects which
might move during garbage collection. However, there was a bug in the
following case:
1. immortal fixie F is updated to point to a movable object M and
thus F is added to the list of fixies to visit during the next minor
collection (but not the next major one, since all reachable objects
are visited during a major collection, and there's no point in
visiting an unreachable object, whereas during a minor collection we
have to visit F because we don't know if it's reachable or not)
2. a major collection occurs, but F is not reachable and thus is not
visited, whereas M is moved
3. a minor collection occurs, and since F is still in the list, it is
visited, but since it contains a stale pointer to M's old location,
we crash
The solution is to ensure unreachable immortal fixies are removed from
the above list after each major collection, thus guaranteeing they
won't be visited on any subsequent collection.
GCC is a lot more sensitive about -Werror=unused-variable, to the
point that stuff declared in header files but unused in a given
compilation unit is flagged. This may be due to the way we're
here's the fix.
At first, it might look like the atomicIncrement operations here,
since they resolve to OSAtomicCompareAndSwap32Barrier, ought to
provide all the memory barrier guarantees we need; however, it turns
out it's not quite sufficient.
Here's why: Apple's OSAtomic*Barrier operations guarantee that memory
operations *before* the atomic op won't be reordered with memory
operations *after* the atomic op - but makes no guarantee that the
atomic op itself won't be reordered with other operations before or
after it. The key is that the atomic operation is not really atomic,
but rather consists of separate read, check and write steps - in a
loop, no less. Here, presumably, the read of t->m->exclusive is
hoisted by the out-of-order processor to between the read and write
steps of the "atomic" increment of t->m->activeCount.
Longer term, it probably makes sense to replace this with the c11
<stdatomic.h> operations or the c++11 <atomic> types. We ought to
actually use the atomic increment operations provided there. As it
is, our atomicIncrement looks substantially less efficient than those,
since it's actually implemented on arm64 as two nested loops (one in
atomicIncrement and one in the compare-and-swap) instead of one. We
should also evaluate whether the various instances of atomic
operations actually need as strong of barriers as we're giving them.
FWIW, the gcc __sync_* builtins seem to have the same problem, at
least on arm64.
This would theoretically break compatibility with apps using embedded
classpaths, on big-endian architectures - because of the size type
extension. However, we don't currently support any big-endian
architectures, so it shouldn't be a problem.
This improves support for building with openjdk=$jdk8. Note, however,
that the Processes test does not pass, since JDK 8's
java.lang.UNIXProcess uses invokedynamic, which Avian does not yet
support.
OpenJDK 8 includes a core class (java.lang.Thread) which so many
fields that it exceeds the class size limit in type-generator dictated
by the logic responsible for calculating each class's GC object mask,
at least on 32-bit systems. There was no fundamental need for this
limit -- it just made the code simpler.
This commit removes the above limit at the cost of slightly more
complicated code. The original motivation for this change is that the
platform=macosx arch=i386 openjdk=$jdk8 build was failing. However,
there doesn't seem to be a prebuild JDK 8 for 32-bit OS X anywhere on
the Internet, nor is there any obvious way to build one on a modern
Mac, so it's safe to say we won't be supporting this combination
anyway. The problem also occurs on Linux and Windows, though,
so it needs to be fixed.
In afbd4ff, I made a low-risk, but very specific fix for a more
general problem: "bootstrap" classes (i.e. classes which the VM has
built-in knowledge of) need to be loaded from the classpath before any
of their methods are called. Based on recent testing, I found there were
more cases than I previously thought where the VM tries to call methods on
"unloaded" bootstrap classes, so we needed a more general solution to
the problem.
This commit addresses it by closing the last (known) loophole by which
methods might be called on bootstrap classes: invokeinterface, and its
helper method findInterfaceMethod. The fix is to check for bootstrap
classes in findInterfaceMethod and load the full versions if
necessary. This process may lead to garbage collection and/or thrown
exceptions, which made me nervous about cases of direct or indirect
calls to findInterfaceMethod not expecting those events, which is why
I hadn't used that approach earlier. However, it turns out there were
only a few places that made non-GC-safe calls to findInterfaceMethod,
and a bit of code rearrangement fixed that.
As documented at
https://developer.apple.com/library/ios/documentation/Xcode/Conceptual/iPhoneOSABIReference/Articles/ARM64FunctionCallingConventions.html,
the ARM64 iOS ABI differs from the generic ABI in a few important
ways. Specifically, arguments passed via the stack are aligned
according to their natural alignment instead of 8 bytes. The VM's
dynamic call code was aligning each argument to 8 bytes, so native JNI
code couldn't find them in their expected places.
Also, we weren't setting the "os.arch" system property on ARM64, so I
fixed that too.
This method ends up defering to JVM_GetClassDeclaredMethods, which
creates an array of java.lang.reflect.Method instances and then
calling getName on each one through the java.lang.reflect.Member
interface. However, Method is a "bootstrap" class, meaning the VM has
built-in knowledge of it and includes a tentative version built-in but
must load the real version from the classpath at runtime before
invoking methods on it. Normally this happens naturally when Method
instances are created in Java code, but here we're creating them in
the VM instead, which doesn't automatically cause the real class to be
loaded. So we must do so explicitly.
On ARM64, conditional branches to immediate offsets can span no more
than 2^19 instructions. In the case of the stack overflow check,
which wants to do a conditional branch from every non-leaf method to a
handler, this can be a problem, especially when compiled code grows
large as with a bootimage=true build against the OpenJDK class
library. Therefore, we use an unconditional branch to reach the
handler on this platform.
When we initialize the vtables for bootstrap Java classes such as
java.lang.NullPointerException (i.e. classes which the VM has built-in
knowledge of), we assign the superclass's vtable to any class which
has no declared virtual methods of its own. However, that vtable will
be null if we haven't initialized the superclass yet. Therefore, we
must order this process such that no class is initialized until after
all its superclasses.
When we intercept a method (i.e. when the VM wants to run its own code
instead of whatever the classpath provides for that method), we make a
clone of the original method so we can later call it from the
intercepting code if appropriate. We also set the ACC_NATIVE flag on
the original method to ensure that our intercepting code is always
used in preference to the classpath version. However, we need to set
that flag *after* we make the clone, or else the clone will also have
the ACC_NATIVE flag set, which is not what we want.
We never noticed this before because classpath versions of all the
methods we intercept as of Java 7 are either native or are never
called from their VM-specified replacements. However, some of those
native methods are non-native in later versions of Java, so the bug
has become apparent.
Also, replace some preprocessor conditionals with C++ conditionals and
add some todo comments and sample code for future work towards better
ABI compatibility in the JIT compiled code.
