One way to avoid the intense pain of building OpenJDK on Windows is to
let somebody else do it, e.g. https://github.com/ojdkbuild/ojdkbuild.
However, that project customizes the builds slightly, which means we
need to make an effort to stay compatible. This one-liner does that.
Previously, I was lazily initializing this field in addDynamic, but
that only worked if a thread compiles an invokedynamic expression
prior to the first time it executes one, which is not always the case.
This is a bit tricky to write a simple test case for, since it
involves multiple threads, but I've tested it with a big, lambda-heavy
app and it fixes an otherwise consistent crash there.
The main issue was that offsets for GC roots on the stack were being
miscalculated because invokedynamic bootstrap methods are invoked as
static methods but were not being flagged as such.
Also, I forgot to initialize MyThread::dynamicTable in the constructor
(and why the hell doesn't `gcc -Wall` warn me about stuff like that?)
We now detect both metafactory and altMetafactory lambdas when
bootimage-compiling invokedynamic instructions, as well as make
allowance for the same lambda being invoked by multiple invokedynamic
instructions (as is the case for Serializable lambdas).
For lambdas that implement java.io.Serializable, the compiler emits
calls to LambdaMetaFactory.altMetafactory, not
LambdaMetaFactory.metafactory, so I've provided a stub implementation
that ignores that currently ignores the extra parameters it receives.
This also fixes a bug in compiling lambda glue code for lambdas that
take longs and/or doubles.
Per a recent bug report in the hello-ios project, we found that
bootimage-generator would abort with no explanation if it encountered
a lambda invocation and the `-hostvm` option was unspecified. This
commit ensures that a helpful message is printed before exiting.
OpenJDK's java.util.zip.ZipFile.getEntryBytes should return a byte
array that is not null-terminated, but we were giving it one that was
null-terminated, which caused lookups to fail later when
ZipFile.getInputStream was called.
Previously, the following code would throw an IllegalMonitorStateException:
public class Test {
public static synchronized void main(String[] args) {
Test.class.notify();
}
}
The problem stems from the fact that for a long time Avian has had two
representations of a given class: avian.VMClass and java.lang.Class.
It used to be that there was only one, java.lang.Class, but that
didn't play nicely with OpenJDK's class library, so we split it into
two. Unfortunately, we forgot to update the JIT and interpreter
accordingly, so a static synchronized method would acquire the
avian.VMClass instance, whereas Foo.class.notify() would be invoked on
the java.lang.Class instance.
This commit fixes it.
The tower of patches and hacks grows higher. Ideally, we'll just drop
support for JDK 7 soon and clean this mess up a bit, but TravisCI
still hasn't gotten the memo that it's dead, so we muddle onward.
I've tested this on Windows, but not yet Linux or OS X. Wanted to get
a PR before I move on to that.
These expressions are tricky because they rely on invokedynamic, which
normally implies runtime code generation. However, since lambdas
don't actually use the "dynamicness" of invokedynamic, we can convert
them into static calls to synthetic classes at compile time.
Since I had already written code to synthesize such classes in Java
and I didn't want to rewrite it in C++, I needed to add support for
running Java code to the bootimage generator. And since the primary
VM used by the generator is purpose-built to generate AOT-compiled
code for a specific target architecture and is not capable of
generating or running JIT-compiled code for the host architecture, I
added support for loading a second, independent, host-specific VM for
running Java code.
The rest of the patch handles the fact that each method compilation
might cause new, synthetic classes to be created, so we need to make
sure those classes and their methods are included in the final heap
and code images. This required breaking some giant code blocks out of
makeCodeImage into their own methods, which makes the diff look
scarier than it really is.
This allows multiple Avian VMs to share the same process space,
provided they don't try to use functionality that involves global
shared resources (e.g. signal handling).
The two big pieces here are basic invokedynamic support and a working
version of LambdaMetaFactory.metafactory. The latter works by
dynamically building a synthetic class with three methods: a static
factory method, a constructor for the factory method to call, and a
method to satisfy the requested interface which defers to the
specified MethodHandle.
This work relies heavily on Avian's specific MethodType and
MethodHandle implementations, which provide extra, non-standard
features to make code generation easier. That means we'll probably
need to use Avian's versions of java.lang.invoke.* even when building
with the OpenJDK or Android class libraries.
This is a bunch of commits squashed into one per Josh's request.
add dynamicTable field
add invokedynamic instruction
add defaultDynamic bootimage field
add dummy invokedynamic support in bootimage-generator
add defaultDynamic thunk
check dynamicTable offset
comment defaultDynamicThunk to fix unused function
comment defaultDynamicThunk to fix unused function
add dynamicTable / dynamicIndex stuff
comment dynamicIndex and dynamicTable
add invokedynamic instruction impl
stub out addDynamic
unstub addDynamic
don't allow tail calls in invokedynamic
implement stub JVM_GetTemporaryDirectory method
(build broken) begin add InvokeDynamicTest
Revert "(build broken) begin add InvokeDynamicTest"
This reverts commit 77f9c54e32ac66d0803eeab93e4a10d3541987a8.
add InternalError
add URLClassPath.c for openjdk-src builds
implement stub JVM_KnownToNotExist and JVM_GetResourceLookupCache methods
intercept open0 / open for openjdk
add basic java/lang/invoke stubs
remove non-public java/lang/invoke classes
fix invokedynamic example building
<wip debugging>
In a bootimage=true build, we create allocate certain objects as
"immortal fixies", which means they will never been deallocated at
runtime and should only be visited if/when they point to objects which
might move during garbage collection. However, there was a bug in the
following case:
1. immortal fixie F is updated to point to a movable object M and
thus F is added to the list of fixies to visit during the next minor
collection (but not the next major one, since all reachable objects
are visited during a major collection, and there's no point in
visiting an unreachable object, whereas during a minor collection we
have to visit F because we don't know if it's reachable or not)
2. a major collection occurs, but F is not reachable and thus is not
visited, whereas M is moved
3. a minor collection occurs, and since F is still in the list, it is
visited, but since it contains a stale pointer to M's old location,
we crash
The solution is to ensure unreachable immortal fixies are removed from
the above list after each major collection, thus guaranteeing they
won't be visited on any subsequent collection.
GCC is a lot more sensitive about -Werror=unused-variable, to the
point that stuff declared in header files but unused in a given
compilation unit is flagged. This may be due to the way we're
here's the fix.
At first, it might look like the atomicIncrement operations here,
since they resolve to OSAtomicCompareAndSwap32Barrier, ought to
provide all the memory barrier guarantees we need; however, it turns
out it's not quite sufficient.
Here's why: Apple's OSAtomic*Barrier operations guarantee that memory
operations *before* the atomic op won't be reordered with memory
operations *after* the atomic op - but makes no guarantee that the
atomic op itself won't be reordered with other operations before or
after it. The key is that the atomic operation is not really atomic,
but rather consists of separate read, check and write steps - in a
loop, no less. Here, presumably, the read of t->m->exclusive is
hoisted by the out-of-order processor to between the read and write
steps of the "atomic" increment of t->m->activeCount.
Longer term, it probably makes sense to replace this with the c11
<stdatomic.h> operations or the c++11 <atomic> types. We ought to
actually use the atomic increment operations provided there. As it
is, our atomicIncrement looks substantially less efficient than those,
since it's actually implemented on arm64 as two nested loops (one in
atomicIncrement and one in the compare-and-swap) instead of one. We
should also evaluate whether the various instances of atomic
operations actually need as strong of barriers as we're giving them.
FWIW, the gcc __sync_* builtins seem to have the same problem, at
least on arm64.
This would theoretically break compatibility with apps using embedded
classpaths, on big-endian architectures - because of the size type
extension. However, we don't currently support any big-endian
architectures, so it shouldn't be a problem.
This improves support for building with openjdk=$jdk8. Note, however,
that the Processes test does not pass, since JDK 8's
java.lang.UNIXProcess uses invokedynamic, which Avian does not yet
support.
OpenJDK 8 includes a core class (java.lang.Thread) which so many
fields that it exceeds the class size limit in type-generator dictated
by the logic responsible for calculating each class's GC object mask,
at least on 32-bit systems. There was no fundamental need for this
limit -- it just made the code simpler.
This commit removes the above limit at the cost of slightly more
complicated code. The original motivation for this change is that the
platform=macosx arch=i386 openjdk=$jdk8 build was failing. However,
there doesn't seem to be a prebuild JDK 8 for 32-bit OS X anywhere on
the Internet, nor is there any obvious way to build one on a modern
Mac, so it's safe to say we won't be supporting this combination
anyway. The problem also occurs on Linux and Windows, though,
so it needs to be fixed.
In afbd4ff, I made a low-risk, but very specific fix for a more
general problem: "bootstrap" classes (i.e. classes which the VM has
built-in knowledge of) need to be loaded from the classpath before any
of their methods are called. Based on recent testing, I found there were
more cases than I previously thought where the VM tries to call methods on
"unloaded" bootstrap classes, so we needed a more general solution to
the problem.
This commit addresses it by closing the last (known) loophole by which
methods might be called on bootstrap classes: invokeinterface, and its
helper method findInterfaceMethod. The fix is to check for bootstrap
classes in findInterfaceMethod and load the full versions if
necessary. This process may lead to garbage collection and/or thrown
exceptions, which made me nervous about cases of direct or indirect
calls to findInterfaceMethod not expecting those events, which is why
I hadn't used that approach earlier. However, it turns out there were
only a few places that made non-GC-safe calls to findInterfaceMethod,
and a bit of code rearrangement fixed that.
As documented at
https://developer.apple.com/library/ios/documentation/Xcode/Conceptual/iPhoneOSABIReference/Articles/ARM64FunctionCallingConventions.html,
the ARM64 iOS ABI differs from the generic ABI in a few important
ways. Specifically, arguments passed via the stack are aligned
according to their natural alignment instead of 8 bytes. The VM's
dynamic call code was aligning each argument to 8 bytes, so native JNI
code couldn't find them in their expected places.
Also, we weren't setting the "os.arch" system property on ARM64, so I
fixed that too.
This method ends up defering to JVM_GetClassDeclaredMethods, which
creates an array of java.lang.reflect.Method instances and then
calling getName on each one through the java.lang.reflect.Member
interface. However, Method is a "bootstrap" class, meaning the VM has
built-in knowledge of it and includes a tentative version built-in but
must load the real version from the classpath at runtime before
invoking methods on it. Normally this happens naturally when Method
instances are created in Java code, but here we're creating them in
the VM instead, which doesn't automatically cause the real class to be
loaded. So we must do so explicitly.
