Category: languages

Michal Cierniak – In my opinion modularity will not bring significant performance wins.  I agree, but that’s not the point.  Java needs modularity and ahead of time compilation so that it can start fast and share memory between processes, otherwise Steve Jobs has a point.

I don’t hear many folks, that run Java applications, complaining that Java’s performance is poor once the application is running.  However they do complain that the applications take a long time to start and that when running many Java applications LOTs of memory is consumed.  I work for a company that makes use of three significant java applications.  No one would claim that they start fast, but my biggest complaint is that these three applications, that are all built on the same code base, share no memory at runtime, none, zip, nadda, nothing.

It’s like we have gone back 40 years in computing history.  There used to be a time that programs didn’t share any memory but that went away with the introduction of dynamic linking and shared libraries. Most programs use DLLs (Windows) and Shared Objects (Unix) to ensure fast startup and efficient memory sharing and all of this is managed by the OS.  If two concurrently running programs both decide to use the same spell checking library then that library is memory mapped into the address space of the two programs by the operating system i.e. there is only one copy resident in memory. In java this is not true, there is a unique copy resident in each address space.

So why can’t Java programs share memory given that shared libraries are a technology that’s been in widespread use for at least 25 years? It’s really for two reasons.  The first is that up until this point there has really been no way to define a shared library in Java i.e. here’s a bunch of code and here is its external interface. JSR294 (modularity for Java) addresses this point.  The second is that Java has no efficient AOT compilation and so unlike most other statically typed languages it can’t utilize the traditional OS functions to load and share its libraries. I have belabored this point in the other "Worlds Apart" posts here and here, pointing out that this is a key differentiation for CLI implementations such as .net or mono.  Miguel (mono lead) also has an excellent post that provides a good overview of mono’s AOT support and the rationale behind it. JSR294 could be designed in such a way to provide for efficient AOT, and I belabor this point here.

It’s also not like the problem of memory sharing and startup isn’t well understood in the Java world. Sun produced a paper 5 years ago trying some crazy dynamic memory sharing thing and followed that with a paper the very next year that deduced instead that shared libraries were the way to go as the first "dynamic" approach "required complex engineering" while using shared libraries was "simpler" and "relies on existing software and OS mechanisms".

However the opportunity to produce shared libraries from Java code has not been seized upon (IMO it’s too hard with custom classloaders) and there have been many other attempts at sharing memory in Java, non of which have yet proven successful. C# and CLIs, which were developed with the benefit of experience with JVMs, made efficient AOT a priority and reap the benefits of faster startup and better sharing.

So why not take the opportunity that JSR294 is providing and level the playing field with CLIs by using it to provide efficient AOT for Java that in turn allows Java code to compile to shared libraries? As a result Java will start faster and share memory between processes.  Without this CLIs will maintain a significant advantage and they have enough of those already.

Alex Buckley C# itself is no more or less designed for these analyses than Java. (in response to compilation approaches outlined in Worlds Apart – JVMs and CLIs).

I beg to differ, JVMs had to introduce hotspot compilers whereas CLIs have not because of a subtle design difference between the languages and, moreover, other design differences can lead to much more efficient AOT‘d C# code. JSR294 (modularity system for Java), however, provides an opportunity to provide just as efficient AOT for Java, I hope we take this opportunity.

The subtle but vital design difference, that led to JVMs requiring hotspot compilers and CLIs not,  is that non-virtual is the default in most CLI languages which is the complete opposite to Java.  Most methods will never be overridden (by, for example, a method in a subclass) however in Java the programmer must explicitly state this via the final keyword. On the other hand, in C# the programmer has to explicitly declare the opposite i.e. when a method can be overridden, by using the virtual keyword.  If no declaration is placed on a method in C# then it is assumed to be non-virtual i.e. the equivalent of final in java.

So why is this important? Well it turns out that one of the key qualities that must be calculated for a method to be an inline candidate is whether it is overridden.  As most programmers don’t say anything about whether a method is virtual or non-virtual (we’re a lazy bunch), the default applies and as a result many more methods in C# are implicitly non-virtual i.e. they can never be overridden. The compiler can use this information when determining whether to inline. In Java much more complicated calculations must be performed and it is the costly nature of these calculations that created the need for hotspot virtual machines.

Early JVMs (i.e. pre-HotSpot) used to JIT compile every method on first use, just like CLIs do today. However, the early JIT compilers were conservative in making inlining assumptions because they didn’t know what classes might be loaded later, and as such their performance was pretty bad.

So why were they conservative?  Think about what they have to do. They can’t rely on the fact that non-virtual methods are declared as such because most of them aren’t (lazy programmers and virtual is the default in Java). Instead they have to guess.  To make that guess they have to understand the class hierarchy at the time of JITing and check that non of the subclasses override the method. That’s expensive enough for the inline operation, but it gets worse. They have to remember this guess in case a class loaded later overrides the method forcing them to de-optimize the code by undoing the inlining. All of this cost is significant and so hotspot compilers were introduced that only do this on code that gets used a lot.

Contrast that with the CLI implementations (i.e. mono and .net).  There’s a description of the rules that microsoft uses and they are very simple. One rule is even "Virtual functions are not inlined".  Whereas JVMs go to great lengths to determine whether a virtual function can be inlined, .net doesn’t even bother looking and this is all due to differences in the language specifications of Java and C#.  As an aside Visual Basic also has non-virtual as the default and the overridable keyword is used to make a method virtual.

Now the JSR294 expert group (modularity system for Java) can’t make non-virtual the default in Java, but they can probably do enough that it won’t matter. If the goal is to get to efficient AOT compiled code, which is the key differentiation between CLIs and JVMs at the moment, then that goal is attainable via JSR294.

As Patrick pointed out there are various attempts to add AOT compilation to Java.  GCJ and Excelsior are the two front runners.  However, they have to answer this same question i.e. when can a method be inlined? Given that they do compilation ahead of time, the performance of the actual compilation step is not as critical as it is for JIT compilers and so they could use classic techniques such as rapid type analysis (RTA) or class hierarchy analysis (CHA) to determine whether a method is overridden. Unfortunately neither of these classic techniques can be applied ahead of time to Java, although they can be applied to C#, why is this?

Java’s custom classloaders mean that the class hierarchy can only be determined at runtime. This is due to a custom classloaders ability to load classes from anywhere on the classpath at any time. It’s simply impossible to understand ahead of time what the class hierarchy looks like and so RTA or CHA can’t be used for AOT compilation of Java programs. Instead to maintain Java compatibility GCJ, for example, resorts to an ABI that doesn’t even allow inlining. It’s performance suffers accordingly.

This is the problem that JSR294 can choose to solve, it can define a modularity system that allows for efficient AOT of Java code.  This problem has already been well researched (and solved).  MJ, a paper by, amongst others, the inventor of RTA, has already defined such a modularity system and successfully applied it to Tomcat.

Again I urge the JSR294 expert group to consider taking on the requirement of producing a modularity system that allows for efficient AOT compilation of Java code.  JVMs are at a significant disadvantage compared to their CLI counterparts at the moment as efficient AOT of CLI code is already possible. Why not bring this possibility to the Java platform in JSR294?

Glyn Normington: It’s interesting to speculate what would have happened if a static module system had been put into Java much earlier. My guess is that it wouldn’t have bothered to address versioning or dynamicity requirements which only become crucial in the context of relatively large systems with continuous operation.

Indeed it is interesting to speculate. Mono and .net are both implementations of the Common Language Infrastructure.  They have had a static module system from the beginning and they power many applications from best in show MP3 players to relatively large systems with continuous operations such as myspace and shortly second life.

CLI implementations and JVM implementations couldn’t be more different.  While at first glance they appear very similar, they both have an intermediate language and they both use a JIT, the similarities seem to end there.

CLI implementations have been designed to apply classic compiler optimizations using code analysis techniques just like those found in a typical C++ compiler. For example their approach for method inlining is very simple. They also JIT all code once on first use. They can do this as most of the CLI languages have been designed for compilation using classic  analysis techniques.

JVMs, on the other hand, use runtime knowledge to decide what to optimize.  Rather than using the classic techniques they guess a lot (e.g. on whether a method can be inlined) and they have to keep these guesses around in case they load a class that invalidates them.  They also have different levels of optimizations that they perform, saving the most aggressive for code that is getting used the most (i.e. the hotspots). They use these approaches as the class loading rules in java make the application of code analysis techniques virtually impossible.

So which is better, I think the jury is still out, and they both have strong supporters.  However, CLI implementations have one big advantage over JVMs.  As they rely on classic static analysis CLI modules can be ahead of time compiled to native binaries. This brings many advantages such as improved startup and better memory sharing between processes, the latter being a key requirement if multiple applications are being run on a single workstation.

As such, the forthcoming language level changes for Java modularity along with the corresponding changes to the runtime provide a unique opportunity.  They could be designed as a static module system like the one in the CLI.  To do this they would need to change the classloading rules for classes in modules, but this would allow JVM implementations to ahead of time compile java modules to native code bringing all the advantages I mentioned above.

I really hope the expert groups design with this flexibility in mind and I see that I am not the only one that has seen the potential.

Update: I posted a follow up pointing out how subtle language differences have lead to the two different approaches to the runtime.

O’Reilly recently published this chart detailing trends in the computer book market.

Javascript is showing some pretty stellar growth, but Tim also has this to say “The net-net is that C# has definitely passed Java in the book market.”

I wonder if mono is helping drive that demand. The “.net for Unix” is already used by wikipedia for its searches (see here), is shortly to be embedded in second life and runs the mp3 player that won best in show at this year’s CES.

Unlike Java, mono code can be compiled ahead of time and more importantly mono has been designed to allow multiple programs running on the same machine to actually share memory. Kinda radical (for anyone familiar with Java), but yet, somehow, so reassuringly familiar.