Cecil Benchmark Results

Benchmark Descriptions

Name Size (lines) Description
lbr-move 2,400 +
10,600 std. lib.		 An instruction scheduler
typechecker 21,500 +
10,600 std. lib.		 A static typechecker for Cecil
tc2 24,600 +
10,600 std. lib.		 A second-generation typechecker for a more advanced type system
compiler 49,700 +
10,600 std. lib.		 An older version of the Vortex optimizing compiler

Optimizer Configurations

Our experiments were performed by compiling Cecil programs through Vortex to produce C code, then running gcc -O2 on the C code to produce a stand-alone executable. The following table describes the different settings of the Vortex optimizer component of this pipeline:
Name Description
unopt No Vortex optimizations (but full gcc -O2 optimizations on the generated C code)
trad-opt Vortex intraprocedural, non-message optimizations, including elimination of redundant and dead loads & stores and elimination of dead object creations
inl trad-opt plus automatic inlining
i inl plus intraprocedural static class analysis to optimize messages
i-cha i plus class hierarchy analysis to automatically identify non-overridden methods
i-cha-scp i-cha plus class-hierarchy-based static class prediction
i-prof i plus dynamic-profile-guided class prediction
i-cha-prof i-cha plus -prof
i-cha-scp-prof i-cha-scp plus -prof

Performance Data

[A Java applet implementing a bar chart should have appeared here.]

Raw, Detailed Data

Analysis

Overall, Vortex's optimizations have a large impact on Cecil programs, improving their performance by an order of magnitude over the base configuration. Neither the trad-opt nor the inl configurations provide a noticable improvement over base, since virtually every operation in Cecil is a dynamically-dispatched message send. Profile-guided class prediction is the single most important optimization, but each technique results in non-trivial improvements in application performance. For several of the programs the i-cha-scp-prof configuration is actually slower than the i-cha-prof configuration. This slowdown is attributable to bad interactions between the heuristics currently used by Vortex to guide class prediction.

Last updated March 13, 1998.

Cecil/Vortex Project, chambers@cs.washington.edu