http://www.masonchang.com/2008/08/tracemonkey-vs-squirrelfish.html

Got a pointer to the application level tests you are trying? OpenLaszlo sounds like an interesting tool for performance testing since we can compare directly to Flash for functionally similar things.

Also, if you are generating native classes for AS, then isn’t that typed code?

In general though I’d love to profile a heads-up comparison of an app-level benchmark and see if the time is in rendering or script execution or something else surprising.

Two important things to note though are: parsing takes basically 0% of the runtime — you can add fairly significant amounts of additional work to the parser without causing any change in sunspider (although one would hope you were making such changes to enable improvements later :D ). As for the compilation of the global code segment, there is only one test that actually performs the majority of its computation in the global scope and that has a very small amount of code relative to its runtime, so there should not be any significant impact from this.

Finally, even though we do discount the compile time for a very small portion of the tests, profiling (at least in squirrelfish) shows that less than 0.3% of the execution time is spent generating the bytecode, so it really is not significant.

I’m not sure why you believe squirrelfish times don’t include that time.. in fact i’m not sure why you think it is actually possible.

Finally, SunSpider was created to find performance issues, if it did not include compilation time it would not be useful, as we would be unable to make any changes to codegen because we would not be able to measure the resultant performance (there’s no point making a change that makes interpreting a script 2 seconds faster if the compilation takes 3 seconds longer).