HTML 5 Profiler #1

Short Version:

-We’ve started a new program called Innovation Friday and my personal project is an HTML performance analysis tool.

-The goals of this software is to:

• Provide a tool so that developers can test hardware, browsers, and HTML5 content on both canvas and WebGL.
• Provide detail on bottlenecks of each stage of the graphics pipeline, not just the performance of a single scene/game.
• Provide a single benchmark score that is a cumulative sum of its parts.
• Provide the capability to use the tool as part of a continuous build process.

-This will be an Open Source project hosted on GitHub…I’d love for you to join me.

-I have announcements for our products coming soon (please…no off topic posts).

Long Version:

I write code on the weekend. Why? Because it’s fun. And since profit isn’t necessarily my goal, I get to pick whatever I like. This year, I wrote an image processing test grading application, implemented my own SSL, along with other ideas (most of what I pick to do indirectly supports GG). This is how I keep up my programming skills since I rarely get a chance in my day to day routine.

I’m very motivated to work on those projects because of the freedom I have to define direction, goals, and implementation.

Last week, GarageGames started an initiative to bring employee’s creative ideas from their off hours into our walls. We call it Innovation Friday.

And the only reason I’m not posting this on the front page is because we are way behind in telling you how we are going to be more innovative with T3D, iTorque, and Torque 2D and I expect to get a stream of questions regarding the status of our products. For now, all I can say is that announcements are on the way.

So…on to the tool…

It started with my interest in HTML5 as a gaming platform for the future. I don’t think that HTML5 (when I say HTML5, just think Canvas and WebGL) will replace all game development formats since cutting edge games will always benefit from being closer to the metal…but I do think there will be some major advantages to have a standard format that is useable across many platforms. And if we’ve learned anything in the past five years, pushing the envelope with social, collaborative, and connected technology is as valuable as pushing the envelope of graphics and logic…and it’s the former where HTML5 excels.

So…on to the tool…

The goals are:

• I want to intimately learn how HTML 5 uses hardware accelerated graphics (that’s the selfish goal)
• Provide a test system that is useful for testing hardware (driver developers and manufactures), browsers (for browser developers like Google, etc.), and content (all HTML5 developers).
• Provide feedback on how each browser handles different aspects of rendering performance.
• Provide feedback for content developers to understand performance issues in their game/content.
• Provide mechanisms for using this web app as a part of a continuous build system.

Now that we’ve stated the goals, let’s talk about existing HTML5 benchmarking tools.
I know that there’s a need for a robust benchmark tool because the developers at WHATWG told me so. Being a natural skeptic, I decided to look into what I can find. If you know of any other tools, please let me know.



ie.microsoft.com/testdrive/Performance/FishBowl/


html5-benchmark.com/

www.kevs3d.co.uk/dev/asteroidsbench/

www.themaninblue.com/writing/perspective/2010/03/22/



Each of these systems has strengths and weakness, but the missing piece of each is that they only tell you about performance of THEIR application. While that is useful, it doesn’t provide very much information about how YOUR application will run. Certain browsers may do better with certain aspects of rendering and a specific test application may exploit the differences to paint an inaccurate picture about the browsers overall implementation.

For example, if a given browser implements every stage of the graphics pipeline beautifully, but makes a mistake in how textures are implemented; the browser will appear to perform badly if the test happens to use a lot of texture. But, if you understand that a particular browser has issues with textures, you might be able to address that issue as a special case for that particular browser (i.e. Cut your texture resolutions in half).

So how will this project address the issue? The answer is to test each stage of the graphics pipeline. We will do that by isolating the particular stage with a performance test and present its relative performance for that given stage. This is where my background in GPU performance optimization will come into play. By running a test at each stage, we will fully understand the strengths and weakness of each browsers implementation. This will provide developers a much better picture of where bottlenecks lie for a given browser.

We will have an overall score for a browser, but it will be more of a performance index that is built from the sum of its parts. The overall score is helpful to some degree, but one brower’s score of 99 could be very different than another browsers score of 99 and without a drill down into the specifics, you’re stuck guessing.

So that’s the high-level overview. What’s next?

Matt, a web developer at GarageGames, and I will be experimenting and rapid prototyping. Once we are done, we will build a basic framework and publish that framework as an Open Source project (probably GitHub).

If you are interested in following, post a comment and I’ll notify you via e-mail as things progress. After we get the framework up and running, we will invite others to help us if they are interested.

Lastly, I’m in know way an HTML 5 expert. I’ve very comfortable with GPUs, but I know very little about how Canvas and WebGL use the GPU. It’s going to be an adventure…I hope you will join me.