satine.org

by Charles Ying

Posts Tagged ‘Mobile’

The Care and Feeding of the Android GPU

Saturday, January 1st, 2011

Android has two major technical UX problems: animation performance and touch responsiveness.

Android’s UX architecture needs work. UI compositing and the view system are both primarily done in software. Garbage collection and async operations frequently block UI rendering.

Android team members are still in denial on the importance of GPU acceleration. They recommend eliminating garbage collection to improve animation performance. They say drawing isn’t the bottleneck and GPU accelerated 2D drawing won’t yield good results:

“It is very naive to think that using the GPU to render text and bitmaps is suddenly going to fix every issue you may see. There are many things that can be done to improve performance of the UI without using the GPU. Notably improving touch events dispatching, reducing garbage collection pauses, asynchronous operations to avoid blocking the UI thread, etc. A one year old NexusOne (and other devices before) is perfectly capable of scrolling a list at close to 60fps (limited by the display’s refresh rate.) Using GPUs to do 2D rendering can introduce other types of inefficiencies (fillrate can be an issue, some primitives like arbitrary shapes are complicated to render with antialiasing, textures need to be uploaded, shaders compiled, etc.) I am not saying we won’t do GPU rendering for the UI (I have worked on it myself a couple of times to test it) but please stop assuming that this is what has to be done right now.”
Romain Guy, Android software engineer

No one is saying that Android’s “2D primitive drawing” needs GPU acceleration. It’s Android’s view system and animation compositor that needs GPU acceleration. To compare, Core Graphics is still mostly software based while Core Animation is entirely GPU accelerated.

Look at Samsung’s Galaxy S browser. GPU accelerated and tile-based. I’m told it’s a result of Samsung’s PowerVR GPU optimizations. Smooth as butter, runs circles around the Nexus S Gingerbread browser on the same hardware!

Stop executing Dalvik Java VM code on every animation frame. Use the programmable GPU graphics pipeline. Add a scene graph if it makes sense. Run it on a separate thread. You might even get 32-bit graphics along the way.

Android engineers say that better hardware will eventually solve the problem — an insane rationale for the problem. On mobile, power efficiency is king. Throwing dual cores or more GHz at the problem is just going to get you more average performance with zero battery life, and even then, as long as your screen doesn’t get too big.

Wake up, Android team. Windows Phone 7 just lapped you.

Update: Additional discussion on Hacker News.

On Mobile Phone OS Design and GPUs

Wednesday, December 2nd, 2009

Mobile phones have really big screens.

Tablets will have even bigger screens.

If you want your mobile phone OS to matter in 2010, you’d better optimize for rendering graphics on the GPU.

GPUs are far more power efficient and faster at drawing graphics. As mobile device screens get really large, drawing the graphics “old school” on the CPU is moving from “unacceptable” to “impossible” pretty quickly.

Today, Motorola Droid runs at about half the frame rate of iPhone 3GS. The GPU in both is a PowerVR SGX 530 series chip.*

A PowerVR SGX 530 has a fill rate of 200 million pixels per second. That means Motorola Droid’s GPU could theoretically drive 8 full screen refreshes at 60 fps on a Droid-sized (WVGA) screen. All without the CPU’s help.

Parts of Android’s graphics subsystem seem to be GPU accelerated, but more is needed.

Imagine if the Android UX ran at a rock solid 60 fps. Then, I think we’d have something pretty sexy.

I wonder if Apple’s PA Semiconductor unit is designing GPUs instead of CPUs?

* It’s unclear what the exact model of the iPhone 3GS GPU is, and so the exact fill rate is not known… (though if it were, say, a PowerVR SGX 535, its fill rate would be 400 million pixels per second)

Technorati Tags: , , , , , , ,

A Kick Ass Browser Renaissance

Sunday, July 5th, 2009

Why are building browsers in vogue again?

Quite simply, a browser renaissance, started with WebKit, a small and flexible open source code base that used less memory and power, and kicked ass at rendering the Web.

WebKit is really helping folks easily create and extend browsers with its very flexible and understandable code base. It’s truly the Linux renaissance again, driving innovation, and place browsers where they were difficult or impossible to put before. And WebKit’s rivals didn’t sit still, they’re also kicking ass at an insane clip, and we, the users and developers, benefit.

Great hardware needs great software.

The browser is really your OS now – your apps live and run there, and you want them to run well. The browser experience matters again. That’s why Apple is here, making a great browser to sell with their hardware. Google, Nokia, HTC, Palm and many others are here for the same reason.

Closely related to this activity are pure browser vendors like Opera, selling a quality mobile web browser to hardware OEMs that don’t have the time to make their own. This is also Mozilla’s goal with Fennec.

A flashier World Wide Web.

  • Flash: Adobe needs to protect its dominance on the desktop Web interactive media experience. Adobe also wants to duplicate Flash’s success in mobile.
  • CSS Visual Effects: Apple is tired of Flash’s slow pace of innovation, performance and constant crashing on Mac OS X. Steve and Bertrand’s insurance policy. And why the hell not when you make stuff like Quartz, Core Animation, QuickTime X, and iPhones.
  • Silverlight: Microsoft builds their own browser, why not their own Flash?
  • HTML5 Video: Everyone doesn’t want all video to go through a Flash plugin. Plus, cool video reflections!

What your high tech company should think about.

  • If you make consumer hardware products, having a great browser will be a key strategic factor to having great applications, user experience, and long-term extensibility.

  • If you make consumer facing applications, browsers now offer the best user interface framework “toolkit”. There are only very few reasons to use or build a custom UI framework today, primarily availability and performance.

  • If you make pure web applications already, targeting these modern browsers now makes sense, the most desirable consumers now have these types of browsers, and the market share continues to climb.

  • If you make or use software frameworks, innovations in the browser wars have yielded some amazing technology that you should be taking advantage of. The 3 JavaScript engines, WebKit Nitro, V8, and Mozilla TraceMonkey now are starting to surpass other dynamic languages in performance and other languages are starting to play catch up.

  • If you are Google, I’d do this.

While we’re talking about JavaScript …

  • Distributing computation work to browsers is a new frontier. Imagine offloading your search indexing or database indexing to visiting users. Having compute clusters will lower the cost of building large scale distributed services in the near future. Today’s primary barrier for this is bandwidth consumption (distributed applications still have large amounts of I/O) but it will work today for certain applications.

  • JavaScript as a server-side language also makes great sense long-term, where code passes seamlessly between servers and clients. Today’s primary barrier for this is mature system libraries and standards for JavaScript, but there are already great companies working on this, and the concept is far from new. (too many links to share here)

Food for thought …

Thanks to Om Malik for the inspiration to write this post. The one question I wish I had a better answer for is: “Why does Apple have Safari on Windows?” I believe the answer may be something more than, “oh hell no, Google ain’t gonna release no WebKit browser on Windows before we do…” (rewind to June 2007) but what the actual answer is, we may not know for some time.

Updates: Corrected that TraceMonkey is the Mozilla JS engine, not Tamarin. Thanks Dirkjan!

Technorati Tags: , , , , , , , , , , , , , , , ,