satine.org

by Charles Ying

The Care and Feeding of the Android GPU

January 1st, 2011

Android has two major technical UX problems: animation performance and touch responsiveness.

Android’s UX architecture needs work. UI compositing and the view system are both primarily done in software. Garbage collection and async operations frequently block UI rendering.

Android team members are still in denial on the importance of GPU acceleration. They recommend eliminating garbage collection to improve animation performance. They say drawing isn’t the bottleneck and GPU accelerated 2D drawing won’t yield good results:

“It is very naive to think that using the GPU to render text and bitmaps is suddenly going to fix every issue you may see. There are many things that can be done to improve performance of the UI without using the GPU. Notably improving touch events dispatching, reducing garbage collection pauses, asynchronous operations to avoid blocking the UI thread, etc. A one year old NexusOne (and other devices before) is perfectly capable of scrolling a list at close to 60fps (limited by the display’s refresh rate.) Using GPUs to do 2D rendering can introduce other types of inefficiencies (fillrate can be an issue, some primitives like arbitrary shapes are complicated to render with antialiasing, textures need to be uploaded, shaders compiled, etc.) I am not saying we won’t do GPU rendering for the UI (I have worked on it myself a couple of times to test it) but please stop assuming that this is what has to be done right now.”
Romain Guy, Android software engineer

No one is saying that Android’s “2D primitive drawing” needs GPU acceleration. It’s Android’s view system and animation compositor that needs GPU acceleration. To compare, Core Graphics is still mostly software based while Core Animation is entirely GPU accelerated.

Look at Samsung’s Galaxy S browser. GPU accelerated and tile-based. I’m told it’s a result of Samsung’s PowerVR GPU optimizations. Smooth as butter, runs circles around the Nexus S Gingerbread browser on the same hardware!

Stop executing Dalvik Java VM code on every animation frame. Use the programmable GPU graphics pipeline. Add a scene graph if it makes sense. Run it on a separate thread. You might even get 32-bit graphics along the way.

Android engineers say that better hardware will eventually solve the problem — an insane rationale for the problem. On mobile, power efficiency is king. Throwing dual cores or more GHz at the problem is just going to get you more average performance with zero battery life, and even then, as long as your screen doesn’t get too big.

Wake up, Android team. Windows Phone 7 just lapped you.

Update: Additional discussion on Hacker News.

Tags: , , ,

78 Responses to “The Care and Feeding of the Android GPU”

  1. Ariya Says:

    I have not seen the evidence that Samsung flavor of Android WebKit is GPU accelerated. Currently downloading Galaxy Tab source from Samsung site does not show anything radically different than stock Froyo.

    The tiles might speed up the rendering, but that’s also something I’ve done ages ago (see https://www.codeaurora.org/patches/quic/webtech/backingstore.patch).

  2. Charles Ying Says:

    Ariya, the example cited is the Galaxy S phone, not the Galaxy Tab. Different builds, hardware, and screen resolution. The hardware accelerated build in question starts with the XXJPU firmware update.

  3. Ariya Says:

    Oops, sorry. I meant of course Galaxy S. Nowhere in the Android package offered by Samsung reveals that it is using GPU. And of course, the tiling stuff shows remarkable similarity to what I have done.

    I’d be glad if someone can truly prove this GPU claim for once.

  4. Jose Says:

    Samsumg people have Carsten Haizler aka “Rasterman”, creator of enlightenment Linux desktop. http://www.vergenet.net/~raster/

    He is an expert making low level, fast and high performance graphics.

  5. Typefaces: Why is Palatino displaying "2" with a vertical line that makes it look like "4"? - Quora Says:

    […] this on several pages recently and it's independent of what browser I use. You can see it here http://www.satine.org/archives/2… for example.Palatino has been around for ages (I used it to typeset my thesis!) and it didn't […]

  6. kay Says:

    I think you meant “Garbage collection and sync operations”? Definitely agree, even just scrolling around the menus is a far superior experience on an iOS device than it is on Android. The smooth screen-to-screen animations and low-latency, high FPS scrolling generates a sense of “connectedness” you just don’t get with any Android device I’ve tried.

  7. Sivan Says:

    These issues will also be a constant drag on WP7, BlackBerry and webOS platforms. All of which have chosen the easy route of interpreted/managed languages and/or using webKit as a runtime.

    The reality is that these platforms are significantly behind iOS in ways that haven’t yet been called out but manifest themselves in subtle UX advantages.

  8. sha Says:

    fwiw the latest skia commits (the 2d drawing and animator library used on Android) added a new gpu backend… maybe a sign of better things to come. Also I would agree with Romain Guy that boost the GC (which they did in 2.3) is a major contributor to better responsiveness

  9. Andrew Says:

    The funniest thing is that the original iPhone had no touch-latency problems and it came out in 2007 as the first multi-touch phone. It’s 2010 and Android still struggles in this area. I think the “feeling” of using an Android device is broken in many respects after using an iPhone because everything seems slower. Little things you take for granted in iOS such as the rebounding at the end of menus/apps really comes out when using Android. My brother got a Droid and I remember the first time I gave it a try; I put my finger down to scroll the home screen and my mind said “YUCK! That doesn’t FEEL right.” I made up my mind from there that I would never buy an Android device. I’ve tried the Droid 2 and the Galaxy Tab and it still has that YUCK feeling. There’s a certain joy that comes from using an iOS device in that it just feels right.

    One thing I do want to stress is 99.9% of consumers are non-tech-savvy (unlike us) and go off first impressions. These consumers are your mothers, fathers, sisters, brothers, grandmas and grandpas. They just want something that works and if you gave them an Android device and it stuttered, they would ask “Why is it stuttering? I put my finger down and made the gesture, then why did it jump like that?”. That’s a bad first impression to make.

  10. Pierre Lebeaupin Says:

    So, not only is Android animation (the rendering of each frame, to be more accurate) not pervasively GPU-optimized, but they’re actually running Java code/Dalvik bytcode at animation time? Dear Lord. Guess why Core Audio and the guts of Core Animation are implemented in C(++), not even Objective-C (and that’s not because the engineers who implemented these are masochists). In these slides the Android guys tell application developers not to use temporary objects and other stuff in their UI code (well, it’s unclear in the slides the scope of the code that should be Garbage Zero, but I interpret it as any code that may run concurrently with an animation – in other words, most code on the UI thread) to avoid generating garbage so that no GC pauses occur at animation time (hopefully? I don’t know the design of the Dalvik VM, so I don’t know whether this works); this is so limiting for Java that coding in C/C++ would in fact be easier.

    It would be better for the Android framework code running at animation time to run in a real-time* environment where pauses cannot happen – which means C/C++ on a separate thread; even just that would free application developers to generate garbage in their UI code (which many third-party programs must be doing already) and improve perceived animation performance already. To be clear, this is not so much a performance problem as much as a real-time* problem, and the latter cannot be solved with better hardware, regardless of the viability of such a thing. Also, it’s no problem for Android drawing code (or any other application UI code really) to be not real-time*, as this counts as setup (as long as it doesn’t interfere with animation or the pauses are long enough to hamper responsiveness).

    *in case automotive or other “real” real-time programmers read this, by real-time here I mean “real-time enough to support video, animation and sound”. It’s obvious neither iOS or Android, or their kernels (XNU/Linux, respectively), are hard real-time OSes like QNX or VXWorks.

  11. Tochi Says:

    I’ve been following Romain Guy since his days at Sun (filthy rich client etc) and I’ll take his word over that of a random blogger who does not appear to have a clue.

    ps dual core actually uses less power

  12. The Care and Feeding of the Android GPU (Charles Ying/satine.org) | BuyElectro.com Says:

    […] Ying / satine.org: The Care and Feeding of the Android GPU  —  Android has two major technical UX problems: animation performance and touch […]

  13. Dennis Forbes Says:

    Just out of curiosity, is this yet more pontifications of an iPhone guy about what ails Android? Such tends to be the norm when it comes to such critiques of issues that just aren’t issues to people who use Android day to day.

    I highly suspect the author picked this topic and many of the talking points directly from Hacker News.

    http://news.ycombinator.com/item?id=2058375

  14. Clarence Odbody Says:

    @Tochi

    I wouldn’t say that Charles Ying is a “random blogger.”

  15. JasonK Says:

    Tochi-

    I’d suggest next time take a look at someone’s resume before commenting on whether or not the person has a clue. His resume is linked right up there on top to the right.

    I’ve been following Romain Guy since before his days at Sun. He’s a smart guy. It doesn’t mean he knows everything.

  16. Andrew Says:

    @Dennis Forbes

    I’m sure it doesn’t bother people who use it day to day, but for many consumers who have used iPhones, or are coming from feature-phones, the unknowns of whether the device will stutter or even register a touch can make or break a deal. It makes the user feel like they’re doing something wrong.

  17. Andrew Says:

    I also wanted to add that these devices are touch based. They have huge screens and touch UI’s and that the centerpiece of these devices. If you put a finger on the screen, should it not feel like you’re actually interacting with the object? Otherwise, what would be the point of having a touch screen at all?

  18. Dennis Forbes Says:

    “Otherwise, what would be the point of having a touch screen at all?”

    That’s a bit hyperbolic, isn’t it? What we’re talking about here are very minor stutters during longer running interactions such as scrolling a list. Interactivity is seldom impacted by the GPU issue. Event dispatching itself was an extremely heavyweight event, often causing interaction issues (having nothing to do with the GPU), however that is largely resolved in 2.3.

    Regardless of Mr. Ying’s many credentials, I think the Android crew have a far better understanding of the pitfalls and benefits of using the GPU versus the CPU. Further, assuming that firing up a GPU (often one of the most power demanding parts of a device) to scroll a list yields energy savings seems remarkably presumptuous.

    I have no doubt that Honeycomb will start to make better use of the GPU to add bling that does a better job selling the devices to a new user, while having marginal value on longtime users. Nonetheless, the problem is overstated and the benefits of the solution are oversold.

  19. bms Says:

    The problem is that Android is written in Java.

    I have yet to see a desktop application written in Java that performs well. Every time I use eclipse, it pauses unacceptably whenever I click on a menu or perform any other GUI operation.

    I have never seen this with a Qt or GTK compiled application.

    The same problems appear on Android.

    -bms

  20. Janne Says:

    @ Dennis Forbes:

    “”Just out of curiosity, is this yet more pontifications of an iPhone guy about what ails Android? Such tends to be the norm when it comes to such critiques of issues that just aren’t issues to people who use Android day to day.”

    Sure, jaggy UI might not be an issue for Android-users. But that doesn’t mean that it’s not a real problem. Some people are simply willing to overlook certain issues. you car might be a gas-guzzler, but you might be willing to overlook that problem. But that does not mean that the problem does not exist.

    I have used Samsung Galaxy S, and I have an iPod touch, and my wife has an iPhone 4. And the UI on the Samsung does seem jaggy and unsmooth. Sure, some people might be willing to overlook that, since they get some other benefits with Android. But to me it feels jarring. It feels like it ruins the suspension of disbelief, so to speak. When that UI moves around unsmoothly, I’m reminded that I’m operating a piece of technology, and unoptimized, clumsy piece technology to boot. It feels like the device is struggling to do the things I tell it to do. On the iOS-side, where the animations are butter-smooth, you lose that feeling of clumsy technology. things just happen, and they happen effortlessly, smoothly and without problems.

    Sure, you might say that this does not matter, since we are talking about “intangible” things. this is not a feature on a spec-sheet. But those intangible things is what makes the device pleasant (or unpleasant) to use.

  21. Dennis Forbes Says:

    @Janne,

    No, I’m saying that once its your day to day device it simply fades into the ether and is no longer of significance. I live with those minor lag interruptions daily, and yet I bravely manage, and it tends to be the people who don’t and wouldn’t use the device who consider it such a critical failure.

    Not only do I manage, but I love my device, and it fulfills my needs perfectly.

    There are certain things that sell a product at first impressions, whether it’s a car, a television, a smartphone, etc, where those same qualities just don’t matter when you actually own the device. For instance the Kindle is lucky that it’s generally sold online, because every single user when first using the device will note “OMG look at how slowly the page flips…I’m losing the illusion that this is a virtual book as I sit here mashing forward and backwards”. But then someone gets the device and reads books on it, and that initial complaint disappears and is forgotten: It just isn’t meaningful day to day.

    I’m not saying GPU acceleration wouldn’t be nice: It matters to a small degree for actual users (though I find the whole “it broke the 4th wall and my illusions are shattered” argument positively ridiculous), and it matters a hell of a lot on the sales floor where people have naive purchase requirements.

    @bms,

    Android is written in C. Go ahead, download and build it yourself – http://source.android.com/source/download.html

  22. Nick Says:

    There seems to be a disconnect in this discussion between those who feel that UX is vitally important and those who feel it’s a relatively minor issue. This is typical of any discussion of Apple technology versus the alternatives. Those who really value UX gravitate toward a company aligned with that. Those who don’t see so much value in UX dismiss them as fanbois, etc.

  23. Dennis Forbes Says:

    @Nick,

    False dichotomies only convince the already converted.

    What we’re talking about are infrequent, minor stutters during animation events. That’s it. This is the current “why Android is wrong” talking point among the anti-Android crowd. If and when they completely resolve that, then the pounding point will be the kerning of some random font, or other such inanity.

    Speaking of user interfaces, though, given that I care about them: the iPhone has a notoriously horrendous notification system. To me, like with most users, that is really, really critical given how important notifications are to smartphones (these are staying connected and notified devices). I mean galaxy-sized relative to a minor stutter during a scroll. If you care about user-interfaces at all, clearly you wouldn’t be able to tolerate such a device. And speaking of terrible designs, how about that visual garbage dump that is the iPhone’s SMS app? That thing is absolutely appalling. Anyone who cares about user interface clearly couldn’t align themselves with such a company, right?

    No device is perfect. Every device has many, many faults. The iPhone is not the pinnacle of design, and for all of its elegant and smooth GPU-using animations, it hosts a number of user interface abominations.

  24. Janne Says:

    @Dennis Forbes

    I’m not saying you should hate your device. There are millions of happy Android-users, and to them, the jagginess is obviously not an issue. Biut there ARE people who do think that is an issue, me included.

    I use a Nokia phone, have used for years. They are crap, but before iPhone it did not bother me since there was nothing better. Yes, the UI was crap and and jaggy. iPhone changed that, it showed how a phone could work.

    Using Android reminds me of Symbian: jaggy and overtly complicated UI. That might not be a problem to some. After all, Symbian has it’s fans as well. But to me that complexity and jagginess makes the system feel awkward and clumsy. To you that might not be a problem, it is to me. And no, placing value on smoothness and fit and finish is not “naive”.

    Am I saying Android is crap? Not at all. It’s an excellent system wig many good features. But smooth UI is not one of them, and it would drive me crazy. That’s one thing Apple does: they are insane about the little details. You barely notice the (un)smoothiness of the UI, but deep down you do feel it. Having it work smoothly creates a more pleasant system. Google might be comfortable in cutting corners here and there, but Apple is not. Both approaches has it’s pros and cons, it depends on what you value. I value user experience, you might value something else. Neither of us is right or wrong, we just have different preferences.

  25. Charles Ying Says:

    To clarify, there are parts of Android that are GPU accelerated, the window-level UI compositor, SurfaceFlinger, the live wallpapers running RenderScript. But it’s important to highlight the most core and used UX scenarios (scrolling, panning, zooming) that Android has not addressed yet, and I was shocked to see that they’re still not addressed in Gingerbread. OEMs have been petitioning Google to fix these problems for over two years now. They’re now taking matters into their own hands.

    Honeycomb may hold the promise of addressing some of these core UX issues. I just hope Android marketing team spends some time scrolling in the browser, rather than the 3D Google Maps application (which is hardware accelerated).

    I know a few folks on the Android team; they are extremely smart, and I’m sure they will fix these problems in the future.

  26. Janne Says:

    @Dennis Forbes

    “What we’re talking about are infrequent, minor stutters during animation events. That’s it.”

    Um, no we are not. I notice it every single time I switch screens in Android. It’s not something I see rarely, it’s something I see all the time when I move about in Android. Switching screens, scrolling… Basic UI -manipulation.

    No, you do not need to defend your choice of phones to us, nor do you have to start point out various shortcomings in iPhone (it does have them), since those are besides the point. Like I said: Android is a fine OS. It has it’s shortcomings. To you they are not that important, but they are to others. Are you now going to say how those people are “misguided” or something?

  27. Dennis Forbes Says:

    @Janne,

    “I see rarely, it’s something I see all the time when I move about in Android”

    It does depend upon the device.

    “No, you do not need to defend your choice of phones to us”

    You have some illusion that such is my motive? As a developer I understand some of the decisions the Android team made, particularly since the platform targets a extremely broad range of devices. I comment based upon that. If you feel some need to chalk that up to defensiveness you do yourself no favors. Or it’s just a trite tactic to try to disarm a position.

    “nor do you have to start point out various shortcomings in iPhone”

    To be clear, I was replying to someone positing that you either care about user interface concerns, or you don’t, following up with the statement that the former is in the Apple camp, and the latter is “others”. That is preposterous. I, like most users, care strongly about user interfaces, but can accept that a stuttering UI isn’t a significant concern in day to day use.

  28. Dennis Forbes Says:

    @Janne,

    “Google might be comfortable in cutting corners here and there”

    This is an interesting tangent. From a developer perspective I would say it’s quite the opposite.

    Google developed for the lowest common denominator, scaling up as appropriate. Really they started too low, targeting even ARMv5 devices, of which I don’t think any were ever even made. Their development was made much more difficult by that effort: They couldn’t simply pick a chip+GPU combo and a screen resolution and develop a straightforward solution, but they had to accommodate many variations.

    Maybe you have NEON instructions…maybe you don’t. Maybe you’re actually running a Moorsetown x86 processor. Maybe you don’t have a GPU, or maybe you have one that is limited to a single surface. And so on.

    Apple, in contrast, had the luxury of building iOS against a single, perfectly defined target (with minor adaptations in later versions). As many of us are developers, I think we would all find the second option much more enjoyable and productive, and we can see how much it would lubricate and ease a choice like “should we use the GPU?”.

    Now that Android has legs, Google is retrenching a bit with Honeycomb and we will likely see a lot more specific hardware exploitation — note how everyone is revolving around Tegra2 for that platform. I would not be surprised if we see Android split off into a targeted platform and a general platform post Honeycomb.

  29. gpu_mystery Says:

    Charles – You seem like a know-it-all guy to me from your resume. Would it be too much of work for you (or any one else for that matter) to explain in as much laymen terms possible the following –

    a) What actually happens when you do scrolling – step by step. a) You touch a list on screen and move it up/down b) A Scroll up/down event is generated c) List implementation catches the event – etc. all the way down. b) What operations a GPU can offer to aid in a scrolling operation and why that would be faster than the CPU

  30. Michael Says:

    @Janne :”Um, no we are not. I notice it every single time I switch screens in Android. “

    Um, I’ve never seen it on my OG Droid or my gf’s EVO. Um, are we 14 now, and start sentence with “Um”? lolwut?

  31. Mister Snitch Says:

    “I, like most users… can accept that a stuttering UI isn’t a significant concern in day to day use.”

    It isn’t a concern TO YOU. But you don’t come across as a guy who’s big on listening. Beating others over the head with his superior opinion, condescension – that’s more your style.

    Meanwhile, the REST of the world (you know, the millions of people whose view differs from yours who are, therefore, insignificant) cares a whole lot about those little things that aren’t “significant” to you.

    You are NOT “most users”. You are not even “like” most users. And you are certainly not better informed or educated than most users. Get over yourself.

  32. gpu_mystery Says:

    @Janne – I have used 4 Android phones so far – Nexus One, G2, Epic 4G, EVO. It never bothered me any time that the UI was less smooth or scrolling was jerky. And I am a fairly heavy and technically inclined phone user. I had to read the Interwebs to ‘know’ that Android has UX issues!

    That is not to say it could not get better – the Animations part when switched off make my N1 a bit faster – so they could improve that part and make it all more smooth. But it’s not an show stopper or even anything that normal users would notice in day to day operations.

  33. Dennis Forbes Says:

    While I appreciate your illuminating perspective, Mister Snitch, let me reiterate the observation that most of the people complaining about Android not using the GPU are not Android users at all.

    Their experience with the device is almost always limited to “I tried a coworkers once and…”, or “I read on Apple Insider that…”.

    It’s a tourist from Bodumkim, Idaho declaring how New York City should fix their subway system, because clearly insufficient lighting on the subway platform is the primary issue the riders face. It’s a valid observation and would probably make it a better system for tourists, but it lacks the context to understand how it really fits in day to day living of the people who actually use it.

  34. Haemish Bew Says:

    I agree that GPU compositing is an absolute must. But garbage collection isn’t a problem: a good collector (either JDK7 G1 collector or the RTSJ collector) has undetectable pauses. The problem is the specific bogus collector that Android uses.

  35. Craig Hunter Says:

    repost from: http://hunter.pairsite.com/blogs/20110104/

    I remember many years ago when Apple introduced hardware accelerated UI operations into Mac OS X. At the time, I knew this would be a big deal someday. In the short term, it paid off in all kinds of smooth, cool animations in OS X. But many of these were eye candy, and the benefit to UI interaction often went unnoticed to end-users (things just worked better and smoother and nobody cared why). With a mouse-driven UI, there’s a disconnect between input and display, so users are less apt to notice responsiveness on a micro level.

    When they released the very first iPhone in 2007, Apple fully leveraged their experience with OS X to create a mobile OS with a hardware accelerated UI. I doubt anybody outside the developer community cared about the details, but it was obvious that the very first iPhone was benefitting extensively from GPU acceleration when you scrolled lists, panned the viewport, etc. And UI animations were fantastic. It was no longer eye-candy — it was magic that made a touch interface functional and responsive. To this day, I give a lot of credit to Apple for getting the touch UI right, and a big part of that was GPU acceleration.

    Many mobile touch screen OSes have come along in the time since, but few have offered the smooth slick feel of iOS, despite often running on devices with big guns for CPUs. I can pick up a Blackberry, WebOS device, or Android device, and immediately tell that OpenGL and GPU acceleration are not present in the UI. Once you know what hardware acceleration offers, software rendering sticks out like a sore thumb. It’s slow, it’s laggy, and it’s at the mercy of anything else that may be using the CPU (which happens to be a lot of things on a typical smart phone with apps that use location services, camera, accelerometer, compass, gyro, etc). In short, it’s just not a good way to do a touch interface because the graphics will never keep up with user inputs in a responsive, smooth, or consistent way. I think consistency is perhaps the biggest factor, and it’s the biggest problem I see when using Android. With a touch interface, consistency is as important as accuracy and precision. Things need to respond the same way all the time, or the interface feels like a kludge.

    The notion that Android’s solution lies in faster CPUs is wrong for many reasons. First, it’s inefficient. Software rendering with a CPU is like mowing your lawn with a pair of gas-powered scissors, one leaf of grass at a time. Sure, faster scissors will improve performance, but the experience will still suck. A lot. Hardware accelerated rendering is like firing up the lawn mower, which can cut entire swaths of grass with a single sweep of the blade. You don’t need to be an engineer to envision which approach will take less gas overall. GPUs are inherently more efficient at rendering, and you want to use them over the CPU whenever possible. The benefits in heat generation and battery life are significant, and compound. Every developer knows this. You just don’t use the CPU for important rendering when you can access the GPU. The GPU will always be more efficient.

    And that leads to the second reason that this dependence on software rendering in Android is wrong — it shows a lack of commitment to doing things the right way from step one. Google’s casual attitude towards hardware rendering is not surprising to me. If the Android UI was not designed from the ground up to take advantage of the GPU (like iOS was) then it’s not going to be an easy task to put it in now. Trust me, I’m a reluctant developer sometimes, and I can spot it a mile away. Revising an OS to include low level GPU acceleration in the UI would be one hell of a grind. But it’s something I believe Android must do if it really wants to go toe to toe with iOS.

    Look at it this way — CPUs are getting faster across the board, and Apple is no slouch with custom ARM hardware (not to mention that Apple no longer clings to CPU architectures courtesy of their PowerPC experience; they’ll go with different silicon if that’s what it takes to lead the game). iOS devices will match any increases in horsepower on Android devices. But those iOS devices will also be leveraging the GPU on top of that, and GPUs have been even farther ahead of CPUs in their pace of advancement. So this is a race that can’t be won when relying solely on software rendering. No amount of gas will make those scissors catch up to the mower.

  36. The Care and Feeding of the Android GPU | JetLib News Says:

    […] to subscribe to the RSS feed for updates on this topic.bonch writes “Charles Ying argues that Android’s UX architecture has serious technical issues because, unlike the iPhone and Windows 7, Android composites the interface in software to retain […]

  37. Corbin Simpson Says:

    As I’m sure you’re aware, some things suck when GPU-accelerated. Glyphs are the traditional example, but with things like the SGX, nearly all 2D operations are slower on GPU than CPU. I think that the Android team is fairly on-the-dot with their remarks and opinions.

    Moreover, none of us really know what the SGX chipsets are capable of, because we have no documentation, no benchmarks, no free access to the drivers, and so on. It’s entirely possible that GPU acceleration is worth it, but we need to actually be able to write the damn drivers before we can see for sure what they’re capable of.

  38. Aaron Says:

    Great read!

  39. SleepyDaddy Says:

    C# and .Net have certain low level features, such as value types, that make it more suitable than Java/Dalvik VM for doing managed UI with hardware accelerated compositing. For example, drawing a bitmap in software to a bitmap cache and transferring that bitmap to the GPU can be done more efficiently in .Net than in Dalvik/Java.

    Garbage collection pauses are not as much of an issue in .Net as they are in Dalvik, because of the heavy use of value types in the core Silverlight framework, especially in the transforms, colors, coordinates, etc..

    Sync issues are also not as much of an issue. Compositing is done on a separate thread, plus the entire framework is designed to avoid Sync issues. In most cases a synchronous version of a potentially blocking API call is not even available – the asynchronous version is the only one available.

    All that being said, it doesn’t really matter whether you are running on Dalvik, .Net, or native C++ runtimes – you can write clunky/slow/jerky UI code in any one of them. It’s up to the developer to optimize and streamline their code.

  40. Winston Edmondson Says:

    A little perspective from a non-technical lover of gadgets. It’s interesting to see how emotional this topic is. It makes me want to learn all the ins & outs of mobile operating system and UX programming so I can see what’s really going on.

    It seems to me that many of you (@Dennis Forbes) are not empathetic to the consumer experience. Little annoyances (animation stuttering while scrolling through a list) can sabotage all of a business’ brand loyalty efforts. Furthermore, little annoyances that we “live with” and tolerate often wait patiently in our subconscious. We may not even realize we’re annoyed. As soon as we get a chance to play with a friends device that has no such issues, we can go from contentment to anger in the blink of an eye.

    My T-Mobile (HTC) Wing was the most expensive phone on the market at the time. I loved it. Then the HTC Touch Pro 2 came out, and I had to have it because it solved some of the problems that I didn’t consciously realize my old phone had. Then I got an iPod Touch, because WinMo6.5 had an awful browser. Today, I have a Samsung Vibrant & a Galaxy Tab. Within the last week, I purchased an HTC HD7, and although I still use the Vibrant as my main phone, I’m in love with the Windows Phone 7 UX. As soon as they add a few must haves (copy & paste, etc.), I’ll make the switch.

    So yeah, sometimes the investment, or the data they have in the phone, or the exclusive apps effectively trap consumers into tolerating a phone, but they won’t tolerate it forever. Why wouldn’t Android engineers experiment with suggestions like this in order to improve the user experience? Why would an Android fan become upset when someone is offering useful suggestions?

    Quick question…is there not a way to independently produce a proof of concept to prove or disprove Charles Ying’s theory? Sorry if that question is naive. I’m just a consumer.

    -w

  41. Venkatesh Srinivas Says:

    a) What actually happens when you do scrolling – step by step. * a) You touch a list on screen and move it up/down * b) A Scroll up/down event is generated * c) List implementation catches the event – etc. all the way down. b) What operations a GPU can offer to aid in a scrolling operation and why that would be faster than the CPU?

    At a very low level, a scroll operation can ultimately be expressed as a few memory copies — one copy to move part of the list that will still be visible up/down and another to copy data from a(n) offscreen buffer(s) onto their final location in a framebuffer. CPUs can move this data around themselves via lots of memory copies or they can ask for a GPU to do so. The GPU can do more — while it is copying the data, it can apply simple transforms or blend the image with other images.

    … garbage collection blocks UI thread …

    :\

    Remember Android is not running Java code under the Hotspot JVM; instead it is using the Dalvik VM (not a JVM!). The Hotspot JVM has a number of garbage collectors, one of which is even designed to reduce pauses (G1); other designs for excellent collectors are also floating around. I don’t know what Dalvik is doing, but if it is perceivably interfering with the UI in 2011, something is wrong.

  42. Dennis Forbes Says:

    “It seems to me that many of you (@Dennis Forbes) are not empathetic to the consumer experience”

    It’s the scale that we’re talking about. Pre-Froyo even the Nexus One had frequent, irritating pauses. The 1.5 hosting G1 and the Magic — close to unusable, with horrible multi-second pauses that were reminiscent of the Windows Mobile days.

    I wouldn’t dare recommend Android to friends or family then. I knew they would have a miserable experience. I stuck with it as a developer, and because it really wasn’t worse than the Moto Q, but didn’t enjoy it.

    Now, with current devices and Froyo or above, however, it is so incidental and ignorable that seeing hay made out of it now is just…odd. Gingerbread goes even further in minimizing those minor hesitations, and the Android team has been quite explicit and reasoned in their explanations. Assumptions that the GPU guarantees either performance improvements or power savings are both presumptuous.

    And again, there are the questions of motives. When most of the people talking about Android’s lack of GPU acceleration are iPhone advocates, iPhone developers, and iPhone fans, you have to recognize how this became the talking point.

  43. SleepyDaddy Says:

    @Venkatesh Android has a concurrent garbage collector in Gingerbread. But, considering that only 43% of Android devices with access to the Android Marketplace even have Froyo installed (the first version of Android with a JIT), several months after its release, it will probably take another year or so before enough people have Gingerbread for that to even matter to the developer. Developers have to target the largest audience possible. Currently that means more than half of Android devices out there don’t even have JIT, let alone concurrent garbage collection.

  44. The Care and Feeding of the Android GPU « marcussbarber Says:

    […] Source: http://www.satine.org/archives/2011/01/01/the-care-and-feeding-of-the-android-gpu/ […]

  45. RKDavies Says:

    Samsung Galaxy S devices WILL always show a bit of lag and hesitation after being turned on and running for a while. It’s an issue with RFS (The filesystem Samsung chose to put Android on top of). There are fixes and workarounds (OCLF) that will remove 99% of any delay seen while navigating between menu’s and other on screen elements.

    iPhone may have done a better job seemlessly bouncing between apps and menu’s, but I see the same type of stutters and lag on iPhones and iPods just as much as I have seen the same behavior on older Android devices.

    What does an iXXX have that Android doesn’t? Netflix and a ton of fanboi’s in black turtlenecks.

  46. brooklyn_wry Says:

    The reality of using a technology product, or any product, is that a user must sometimes accept less-than-perfect.

    My opinion, and I realize it is only that, is that for a developer to aim for less than perfection only assures that they’ll never hit it. I’ll be the first to admit that these guys are a lot smarter than I am, but when I conceive or strive to build a piece of software, I want it to be perfect.

    The goal of human innovation and engineering is to build things that are remarkable, that defy previously held expectations or assumptions. To take what we have (not what’s coming) and make it as good as possible. For software, I think this means an interface should be as fluid and responsive as possible. It should place as few demands on the underlying hardware as is possible, period. To simply say “the processors of the future will take care of that” is bullshit, not because it isn’t true, but because it simply dispenses the idea that something better is attainable now. It is counter to the spirit of innovation and even broader human progress. Putting off until tomorrow what you could achieve today is a sad excuse, nothing more.

  47. Luis Nell Says:

    I think Dennis brought up a very important topic – platform support.

    Apple always had, and always will have, the luxury of knowing which components are inside the products their software runs on (that’s also the reason osx86 is only possible thanks to custom kernel patches introducing drivers etc.). In contra we have f.e. Microsoft. Microsoft needs to support many components so their system can run on each one of them, at least in some basic mode. Remember running XP without GPU drivers resulted in some 100% supported mode for devices (usually 640×480 16bit). When you installed GPU drivers you got max. screen resolution, GPU acceleration etc.

    We can observe the same problems with Android. Google wants to offer a reliable, secure, and most important, open system. Therefore they can’t introduce optimizations as easily as Apple (which nowadays drops iOS support for older iPhones/iPods). I think that is the true problem. GPU acceleration is something that can help, however the Android team also needs to keep the code base as compatible as possible, – which by the time is a truly difficult job.

    Another factor is people. See, the Apple crowd (I’m also using OSX and iOS) is glad to see a new product – it doesn’t matter much if your current hardware is compatible with the new and shiny software that is coming. If it isn’t compatible, you have to buy something new. Sure, this annoys a lot of people (myself included, iOS4 is really laggy on my iPhone 3G). However, Apple gets the marketing right in this case. The Apple-Crowd is accustomed to need the newest product to run the newest software smoothly.

    On the other side we have Google. I will take Microsoft as some kind of analogy. Highly compatible systems need a lot more transition time from old to new platforms. See how long it took Microsoft to prepare users for Windows 7 – a whole OS between XP and 7. Now, people is happy to see a nice Windows, without all the slug that Vista introduced. A lot of people consider the migration to 7, with new hardware (I exclude companies here).

    It is tough to navigate in a fragmented market. Google does a great job and it will take some time until the Android Team can truly create the system many people want to see. They have done hell of a job the past years and I am sure we will see many exciting things. Note that Google does something different than Microsoft (besides openness etc.), they also throw out their own phones. However, they do not abuse this fact (they could bring out their own phone with full fledged support for GPU acceleration etc. since they know their hardware in this case too).

    Where this leads, we will see.

    Note that I completely ignored the fact of custom ROMs, which would blow the size of this comment a LOT.

    Just my 5 cents #offtopic. ;-)

  48. gpu_mystery Says:

    @brooklyn_wry – Your thoughts on crappy iOS notification system that is there since 4 major releases of the OS?

  49. Jnoonan Says:

    I’m probably going make some people mad, but I am going to state my experience. For starters, I am a droid x user. I was an iphone user, but defected to the android world so that I could be on verizon since at&t’s network is as bad a advertised where I live. I like my droid x, but this operating system is very inferior to ios. It is laggy. Brief pauses occur nonstop. Everything is extremely slow. I also have used my collegues droid x and it is the same way. The evo and galaxy are the same too, but I do not own one, I have compared the ones my friend’s own. Android has a long way to go to catch where the iphone was in 2007 when it comes to responsiveness.

    The only real problem with the iphone is apple’s control (if you consider that to be an issue) and the att network. Unless there is a severe change I will either go back to the iphone when it lands on verizon or I will consider the win phone 7. I have always disliked java programs because they are slow….my phone has enougg hardware that menus should not be so jerky and non-fluid.

  50. Jnoonan Says:

    typos are present from typing that on my phone.

  51. gpu_mystery Says:

    @Jnoonan – Does your Droid X look like the bigger phone in this video -http://www.youtube.com/watch?v=yexYJDzUKng ? I am worried some one sold you a pet rock with “DoridX” written on it.

  52. brooklyn_wry Says:

    @gpu_mystery

    Push notifications on iOS are rubbish, and that’s far from the only problem. But on the topic of UX animation I think Apple’s work is solid.

  53. eric Says:

    I can see the lag in this vid. http://www.youtube.com/watch?v=B0lngKZ-bgU

  54. Aubrey Anderson Says:

    Thanks so much for calling this out, Charles! This has been our experience exactly when trying to get even close to passable WebKit rendering on android when porting even super lightweight apps over from iPhone / iPad. We had visions of you trying to port Flipboard and thinking “WHAT THE HELL IS IT DOING??? DRAW!”

    My kingdom for translate3d on Android!

  55. Nobomaniac Says:

    Sure, native code almost always produces faster execution than managed code. However, the VM layer provides many many benefits for Android phones. Just think what would happen if Apple decided to swap architectures on IOS? Rosetta or fat binaries, that’s what!

    If you have a problem with the speed of execution on a VM, just imagine a full translation layer working to emulate a differing architecture.

    P.S. Romain Guy is about what you would have if you cubed my intelligence, so take this post with a grain of salt.

  56. sl_yamma Says:

    Average battery life? Does that mean you are suggesting something like the iPhone has ‘above average’ battery life?

  57. iinlane Says:

    Just implementing the draw functions might not be enough – i do not think it solves jerkiness. Move to retained mode graphics would solve the problem even when GPU is not used.

  58. gpu_mystery Says:

    @brooklyn_wry So what u said below is not applicable to Apple/notifications?

    “The goal of human innovation and engineering is to build things that are remarkable, that defy previously held expectations or assumptions. To take what we have (not what’s coming) and make it as good as possible.”

  59. Andruid Says:

    The idea that it is o.k. to not use GPU acceleration is sad. The biggest advances in computer performance over the past 5-10 years have come in the GPU. Its likely that this will continue over the next cycle, and perhaps forever. The GPU plays a crucial role in making computing an immersive media experience.

    Not incorporating the GPU wherever reasonable into graphics is the opposite of having a computer architecture strategy.

  60. Richard Says:

    Nevertheless, the number one performance drain, by a factor of 10 over everything else, is the screen. 2 hours of ebook reading (indoors) is enough to totally empty the battery. So I do agree with Google to the extent that there is plenty of spare power available for the GPU, in the context of how atrociously hoggy the LED screen backlight is.

  61. Joe Says:

    I’ve had a Galaxy S for three months now and have never noticed any of this purported lag in my heavy use of it. My screens/lists scroll and rebound silky smooth all the time.

    Perhaps this is a problem for some devices, but clearly some vendors have gotten it right.

  62. sunil agrawal Says:

    i agree with you Ying that view and animation should be accelerated but not through GPU, it should be through 2D GFX HW. Using GPU for accelerating 2D Gfx APIs will be disaster. Good news is that most of android devices coming in the market seem to carry powerful 2D GFX HW but unfortunate thing is that it is not being used at all.

  63. Janne Says:

    @gpu_mystery

    “I have used 4 Android phones so far – Nexus One, G2, Epic 4G, EVO. It never bothered me any time that the UI was less smooth or scrolling was jerky. “

    Yes, it does not bother YOU. It bothers ME. We all have different priorities, and we value different things. Android-users obviously dislike some aspects of the iPhone, and they choose Android instead (which does not have those shortcomings). I for one dislike certain aspects of Android, and choose iPhone instead (well, in reality I have company-issued Nokia-phone, but still).

    It’s interesting that the Android-folks are not saying “No, the UI is perfectly smooth on Android”. What they ARE saying is “It does not bother me”. That means that we are talking about personal preferences. Android-users are not bothered by the unsmooth UI, and I respect that. I’m not going to start telling you that you are stupid or misguided, because you are not. You have your set of priorities, and I have mine. They obviously differ. So are you now going to start telling me how MY priorities and preferences are wrong and misguided, and how YOUR priorities and preferences are obviously better? I don’t think it quite works that way….

  64. Janne Says:

    @Dennis Forbes

    ““I see rarely, it’s something I see all the time when I move about in Android”

    It does depend upon the device.”

    I have no interest in searching for the one Android-device that does have smooth UI. And Samsung Galaxy S is one of the top-notch Android-phones available. And, FWIW, the UI is smooth on the older iPhone/iPod touches, so I fail to see why state-of-the-art Android-phone struggles with the UIl, while older iPhones/touches do not.

    “let me reiterate the observation that most of the people complaining about Android not using the GPU are not Android users at all.”

    I’m not a day-to-day user of Android, but I have tested it as part of my job. But are only Android-users allowed to critique Android? How many of the people who critique iPhone (in this discussion even) are iPhone-users? You are obviously an Android-user, yet you critiqued iPhone’s notification-system. Why do you have the right to critique iPhone, but iPhone-users do not have the right to critique Android?

    And let me re-iterate: this is not a “my phone is better than your phone!”-thing. Nor is this about Android being crap (it’s not). It’s about Android having unsmooth UI. And interesting point is that Android-folks here are not disputing that claim, what they (you included) are saying is that “it does not bother me”. Great, I’m happy it does not bother you. It bothers me. It bothers others. What are you going to do about it? Come over to our home and tell us in detail how stupid we are?

    All phones and OS’es have various shortcomings. iPhone is only available on AT&T in USA (for now at least). Yes, it’s notification-system is poor. Yes, you only have one model to choose from. And so forth. To some, those might be dealbreakers. To others, they might not be. Android has jaggy and complex UI, crappy copy&paste, lower-quality apps, carrier crapware… To some, those might be dealbreakers, to other, they might not be. We all have to pick our poison, some choose iPhone, others choose Android. Neither of those are crap, they are both great. They both have their own set of problems. It just happens that the unsmooth UI os one of Androids shortcomings, and that is a problem for some people. And the opinion of those people is perfectly valid.

  65. tania Says:

    @Dennis Forbes and @gpu_mystery

    It’s rather juvenile how you sum up all Android critiques here as iPhone fans. It is people like you that degrade any well written topics regarding Android down to fanboyish level. What a shame.

    If you both would stop being overtly emotional and act a little mature you would have noticed that Charles Ying’s (and most posters here) main purpose by pointing technical issues in the Android platform was to improve the Android development as an OS. Or in layman’s term, to make it better than it already is.

  66. Storybird Games » Android UI on the GPU Says:

    […] Ying talks on his blog about the technical design of the UI on Android vs iPhone GPU-accelerated Core Animation. According […]

  67. Dennis Forbes Says:

    Weak response, tania. There is nothing “juvenile” or fanboyish in anything whatsoever that I have said on here. Go ahead, please review again and cite examples.

    What you and others with the same tired, boorish response tactic don’t like is that I’m right, interrupting your little critique fest.

    Ying’s observation is nothing new or unique — we’ve had this conversation since Android first appeared, and people immediately noticed the superior bling of the iPhone. It is old news. So when someone wallows in and makes such a proclamation, assuming that it is somehow novel or insightful, that’s really just kind of funny.

    And finally, the makeup of the critics are important. John Gruber linked here — unsurprisingly — and the guy is the raging firehose of iPhone fanboys: Where John Gruber links you can usually be sure to find the same sort of, excuse the wording, circlejerk of people desperately trying to argue themselves a new reality in the face of overwhelming evidence to the contrary. So I’m not surprised to see posts such as yours.

  68. 2011: Year of the iPad | Lessons of Failure Says:

    […] for nearly the same cash outlay.  Never mind that engineers looking at the Android have discovered major issues with the compositing and view system which are primarily software-based, giving extremely poor responsiveness in the touch interface and […]

  69. gpu_mystery Says:

    @Dennis Forbes is right on. Listening to Apple fanboys talking about Android deficiencies is like Bin Laden pointing out flaws of democracy – sure, democracy isn’t perfect but the alternative is worse – however much shiny it looks from distance ;)

    If you understand Jobs better – Things are packages of emphasis. Some things are chosen to be done in a product and some or not. Android has so far focused on doing things that they feel were higher priority for the platform – cloud integration, JIT, Concurrent GC, Tablet UI for example – as Charles himself said he is sure the UI Acceleration problem will be solved in time. But so far it hasn’t been a active show stopper issue like the iPhone fans are making it out to be – just as the lack of useful notifications haven’t made a significant dent in iPhone adoption.

  70. tania Says:

    @Dennis Forbes

    Simply put, if you can’t contribute to offer solutions to make Android OS a better experience I suggest go to a more user oriented site as Charles’ article is geared towards developers. Stop polluting this site with your fanboyish rants. You’ll achieve absolutely nothing. That goes for you too gpu_mystery.

  71. gpu_mystery Says:

    @tania – You can’t be serious. Neither Charles nor you or any of the other fanboys seem to have given any solution to make anything better. On the contrary it seems like you are all here to tout Apple’s superiority. (I would excuse Charles from this however).

    So you telling us to go elsewhere achieves only one thing – lets you spread FUD without any challenge. Stop claiming that you are offering any solutions – you aren’t.

  72. Gorilla Logic Blogs » Blog Archive » 2011: Year of the iPad Says:

    […] for nearly the same cash outlay.  Never mind that engineers looking at the Android have discovered major issues with the compositing and view system which are primarily software-based, giving extremely poor responsiveness in the touch interface and […]

  73. Honeycomb Says:

    http://www.youtube.com/watch?v=hPUGNCIozp0

    Everything is smooth and nice on Android 3.0 Honeycomb thanks to the power of the GeForce GPU inside the Tegra 2 SoC.

  74. Dennis Forbes Says:

    @tania,

    Cute retort. The premise that the discussion here is technical is adorable.

  75. will29 Says:

    Android has two major technical UX problems: animation performance and touch responsiveness. Android

  76. AndroidFan Says:

    Android is a very useful thing as it sparked competition and forced more investment in better devices from all smartphone contenders. Google didn’t get everything right (nor has anyone else) and Android graphics stack is not even an optimal raster implementation. There are real performance and battery life gains to be had from improving this stack (and GPU is one of the things that can be done but it isn’t a silver bullet). GPUs are good with moving/transforming a lot of data not very efficient for imperative mode primitive painting and each chip/driver combination has different strengths/weaknesses which does make it hard to have a single open implementation. On the other hand I wish Google decoupled their platform from their services and provided equal opportunity to all platforms.

  77. AndroidDev Says:

    P.S. Skia is GPU accelerated in Honeycomb thanks to Mike Reed.

    http://code.google.com/p/skia/source/detail?r=649

  78. PhilH Says:

    Just stumbling through and wanted to add my two cents. While the technical debate over GPU acceleration is fine theres quite a bit of FUD mixed in from iPhone fans on this topic as well. Actually I think there are two common problems that occur in these discussions.

    First I think some may be confusing the need for hardware acceleration with poorly developed apps and especially home screen apps. For instance the home screen of a Droid X is sluggish. Try replacing it with ADW Launcher or the new Go Launcher and see the difference. Theres none of this stuff about lack of “connectedness”. The only time I may see a glitch is if theres a lot of app updates going on in the background and a rare unexplained stutter. I’ve seen that much on some people iPhones honestly.

    The other problem is the FUD. Its comments from people that I don’t think have ever even tried an Android device. They’ll say my phone is slow and I even tried a Galaxy S and its still slow. Now this completely contradicts all the reviews and video of phones ever since the Desire and Incredible were released and especially after Froyo. Theres video on top of video of Android phones and pretty much none of this lag is apparent. The only lagging lists are those that load data as you scroll for the most part. So somehow these phone were fast at launch from all the reviews and videos but over time they magically get slower according to people that tried them but aren’t happy with Android. People claim Android fans are denying things but the proof is right there on the internet. The naysayers really do more to blow up 1 stutter in 100 swipes to be a serious UX problem. But as someone else here mentioned not being able to deal with notifications on my own time or dismiss then when I’m ready are far greater UX problems. But as I’ve noticed for some time now Android users and iPhone users are totally different. iPhone users seem to just consume content and things like the notifications don’t bother them. Android users use their phones like computers and can’t deal with having to respond immediately or loose notifications.