satine.org

by Charles Ying

The PlayStation Web App

September 27th, 2011

This is the new PlayStation Video Unlimited service. This PlayStation app runs at a full 60 frames per second (when you see it on a PS3), has tons of 3D graphics effects, full-speed 1080p video playback, and a fluid, hardware accelerated, animated user experience. What you may not know is that this is a web app.

A Web App? On A PlayStation?

The Video Unlimited service is a JavaScript application with a carefully designed runtime platform and very lightweight APIs to access hardware accelerated 3D graphics and shader effects, video playback engine, and other aspects of the PS3 hardware.

Two years ago, I helped start this project at Sony. In six weeks, our team took a working Flash UI prototype and recreated it on a PS3, complete with an early version of the platform, now internally called Trilithium. Alex Bustin, the same UI developer who built the original UI prototype, also wrote the Trilithium port.

The release of Video Unlimited was delayed until now, but Trilithium was used to build another of Sony’s partner’s apps, Hulu Plus for PS3. (See video at the end of this post).

The Trilithium Platform

Trilithium’s strength comes from taking full advantage of the PS3 hardware and existing well-optimized frameworks to do everything from graphics to video playback, leaving the decisions about the high level application to a very flexible JavaScript core API.

We built Trilithium for several reasons:

  • Make good use of the complex 8-core + GPU PS3 hardware without killing ourselves.
  • Give this power to our UX developers and designers.
  • Let partners easily build their own PS3 apps with little knowledge of PS3 architecture.
  • Rapidly develop with a flexible environment.

True, there’s no hyperlinking and Trilithium isn’t open (for now).

But Video Unlimited, Hulu Plus, and future Trilithium apps do show what’s possible when you bring the best parts of web and native technology together.

Hulu Plus for PS3

The Care and Feeding of the Android GPU

January 1st, 2011

Android has two major technical UX problems: animation performance and touch responsiveness.

Android’s UX architecture needs work. UI compositing and the view system are both primarily done in software. Garbage collection and async operations frequently block UI rendering.

Android team members are still in denial on the importance of GPU acceleration. They recommend eliminating garbage collection to improve animation performance. They say drawing isn’t the bottleneck and GPU accelerated 2D drawing won’t yield good results:

“It is very naive to think that using the GPU to render text and bitmaps is suddenly going to fix every issue you may see. There are many things that can be done to improve performance of the UI without using the GPU. Notably improving touch events dispatching, reducing garbage collection pauses, asynchronous operations to avoid blocking the UI thread, etc. A one year old NexusOne (and other devices before) is perfectly capable of scrolling a list at close to 60fps (limited by the display’s refresh rate.) Using GPUs to do 2D rendering can introduce other types of inefficiencies (fillrate can be an issue, some primitives like arbitrary shapes are complicated to render with antialiasing, textures need to be uploaded, shaders compiled, etc.) I am not saying we won’t do GPU rendering for the UI (I have worked on it myself a couple of times to test it) but please stop assuming that this is what has to be done right now.”
Romain Guy, Android software engineer

No one is saying that Android’s “2D primitive drawing” needs GPU acceleration. It’s Android’s view system and animation compositor that needs GPU acceleration. To compare, Core Graphics is still mostly software based while Core Animation is entirely GPU accelerated.

Look at Samsung’s Galaxy S browser. GPU accelerated and tile-based. I’m told it’s a result of Samsung’s PowerVR GPU optimizations. Smooth as butter, runs circles around the Nexus S Gingerbread browser on the same hardware!

Stop executing Dalvik Java VM code on every animation frame. Use the programmable GPU graphics pipeline. Add a scene graph if it makes sense. Run it on a separate thread. You might even get 32-bit graphics along the way.

Android engineers say that better hardware will eventually solve the problem — an insane rationale for the problem. On mobile, power efficiency is king. Throwing dual cores or more GHz at the problem is just going to get you more average performance with zero battery life, and even then, as long as your screen doesn’t get too big.

Wake up, Android team. Windows Phone 7 just lapped you.

Update: Additional discussion on Hacker News.

Dashing Xcode

July 6th, 2010

Jesper thinks that Apple is working on a new language. I’ve posted something similar in 2007, but a lot has happened in 3 years.

I believe Apple’s new runtime language successor to be (or very close to) JavaScript with an Objective-C bridge. Here’s why:

  1. JavaScript is already one of the iOS platform languages.
  2. JavaScriptCore Nitro is fast.
  3. JavaScriptCore already bridges with Objective-C.
  4. Apple uses JavaScript in deep, key areas: iAd, iTunes 9, Mobile Me
  5. DashCode (and PastryKit).

Apple is probably still sorting out:

  1. Security + sandboxes. I once asked Doug Crockford what he thought was in JavaScript’s future and this was it.
  2. LLVM compilation. For Apple, it makes sense to have a compiled language story that improves JavaScript performance even further via compile-time analysis and optimization, courtesy of LLVM. It’s great for iOS code validation too.
  3. Framework design. Apple is known for taking time to try out features and APIs to get them right before publishing them.

Update To Jesper’s point about replacing Objective-C. I believe this successor adds a new level to the Apple language family and doesn’t replace Objective-C. Use JavaScript where appropriate: high level memory safe, dynamic, functional code with a productive, concise syntax. Drop into Objective-C to do performant, memory efficient, or security-privileged things. Drop into C for more speed, more efficiency. It’s turtles all the way down, and in many ways, we’re already there.

Technorati Tags: , , , ,