Enabling Video for Handset and Handheld Devices

There’s more to mobile phones than making phone calls

Increasingly consumers are relying on their mobile phones to do more than make phone calls. Phones can be used to play games, listen to music, send emails, and watch video. However, the market is currently limited by economic realities. The more features a phone has, the more a phone costs, and most consumers do not want to pay a lot of money for a phone. Also, in markets where operators offer low prices for phones in return for long-term contracts, operators are interested in protecting their margins with low-cost phones. At the same time, they don’t want to give up the opportunity for incremental revenue that comes with added features. Among the hottest markets today are 3D gaming and video for phones, which happen to be the most resource-demanding features for phones.

Video is connectivity

The fact that people carry a mobile phone with them everywhere they go means they can rely on it to keep them in touch while they are on the move, and keep them entertained at the same time. Increasingly, people are coming to rely heavily on video for information. In fact, in some parts of the world more people have televisions in their houses than have a telephone line. It’s only natural that as people get access to still pictures, music, and Internet connectivity, that they see video content as a logical addition. The rapid advance of the camera phone into the mobile handset market has made even video creation seem practical.

Video can consist of entertainment such as movies and prerecorded and/or live TV, and it can be educational such as training and company information. The usage model for video is well understood. It’s been 50 years in the making.

Market requirements

Video on mobile phones has been impractical until the introduction of third generation (3G) networks. Second generation (2G) cellular networks are capable of data rates up to 9.6 kbps (kilobits per second) for upload and 29.8 kbps for download—either way, that’s not very much. 3G wireless networks deliver data rates from 64 kbps for upload and 200 kbps for download—making video much more practical.

The accompanying chart describes different wireless technologies in use today.

3G networks have been rolled out in Asia and Europe and, after what seems like a very long wait, they are coming to the U.S. The major carriers in the U.S., including Sprint, Verizon, and Cingular Wireless, plan to roll out new services throughout 2005 and 2006. T-Mobile plans to start deploying HSDPA, a high-speed service within W-CDMA, in 2006.

The time is right, but challenges persist

Just because many consumers want video services, that does not mean they’ll be happy with anything they get. People are accustomed to very high-quality TV broadcasts and they will be disappointed with anything less on their handsets. This is not speculation. Consumers have rejected low-cost digital video recorders and computer-based TV cards that have not been able to deliver the smooth frame rate and high level of detail that consumers expect from anything labeled a TV. With the introduction of DVD and highdefi nition services, the bar is being raised even higher.

All approaches to providing video on handsets are based on MPEG-4, an ISO/IEC standard developed by the MPEG (Motion Picture Experts Group) Working Group. As the name

Table 1: Worldwide rollout of 3G networks has begun.

Figure 1: The transition to higher speed networks is well underway. (Source Jon Peddie Research)

suggests, it is a refi nement of earlier MPEG standards, MPEG-1 and MPEG-2. MPEG-4 enables digital television with a high rate of compression and it also defi nes standards for interactive graphics applications and interactive multimedia. H.264 is a further refi nement of MPEG-4 written by the Video Coding Experts Group (VCEG) together with the MPEG group. It is synonymous with MPEG-4 Part 10 and the name generally includes AVC, for Advanced Video Coding. Thus the standard is often called H.264/AVC, or even MPEG-4/ H.264 AVC to communicate the participation of all these groups. The name alone is indicative of the complicated nature of international video standards.

H.264/AVC enables very high compression rates with low bit rates and high quality. The bit-rate requirements of H.264/ AVC can be half that of MPEG-2. However, the drawback to H.264/AVC is that it requires more processor power for code/ decode (codec) operations. As a result, video quality on today’s phones can be very disappointing since codec operations are often performed in software relying on the phone’s CPU. It is therefore necessary to use a video hardware accelerator to meet the expectations people have for video.

The handset market is very volatile at the moment. Handset manufacturers are building phones with a variety of hardwired and programmable components in order to meet customer demand for more features. These manufacturers desperately need new ways to advance their technology without building completely new products every six months.

One way to build a handset/handheld device that has the performance and feature to please the most people is to add hardware acceleration components for every possible feature consumers will want—whether it’s still pictures, games, audio, touch screen, or video. In practice, of course, this results in products that are undesirable in terms of cost, weight, and power consumption for the mainstream of users.

The Falanx approach

The Falanx IP product line includes the Mali Pixel Processors and the MaliGP Geometry processor. The processor cores can be used to build co-processors or SoCs (system on chips). In addition, Falanx’s patented technology gives developers a full range of choices for product lines. For example, in addition to entry-level products with baseline features such as video and/or 2D graphics, product designers can “turn on” additional features such as advanced 3D graphics for higherend products.

Mali Pixel Processors

The Mali Pixel Processor family includes the Mali200, Mali110 and the Mali55. These are single-pipeline 2D/3D/Video IP rasterizer cores, which means that the cores are capable of full graphics performance and also video. As a result, developers do not have to choose between 3D graphics or video. The Mali110 and Mali55 deliver 30fps video encode and decode, they support MPEG-4 encode and decode and also H.264. The Mali Pixel Processors also support 4X Full- Screen Anti Aliasing (FSAA) with no performance loss and 16XFSAA and texture fi ltering for advanced performance in games, graphics, and multimedia. Falanx’ pixel processors come in two designs—the small die-size Mali55 is for the value market and the high-performance Mali110 is designed for smart phones, feature-phones, etc.

Falanx has optimized their IP cores to function effi ciently with ARM cores, the leading processor core for handhelds and handsets in this industry. Building on the Mali product line features, Falanx has taken advantage of industryleading codecs for better performance in encoding and

Figure 2: Today, most mobile phones do not have the ability to handle video content. Over the next three years however, the mix is going to change as people buy high-performance multimedia phones. Source: Jon Peddie Research

Table 2: The Mali Pixel Processors offer video features as well as anti-aliased graphics. Falanx’ technology includes early z occlusion, which the company says can boost performance by as much as 70%. Video features include motion estimation, motion compensation, video scaling, Image Diff, and Color Space Conversion.

decoding, and for load balancing. Through their collaboration with an industry leading codec provider, Falanx has taken advantage of this company’s expertise to develop ARM side scheduler for parallel image capture, encoding and decoding operations, and to leverage the abilities of the ARM processor for Huffman variable length (VLE) encoding and decoding (VLD).

The Mali55 can be implemented in a 2.0 mm2 die area with 190k gates, with average power consumption at 0.4 mW/ MHz running at 200 MHz for a very small-footprint, lowpower solution that has full video functionality. The Mali110 is designed for higher performance devices and may be teamed with the MaliGP to add advanced 3D performance to the Mali110’s video and graphics capabilities.

MaliGP

The MaliGP geometry processor core accelerates integer, fl oating point, and 3D graphics. It offers high performance but maintains a low gate count. In addition, the MaliGP improves the performance of other media algorithms including imaging (JPEG2000), audio and video compression and decompression. A chip using Falanx’ MaliGP and running at 150 MHz can deliver 5 million triangles per second (measured in transform operations) and MPEG-4 encode and decode at VGA resolution at 30 fps. The combination of the MaliGP with the Mali 110 enables a system capable of H.264 decode and encode.

Falanx delivers both quality and performance

Figure 4 gives a clear-cut picture of video decode performance relying on the CPU alone, adding only the MaliPP and also the MaliPP with MaliGP. As you would expect, both processor cores contribute to performance and make a signifi cant difference in the ability of a product to handle video.

When it comes to H.264 the challenges presented to hardware in handheld systems are even more dramatic. All the ingenuity of the industry’s video experts has been brought to bear on the creation of H.264, a video technology for highly compressed video that maintains the highest video quality. The approaches for decoding H.264 are standard but when it comes to encoding video for recording there are different approaches that can be used with varying complexity and, obviously, varying quality. This is not a game that anyone can play and video quality will remain a differentiating feature on the mobile phone for some time to come.

Figure 3: Falanx’ Mali IP products work with ARM processors to provide complete solutions for low gate counts and low power consumption.

Figure 4: As this chart indicates, MPEG-4 encode is very demanding of system resources. Current ARM processors cannot deliver the performance required for the high level of video quality expected by the typical consumer. ARM tested at 200 MHz, MaliPP tested at 120 MHz, MaliGP at 80 MHz.

Summary—beyond video

Video is just one element of multimedia and it is only part of the story for handheld devices. The primary function of most handheld/handset devices is voice communication. The introduction and runaway success of cameras for cell phones proves that people want multimedia functions including video. It also seems clear from current usage models that users want games and they especially want 3D games. However, consumers have also demonstrated that they want long battery lives, and they show little inclination toward giving up the subsidized models offered by carriers today, which make phones relatively inexpensive with service contracts. An approach such as that offered by Falanx gives phone manufacturers the ability to add features such as 3D gaming without sacrifi cing video. Manufacturers can also build a line of phones—from mainstream multimedia phones to advanced smart phones—on a single platform.

Falanx Microsystems AS
Postboks N-2182
7412 Trondheim, Norway
Phone: +47 4000 5757
E-mail: info@falanx.com