EventDV.net
Search EventDV

EVENT-DV 25
2010 Awards Show
2009 All-Star Team
2008 All-Star Team
2007 All-Star Team
2006 All-Star Team


RELATED SITES
Streaming Media Producer
OnlineVideo.net
Streaming Media
EMediaLive Archive


PRIVACY/COOKIES









Copyright © 2004 -
Information Today, Inc.



Benchmarking Adobe Premiere Pro CS5
Posted Oct 1, 2010 Print Version     Page 1of 1
  

Last year, my studio filmed a software user group conference in the ski resort and Winter Olympics host town of Whistler, British Columbia. We filmed sessions in two rooms simultaneously, live mixing PowerPoint and software demonstrations with our two single-camera feeds. We used a pair of EDIROL V-8 video mixers, and to avoid long capture times, we captured the mix live to a FireStore in one room and an editing system running Adobe Premiere Pro CS4 in the other. Two days later, I had 28 1-hour sessions that needed to be exported as soon as possible to Flash video (On2 VP6) and uploaded for my client. My client didn’t want to stagger the release of the videos for fear of implying that any one presentation was more important than one that was released later. So I was under a lot of pressure to perform some quick edits for the intros, add some titles and voice-over, trim the in and out points, and wait for the exporting to finish.


It took an entire week to complete the rendering. As a result, I missed my deadline. I attribute my failure to a few key mistakes. I started rendering with the computer that I took to Whistler—my AMD 6000+ dual core system, running Adobe Premiere Pro CS4 and using Adobe Media Encoder. In hindsight, I should have taken the initial hit on additional transfer time and used my Intel Core i7-920 system to do the encoding, as it could process the files almost four times faster. Eventually, I did switch systems. But at that point, I had lost too much time to recover. But my biggest mistake was underestimating just how much time it took to render that much video to that specific codec.

Over the past few years, I have become less concerned with how much time it takes to render video from one format to another simply because most of my projects were still in SD—albeit now in widescreen—and computer processor advancements made quick work of rendering SD video to my two most common deliverables at the time: full-length videos to MPEG-2 for DVD and highlight clips to Flash video for the web. But following this failure, I became more concerned with speed on workflows that required exporting full-length productions to H.264, my new favorite web format, especially because the amount of HD productions I was delivering was increasing.

Prior to NAB 2010, I was contacted by Adobe’s PR company to get a sneak peak at the new CS5 release. The highlights, as they pertained to speed, were a native 64-bit architecture (meaning Premiere and Photoshop could address more RAM than previous 32-bit versions), the new Mercury Playback Engine, and CUDA GPU acceleration (see Jan Ozer’s “Five Alive” article for a full overview of the suite’s highlights). The Mercury Engine is software-driven and doesn’t require any hardware. The promise was that with an upgrade from CS4 to CS5, users would get faster playback and rendering performance.

Adobe Premiere Pro CS5

House of Cards
CUDA GPU acceleration builds on the Mercury Playback Engine, is hardware-driven, and requires the purchase of a compatible NVIDIA video card. The lineups of supported cards at the April launch included NVIDIA’s workstation Quadro line. Quadro-supported models include the $3,000 FX 5800, the $2,000 CX, the $1,500 FX 4800, and the $800 FX 3800. Premiere Pro 5.0.2, introduced in late August, added support for these new NVIDIA cards featuring Fermi architecture: the GTX470, Quadro 4000, and Quadro 5000. An $800 starting point does sound appealing, but support also extended to a single GeForce video card model—the GTX 285. When this card launched in 2009, it had a MSRP of $399. Unfortunately, for CS5 hopefuls who were looking to gain CUDA support, it was replaced by the newer and as-yet-unsupported GTX 470 model a few months prior to the launch of CS5, so finding a new one is very difficult. I ended up buying one used on craigslist (from a gamer who was upgrading his system) so that I could compare it with the FX 4800 for this review.

NVIDIA Quadro 5800

Incidentally, these two models are the only Mac-supported ones, although I did all my testing on PCs. My source at Adobe tells me that the GTX 470 will be officially supported sometime in the future, so be sure to check Adobe’s Premiere Pro CS5 page for an updated support list.

Testing Parameters
Armed with the lofty promises of improved speed, both with and without new hardware purchases, I set out to see for myself if, and by how much, CS5 was faster than CS4. I tested two very different timelines using the HDV codec at 1080i60. The first timeline was a 48-second timeline with two clips and three cross-dissolves. This was pretty close to the the most basic timeline you can have. It was designed to see how much time the actual encoding took if rendering was not a factor. 

The second timeline was much more complicated; admittedly, with eight layers, it was much more than anything I’ve ever compiled outside of After Effects. Four of the layers were HDV video. Layer one was chromakeyed with a second scaled video layer, and the two additional video layers were scaled as picture-in-picture layers. I also added a title layer, a matte layer with opacity to form a lower-third title background, and, finally, a layer with my company logo with a 3D rotation. All the effects I applied were identified by Premiere as GPU-accelerated effects.

Time in seconds—lower is better

This project was 48 seconds as well and used HDV footage. But peaking at eight layers deep, it was designed to push my test systems to the limit. When compared with the results from the first timeline, it would show how much additional time the complicated rendering took. I conducted my tests in both CS4 and CS5, in order to determine if and by how much the software Mercury Playback Engine and the hardware CUDA GPU acceleration improved performance. And iif this weren’t enough, I conducted the tests on desktop systems with three different video cards: a non-CUDA-supported ZOTAC 9800GT and a pair of supported cards, the Quadro FX 4800, and the GeForce GTX 285, to see just how much of a difference the video card made. Once the footage was on my timeline, I exported to Adobe Media Encoder and chose one of three H.264 custom presets I created for 360p, 720p, and 1080p. I then recorded the encoding time on a spreadsheet. In general, the higher resolutions took longer to encode than lower resolution formats. So in the interest of simplicity, I’ll only discuss the 1080P results in this review. I also want to mention that I repeated these tests with Canon 5D Mark II footage, and the times were very similar.

Adobe Media Encoder CS5

SIDEBAR: Don’t Trust the Presets
I used to blindly choose presets in Adobe Premiere Pro based on the preset name without paying too much attention to the encoding parameters. Who was I to question Adobe’s preset settings when I was exporting my video to YouTube, Vimeo, and my iPhone? I figured if Adobe called it the YouTube preset, it must be the best setting for video that would end up on YouTube. I was wrong.

It wasn’t until I started benchmarking for this review and noticed major anomalies in the encoding times that I took a closer look. Adobe’s frame rate settings were all over the place, and this was causing problems. Besides the fact that encoding with a different frame rate from acquisition is probably not what you were intending to do when you select an export preset, encoding theory dictates that changing the frame rate from the acquisition to the delivery can introduce problems and, most importantly, it takes time to add the additional frames. Now let me clarify that this wasn’t a case of Adobe designing a shortcut for performing a 60i/30p transfer to 24p in an attempt to achieve a “film look,” but rather a case of selecting the wrong frame rate with absolutely no benefit and lots to lose.

I ran down the Adobe H.264 preset list and quickly got confused. I even questioned if I was missing something. But I checked with Apple, YouTube, and Vimeo and concluded that, although they all help to add confusion to the matter, there is still a problem with the Adobe presets, not with my understanding of frame rates and encoding.

Inexplicably, the Apple iPod/iPhone preset was set to 30.0fps for SD, while the widescreen (WS) preset was set to 29.97fps. Why was there a different frame rate depending on the aspect ratio? It didn’t make sense; then it only got more confusing. The Apple TV preset for 480p was 29.97, but the 720p preset defaulted to 23.976. The HDTV presets conveniently included 24, 25, and 29.97 in their preset names to denote the frame rate for both 720p and 1080p options, but they used 24.0fps instead of 23.976. The Vimeo preset was no better, with the HD setting defaulting to 29.97 and the SD preset using 30.0fps. Finally, the YouTube preset got it all wrong and used 30.0fps for SD and WS SD and 24.0 fps for WS HD (and, yes, I too thought the WS was redundant for the final HD setting).

Why are the frame rates all over the place? In North America when we record video in 60i/30p and 24p, we almost exclusively use the drop frame rates of 29.97 and 23.976. So that is the setting we should use when selecting our project and sequence settings, as well as export settings. Yes, drop frame rates can be a bit confusing. But when it comes to encoding settings, there is a big difference between drop frame rates and nondrop frame rates, and the two can’t just be used interchangeably. Yes, there are some examples of actual 24.0fps and 30.0fps shooting—notably on the 5D Mark II before Canon updated the firmware and, most recently, on the Apple iPhone 4G—but those are the exceptions and not the rule. So shame on Adobe for creating export presets with nondrop frame rates, such as 24.0fps and 30.0fps. For the record, Adobe did a really good job of using the correct drop frame rates for its project/sequence settings, including offering both 24 and 23.976 frame rate preset options for digital SLR acquisition. So the frame rate problem is only evident in its encoding presets. I checked back as far as CS3, and although Adobe had fewer presets two generations ago, as some of the playback options had not been invented yet, the frame rate problem was still there. So this has been going on for a while unchecked.

Is Adobe solely to blame for this confusion? Not entirely. It didn’t create the drop frame standard. But the company should know better. Although, as these examples show, it isn’t the only one contributing to frame rate confusion.

Here is what YouTube has to say about frame rate:
“The frame rate of the original video should be maintained without re-sampling. In particular pulldown and other frame rate re-sampling techniques are strongly discouraged.”

Despite this, in its own settings for Apple users, it uses 30.0fps. So YouTube got it right by encouraging users to maintain the original frame rate of the video, but its example uses the very unlikely acquisition frame rate of 30.0fps.

Here is what Vimeo has to say about frame rate:
“If there is an option that says ‘current,’ it is best to just go with that. Otherwise, this is usually 30 fps (frames per second) for the USA, Canada, and Japan, while in Europe and rest of the world it’s usually 25 fps.”

Vimeo gets it right by saying “current” but gets it wrong by stating that it usually should be 30fps in USA, Canada, and Japan.

In its product technical spec pages, all the Apple products list their frame rates as 30fps. But here is what Apple has to say about frame rates in its more-detailed developer technical notes:
“The output frame rate is determined by the source movie’s frame rate, limited to 30 fps using adaptive frame sampling.”

The same 30fps confusion happens with the Apple products, which all state they have a frame rate of 30fps in the summaries but should more accurately be described (as it is above) as having a maximum frame rate of 30fps (which would include 29.97 and everything below).

So the big three all give the correct advice—keep the source frame rate—but they give misleading examples that demonstrate otherwise. My advice is to make sure you use the same frame rate on acquisition as you do to edit and ultimately to export. This means modifying the majority of the factory presets to match the frame rates of your acquisition.

Still need more convincing as to why the frame rate confusion is a big deal? I encoded the same two test timelines of 29.97 HDV footage using my slowest laptop, a dual core AMD with a TL-50 processor, using the YouTube widescreen preset. This preset uses 30.0fps. Then, I repeated with a modified 29.97fps frame rate version with the same settings. The 30.0fps preset took 27 minutes, 42 seconds and 6 minutes, 11 seconds, and the modified 29.97fps preset took 13 minutes, 18 seconds and 3 minutes, 4 seconds to encode the two test sequences. In this scenario, selecting the wrong frame rate more than doubled the encoding time.

Test System 1
My first test system was considered a cutting-edge dual core system in 2006. That’s when I first put it together, based on the Videoguys.com DIY 4 build. But being 4-years-old, it is hardly expected to compete with my 2009 build—an Intel Core i7 system. Since its original build, the AMD system has been upgraded from its original AMD X2 4400+ 2.2GHz processor with 2GB of RAM to the more powerful AMD X2 6000+ 3.0GHz processor with 6GB of RAM.

Time in seconds—lower is better

This test demonstrates that upgrading even an old system from CS4 to CS5 improves encoding times on both the single layer and the eight-layer timeline using the software Mercury Playback Engine. The story is much different, however, when enabling the GPU acceleration.

There is no benefit for the simple timeline. But on the multilayer timeline, the GPU acceleration is three times faster than the baseline CS4 encoding time and two times faster than the software-only CS5 encoding time. So if you are running a system that is not quite cutting-edge and are hesitant to upgrade to CS5, this test shows you that the sweet spot for upgrading is to upgrade to CS5 with an approved CUDA card that allows GPU acceleration.

Test System 2
I built test system two in 2009, based on the videoguys.com DIY 7 system. As expected, it’s much faster than my AMD system. The results show that even in its slowest configuration—CS4 with an inexpensive non-CUDA card—it beats the AMD system’s fastest time, when it was equipped with CS5 and a GPU-accelerated CUDA card. But the point of this comparison is to determine how much benefit each upgrade provides. So here goes.

With CS4 my results show that for my test timelines, it doesn’t make any difference which video card is being used. This is consistent with what I experienced when I used previous Premiere Pro versions, and, specifically, when I dropped $500 for a now-discontinued Quadro FX 1500 card, with which I didn’t notice any improvements when I installed it into CS3. 

My video card performance experience was unchanged initially when I upgraded to CS5, and the encoding times were all the same, whether I was using an approved card or not. This all changed when I enabled the GPU with the approved cards (GPU acceleration is not available on nonapproved cards), as the encoding times for the eight-layer timeline dropped even further. Predictably, the times for the simple timeline remained unchanged as the GPU acceleration speeds up the rendering time but not the encoding time. 

SIDEBAR: Upgrading Your System
Purchasing a new system can be very costly. So upgrading an existing system can be a cost-effective way to increase performance without breaking the bank. You might be surprised by how inexpensive it is to buy upgrade components now, compared to when your system was new.

Before any new purchase or upgrade, take a few moments to review your minimum system requirements. Premiere Pro CS5 requires a 64-bit OS such as Vista or Windows 7 on the PC or Mac OS X v10.6.3 on the Mac. A 64-bit OS allows your computer to address (use) more than 4GB of RAM, so you will want to increase your RAM to take advantage of that. If you’re like me, you probably purchased the next-to-the-fastest processor that your motherboard could handle because when you purchased your system, the fastest probably cost two to three times more. Since then, even faster CPU models have probably been released, so upgrading your CPU should be at the top of your upgrade list.

Installing a CS5-approved CUDA card might be more involved than you anticipate. The approved CUDA cards are all full length and might require modifications or may simply be too large for some computer cases. Along with your other new components, they may also require more power, so you may also need a power supply upgrade while you are at it.

On my AMD system that is housed in a Cooler Master Centurion midtower case, I was only able to physically fit the full-length video cards when I moved around some of the hard drives. My Core i7 is housed in the slightly larger midtower Cooler Master NV690 case, and it fit without modifications. However, on my next system, I’ll be sure to build it with a larger full-tower case.

So is upgrading still worth the costs? It all depends on the number of existing components you can still use and the cost of the upgrade. As test system 2 will show, the speed on an upgraded older system won’t compete with the performance a new top-of-the-line system. But the cost of upgraded computer parts is probably in the hundreds of dollars, whereas a new system will cost a few thousand.

Comparing the CUDA Cards
When Adobe launched CS5, I was told that the GTX 285 was an approved CUDA card that would allow Premiere Pro to use the card’s GPU to speed up rendering, but that its performance wouldn’t match that of the Quadro line. When I pushed for an explanation, the Adobe rep told me that the lone supported GeForce card was limited to six layers, which didn’t seem like much of a limitation to me. But at the end of May, I received notice via the Adobe Updater that an update was available for Premiere Pro.

The update read: “The Adobe Premiere Pro CS5 5.0.1 update allows the Mercury Playback Engine to support additional layers on the NVIDIA GeForce GTX 285 GPU card.” The tests for my workflows show that it doesn’t make any difference to the export time which video card you used for a simple timeline or for a complicated timeline using the software-only Mercury Playback Engine for acceleration. On the other hand, because the GPU acceleration is available only when using approved CUDA cards, enabling the GPU acceleration made a big difference for multilayered and effect-heavy timelines, bringing their times very close to the times of the simplest timeline (although no benefits were observed when using GPU acceleration for a simple timeline). In fact, the encoding times for a simple timeline with CS5 were all within an acceptable margin of error, and I consider them to be equal, regardless of video card or GPU acceleration.

So what more can I say about Premiere Pro CS5? Adobe made big promises with two levels of speed improvements in this CS5 release, and the company lived up to them. With the first level, the software Mercury Playback Engine, encoding times dropped when compared to CS4, and these numbers dropped even further for multilayer timelines and timelines with effects, using a supported CUDA card.

Now as I close off this review I think it’s worth pointing out that the benefits of Premiere Pro CS5’s newfound speed go beyond just encoding times. The same engines that power the export encodes also work to accelerate even the most complicated timelines so that they play back in real time. As an example of just how fast the GPU acceleration is, my eight-layer timeline Premiere Pro took 15 frames to get up and running to full speed (which corresponds with HDV’s long GOP structure). But after that, it dropped only three frames for the remaining 47.5 seconds.

When it comes to speed on CS5, Adobe promised and delivered with the combination of the Mercury Playback Engine and GPU acceleration—and, in doing so, it has allowed us to edit and encode in a whole new world of speed.

Shawn Lam (video at shawnlam.ca) runs Shawn Lam Video, a Vancouver video production studio. He specializes in stage event and corporate video production and has presented seminars at WEVA Expo 2005–9 and the 4EVER Group’s Video 07. He won Creative Excellence Awards at WEVA 2010 and 2008 and an Emerald Artistic Achievement Award at Video 08.



EventDV Spotlight is now:
Email:
more info
more info

Print Version   Page 1of 1