I pitted the Canon XF300 against my 5-year-old Sony HDR-FX1, the first prosumer HDV camcorder to gain wide acceptance in the market, and the Canon 5D Mark II, the latest “video” camera (which, of course, is not a video camera in any traditional sense) to see wide adoption in numerous markets. Both the XF300 and the 5D have nearly identical 50Mbps bit rates. But the 5D uses the more efficient H.264 codec as opposed to the XF300’s MPEG-2-based codec. (Note that while both HDV and the new Canon codec are both MPEG-2-based, the Canon MXF codec is much more generous with image quality, with full 1920x1080 resolution, twice the data rate, and more than twice the color resolution of HDV.) But what will the H.264/MPEG-2 dichotomy mean when noise from gain fills the image?
For this test, I visited the studios of PixelPops Design, home of EventDV Graphic Thoughts columnist Lance Gray and his partner Brian Gunn. Gunn and I set up a static scene and used an external waveform scope to assess the light levels from the different cameras (Figure 2, below). I set the back wall to 80% luminance. Then I would ramp up gain or ISO settings at standard intervals and use other camera settings (shutter, ND filter) to bring down the light to keep the back wall at a consistent 80%.
Figure 2. Testing the cameras at the PixelPops studio
I exported uncompressed images from each of these video clips. Because this test was not done simultaneously on all cameras (the focal lengths of the 5D prevented that), you should use the individual images only to assess the image quality delivered by each camera as noise increases. Here are a couple comparison images from these three camcorders showing both mid-level and extreme samples of gain (Figure 3, below). The top image shows that each can produce some very clean-looking video at moderate gain. The second, more extreme example shows that high noise causes problems everywhere.
Figure 3. Mid-level (top) and extreme (bottom) gain levels compared on the XF300, the 5D, and the FX1
In the high-gain image, bottom left of Figure 3 (above), the 21 dB of gain in the XF300 kicks in some noise reduction that’s not present in the 18 dB image. So it looks cleaner, but now the color component becomes blotchy with blue and yellow patches. In the center slice, the 5D looks pristine, until you look closely and realize that so much fine detail is gone—smoothed out of existence. Also, the wood most clearly shows off the 5D’s blotchy green and orange compression artifacts. Lastly, the old FX1 looks noisier than the other two, but the level of visible detail surpasses either of the two newer cameras. The noise is restricted to the luminance channel, and the FX1 does not have anywhere near the level of color blotchiness of the XF300 or 5D.
Lastly, we used a resolution chart to try to differentiate between the three camcorders (Figure 4, below). The difference was not as dramatic as you might think.
Figure 4. Comparing the XF300, the 5D, and the FX1 on the resolution chart. Here, the newer cameras really shine, and the XF300 camcorder hangs in well with the 5D.
On the left, the XF300 looks phenomenal well beyond 800 lines—the camera softly resolves the vertical down to where the lines blur together. Horizontally, the camera even resolved the stair stepping evident in the printed glossy chart. The Canon 5D also did well, showing no aliasing that I could discern. The FX1 did not do as well here, limited by the subsampled HDV resolution (1440x1080) and the lower bitrate (half that of the other two camcorders). The FX1 has some interlacing evident in the circle. This came about because the only way to match the size of the chart was to pull a still from a slow zoom. The FX1 was the only camera shooting interlaced video. I reshot this later, but even without the zoom, the resolution of the fine detail in the middle was about the same.
For the second set of tests, I visited Dallas-based video production house GlobalWebHQ Productions. Production manager Adam Oas and I put four cameras side by side to shoot the same static scene and lowered the light on the scene to see which camera and which codec could hold up as the light went away.
Contestants in this gambit were the Canon XF300, the Sony FX1/Z1U, the Sony FX7/V1U, and the Canon 7D DSLR (Figure 5, below). All shot 1920x1080p30, save for my Sony FX1, which shoots 1080i30 (60 interlaced fields). The Sony V1U shoots 1080p30, but it records an HDV-compliant 1080i30 stream. Put the two fields together in software and you get your 30p back.
Figure 5. The Day 2 lineup, from left: the Canon EOS 7D, the Sony V1U, the Sony FX1, and the Canon XF300
We started with 80 foot-candles (fc) of light to give us a good overall exposure on our Barbie and Mr. Potato Head scene as judged on an external waveform scope. This placed the background greenscreen at 30%. We used this arrangement to set all the cameras as evenly as possible as we dropped the light levels down from 80 fc to below 3 fc, as measured with a Sekonic incident light meter.
We used 1/60-second shutter and kept the f-stop settings fixed to isolate gain adjustments, until we ran out of gain and used what little bit of iris we had left to suck in more light. The one exception to this was a 3 fc. We used the incredible 33 dB of gain on the XF300, which actually forced us to close down the iris to keep the overall exposure consistent.
The gain settings are not an accurate comparison between cameras because camera manufacturers can set 0 dB of gain wherever they want to. Indicative of this, the Canon XF300 offers -3 dB and -6 dB of gain for slightly cleaner images than the standard 0 dB gain setting. The 7D adjusts ISO higher, and there is no direct correlation to gain setting. Moreover, the higher bit rates of the XF300 and the 7D, not to mention the three different codecs used in these cameras, all play a part on image quality and compression artifacts due to noise. So the comparison images based on incident light readings are the only way to qualitatively compare the images from the different cameras.
In these select images from the 80 fc test (Figure 6, below), you can see some clear differences in image clarity starting with 0 dB of gain. The HDV cameras have more artifacting in the colors of the face compared to the XF300 and the 7D.
Figure 6. Image comparison at 80fc, o dB gain
When you get down to 15 fc and the various gain settings that have to be used to achieve the same image brightness, you start to see how clean and crisp the XF300 and the 7D are (Figure 7, below). The FX1 has a lot of gain noise, but as I noted in Day 1 tests, its luminance noise is evenly dispersed. The V1U has the poorest image. Please excuse the guest appearance by the viewing loupe for the 7D.
Figure 7. Image comparison at 15 fc
When you get down to 7 fc, the differences really become apparent. The XF300 may have noise, but the clarity and resolution of the image is unquestionable. You can see patterns in the flat green field up top, but I’d hazard a guess that it would still cut a clean key, especially with 4:2:2 color space to back it up. The FX1 has an overall grainy feel, but the edges of the figure are clearer than the V1U, which seems to be overcompensating with noise reduction, which makes it appear to have a cleaner image at the expense of edge sharpness. The 7D continues to have a nice bright image at the expense of edge sharpness. The shallow depth of field would hurt your ability to get a clean edge key. Compression artifacts and noise are very low, even at this high ISO.
Lastly, at 3 fc, I had to gain the XF300 up to 33 dB to get enough light. But then I had to iris down to 3.4 to keep the image consistent with the others (Figure 8, below). You can see that the noise from the gain here basically makes this image unusable in terms of quality. But if you absolutely had to get any image, the XF300 could do it. The FX1 and V1U were already maxed out at 18 dB gain, but they had a bit more left in the lens, so they were opened up from ƒ3.1 to ƒ2.0. This put them under the luminance levels we had set, and the issues from gain noise and noise reduction remain. The 7D image now starts to strain. Visible blockiness and color artifacting start to appear as the camera’s noise reduction compensates for increased noise. However, it still has the cleanest image of the group and is able to gather enough light to maintain the image brightness we had set at the beginning.
Figure 8. Settings necessary to get anything approaching a clean image at 3 fc
For kicks, we tried one more test. We dropped the light levels down to about 1 fc. The needle on the incident light meter barely twitched. But it did move, and there was light on the subject. The FX1 and V1U would be too dark to be usable, so we only shot the XF300 and 7D. You wouldn’t want to use either clip of video in a polished production.
The temporal artifacts of the noise and compression are understated in these still images. But the XF300 continues to maintain the sharp focus throughout the image that the small imager provides by design (but the 7D provides the cleaner and more pleasing image).
I’d have to say I was impressed that the XF300 got there, but the 7D proved that it could get there with better style and grace.
One request I got from a reader was to test skin tones and highlight handling as well as to test the XF300 against Panasonic’s HPX370. So with the assistance of Barak Epstein of Videotex Systems, who let me use the company’s HPX370 and studio, I put the two camcorders to the test as a Caucasian face was ramped up from 75 IRE on the skin tones to 100 IRE to see how Canon’s 4:2:2 50Mbps codec and image processing held up against Panasonic’s 4:2:2 112Mbps DVCPRO HD codec.
The HPX370 was a bit soft—an unfortunate situation—but the skin tones are still there for comparison between the cameras in these images. The skin tones hold up well as you go up the scale. Obviously, there seems to be a big change once the face hits 100%; neither looks good. But I’d have to say that even at 95%, the Canon XF300 looks better than the Panasonic here (Figure 9, below).
Figure 9. Skin tones comparison, with the XF300 images on the top and the HPX370 on the bottom
As I write this, Canon just announced the XF100 series with the same 4:2:2 and 50Mbps in a smaller and potentially more affordable package. I suspect that small camcorders like this are the death knell for HDV. Yes, there is AVCHD, but that’s trading one color space-reduced MPEG-2 codec for a color space-reduced MPEG-4 codec.
The XF camcorders more than double the color resolution, and they give the whole image a lot more breathing room in the data rate. This is a distinct advantage over HDV and AVCHD, and it bodes well for color grading, keying, and overall image quality.
You can read a lot about “shooting for grading” online and see samples of video that looks dull and washed out—which ensures that all of the black and white detail that the chips can record is saved to your video file. Video that looks “snappy” right out of the camera has settings that lose black detail and crush some whites. So tweaking the camera, any camera, to ensure that you get what you want means that every possible adjustment changes the results of a test.
After trying three times to conduct seemingly scientific tests comparing disparate camcorders and then seeing the great disparity of the images that result, I’ve come to the conclusion that entirely objective tests are impossible. There are so many ways to adjust the internal image control settings any more that any adjustment, or nonadjustment, can have a big effect on the resulting images. I counted 68 different internal settings on the canon XF300 that can be tweaked for any given custom picture profile.
So the results of all this testing tell me that the cameras continue to improve, look better, and offer more capability and faster access to your data.
You can see clear depth of field differences between the XF300, or any video camera for that matter, and a DSLR such as the 5D or 7D. In deference to shallow depth of field purists, in my testing, having various parts of the foreground subject be out of focus was actually a problem. In terms of cutting a key, you need that edge to be sharp. I needed more depth of field. I know that can be achieved with a higher ƒ-stop, but that loses light and forces the DSLR into higher ISO’s noise reduction and artifacts faster. That makes it less useful. It was actually quite nice to be able to shoot at ƒ2.0 and have everything in focus for the still-life tests. But that was only possible with the video cameras, not the DSLRs.
There was a slight blue shift with the XF300 as the gain increased, but I saw it only at higher gain settings; 9 dB of gain proved surprisingly clean. Compare the CMOS-based XF300 to the CCD-based FX1 or the CMOS-based V1U, and you’ll realize that the type of chips used don’t make anywhere near as much difference as the amount of light the lens can gather and the quality of the video processing inside the camcorder. In both these respects, the XF300 excels. The massive light gathering of Canon’s sizable integrated lens (83mm diameter threads), the 4:2:2 color space, and the generous 50Mbps data rate of the MPEG-2 codec make for some beautiful and extremely sharp images. Of course, you can dial down the sharpness with several of those 68 in-camera adjustments.
I put the XF300 up against both a Canon 5D and a Canon 7D expecting the XF300 to be trounced when it came to low-light capability. In the end, the difference wasn’t anywhere near as great as I expected. Yes, the DSLRs can shoot a little cleaner at lower light levels. But at light levels you’ll see in typical productions, the XF300, like the 5D and 7D, is extremely clean, the only appreciable difference being the greater depth of field offered by the XF300.
The DSLRs had a surprising amount of artifacts in the video, considering they were shooting the same 50Mbps as the XF300 but with half the color resolution (4:2:0) and using an interframe codec that’s supposedly twice as efficient (H.264). Given those distinct advantages, I would have expected the DSLR footage to blow away the XF300, but it just didn’t.
If you can add one or two lights, would like to record high-quality audio into the same file as the video, and have ready access to all the camcorder settings for adjustments on-the-fly—like a feathery smooth zoom rocker—the XF300 offers an incredibly capable camcorder with many innovative features and image quality that is very hard to beat until you more than double the camera’s $6,799 MSRP.
The MX format needs editing software from the last year or so, but the advantages offered by tapeless ingest are well worth it. Canon clearly did its homework with this camcorder. Adding the new, smaller XF100 series to the family only further solidifies the format’s future growth. Now there’s a huge Canon fan base looking for the same camcorder wrapped around the Canon 5D imager (XF500?). If Canon wants to quickly take over a significant share of the prosumer camcorder market, adding a camcorder like that to its XF lineup will certainly make it happen.
Anthony Burokas (VidPro at ieba.com) of IEBA Communications has shot award-winning corporate video internationally and recorded events since the days of 3/4" tape. He is currently technical director for the PBS series Flavors of America and resides just outside of Dallas.