Video Compositing 101
Posted Jun 2, 2004

Compositing isn't as tough as you might think. With the right tools and some basic skills, creating video overlay effects and even "virtual sets" is easier than ever.

Compositing sounds complicated, but it's really not. Simply put, it's the process of merging two videos (or a single video and still image) to display one combined video. Compositing is also called "overlaying," as in overlaying one video over another, and "blue screen" or "green screen," which are specific compositing techniques used to combine two videos.

The simplest application of compositing is inserting a logo or other bitmap into a video. Much more exciting is video compositing, a technique used to accomplish a variety of goals, including inserting your subjects into videos of different locations, and even placing them in "virtual sets."

The production side of video overlay, both the shooting and editing, may seem functionally demanding, technically complex, and prohibitively expensive, but it's really none of the three, and non-technical, small-outfit users can efficiently and effectively accomplish both.

Key Notes
Video compositing offers far greater creative potential than still image overlay, and with a little coaching, you'll find the process of shooting and compositing video relatively straightforward. Even better, you don't need to spend a lot of money to get a good result. Let's examine what actually happens when compositing one video over another, and then move to the equipment, software, and techniques used to produce a good, clean result.

At a high level, compositing combines two videos by eliminating portions of one video, and placing the remaining portions over the other video. The background behind my image was green, so I used a process called chromakeying (also called color keying) to remove everything that's green in the video and place the remaining portion, just the video of me, over the video of the school.

What are the keys to good keying? There are three worth noting. The first is a simple rule of thumb: the object being keyed can't contain any colors that are similar to the background, otherwise, those will be eliminated along with it. So if you have a bright green background, and wear a bright green shirt, you'll look like the invisible man from neck to navel.

Second, the background has to be relatively consistent. If you have a cloth background, this means it's flat, smooth, and wrinkle free. If the background is painted, this means there are no chips, bumps, or uneven sections.

Finally, you need a clean video signal, which means a decent camera. If you attempt to shoot green-screen video with a $500 single-CCD DV or analog camcorder, you'll likely be disappointed in the results.

Now that we've got the basics down, let's take a closer look at the details.

Back Issues
Backgrounds are typically blue or green. I choose green because blue is plentiful in my wardrobe and green non-existent, but I'm not religious about it. Either color will do just fine, so long as you purchase fabric or paint backgrounds specifically designed for video-compositing applications from a reputable source. Hanging up a blue cotton sheet from Wal-Mart just won't get the job done. As you'll see, you don't have to spend a lot of money—we spent all of $20 for one of our backgrounds—but the color has to be consistent.

Your choice of fabric or paint depends upon application. Paint is probably better if you'll be shooting consistently from one location, and is cheaper from a coverage perspective, though probably not if you have to pay someone to paint the wall. Fabric is faster and more portable, and you can take it down and store it when not in use, which is lovely given that bright green and blue don't feng shui very well.

Most chromakey fabric is made of muslin, a heavy fabric that hangs very smoothly. When hanging fabric, be careful to avoid creases and folds, and if the subject will be moving, or you're shooting outdoors, secure the fabric top and bottom to avoid motion from the wind.

For true portability, consider purchasing a collapsible background. Depending upon where you buy and the size you choose, these can cost anywhere from around $100 to several hundred dollars. Two good places to check for products and prices are www.markertek.com, for name-brand products and prices, and http://stores.ebay.com/J-and-K-Group, for less well-known products and much cheaper prices.

Background Size
The conventional wisdom of chromakey lighting and positioning is that the chromakey background must be lit separately from the foreground speaker and that the speaker should be positioned between six to ten feet from the background. The obvious concern is that using one light source for both subject and background would cast shadows on the background, introducing inconsistencies that complicate the chromakeying process. Unfortunately, forcing the speaker to stand this far from the background significantly increases its required size.

However, you can place the speaker close to the background (and use a smaller screen) without compromising quality if two conditions are met. Specifically, if you have a single light almost directly above both subject and background, and limited motion in the shot, shadows won't be a problem. Since the double fluorescent lamp in my tests lit both the background and me from such a high angle, any shadows are either directly behind me, or so low that they're out of the shot. Either way, the background consistency is maintained for keying purposes.

Conversely, once you introduce motion into the scene, the background size requirements increase very significantly, with precise requirements depending upon the range of motion, height of the subject, and camera location. For most corporate or academic talking head shoots, however, you can produce very high quality with a 3x5' swatch that costs just $20.

Lighting Cues
If you go with a larger background, producing flat lighting over the complete background is critical to achieving a clean chromakey. Typically, you'll need at least three lights, one on each side and one shining down. All lights must be soft lights with no hot spots, so use either fluorescent lights or bounce the lights off of a white surface. Obviously, the lights must be placed behind the subject to avoid shadows on the background.

Then you'll need to light the subject. These lights should not shine on the background, since this could compromise its consistency. Backlighting is critical here because it helps avoid a halo effect where the background color "blooms" around the outline of the subject.

If you're attempting to insert the subject seamlessly into a specific background, consider how the background scene is lighted and attempt to match that in your studio. At the very least, make sure the primary lights are shining from the same direction so that the shadows all point in the same direction.

On Camera
Even if your background and lighting are perfect, if the image acquired by the camera is noisy, the quality of the overlay will suffer. We used two cameras in our tests, a Sony DCR-VX2000 (2000-vintage, 3-CCD prosumer) and an older Sony DCR-TRV9 one-chip consumer camera. We got very good results with the VX2000, but the TRV9 was totally inadequate for two of the three applications that we tested, and marginal for the third.

It's possible that some newer one-CCD camcorders can produce sufficient quality for a clean chromakey, especially if the scene is really well-lit, which somewhat negates the poor low-light performance of most consumer camcorders. Under most real-world conditions, however, consider a three-chip camcorder in the Canon GL2/Sony VX2100 class or better.

Before shooting, white balance the camera, or set it to indoor lighting if manual white balance isn't available (even if you'll be compositing the video over an outdoor scene). Use manual focus in these shots, especially if the subject is off center, since the detail-less background can send autofocus mechanisms endlessly searching for an accurate focus. Manual exposure is also best to avoid against inadvertent exposure changes caused by reflected light or lighter objects that temporarily appear on screen.

As previously mentioned, your subject shouldn't wear clothing with colors similar to the background color. In addition, avoid white clothing, which can easily reflect the background color, and have the subject remove watches, bracelets, and other items that can reflect the background color.

The Software Side
In many ways, the software side is a breeze after the hassles of setting up and shooting the video. To briefly recount, during the chromakey process, you combine two files, one containing the background video and one containing the video to be superimposed over the background, often called the overlay video.

Operationally, you load both files, and then tell the editor which color to ignore in the overlay file. There are many keying techniques, the most basic of which is chroma or color keying, which is the technique we will use in our examples. Apple iMovie Virtually all chromakey software share a common set of basic tools. At the consumer level, you can do chromakey with iMovie with the assistance of an eZedia plug-in called eZeScreen ($39, or you can buy eZeMatte, eZeScreen, and eZeClip, which creates picture-in-picture and split screen effects for $69). To get to the point shown in Figure 3, drag your background clip to the timeline, open the Effects library, select eZeScreen and click the Configure button.

When you select the effect and overlay clip, eZedia attempts to set the optimal parameters for you, and does a pretty good job, though in testing I set the controls to zero to make it easier to demonstrate the process. The first step is to identify the background color. Most programs do this with an eyedropper control similar to the Pick control in eZedia; simply click the eyedropper, then touch the background. The hunt is on.

The next control shared by most chromakey tools is the Tolerance control, which is often called Similarity. This tells the editor to expand the search from the one color chosen with the eyedropper to an increasing range of similar colors, helping to eliminate minor differences in lighting and shading on the chromakey background.

Typically, if you've done a good job setting up and shooting, increasing the Similarity will remove most, if not all of the background color, and produce a clean overlay. The process is simple, you simply increase the Similarity value until the background disappears. Go too far, and bits and pieces of the subject start to erode as well, so back off until you find a good balance.

Your next concern is the edges, where the video overlay meets the background video. You'd like the edges to be as smooth as possible because jaggies make the overlay obvious, spoiling the illusion. As with eZeScreen, most programs include a smoothing control to smooth the edges, though they are not always effective. For example, with the eZedia plug-in, even subtle adjustments to the smoothing control started eroding the foreground image, complicating their use and limiting their benefit.

Finally, many programs also provide a transparency control that allows you to make the video partially transparent, a la Patrick Swayze in the movie Ghost. In use, eZedia's control proved a touch counter-intuitive, affecting the background wall, not the foreground subject. Overall, however, the plug-in did a credible job, producing a generally clean key with just a hint of a greenish halo over most edges, and occasional jaggies. Definitely well worth the price.

Adobe Premiere Pro
Frankly, Premiere Pro is very disappointing in the chromakey area, probably because Adobe expects most serious videographers who chromakey to use Adobe After Effects. And it's not an unreasonable expectation, since Adobe makes the Video Collection production bundle so inexpensive. Let's take a quick run through Premiere Pro for a few more concepts, and then we'll move on to After Effects.

As with most prosumer timeline editors, you start by loading the background video on the bottom track (Video 1) with the overlay video above it. Then you drag the Chromakey effect from the Effects palette down onto the clip. Though Premiere provides a specific Green Screen Key effect, I generally find that these preset effects don't work very well in real-life situations, so you're better off using the more general Chromakey control.

Click the Effect Controls tab in the Monitor window to view the Chromakey controls. Several of the Chromakey controls, like the Color, Similarity, and Smoothing controls, should appear instantly familiar, performing the same functions as in iMovie, while the Opacity effect, directly above the Chromakey effect, controls transparency.

The other controls—Blend, Threshold, and Cutoff—are vaguely defined, which complicates operation. For example, regarding the Cutoff control, Premiere's help file instructs, "Drag to the right until the opaque area reaches a satisfactory level." Hmmm. Basically, my approach with controls like these is to wiggle them a bit, see if they help or hurt, and then move on. With most programs you'll quickly zero in on the controls that really matter.

The other effect shown in the Video Effects panel is the Garbage Matte, which performs a critical role in many keying effects. Briefly, for a number of reasons, the edges of the overlay video are often a problem area, usually slightly darker than the middle regions and therefore difficult to eliminate. The Garbage Matte lets you draw a free-form trapezoid around the subject, like that shown in the preview window on the right, with all segments outside the drawing in the overlay clip discarded like, well, garbage. Hence the name.

Virtually all prosumer programs have a capability like the Garbage Matte, though it may have a different name. If you find yourself going crazy trying to eliminate the outer regions of your overlay clip, find the Garbage Matte (or its equivalent).

Premiere's smoothness controls leave a good deal to be desired, which is especially obvious during real-time preview or after you render the clip. For this reason, we'll pay a quick visit to Adobe After Effects.

Adobe After Effects
If you're editing in Premiere Pro and find the chromakey effect inadequate, load both background and overlay clips into After Effects, apply the chromakey effect, then render a file for re-importing back into Premiere Pro for further editing. Adobe promises a smoother workflow with the next release of the Adobe Video Collection, including the ability to copy and paste tracks from Premiere Pro to After Effects.

Incidentally, the Adobe Video Collection is available in two flavors, the Standard Edition (MSRP $999), which includes Premiere Pro, After Effects Standard Edition, the authoring program Adobe Encore DVD, and the audio editor Adobe Audition. The Adobe Video Collection Professional ($1,499) includes After Effects Professional, Premiere Pro, Adobe Photoshop, Encore DVD, and Audition. One of the most significant differences between After Effects Standard Edition and After Effects Professional is a keying plug-in.

Space doesn't allow me to provide a step-by-step workflow in After Effects, which is generally similar to Premiere Pro, though unfortunately not nearly as user-friendly. In After Effects, the basic work unit is a Composition, which you create by choosing Composition > New Composition from the main menu. After you name the composition, After Effects opens a new timeline where the work actually occurs.

Import assets by choosing File > Import in the main menu, which loads the files into the project bin. From there, drag the background video to Track 1 on the timeline, and then drag the overlay video to the timeline, dropping it above the background video into Track 1, which pushes the background video to Track 2.

Select the overlay video by touching the Source name in the timeline, then add the Color Key effect by choosing Effect > Keying > Color Key from the main menu. After Effects loads the effect controls into an Effects Control window. By this point, the Color Key controls should be very familiar; an eyedropper to select the color, a Tolerance control to expand the range of values, and Edge Thin and Feather controls to smooth the edges.

Beneath the color key is another tool called Spill Suppressor, inserted by choosing Effect > Keying > Spill Suppressor. Operationally, the Spill Suppressor hunts out any background coloring that spills over from the Color Key process and converts it to gray, making it much less visible.

Many color-suppression tools are integrated into the keying process, automatically suppressing the color selected with the Color Key eyedropper. Because you can use After Effects color suppression tool with or without the Color Key effect, it has its own eyedropper for suppressing the color to suppress.

Obviously, you want spill suppression to work on the same color as the Color Key effect. To make this so, click the spill suppressor's eyedropper to make it active, then click the color swatch next to the Key Color effects. Then fiddle with the other controls until you achieve the desired result.

Spill suppression is one of the marks of a professional compositing program, and combining the two effects produces a very powerful result, the best you can get with the Standard Edition of After Effects. Step up to the Professional Edition, however, and you get the Keylight plug-in, which is one of the most powerful chromakey tools available, in part because it does much of the work for you.

Interestingly, however, there is always some risk in simply accepting the automatic settings of any keying program. This is apparent if you compare the Clear section of the preview window on the right, with the Grayed are on the left. Whenever applying a chromakey effect, it's good practice to shift the overlay video a bit to the side so you can see if the effect is leaving residue on the background video.

It's easy enough to fix. However, since the residue is so subtle, you probably wouldn't notice it unless you actually shifted the overlay video and looked for it. This is no knock on the Keylight plug-in; it's a must-have tool for chromakey producers using the Adobe Video Collection. Still, just remember, anytime an effect works "automatically" it should trigger a thought to verify the results.

Serious Magic ULTRA
Serious Magic's ULTRA (MSRP $795) is a compositing tool that wears two hats. First, you can use it solely for its extraordinary compositing capabilities, inserting your background and overlay videos and producing a combined clip for additional editing. Also, you can use the product to insert chromakeyed subjects into "virtual sets" that simulate offices, living rooms, or space stations, and even move the subject around the room. I'll briefly touch on both functions.

As a compositor, ULTRA offers the familiar set of tools, with one significant difference. Specifically, in addition to allowing you to broaden the search for background colors with a threshold or sensitivity control, ULTRA lets you choose multiple points to eliminate in the background, which helps with inconstancies like shadows and other background imperfections. These points are shown as plus signs on the screen.

This, plus comprehensive keying, spill suppression, and edge softness controls (Soften Matte, Sharpen Matte), and the ability to zoom into the image to work at high magnifications (via the Magnify Preview slider on the bottom) produced some of the smoothest, clearest keying in our tests. Even better, you don't need a computer science degree to run the software; it's very easy to use.

ULTRA offers exceptional virtual set capabilities. I used the software is compositing a video of me into a virtual newsroom and then showing a different video in the television screen over my left shoulder. (If you must know, I was converting some 8mm film to DV tape; this was a Halloween video from the early Sixties). These effects are very easy to produce, and introduce a whole new level of creativity into your video productions, especially considering the ability to pan and zoom around the virtual set without distortion.

If you're seriously into chromakey effects, and your video editor isn't getting the job done, consider Serious Magic ULTRA, especially if you're a less than sophisticated user who may not be up to the challenge of the Premiere Pro or After Effects interfaces. You'll produce very high-quality effects, with the added bonus of the virtual set capabilities.

Sidebar
Composite and Spin: Inserting Spinning Logos
Our whirlwind tour has taken us halfway around the compositing world and the question that's burning in your brain somehow remains unanswered: "How in the world do I create and use cheesy spinning logos?" The product of choice here is Ulead's Cool 3D Production Studio (MSRP $129).

Operation is easiest when using objects and effects from Cool 3D's EasyPalette. For example, to create the spinning logo shown in Figure 12, I dragged in a model, customized the text and rendered as a 320x240 resolution, 32-bit AVI file with an alpha channel, which designates the background as transparent. Chose File > Export Video Overlay > AVI from the Cool 3D menu to produce a similar file.

When inserting a logo like this into most editors, treat it much as you would a still image, placing it on a track above the background video and sizing and positioning the video as desired. Most editors, like Premiere Pro and After Effects, automatically key out the background, so you may not even have to apply an effect. If you need to, use an alpha channel key, which should eliminate the background without any adjustment.

Companies Mentioned in this Article
Adobe Systems, Inc., www.adobe.com
Apple Computer, Inc., www.apple.com
eZedia, Inc., www.ezedia.com
Serious Magic, www.seriousmagic.com
Ulead Systems, Inc., www.ulead.com