One thing that television and the movies have in common is that both portray motion by capturing and displaying a series of still pictures, albeit in very different ways. The capture and display of moving images in these ways can generate artifacts, which are visual effects created by the capture/display process that were not in the original scene.
Some of these artifacts are reversible and may be removed by artful processing, while some cannot be removed at all. Some artifacts are less obtrusive than the effects of their removal would be. It is possible, for example, to remove the visible scan lines from a television screen by placing a ground glass filter over it, but this will also remove enough detail from the picture that we would rather live with the scan lines.
FREQUENCY RECONSTRUCTION
Temporal sampling artifacts may be generated in both film and video capture. We know that sample theory dictates that we need at least two samples per second of any frequency that we capture, in order to perfectly reconstruct that frequency. In digital audio, we sample at a rate faster than twice the highest frequency of interest, and filter the input signal so that no frequency above the highest frequency of interest enters the sampler.
For example, we sample at 48 kHz, while limiting the highest input frequency to 20 kHz. We know that filters have finite slopes, and we have therefore chosen a sampling frequency sufficiently high that we may ensure flat response to 20 kHz; the frequencies within the filter's attenuation slope are well-below half the sampling frequency. Sampling at exactly half the sampling rate, in this case 24 kHz, is called critical sampling and is avoided in digital audio.
Sampling frequencies greater than half the sampling rate result in the replacement of these frequencies with aliases, which are frequencies that were not actually present in the original sampled material. Once aliases have been created, they may not be removed from the reconstructed material, as they are within the band of frequencies of interest.
THE SAMPLE THEORY
Any kind of sampling is subject to the sample theory, and this includes moving pictures. Motion picture photography may be used to illustrate the temporal aliasing phenomenon. A film camera that shoots 24 frames per second has a temporal sampling rate of 24 Hz. This means that any temporal frequency this camera photographs that is above 12 Hz will generate a temporal alias.
Because Westerns, America's own epic dramas, have been so prevalent in our cinematic and television-viewing experience, backward-turning wagon wheels are one of the most-frequently seen temporal aliases. The spokes in a wagon wheel form a repetitive pattern when sampled - that is, each snapshot of the wheel turning "freezes" the spokes at some given position. The frequency at which spokes pass a given point depends on how many spokes the wheel has and how fast it is turning.
WAGONS, HO!
If a wheel has 24 spokes, for example, the spokes are spaced at intervals of 360/24 = 15 degrees. This wheel may turn at a speed such that at the instant each spoke in turn reaches "zero degrees" (or straight up), a photograph is taken. In this case, the wheel is turning 15 degrees each 1/24 second, making one complete revolution each second.
For the moment, we will ignore the aperture effect and pretend that each snapshot is taken instantaneously, knowing that in reality the camera's shutter must remain open for a finite period of time, causing higher-speed motion to blur. Because all the spokes look alike, we cannot tell which particular spoke is passing zero degrees, or any of the other 23 points, in any given snapshot.
In fact, we cannot tell that anything is happening at all, because the wheel appears to be stationary. In this case, only one sample is being taken each second, and sample theory tells us that we cannot accurately reconstruct this frequency. We find this to be true, because the 24 Hz spoke travel frequency has been replaced by the alias frequency of zero Hz.
If the wheel is sped up to two revolutions per second (with spokes still miraculously in-sync with the camera shutter), a snapshot will be made as every other spoke passes zero degrees, and the wheel will again appear to be stationary. Another way to say this is that every integer multiple of 24 Hz will generate a zero Hz alias.
TALKIN' ÎBOUT A REVOLUTION
If the wagon wheel travels at one-half revolution per second, the frequency of spokes passing a given point is 12 Hz, or exactly half the sample frequency. As mentioned earlier, this is called critical sampling. The visual result of this will be that a given spoke will be at zero degrees for a given snapshot, while the spoke immediately ahead of it in the direction of travel will be at 15 degrees and the spoke immediately following it will be at 345 degrees.
At the next snapshot, the first (or "zero") spoke will be at 7.5 degrees; the spoke immediately ahead of it will be at 22.5 degrees; and the spoke immediately following it will be at 352.5 degrees. At one-quarter revolution per second, the spoke repetition rate will be 6 Hz, and the spokes in a given snapshot will be advanced by 3.75 degrees in the next snapshot.
The visual effect for spoke frequencies between zero and 12 Hz (the wheel is turning between zero and 1/2 revolution per second) is that the wagon wheel will appear to turn normally. If the wheel turns at three-quarter revolution per second, the spoke frequency becomes 18 Hz, and the "zero spoke" in our reference snapshot will appear advanced by 11.25 degrees in the next snapshot, while the spoke immediately behind it will be at 360 ö [15-11.25] = 356.25 degrees, or 3.75 degrees before the zero degree point.
The net result of this is that when the 18 Hz spoke frequency is sampled, it is replaced by an alias of 6 Hz. Further, the zero spoke will have advanced by more than half the spacing between two spokes, while the spoke behind it will be 3.75 degrees behind zero degrees.
To the eye, it will appear that the zero spoke has moved not forward but backward by an amount equal to a spoke travel frequency of 6 Hz. Visually, the wagon wheel appears to turn normally as its speed is increased between 0 Hz and 12 Hz - and as its speed is increased above 12 Hz, the real spoke frequencies are replaced by aliases, and the wheel appears to turn backward rapidly at just above 12 Hz. And as the wheel speeds up (progressively more slowly until, as a spoke frequency of 24 Hz is reached), the wheel appears to have stopped.
SPINNIN' WHEEL
Thus, the pattern between 12 Hz and 24 Hz is visually a mirror image of the pattern between 0 and 12 Hz. As the spoke frequency is increased above 24 Hz, the pattern repeats itself, but the wagon wheel will appear to be traveling at half its real speed, until at 48 Hz, it will appear stationary again. This pattern will repeat ad infinitum (or until the bearings burn out), with the wheel appearing stationary at every integer multiple of 24 Hz.
All the visual representations above a spoke frequency of 12 Hz are invalid, as the real frequencies have all been replaced by aliases, just as the sample theory tells us they will be. Further, none of these aliases may be removed, because they all fall within the 0-12 Hz frequency band.
The only way to prevent this aliasing is to filter out all the frequencies above 12 Hz before sampling, analogously to the way digital audio is sampled. However, this cannot be accomplished in film photography, as the requisite sharp-cutoff optical filters cannot be realized. We must, therefore, live with the aliases generated by optical sampling, and we do live with them on a daily basis - both in cinema and television.
When 24-frame film is transferred to 60 Hz video, another form of artifact is generated that we know as 3/2 judder. This column has discussed 3/2 pulldown in the past, but the essence of the process is that in order to transfer 24-frame material to 60-field interlaced video, 60 fields must be made from 48 discrete fields; that is, for every four film frames, not eight but 10 fields must be generated.
This is done by repeating a field after each four fields. For 60-field interlaced video (30 frames per second), one pair of fields is made from each film frame, and one of the fields generated by every second film frame is repeated, so that four film frames comprise five interlaced video frames. An analogous process is used to generate 60 Hz progressive video, except that full video frames replace fields.
FRAME REPITION
When 3/2 material is displayed, the repetition of film frames (as fields or progressive video frames) produces a distortion of what is sometimes called the optical flow axis, which interferes with eye's tracking of motion. As an object moves through the field of vision, the eye attempts to track it. Repetition of frames means that as the object moves, instead of each picture appearing at a unique place on the eye's retina, identical versions of each picture appear at different places on the retina - three identical pictures followed by two identical pictures, followed by three identical pictures, and so on.
This interferes with the eye's ability track the object, causing it to appear to "stutter" as it moves through the field of vision instead of moving smoothly as the eye knows it should. As one in five fields of progressive frames is repeated, the stutter frequency is 60 x 0.2 = 12 Hz.
When the camera pans to track a moving object, the background is not tracked, and the background judders. Judder is readily visible during pans, making cinematographers keenly aware of panning speeds. Unlike aliasing, this artifact of 3/2 pulldown is reversible.
Next month, we will explore these temporal artifacts in greater depth and look at some other artifacts that may be seen when film and video are displayed.
