Two editing trends have come into conflict with the release of Intel's Dual Core 2 processor. Laptops now have the computer power to support online HD editing. This has led many to abandon tower computers in favor of laptops such as Apple's MacBook Pro. Editing multistream HDV, with its 3.5MB/s data rate, is well within the capabilities of these lightweight systems.
However, with Apple's release of its ProRes 422 codec, along with increasing use of the Apple Intermediate Codec (AIC), hard disk bandwidth requirements have increased so significantly that laptop-based, multistream, real-time editing is no longer possible.
One solution is to store media on an external RAID that plugs into a laptop whenever users need to edit. CalDigit's HDPro is a plug-in RAID that offers capacities from 2TB to 8TB.
System features
The entry-level HDPro has a 2TB capacity and includes a PCIe interface. All HDPro systems are compatible with MacBook Pro, Mac Pro, G5 and G4 computers, as well as Linux and Windows workstations.
The chassis can hold up to eight drives and uses load-balancing power management to divide the workload between two power supplies. (See Figures 1 and 2.) Power management enables the system to continue working in the event of a power supply failure. It supports eight, 3Gb/s, SATA channels. An integral Intel XScale core that includes up to 2GB of 64-bit DDR333 cache memory drives these channels. (The default cache is only 256MB.)
Unlike Fibre Channel, SCSI, FireWire and eSATA RAID storage, the PCIe interface connects directly to the host computer's bridge chip. A direct link to system RAM eliminates the latency introduced by these other interfaces.
The system's integrated controller includes an environment monitor processor that monitors and manages all aspects of the system, including the hard drives, RAID set controller, controller cache, fan, power supply and enclosure temperature. A front-panel LED display provides visual feedback, and an audible alarm signals a controller or drive failure. It can also send users an e-mail error notification.
CalDigit's RaidShare utility is used for both setup and monitoring. Because it is based on TCP/IP, users can monitor and manage the RAID storage from anywhere.
With a four-lane interface to the host, and each lane delivering up to 250MB/s, a RAID 5 system delivers up to 200MB/s. Of course, that performance depends on a long list of other factors, including PCI bus traffic, host RAM available for I/O buffers, tasks — including decompression/decoding — competing for CPU cycles, stripe size, and the type of RAID employed.
RAID 0: Performance
A RAID 0 provides maximum performance and requires at least a pair of drives. A software or hardware controller will sequentially cycle data blocks across all available drives. The probability of a RAID 0 failure increases in direct proportion to the number of drives. Thus, the chance that users might lose data with an eight-drive RAID 0 array is eight times higher than with a single drive used for media.
RAID 1: Safety
To increase RAID reliability, a RAID 1 configuration can be used. The software or hardware controller mirrors data on other drives, thereby cutting the chance of data loss by half. Unfortunately, storage capacity is also cut in half.
RAID 3, 5 and 6: Performance and safety
The weaknesses of RAID 0 and RAID 1 are overcome by configuring drives as RAID 3, RAID 5 or RAID 6. All three types of RAID distribute data across multiple drives. RAID 3 employs controller-moderated parity bits written to and read from a single drive dedicated to parity data.
The system supports RAID 5 by distributing parity bits across all drives in the array. Performance is increased compared with RAID 3 because parity data are stored using RAID striping. Because parity data requires disk space, a system's data storage space is less than the RAID's total capacity. (See Table 1.)
A RAID 5 array will survive one defective drive. For example, I pulled a drive from the HDPro while playing a nine-stream loop. After notifying me of the failure, playback continued without issues.
A RAID 6 system, where parity bits are distributed on a pair of dedicated drives, can sustain two hard drive failures. For this reason, RAID 6 is a valuable option for video production.
My 2.56TB review unit had eight 320GB drives — each with a 32MB cache. It was shipped with a four-lane XpressCard/34 interface that I plugged into my MacBook Pro.
Using AJA's benchmark program, I measured performance using the RAID 5 shipped configuration. Measured performance was, as promised, near 200MB/s. Write speed was measured at 165MB/s and the read speed was 171MB/s.
After I configured the HDPro as a RAID 6, its capacity fell to about 1.9TB. Measured performance, however, remained the same with a write speed of 161MB/s, and a read speed of 171MB/s.
Multistream performance
Despite its wide use for media storage, RAID has a weakness. Data access times become longer when there are more drives in the RAID. This is because whenever a disk seek is required, typically all drives must initiate a seek. Multiple stream audio and video playback compound this source of RAID performance loss. For this reason, I tested the system to determine its ability to play multiple data streams.
Multistream playback, of course, increases playback engine overhead. Overhead is also significantly increased when interframe or intraframe decompression is required. Both sources of overhead decrease the number of supported playback streams.
Table 2 shows measured Final Cut Pro 6.0.2 multistream performance for the HDPro running as RAID 5. (All tests were conducted with six audio channels.)
These data are important because intermediate codecs, such as Apple's ProRes 422 and Avid's DNxHD, are increasingly being used because they eliminate uncompressed video's massive storage burden and huge disk bandwidth requirement.
One might assume that because this system supports only a single uncompressed stream, users would be limited to cuts-only, real-time editing. Thankfully, this is not entirely true.
HDPro bandwidth beyond that required for a single stream supports prefilling the memory buffers maintained by the NLE. To confirm this, I checked the ability of the system to support dual-stream transition effects. (See Table 3.)
To provide a comparison for the HDPro benchmarks, I measured read/write performance of the 5400rpm, 120GB drive in my MacBook Pro 2. These data clearly show why the drives shipped in most laptops are inadequate for intermediate codec, multistream, editing.
Conclusions
From a reviewer's viewpoint, the pair of PDF files CalDigit supplied did not inspire confidence. Not only could I not find documentation on the system's multicolor LEDs, the two files provided conflicting coverage of the RaidShare utility.
Thankfully, I encountered no problems with the HDPro RAID itself. There are, however, inherent problems with the CardExpress/34 interface card. First, the card is not hot-swappable. I had to restart my laptop each time I wanted to use the HDPro. However, that was a mere annoyance compared with the second problem.
Once the CardExpress/34 is seated, its connection can be broken far too easily. A slight inward pressure causes a bus disconnect in preparation for ejection. Conversely, a slight outward force causes disconnect, as part of a cable safety release. During testing, the interface accidentally disconnected several times. In a darkened editing suite, it would be almost impossible to avoid disconnects.
Laptop owners may find CalDigit's chainable, 2TB, dual-drive, FireWireVR RAID a safer solution even though its performance is only half that of an HDPro. For Mac and PC owners with tower systems, an HDPro with either a PCIe or PCI-X interface provides reliable, high-performance, data storage.
Steve Mullen is owner of Digital Video Consulting, which provides consulting and conducts seminars on digital video technology.
Eight drives 250GB 320GB 500GB 750GB 1000GB RAID 3 2.00TB 2.56TB 4.00TB 6.00TB 8.00TB RAID 5 1.75TB 2.24TB 3.50TB 5.25TB 7.00TB RAID 6 1.50TB 1.92TB 3.00TB 4.50TB 6.00TB
Table 1. HDPro eight-drive maximum data capacity
Intel 2.33GHz Core 2 with 2GB RAM Number of streams Dynamic real-time aggregate data rate Codec Sample width Per-stream data rate Dynamic real-time-enabled Safe-enabled HDV
1440 × 1080
4:2:0 8-bit 3.5MB/s Nine Eight 32MB/s § XDCAM EX
1920 × 1080
4:2:0 8-bit 4.9MB/s Nine Six 44MB/s § DVCPRO HD
1280 × 1080
4:2:2 8-bit 14MB/s Three Three 42MB/s ‡ ProRes 422 HQ
1920 × 1080
4:2:2 10-bit 26MB/s Three Three 78MB/s ‡ AIC
1440 × 1080
4:2:0 8-bit 14MB/s Two Two 42MB/s § Uncompressed
1920 × 1080
4:2:2 8-bit 125MB/s One One 125MB/s Uncompressed
1920 × 1080
4:2:2 10-bit 166MB/s One One 166MB/s
Table 2. Measured Final Cut Pro 6.0.2 multistream performance for the HDPro running as RAID 5. The § represents Interframe decoding, and the ‡ represents intraframe decompression.
Color correction on both clips Maximum dissolve duration Codec Resolution Sample width Dynamic real time Safe Uncompressed 1920 × 1080 8-bit 2.5 seconds 2.0 seconds 10-bit 0.5 seconds 0.5 seconds
