EIGHTY-FOUR, PA.—Long ago, storage components were principally ranked by the individual physical capacity of the device, or in the case of JBODs (just a bunch of disks), the array. As technologies evolved, hard drives began to approach the 4 GB level (circa early-1990s), video servers began to emerge, and then protected storage became a commodity reality.
As RAID became more affordable, the storage description perspective shifted to include a RAID-level specification with a perceptually incorrect indication that “the higher the RAID level, the better it was.” Thus RAID 5 seemed to be better than RAID 3, and when you combined striping (e.g., RAID 50) the value proposition increased “many fold.”
Today, as magnetic spinning hard disks approach the 10TB range in 3.5-inch form factors, we also find greater need for smaller form-factor memory. Users are now looking more toward USB flash drives and other flash memory devices to support consumer as well as commercial storage needs. Collectively these USB, SD (Secure Digital) and similar extensions are called “flash memory products” and are generally composed of NAND flash memory components. In support of HD and 4K resolutions, one can now find these SD card versions with published storage at upwards of 128GB and solid-state drives (SSDs in 2.5-inch form factors) at half-a-terabyte and above.
Fig. 1: UHS-II memory cards have additional contacts for data transmission and width improvements.AVAILABLE SPACE
Regardless of the media type, the available space reported to a user by an operating system or camera device almost never matches the physical (raw) storage device capacity listed on the nameplate. The reasons are often confusing or misunderstood, sometimes causing users to roll the dice on what they’re really getting for the precious “pennies on the gig.”
In all cases, some storage memory cells for any flash memory product will be utilized for “device formatting” during manufacturing. Subsequently, additional cells are consumed during formatting by the operating system (OS) and are used for administrative computing operations either for computing inside the card or computing within a host device, such as the camera or Windows or Mac OS. This so-called “hidden space” becomes unavailable to the user for data storage. However, it does serve to improve the performance and/or the reliability of the storage medium when used with host devices.
Controller firmware, defined as the code that runs inside the storage device, is an allocation set aside for updates and other internal administrative functions. In some applications, memory cells are reserved in order to administer copy protection or other security feature requirements found in SD cards.
Function-wise, the hidden space may also be allocated for such things as testing the memory cells to identify useable versus unusable cells. In this case, the system reserves (or deactivates) the unusable cells to avoid reliability issues or performance degradation in the user experience. Essentially, these cells cannot be written to and remain inactive for any user data.
FILE-TRACKING
During OS configuration for the device (which varies for Mac versus Windows operating systems), the device-formatting operation will allocate space to track user file locations, to place labels for dates and times, and for other metadata such as filenames. The device also stores administration records describing used versus free file system space, which is then reported to users through the operating system.
For example, on a Windows OS, a card or drive will (usually) create a file allocation table (FAT), with root directory and subdirectory elements carving out some portion of the storage as unavailable for actual user data.
Flash memory devices are known to “age,” that is, they deteriorate based upon the number of erase/write cycles they are subjected to. Spare storage cells are allocated in the NAND for administrative purposes, which will extend over the life of the card or drive. The “spare cells” are available for use as substitute cells when an in-use memory cell should fail to store (or successfully read) a user data item. If a non-functioning cell is detected, the cell is effectively “retired.” Then, a spare cell from the reserved pool is activated and the user is allowed to place data there—without ever knowing the previous cell was deactivated.
Overall, the available user space remains unchanged. As the spare cell is activated it equalizes storage capacity as the non-functioning cell goes dormant (deactivated).
ULTRA HIGH-SPEED PERFORMANCE
The advent of streaming media including higher-resolution/higher bit-rate imaging (both for moving and still images) is bringing an increase in performance requirements for flash and other high-speed memory components. The nomenclature for flash Secure Digital memory now includes designations for Ultra High Speed (UHS) II and UHS Speed Class 3 (U3) (see Fig. 1 for comparison). Read speeds are now reaching upwards of 300 MBps, with write speeds up to 260 MBps.
With transfer and read/write speeds of this dimension, memory device manufacturers may recommend specifically developed high-speed readers, which enable the high-volume transfer of data (i.e., RAW photos and 4K video) between the card/device and a Mac or PC. Adding this component can dramatically accelerate workflow and offer users performance improvements straight out of the package.
As soon as you set a target for one device, others emerge that drive the performance capabilities even higher. As evidence, formats such as SD, SDHC or SDXC are now superseded by a new XQD card format. Sony’s recent G-series XQD Format Version 2 technology states transfer speeds of 400 MBps (read) and 350 MBps (write). Of course transfer speeds will vary depending upon the interface and other influences revolving around the host devices and/or the receivers.
When employing other interface technologies (e.g., PCI Express Gen.2 and USB3.0 interfaces), coupled with these UHS (ultra high-speed) cards, support is now available for stable recording at up to 600 Mbits (75 MBytes) per second. Depending on the device (camera/camcorder) being used with the card, a maximum read speed of 1.44 Gbits (180 MBytes) per second is achievable, enabling faster ingest performance.
Where once we specified storage only by capacity, we now must factor in speed—with both read and write consideration being given to workflow performance. UHD/4K production continues to drive the thirst for both capacity and performance, and are accessible to the professional as well as the casual shooter.
Karl Paulsen, CPBE and SMPTE Fellow, is the CTO at Diversified Systems. Read more about this and other storage topics in his book “Moving Media Storage Technologies.” Contact Karl atkpaulsen@divsystems.com.
