In this era of ever increasing file sizes, megalith like storage requirements, rotational disk drive capacities exceeding terabyte proportions, and an astounding dependence upon storage in dimensions previously unobtainable comes a restoration of a nearly lost technology—solid state memory for mass storage.
The evolution of flash-based memory for consumer and professional applications is helping to drive a renewal in solid state, scalable enterprise storage. Flash, a nonvolatile memory (NVM) that can be electrically erased and reprogrammed, has most recently reappeared in products for professional video media serving applications including video on demand. SeaChange and Toshiba have demonstrated systems at NAB, SMPTE and other industry trade shows. We now find fast data storage systems in the 10–20 TB of flash storage with upwards of 20 GBps of read or write bandwidth currently available from companies (e.g., TMS Inc.) that are structured in both RAM-cache and flash-based SAN architectures.
FLASH FORMATS
Flash memory has become the core element in what is now referred to as "solid state disks" (SSD). Flash development began in the mid-1980s when Dr. Fujio Masuoka, working for Toshiba, presented the concept to industry trade conferences. Intel soon recognized the tremendous potential and began aiming commercial production for the late 1980s.
There are two forms of flash: NOR—first available around 1987, and NAND—developed in the 1988–1989 time frame. The fundamental distinctions between the NOR and NAND include their speed of access and/or erasure; how its individual memory cells are connected (NOR is parallel with NAND as serial); and how their read/write interfaces (page based versus randomly accessible) are facilitated.
NOR-based flash memory has long erase and write times. NOR flash provides full addressing to data buses, permitting random access reads and programming to any memory location. Its drawback is that complete random access rewrite or erase operations are encumbered because an individual memory cell's state can only be changed from its native state (logical 1) to its opposite state (logical zero). Successive writes that change a cell's bit from logical 1 to logical zero are permitted, but returning a single memory bit in a cell to its native state (logical zero) must be accomplished by an erasure of an entire block; that is, all the bits must revert to a logical 1. NOR-based flash was the basis of early flash-based removable media including CompactFlash, although it too later migrated to the less expensive NAND flash.
NAND FLASH MEMORY
NAND gate flash uses tunnel injection for writing and tunnel release for erasing. Tunneling is the principle of injecting charge carriers (i.e., the electrons and holes of solid state physics) to an electric conductor through a thin layer of an electric insulator. The I/O for NAND flash memory cells are not individually random accessed. Data, arranged as pages, must be addressed (i.e., read) in blocks, much like hard disk drives. Reading and programming is accomplished on a page basis, however erasure must occur in complete blocks. The core of most removable USB flash drives used the NAND flash memory format.
Flash is nonvolatile and consumes as much as 10 times less power than the same amount of hard drive storage. A distance technology of some 10-plus years ago showed some promise for basic forms of nonrotational storage, yet the technologies were seldom employed exclusively. Flash storage technology, while more expensive than hard disk storage, has a lower power requirement, achieves greater reliability and also runs cooler.
Supporting hard disk drives with ancillary RAM caches remains common practice. Still, any intentions to completely replace hard drives with DRAM (dynamic random access memory) devices was not on the agenda due to the storage limitations of DRAM, its high cost and high power consumption and its volatility. DRAM is subject to failure or complete memory loss should the on board or main power fail or if maintenance is required. If used in this type of application, most critical storage assets would need to be backed up to another form of physical media, adding cost, operational burden, and additional space and cooling.
As recent as 2006, driven by the explosion in portable handheld devices, hybrid 1.5-inch microdrives began to support 256 MB of onboard supplemental flash memory. Some products placed cache and controllers directly onto the motherboard. In some recent laptops, flash memory has become the only form of mass storage—completely eliminating spinning magnetic drives.
FLASH VS. DRAM
Flash memory versus DRAM or disk storage has some obvious advantages. Flash can store orders of magnitude more data in the same physical footprint as DRAM. Flash is nonvolatile and consumes as much as 10 times less power than the same amount of hard drive storage. Its reliability is extremely high, as much as 100 times better than a disk drive. Another key rationale is flash's access time, with improvements approaching 250 times faster compared to disks.
When looking at the system architecture, a mixture of RAM-cache with flash support is not uncommon. In this form, the advantages of RAM with its extremely fast read and write performance supports 10–15 percent of the temporary storage cache with the balance of the SAN utilizing flash only. This model uses the RAM for buffering with the principle data content residing in nonvolatile flash memory. Since flash can be read very fast, compared to its write times, moving assets from "permanent" flash storage to RAM can be managed on a demand basis. Populating the flash storage component can happen directly from the input-stages of the storage system, or when demands or vulnerability warrant, using a combination of the RAM buffer into the flash memory.
Flash-based stores can be protected in similar means to their rotational counter parts. RAID techniques are deployed using specially purposed single or dual controllers that manage the striping in similar fashion to RAID 3 for hard disks. Complete mirroring is another alternative but carries a price penalty.
Today, there still remain some arguments which are difficult to dismiss. Even with 64 GB flash, for a single stage environment, it is hard to justify flash versus the cost of a single 1 TB hard drive. Yet in order to gain the bandwidth and throughout necessary for a VOD or conventional video server storage system, several other sets of components must be included—such as multiple sleds of hard drives in a fiber channel RAID configuration, specialized controllers and the like.
A "flash only" solution may not yet be viable for those mission critical, large file size systems found in broadcast play-out operations—but over time, the flash-based supplemental edge server, cache or VOD application may turn out to be far more than just a niche application.
Karl Paulsen is chief technology officer for AZCAR Technologies, a provider of digital media solutions and systems integration for the moving media industry. Karl is a SMPTE Fellow and an SBE Life Certified Professional Broadcast Engineer. Contact him at karl.paulsen@azcar.com.
