Flash memory provides a needed level of storage performance for media and entertainment applications which now command, for UHD and beyond, massive amounts of storage and speed to meet the growing amounts of content being generated. New high-res applications demand much higher storage performance than conventional enterprise back office applications. Flash memory is helping enable that performance—but with it, the physical media also commands interfaces that provide the best value for the applications and its associated storage.
Non-volatile memory express (NVMe) is a trending technology that utilizes Flash storage more effectively and efficiently. The harmony of solid state and rotating storage media will continue for the near term, and for the undefined future. These respective storage mediums will endure, providing complimentary values for ambitions such as more storage, better storage, and faster throughput with a reduced hardware footprint, at less cost.
[Read: A Solid State of Non-Volatile Memory]
MORE THAN A 2X BUMP
At the previous two Flash Memory Summits (2016-17) both NVMe and the PCIe 4.0 bus were hot topics. Yet right alongside the 2016 Flash promotors, the SCSI Trade Association reminded the industry that “a new serial-attached SCSI (SAS) technology was on the way.” That new technology took the SAS ecosystem from the 12G level to a usable 24G SAS (24GBps, serial attached SCSI). Promoters unveiled this advancement as more than just a “two times bump in speed over the previous data rates for 12G SAS” (refer to Fig. 1 for the SAS technology roadmap).
The interface of choice for mission-critical storage applications remains SAS; moving the interface from 12GB to 24GB yielded a major refurbishment in the technology. Updates included more efficient 128b/150b encoding, with SAS Protocol Layer (SPL) packets and Forward Error Correction (FEC). The transmission signaling rate is specified at 2.4GBbs (i.e., 22.5 gigabaud rate) which still retains compatibility with earlier 6G and 12G solutions.
Changes which help achieve this faster throughput include an FEC field, 20 bits in the SPL packet that aid in error detection and recovery. An SPL packet is a 150-bit block that includes a 2-bit header and a 128-bit packet payload, plus the FEC bits. At the deep-dive level, additional features added include changes in: binary primitives (in the SAS Link Layer); primitive parameters; serial management protocol (SMP); open priority; and inter-expander fairness arbitration enhancements. Details can be found in a technical overview document for Serial Attached SCSI, which is roughly 1,000 pages and whose details are beyond the scope of this overview.
LOWERING LATENCY
Regardless of the application for media and entertainment or others such as transactional trading on the stock exchange, latency has a tremendous impact on storage performance. Recalling from our previous article—NVMe is an industry standard that is optimized with a new storage stack featuring a low latency, efficient and scalable protocol streamlined with a revised drive command set that uses fewer clock cycles per IO operation.
Comparing spinning magnetic drive latencies for hard disk drives (HDD) to those of Flash SSDs, conventional Flash NAND technology offers a 100x reduction in latency over HSDs.
Latency figures decrease further when NVMe eliminates the 20 microseconds of latency found in the SSD NAND (whether SAS or SATA) implementation. “Next Gen NVMe” will now drive NVMe to deliver “4KB operations in under 10 microseconds,” according to a presentation at the 2016 Flash Memory Summit.
CHANGING THE MESSAGING
What do these memory improvements provide to the media and entertainment (M&E) industry? For starters, it enables audio/video to become the dominant medium for the carriage of information and content. In a February “New York Times” text message to mobile subscribers we saw, “What you are doing now (i.e., ‘reading text on a screen’) is likely going out of style.” The Times’ text infers that A/V will replace the text messaging we do today and in the not-too-distant future.
One of the tasks necessary to achieve this change will be to increase the speed of the delivery while at the same time improving the processing and memory requirements of the channels which deliver the information. The new 5G networks may help this, but more is needed. This same prolog can be applied to production and post-production workflows for M&E.
[Read: Non-Volatile Memory Grows In Popularity]
We have already seen the impacts of 4K over HD in terms of resolution, quality and image perception. While well-produced 1080p content can be wonderfully upscaled for 4K displays; in many cases there can be very satisfying results when the content is originated as 4K and then downconverted to 1080p and displayed on a high-end HD (1080p60) television system.
The real impact for UHD/4K, once available in more delivery systems, will be in the use of higher frame rates (1/120 second frames instead of 1/60 second) especially for sports. Add the higher resolution per frame and a higher density of pixels per unit screen area, and it could make today’s conventional HD images look like older analog 1-inch videotape, comparatively.
IT TAKS A LOT OF MEMORY
To reach these goals, it takes memory, a lot of it; and fast memory coupled with much higher bandwidth (selected video formats described in Fig. 1). Compression helps to a degree, but with the improvements in compute and networking technologies, more productions are switching to full-bandwidth, uncompressed video for their editing and post-production workflows.
Digital storage for media workflows will continue to advance in these areas. Flash, coupled into NVMe interfaces, make these higher capacity, faster and better memory solutions possible. We’ve seen Seagate produce a 12TB “BarraCuda Pro” 3.5-inch HDD with SSD-like performance, and SanDisk’s “Extreme PRO” CFast 2.0 solution in a 256GB form factor capable of 525MBps read (450 MBps write). These devices are the tip of the iceberg, as the products become the workstation’s local drives (i.e., the Seagate HDD) and the camera capture’s memory solution (i.e., the SanDisk memory)—key components needed to work in higher resolution and higher frame rate production workflows.
ACHIEVING THE TARGET
Along with this perspective on higher resolution, higher bandwidth production—another element which will consume massive amounts of memory is video on demand. According to Coughlin Associates, the shipping capacity for VOD storage will increase 8-times, from the 2,500,000 TB (terabytes) in 2016 to a whopping 20,000,000 TB (per year) in 2021. Add the other data sets, which aren’t getting any smaller, and the requirements necessary to fulfill all these ambitions is quite an undertaking. When AI and VR take hold, cloud services may not be enough to satisfy such exponential changes in storage volumes—let alone the means to transmit (wirelessly) the demands to the end users.
To manage the physical maintenance issues of storage using HDDs could require small armies of support teams (quite possibly robotic) just to change the drives in even a modest data center. Flash memory would need less support, with the R&R (rescue and recovery) cycles for SSD upkeep probably hundreds of times less. Technological changes would likely outpace the life of the SSDs in service, making storage a disposable commodity.
This is the future for storage. Mechanical drives, besides reaching physical capacity thresholds that may limit their overall performance, are probably in their last decade of useful life—at least for large scale M&E solutions. Single unit workstations may continue with HDDs, where cost-to-capacity requirements are different than in datacenters—although the migration to all SSDs is very much apparent in tablets and mobile devices. Watch for some dramatic changes in storage and Flash as more video enters into our lives and better images become an expected case and not a “luxury once in a while” alternative.
Karl Paulsen is CTO at Diversified (www.diversifiedus.com) and a SMPTE Fellow. Read more about this and other storage topics in his book “Moving Media Storage Technologies.” Contact Karl atkpaulsen@diversifiedus.com.
