Two decades after its beginning in 1988, the ubiquitous Fibre Channel technology has become engrained in most of the advanced storage and transport applications where superior performance, high bandwidth and ease of accessibility is important. Since its 1994 ANSI approval, Fibre Channel has made great strides from what was its original intent, a means to simplify the then HIPPI system which at that time was the predominant interface employed for high bandwidth, mass storage connectivity.
At the time, around mid-1995, IBM's proprietary Serial Storage Architecture (SSA) interface was a viable competitor for storage system connectivity since it reduced the complexities of HiPPI's bulky, 50-pair cable harness interface. The SSA interface lost out in the marketplace, as the industry sought to extend the distance between the host and storage devices. Fibre Channel adoption was further fueled by an industry need to support the carriage of data over new and emerging upper layer protocols, such as ATM and IP. SCSI (small computer system interface), the predominant baseline for most storage and peripheral interfaces, was the most visible and obvious choice, but was encumbered by its parallel connections and limited number of connected devices.
Originally developed as a backbone technology for the interconnection of local area networks (LANs), Fibre Channel had as its primary objective the superseding of 100 Mbps Fast-Ethernet and the Fiber Distributed Data Interface (FDDI) which until around 1998, was utilized as a campus backbone technology that could extend LANs to as much as 200 kilometers.
TOPOLOGIES IN TRIPLICATE
The Fibre Channel standard defines three topologies:
- • FC-P2P, used when a point-to-point bidirectional connection between two devices, e.g., a host and a peripheral, is required.
- • FC-AL (arbitrated loop), defines a unidirectional ring whereby only two devices on that ring can exchange data at any one time.
- • FC-SW, associated with a switched fabric and defined as a network mesh in which several devices can simultaneously exchange data at full bandwidth.
The "port" is the common element found in each device utilizing the Fibre Channel topology. A port must always consist of two channels, one for input and one for output. Servers, storage components and Fibre Channel switches all use the port interface. Servers will typically communicate by means of a host bus adaptor (HBA), such as those found on PCI cards in computers. Any connection between the ports is provided by a "link." Depending on the topology, the link may be unidirectional or directional (Fig. 1); with the fabric and arbitrated loop topologies realized by incompatible and differing protocols.
Fig. 1: Fibre Channel TopologiesTHE PROTOCOL STACK
Fibre Channel design goals were set to achieve in essence a long-distance, high-speed serial transport methodology for the interchange of data. The objectives included a low rate of transmission errors, low latency (delay) of that transmitted data, and a means of hardware-based interconnection that would free up the CPU cycles through the use of those HBA devices.
Like most other network architectures, Fibre Channel employs a protocol stack, which consists of five layers. The lower layers define the communication fundamentals (e.g., the physical medium) and the upper layer defines the applications called the upper layer protocol (ULP). It is the ULP that defines how the actual Fibre Channel network functions, say, for example, if it operates as an IP-network or as a SAN (for storage) or both.
The lowest layer, FC-0 is known as the physical layer. FC-0 describes the plugs and cables parameters for implementing Fibre Channel. Whereas SCSI has discrete parallel lines, each of which describes a bit on the overall data line, Fibre Channel uses a serial transmission protocol which mitigates the skew effects of the parallel data lines (as in SCSI) and allows for the transmission of data at much higher speeds. This layer also describes the medium, whether it be on copper or fiber-optic cabling. The copper cables are further broken into categories for intracabinet (usage within an enclosure) and intercabinet (usage between cabinets, enclosures or facilities). Fibre Channel optical cabling further supports the small form-factor pluggable (SFP) transceiver package often used for higher data rates.
The next layer of the stack, FC-1, defines the encoding parameters for data over a Fibre Channel link, which uses 8B/10B coding and is augmented by ordered sets of transmission words that make up the link control protocol administration. The 8B/10B binary encoding scheme was selected as a compromise (as compared to Manchester encoding) that works by converting an 8-bit byte into a 10-bit character that is carried over the link. Thus, to carry 100 MBps of raw data requires 1 Gbps instead of 800 Mbps without encoding. Note also that the Manchester encoding method would require nearly double that rate, i.e., 1.6 Gbps.
The FC-2 layer defines the data transfer layer. This layer regulates and determines how larger data units, such as a file, are transmitted over the network; and how the flow of that data is controlled. In addition, FC-2, the most comprehensive of the entire protocol stack, defines a set of service classes that are applied to the various applications by which the Fibre Channel protocol can be used. Given that Fibre Channel (in, for example, FC-SW) can handle multiple sets of data exchanges simultaneously, a set of classes is employed that describes connection-oriented (Class 1), and packet-oriented (datagram) services (Class 2 and Class 3). Class 2 uses an "end-to-end" flow control and link control (i.e., each frame is acknowledged); whereby in Class 3 the individual frames of the Fibre Channel transmission are not acknowledged. In practicality, and for general implementation, the service classes are seldom explicitly configured, letting the end devices effectively negotiate whether they communicate as Class 2 or Class 3. This concept may differ in closed systems where data transmission must be optimized for, say, mission-critical operations.
The common services layer, known as FC-3, is essentially left empty. Its intent is to allow for such features as encryption or mirrored and other RAID functionality. Some systems implement these principles of FC-3 in operating system software or in Fibre Channel switches that employ inter-switch link (ISL) trunking.
Fig. 2: Network and Fibre Channel stack The FC-4 protocols handle the tasks of mapping application protocols onto the Fibre Channel network. For SCSI, the application protocol is called Fibre Channel Protocol (FCP), which maps the workings of the SCSI-protocol onto the Fibre Channel network underneath. Drivers are added to the operating system that in turn recognize the Fibre Channel medium and allow for the interchange of data as though it were simply connected over a SCSI-interface or link (Fig. 2).
BANDWIDTH AND DATA RATES
Fibre Channel products are available at 1 Gbps, 2 Gbps, 4 Gbps, 8 Gbps and should be interoperable with one another. The 10 Gbps standard and its 20 Gbps derivative are not backward compatible with any of the slower speed devices. These higher data rates differ at the FC-1 level because they use 64B/66B encoding as opposed to the 8B/10B encoding for the lower data rates). 10 Gigabit and 20 Gigabit Fibre Channel is primarily deployed as a high-speed "stacking" interconnect to link multiple switches.
This installment has only touched upon the basic fundamentals of Fibre Channel for use in a network storage technology. It goes without saying we can well expect to see both data rate requirements and bandwidths continue to increase going forward. Furthermore, we can already see that physical distances between storage system components are growing further apart geographically as disaster recovery, clouds and other remote systems evolve. Thus it is that these forms of networked storage systems, found on both Ethernet and Fibre Channel topologies (deployed as SANs and NAS systems), that will be addressing interoperability and performance harmonization across all segments of the media industry going forward.
Karl Paulsen is chief technology officer for AZCAR Technologies, a provider of digital media solutions and systems integration for the moving media industry. Karl is a SMPTE Fellow and an SBE Life Certified Professional Broadcast Engineer. Contact him atkarl.paulsen@azcar.com.
