Ethernet Networking in the 10 GB Domain

The origin of Ethernet began some 25 years ago, and over time, has evolved to meet the increasing demands of packet switched networks. Today, Gigabit Ethernet (IEEE 802.3z) has become a mainstay standard for networking, yet the expected demand for systems with data rates in excess of 1 Gbps is no longer just a dream.

Summer 2002 brought ratification of the 10-Gigabit Ethernet (IEEE 802.3ae) standard. As the most recent implementation in LAN technology, 10-Gigabit Ethernet's initial goal was to develop a new high-speed technology that expanded, yet complemented, existing Gig-E capabilities.


(click thumbnail)Fig. 1

(click thumbnail)Fig. 2Under the International Standards Organization's OSI (Open Standards Interconnection) model, Ethernet is fundamentally a Layer 2 protocol. 10-Gigabit Ethernet uses the IEEE 802.3 Ethernet Media Access Control (MAC) protocol, the Ethernet frame format, as well as the minimum and maximum IEEE 802.3 frame size. The 802.3ae specification defines two PHY types (where "PHY" is the network terminology for the lowest PHYsical level in the OSI model, Layer 1). The relatively recent development of the 10-Gigabit Ethernet standard as a full duplex, fiber-only technology is shown in the timeline in Fig. 1.

Included in this technology is a "dual networking" capability, which includes higher speed backbones for the LAN and a methodology for direct integration into WAN/MANs. This requires that a distinction be made between which network is addressed, as the port hardware needed is determined by the network type being connected, as shown in Fig. 2.

DARK FIBER

When used in a LAN PHY environment, the 10-Gbps data rate is intended to be carried over dark fiber (or dark wavelengths)-deploying dual networking capabilities imposes new cabling scenarios, as both LAN and WAN PHYs may use the same type of physical cables with potentially different transceivers and fiber. To ensure reliability and proper functionality, good documentation and identification of cabling media should be used. Furthermore, when ordering 10-Gigabit Ethernet products, configuration and part options must be precise.

Laser wavelengths, interface types and media types all play important parts in establishing the proper media links for 10-Gigabit Ethernet. The higher bandwidth data rates of 10-Gigabit Ethernet required different physical interfaces for distribution and connectivity. Mapping the type of media, along with the placement of fiber links in data-centric facilities, now requires that the characteristics of the signals and the networking needs be identified early. Be certain, when planning for higher bandwidth deployments, that the type and wavelengths of fiber systems will support the full gamut of Gigabit and 10-Gigabit Ethernet-working (see Tables 1 & 2).


(click thumbnail)Table 1

(click thumbnail)Table 2Intended for interswitch links versus smaller subsystems, 10-Gigabit Ethernet now employs full duplex-switched communications, with two separate channels for the send and receive ports. The added complexities deterred the early stages of deployment, and the expectations were that few individual systems would receive the full benefit of 10-Gigabit connectivity. This affected the initial development of host adapters, but is expected to change as acceptance and new designs come forth.

One prominent feature in 10-Gigabit Ethernet is its improved coding efficiency in relationship to total available bandwidth. The more familiar Gigabit Ethernet and Fibre Channel both use an 8/10 encoding ratio, which results in a 20 percent overhead in data transmission requirements. The 10 Gigabit Ethernet encoding scheme instead uses a 64/66 ratio, which means that for every 64 bits of data transmitted, only two extra bits of encoding information are transmitted, yielding a much improved efficiency with an overhead equal to approximately 3 percent.

POSSIBLE USES

The possibilities for implementing 10-Gigabit Ethernet for networked storage technologies, both for SAN and NAS implementation, along with the higher bandwidth made available, may offer new opportunities for large-scale and remote-located storage subsystems. Today, the immediate practicality for remote connectivity of the physical storage platforms for media files may not be substantial; but in the same fashion that media storage for video server applications has followed the path of IT-based storage on a local (attached) basis, there may well be an evolution of 10-Gigabit Ethernet that addresses such issues as live remote data protection over high-speed, long-distance connections.

Bandwidth sharing, where storage networks are concerned, requires that a large amount of bandwidth be available for storage I/O. To place into perspective how extensible 10-Gigabit Ethernet might be, consider that most data networking applications work very well with 100 Mb connections communicating between local nodes, which-including overhead-is equivalent to approximately 10 MB of conventional data-only storage I/O. Yet the average I/O requirements for today's large media file storage systems exceeds that number significantly?with data rates in excess of hundreds of megabits per second. Given that perspective, a possible scenario for the merging of 10-Gigabit Ethernet into NAS or SAN storage for the WAN/MAN environment may be through the introduction of redundant, remote data protection at very high speeds.

Although video server manufacturers have developed components and applications that allow an electronic copy of media files to be sent to an alternate or secure remote location, the high price and low performance have limited the effectiveness of its deployment for most real-time operations. Alternative off-site storage of valuable assets, such as entertainment or news media, if placed into the hands of a third-party data warehouse, may be met with resistance because of the lack of control and chances for leaking of that data to other users.

KEEPING IT SECURE

Developing a secure, controllable, accessible and extensible off-site storage solution may eventually fall into the 10-Gigabit Ethernet domain. Despite the fact that 10-Gigabit Ethernet has excellent performance, and its support for maximum distances of 40 kilometers offers new opportunity, the cost of the long-distance technology necessary will probably remain high for some time. However, when compared to Fibre Channel's maximum link distance of 10 km or Gigabit Ethernet's maximum link distance of 5 km, a 10-Gigabit Ethernet solution might have the characteristics to make it appealing for remote storage?and could certainly be implemented in a local or campus (MAN) environment even today.

When placed into the context of high-end networks, such as those deployed by the telecommunications industry that run at data rates of 9.6 Gbps (OC-192), the newer SONET SDH may provide a path to the solution of live, accessible protective storage for disaster recovery or segmented playout to other facilities?by using 10-Gigabit Ethernet technologies. This concept is already made possible in the data world because the WAN PHYs for 10-Gigabit Ethernet are designed to connect to existing or new SONET SDH networks. And with the WAN PHY a superset of the LAN PHY, LAN-interoperability with SONET SDH networks is now possible, in essence creating an extension of the LAN over high-speed, long-distance communications links. The variance between traditional SONET SDH applications, which use synchronous transmissions, is that 10-Gigabit Ethernet WAN PHY uses asynchronous transmissions, and in turn does not use some of the higher level transmission logic associated with SONET SDH networks. Furthermore, by framing and receiving SONET SDH transmissions, basic network management functions can be extended to remote storage solutions.

Today, some of these concepts might appear to be "out in space" and may well be unsuitable for many operations. However, given the phenomenal growth in storage and networking technologies, looking fast forward in time especially when building new facility architectures means we should take notice of the opportunities that may develop, and be prepared for the new horizons in data and media storage technologies.

Karl Paulsen
Contributor

Karl Paulsen recently retired as a CTO and has regularly contributed to TV Tech on topics related to media, networking, workflow, cloud and systemization for the media and entertainment industry. He is a SMPTE Fellow with more than 50 years of engineering and managerial experience in commercial TV and radio broadcasting. For over 25 years he has written on featured topics in TV Tech magazine—penning the magazine’s “Storage and Media Technologies” and “Cloudspotter’s Journal” columns.