As a younger person, I thought I knew everything — especially about what appeared to be simple technical topics. But as the old adage goes, the older I get, the less I understand. This month we will look at the core protocols that enable your media network, or for that matter, any network. You may find that there is more going on than you thought.
Internet Protocol (IP)
When it comes to core protocols, it would be hard to argue against Internet Protocol (IP) being at the top of the list. We all know that everything these days travels over IP networks, but what is Internet Protocol exactly? To answer that, we first should describe what a protocol is.
Long before there were computer networks, protocols described formal agreements between two parties about how they should interact. Protocols were often employed in affairs of state — interactions between nations. It is not too hard to see how agreements between computer systems about how they should interact got the name “protocols.” Protocols describe the expected messages and allowed responses on a computer network.
IP's job is to put information in a header, which allows data to be routed from one device to another. IP is low on the protocol stack, so it is closely related to the physical and electrical media that will be used to carry the data. IP prepares data sent to it by higher protocols for transmission across a network, taking into account things such as the packet length, hardware addressing structure and how data should be split across multiple packets (if this is allowed). IP addresses can be assigned by a network administrator, and this allows that administrator to group computers together and place them all on the same network.
These days, Ethernet is the dominant electrical and physical networking technology, especially in the broadcast environment, but remember that datagrams sent to you across the Internet may be moved using wire, optical fiber, satellite and wireless. The specifics of these physical mediums are standardized at a lower level. The IP header contains 13 different fields. Two very important ones are the IP source address and the IP destination address. IP addresses allow datagrams to be routed automatically from sender to receiver across different physical networks. But how does that work exactly?
Address Resolution Protocol (ARP)
ARP allows network devices to associate a particular IP address (192.168.30.20, for example) with a specific piece of hardware. Behind the scenes, routers build ARP tables that contain the IP address of a device and the hardware address of the device. For Ethernet networks, the hardware address is known as the Media Access Control (MAC) address. While IP addresses can be assigned by anyone, MAC addresses are assigned by the equipment manufacturer and are unique for the device. Once the router associates an IP address with the unique hardware address, it can transfer the data to the correct device. You might ask: Why have two addresses? The answer is because not all IP networks are Ethernet networks. For example, you can run IP over Synchronous Optical Networks (SONET) or Token Ring. The identity of physical devices on SONET or Token Ring networks are established with other mechanisms. This allows us to deliver the same IP datagrams to the end user regardless of whether the datagrams travel over different physical networks.
Ports and port numbers
Port numbers are used in several protocols, but perhaps most importantly in Transport Control Protocol (TCP) and User Datagram Protocol (UDP). The concept of port numbers originates from the time when mainframe computers were physically wired to terminal devices such as teletype machines. (We are talking about a long time ago.) The physical connections to the main frame were called ports (thus TTY0, TTY1, etc. still found in Unix today). IP addresses allow you to send data from one computer to another.
But once a packet is delivered from source to destination, how does the destination computer know what to do with it? It could open the packet to examine the contents, but a better way might be to add a tag that tells the computer what it is. You may have heard that HTTP (Hypertext Transfer Protocol) is normally on port 80. What this means is that HTTP packets are marked as going to port 80 so that the receiving computer can automatically route those packets to a Web browser listening on that port. But how do you get the port number into the packet?
Transmission Control Protocol (TCP)
TCP is used to collect data from an application, perhaps a Web server, and to send that data to a Web browser at the other end. The TCP header contains a source and destination port number field that allows the application to mark the datagram appropriately. But TCP provides another vital function: TCP attempts to guarantee delivery of the datagram. TCP does this by stamping each datagram with a unique sequence number. It then looks for the receiver to acknowledge that it received the datagram. If the receiver takes too long to respond, or if the receiver asks, the sender will resend the dropped packet. TCP also implements a number of rate-control mechanisms to deal with rate limits imposed by the receiver and to deal with congestion issues on the network.
TCP does one other nifty thing besides handling lost packets. It reorders packets that have been received out of sequence. Once a packet is launched onto the Internet, it is on its own, and there is no association between this packet and the one that comes before or after it. Packets can and do arrive in a different order from the order in which they were sent.
TCP is extremely common, and it almost always runs on top of IP, so the notation TCP/IP is common. But bear in mind that TCP/IP specifies two separate protocols: TCP and IP.
User Datagram Protocol (UDP)
UDP operates at the same level as TCP and provides the same basic functionality. As with TCP, UDP collects data from an application and directs it to a particular port on a destination device. But UDP is a much lighter weight protocol. UDP simply collects the data from the application above, stamps it with a source and destination port number, and sends the datagram on its way. Nothing in the UDP protocol guarantees that packets sent across the network will reach the receiver. In fact, UDP explicitly does not check to see that packets have been received. UDP is a “fire and forget” protocol.
UDP is often employed in professional video-over-IP contribution devices. UDP's light weight is a benefit when dealing with the high bit rates required for live video transmission. Also, with professional video, requesting a retransmission of lost packets may not be practical. By the time the whole retransmission process completes, the time for the video to be displayed may have passed.
UDP packet sizes can vary, and in some cases UDP packets can be very large. This brings up the issue of fairness. Very large UDP packets may hog bandwidth on a network, causing other traffic to suffer. For this reason, and for other security reasons, some system administrators do not permit UDP traffic to cross their firewalls. This can cause headaches for broadcasters who are using UDP for backhaul transport.
File Transfer Protocol (FTP)
As the name implies, FTP is used to transfer files across the Internet. FTP has some excellent features that make it an indispensable protocol. FTP handles lost packets and reordering. It also senses congestion on a link and employs automatic rate control to relieve the congestion.
That said, FTP has some characteristics that make it unsuitable for moving professional video files. First, many FTP applications have a file size limit of 2GB. Professional video files can be much larger than this, so this limitation can be a real problem. Second, FTP has rate-control mechanisms that can interfere with transmission of large files. If FTP senses that the network is congested, it responds by cutting its speed in half. If the congestion continues, FTP cuts its speed in half again. This continues until the transfer either succeeds at a lower rate or aborts due to timeout. If the session has not timed out and the congestion situation improves, FTP increases its speed, but it can take a long time (several tens of seconds) for FTP to get back to its initial speed. You can see this rate reduction on a network traffic monitor as a stair-step pattern. Unfortunately, in some cases, FTP's rate control mechanisms can limit throughput to a low level even though the available bandwidth is high and congestion does not exist.
There are a number of other protocols that are critical to network operations. See Table 1 on page 24 for a quick summary. Also, I strongly recommend that you read “Internet Core Protocols” by Eric A. Hall (O'Reilly). This excellent book will give you much more detail on these protocols.
Brad Gilmer is president of Gilmer & Associates, executive director of the Advanced Media Workflow Association and executive director of the Video Services Forum.
Protocol Use HTTP(S) Hypertext Transfer Protocol (Secure) Used primarily by Web browsers, but increasingly used for the transmission and retrieval of files and other data. HTTPS encrypts data between the server and client. IGMP Internet Group Management Protocol IGMP is the core multicasting protocol. It allows a single host to send out messages to multiple clients. This protocol is used to send one stream to many receivers. POP Post Office Protocol Used to check and retrieve mail on remote mail servers. SMTP Simple Mail Transfer Protocol Used to send mail through a mail server. SNMP Simple Network Monitoring Protocol Used to remotely monitor equipment on a network. May also be used to execute limited remote commands. SSH Secure Shell Secure terminal emulation for use between clients (usually system administrators) and Internet servers. Telnet A nonsecure terminal emulation protocol for use between clients (usually system administrators) and Internet servers.
Table 1. This table lists a number of Internet core protocols that could not be covered extensively in this article.
Categories: Storage and Networking
Send questions and comments to: brad.gilmer@penton.com
