Shadow browse is a method by which a low-resolution copy of broadcast-quality material can be distributed over a LAN. There are many ways to browse a proxy of the broadcast-resolution content. At the most basic, a file is streamed for viewing at play speed. This can be used for review or approval purposes.
For many broadcast applications, something more sophisticated is needed. The remote viewer may want full VCR-like control over the file — fast wind, slow motion, cuing — and frame-accurate tagging of material so that an edit decision list (EDL) can be created. Material then can be viewed and edited on any PC connected to the network. Typically, the edit list created then can be applied to the original broadcast-quality material or used for other purposes.
Figure 1. A browse system comprises an ingest engine, server and network-attached clients. Click here to see an enlarged diagram.
As shown in Figure 1, a typical shadow browse system comprises:
- An ingest engine to capture the broadcast material and encode it to a format suitable for distribution over a network.
- A browse server for the storage of the “shadow” material until a client requests it.
- Several browse clients that provide a means of displaying and editing the shadow material and creating an EDL.
Why use shadow browse over a WAN?
When shadow browse was first introduced, the Internet infrastructure did not have the performance to cope with the amount of data involved when playing video over a WAN. It also did not give journalists and professional editors the real-time control over the playback that they required. However, with the improvements in the Internet that have taken place over the last few years, today's infrastructure now can allow real-time control over video playback.
With this improvement in the Internet, broadcasters now find that they can share digital content with colleagues in remote offices all from a central resource. They realize that they no longer need to duplicate the digital content between offices, either as digital files or as tape. This has important implications when it comes to resource management, efficiency and, of course, costs.
Overview
In principle, shadow browse will work over a WAN. However, there are several network factors and operational issues that need to be understood and considered before attempting to browse over a WAN. For optimal performance, the network needs to have a high bandwidth (preferably at least 100Mb/s and full duplex), a low-latency (a round-trip delay of less than 1 millisecond) and little (preferably no) data loss. Therefore, the network factors that need to be considered before attempting to shadow browse over a WAN are:
- Round-trip delay.
- TCP window size.
- Loss of data.
- Network structure.
- Network bandwidth.
Each of these factors will affect, in different ways, the usability of a shadow browse system over a WAN. For example, it will affect the time it takes for a browse server to respond to requests from a browse client (when browsing a browse server directory for clips, adding a clip to a clip list, requesting a seek, or pausing/restarting a playback stream) and the smoothness of the video and audio during playback.
In the following discussion, although the preferred minimum bandwidth is at least 100Mb/s, it is assumed that the minimum network bandwidth is only 10Mb/s, and it is full duplex.
Round-trip delay
On an otherwise idle network of adequate theoretical bandwidth with no loss of data, the primary factor affecting the usability of shadow browse will be the round-trip delay. The round-trip delay is the time taken for one machine to send data to a second machine and to subsequently receive an acknowledgment for that data.
If the round-trip delay is high, then the overall data throughput will be low, because protocols such as TCP rely on acknowledgement packets to regulate the transmission of the data. Consequently, on a network with a high round-trip delay, the responsiveness of a browse client application will be poor.
TCP window size
The next factor that affects the data throughput is the TCP window size. Window size is the amount of data that the sending machine can send before receiving an acknowledgement that the first data packet sent has been received. The TCP window size starts out small and grows at a rate proportional to the rate of arrival of acknowledgements.
TCP normally allows a maximum of 64KB of data to be sent before an acknowledgement has been received. Therefore, if 64KB of data have been sent, no more data can be sent until there is an acknowledgement that the first data packet has been received at the destination.
Hence, if the round-trip delay is 1 second, then the maximum data rate that can be achieved is 64KB/s. If the round-trip delay is 10ms, then the maximum data rate is 6400KB/s. This still assumes that the network does not lose any packets.
Loss of data
When data packets are lost, the window size (the number of packets sent before receiving an acknowledgement) is reduced by half; hence, the data throughput is reduced by half. If packet loss continues, the window size is reduced by half again, with the corresponding effect on the data throughput. When no more packets are lost, the window size is gradually increased with a commensurate increase in the data rate. Therefore, loss of data may result in a degradation of the browse system. The more data lost, the poorer the browse experience will be.
Network structure
In a conventional shadow browse system, the browse server and the browse client would be the same sub-net, but this is not an absolute requirement. If the browse server and the clients are on different networks — especially when separated by public infrastructure — they are almost certainly separated by one or more firewalls, which will implement restrictive access policies.
Linking machines on remote networks in a secure manner is normally done using a virtual private network (VPN) or network address translation (NAT). A VPN provides a transparent “tunnel” between two remote networks, meaning that nothing special needs to be done in order to communicate between machines on those networks. Firewalls with NAT result in IP address and port values being translated between public and private variants, meaning that protocols that pass addressing information around need to cope with this.
Network bandwidth
Browse systems tend to use different data connections for each of the audio and video clips in a play list. Where playback occurs over a clip boundary, the browse server will send data for the currently active clips and data for those clips that are about to be played at the same time but using different data connections. This happens so that the client machine is primed with data for the following clip and, therefore, can play it seamlessly after the preceding clip.
Each data connection has its own TCP window size. In networks where the round-trip delay is high, this approach could be an advantage. However, at the point just before a clip boundary, there is a short period of increased data traffic because the browse server is sending data for the current and following clips at the same time. With clip lists containing several adjacent short clips, this increase in the network traffic will occur more frequently. In bandwidth-restricted networks, this may lead to a reduction in playback performance.
Also, because using a browse system at fast forward and rewind potentially increases the amount of data being passed over the network, it may be necessary to decrease the playback speed above which such systems switch to dropping data while playing at fast forward and rewind speeds.
As shadow browse files have a low data transfer rate (typically 1.2Mb/s for an MPEG-1 proxy) over a 10Mb/s network with a low round-trip delay, the bandwidth (i.e. the 10Mb/s) will become an issue only when the number of streams being used increases to saturate the network. Over a 10Mb/s full duplex network using 1.2Mb/s files, it should be possible to use three, possibly four, client applications before the network traffic has a significant impact on the data flow.
Using a WAN simulator
On a LAN, even where there are several switches and hubs between the browse server and the client machines, the round-trip delay is typically about 0.5ms. On a WAN, the round-trip delay is likely to be much higher than on a LAN, and a WAN is also more likely to suffer from lost packets.
While a shadow browse system may still be usable as a simple playback device when the round-trip delay is about 50ms (i.e. approximately 100 times slower than the LAN), it is likely that certain operations — such as browsing clips on a browse server, loading a complex clip list, starting playback, and seeking and changing the playback rate — will be slow. The longer the round-trip delay and the more data that is lost, the less responsive the browse system will be. It is also likely that with a round-trip delay of 15ms and greater, the user will experience pauses in the playback of clips from a browse server. Clearly, even with a WAN that has a low round-trip delay, if there is sufficient packet loss, the user will still experience pauses in the playback of clips from a browse server.
While we have tried to investigate the effect of different network conditions on the usability of a shadow browse system over a WAN, the figures quoted are the findings when using a software network simulator. Although we believe the data to reflect accurately what will be experienced on a real network, it can only be taken as a guide to what may be found when using a shadow browse system over a real WAN.
However, the principles stated in this document are correct. Provided that the WAN used has a low round-trip delay and low (or no) packet loss, then a shadow browse system will work as it does over a LAN.
Conclusion
A shadow browse system will work over a WAN provided that certain criteria are met. These criteria are that the network:
- Has bandwidth suitable to the amount of data that is going to be sent, taking into account factors such as clip boundaries.
- Has a low round-trip delay.
- Has very few lost packets.
- Allows direct connections to be made from the browse client machines to the browse server and from the browse server to the browse client machines.
It may be possible to achieve these criteria without implementing any special network, but it may be prudent to use a VPN to achieve the necessary criteria.
Dr. Martin McAvoy is responsible for IPV's worldwide technical service.
