I was asked recently why HDTV content (1080i, 720p) looks so good on 4KTV’s. Shouldn’t HDTV appear as soft on 4KTV’s as SDTV looks soft on HDTV’s? That is a good question and to start we have to go back in history (not too far) to begin to unravel this conundrum.
In the Beginning...
In the 1990’s HDTV was being birthed in labs around the globe. In 1993, the ATSC in the U.S. standardized a new digital terrestrial broadcast system that could deliver an HDTV program via a 6MHz RF channel.
One aspect of this new system (now referred to as ATSC 1.0) is the infamous table of 18 different video formats—"infamous" in that it caused as much of a stir as the fact that ATSC abandoned analog for TV transmissions. What were the broadcasters and manufacturers going to do? Were broadcasters going to produce content that was optimized for each format or were they going to agree to a common HD format? Would the new HDTV sets be able to accept any and all of the video formats or just a select few?
By the late 90’s the landscape looked like this:
Broadcasters:
- 1080i60 (CBS/NBC)
- 720p60 (ABC)
- 480p (Fox) (Fox switched to 720p in 2004).
Set manufacturers:
- Display 1080i60
- 480p60 or 480i60, which allowed for a common horizontal scanning frequency (note that there were some native 720p displays available)
With 720p content, the TV set had to convert the 1280x720p60 signal into either 1080i60 or 480p60. Both approaches required either upconversion or downconversion.
But the tougher challenge that these sets faced was the conversion of analog NTSC (484i60) transmissions during the transition period from analog OTA TV to digital OTA TV. NTSC proved to be a challenge to convert to an HDTV display as it requires good demultiplexing from composite to component color, noise reduction, color space conversion (NTSC to ITU Rec 709) as well as re-scaling as the NTSC images are formatted in a 4:3 aspect ratio, not the 16:9 aspect ratio of HDTV.
One of the first manufacturers, Panasonic, recognized the need for an image processing chip that could do all the required image operations: scale, upconvert, downconvert, de-interlace and noise reduce. Known as the UFC1000 chip, it was used in some HDTV sets to enable the formatting of any of the 18 different formats in ATSC 1.0 to a common display set of parameters. This chip was also incorporated into a professional product known as the UFC1800 (for the 18 different formats!). Other manufacturers followed suit and added this type of image processing to their TV chip sets.
The Limits of CRT
While the HDTV sets of this era did a good job converting all the possible formats, the final display quality was limited by the abilities of the CRT (flat panels had yet to become a mainstream consumer product). This helped in some ways as the CRT spatial response is gaussian and has a smooth roll-off response curve, thus hiding some artifacts and noise.
On the other hand, it limited the maximum spatial frequency response, especially at peak brightness, to 500-600 lines. Also, ATSC 1.0 specified the use of MPEG-2 compression, which is limited to 8 bits for video and while MPEG-2 could reasonably transmit 1080i in approximately 18 Mbps, its compression efficiency is another limiting factor to delivering full HD resolution to the TV (in fact most 1080i compression at that time used a subset of 1080, 1440x1080 to save bandwidth).
With all the above issues, SDTV (aka NTSC) when presented on an HDTV set, was a significantly lower quality experience—better than VHS, but not as good as DVD. Viewers could readily discern the difference between standard definition on an HDTV display.
Flat Panels Arrive
During 2000-2005 the first generation of flat panel technologies became available in the TV market—initially LCD type flat panels followed by plasma types (PDP). These panels were larger and brighter than most CRT TV’s but were limited to 720x480p60.
After a few years, the flat panels standardized on 1920x1080p60 as their HDTV format. The embedded image processing became more sophisticated and thus the internal format conversions, including de-interlacing, improved. While it could not make NTSC look as good as HDTV, it did make 480i60 digital imagery competitive with HDTV.
Some of the technologies that were adapted for upconversion included: Motion Compensated upconversion, Edge Detection and Smoothing and later Adaptive Resolution. By 2010, flat panels were taking the place of CRTs, NTSC was going away and the pictures, both HDTV and SDTV never looked better.
2008 was the dawn of UHDTV. Initially display manufacturers spoke of "QuadHD" panels, meaning that the panels had native pixels arrays of 3840x2160. When these panels become affordable enough to put into consumer products, they were branded "Ultra HD" (or UHD).
As these UHDTV’s became available, there was a call to produce content in this new format. The Blu-ray Disc Association (BDA) standardized UHD for Blue-ray discs in 2015. In 2016 Hollywood studios began to release movies in this format, which was very similar to the 4K spec for Digital Cinema (DCI). Broadcasting standards were limited to HDTV, although cable and satellite TV could deliver enough bandwidth for specialty channels or one-off broadcasts (such as the Super Bowl, Olympics or the World Cup).
For the most part, these new UHDTV displays upconvert the available HDTV programming into this new higher resolution format. In addition to the tools already available for upconversion, improvements to the image processing tool box include: HDR processing (Dolby Vision, HDR10+), Spatial Domain processing, neural network (aka AI machine learning), as well as advanced image compression that supports 10bit video (HEVC).
Viewing Distance
Now that we’ve finished our history lesson, let’s return to the question, "Why does HDTV look so good on 4K displays?" First thing to note is that oversampling, either in the camera or display, provides improved resolution in any down or upsampled image. How do we determine the resolution of a program that provides "sufficient quality" for good viewing on a UHD display?
HDTV was designed for a ‘3x picture height’ viewing distance (aka the "Lechner Distance").
At this distance, the field of view (FOV) is about 32 degrees. This provides, at HDTV resolutions (1920x1080), approx. 120 cycles/degree of spatial resolution to the eye.
As a reminder, normal vision, as measured by a Snellen Chart (the E chart) at 20/20, is 30 cycles/degree. Our eyes are able to resolve, in the Central Island of the Fovea up to 60 cycles per degree. Psychovisual researchers have found that we have something they call "hyperacuity" which accounts for our eye-brain processing to see details beyond the physical limits of the retina. Depending on the context of the image, hyperacuity can be from 120 to 360 cycles/degree. For our purposes, lets say 120 cycle/degree is sufficient resolution for a display to re-produce the image’s spatial details.
UHDTV, having 4x the pixels, is designed for viewing at 1.5x picture height distance, or about a 62 degree FOV. This requires a full 3840x2160 image for the hyperacuity criteria of 120cycles/degree at the 1.5PH distance ("half a Lechner").
Two points to consider here: Are we watching at 1.5 picture heights? For a 65” diagonal display, this is about 4 feet—not the typical living room viewing scenario of couch to wall of 8-9 feet. Also given that the Fovea region of the eye (the area of the retina with the highest spatial frequency response) represents a 6 degree FOV, sitting at 4 feet requires more head movements by the viewer to be able to track moving objects on screen than if viewed from 3 PH. This may not only be uncomfortable, but as well turning one’s head introduces motion blur with reduced resolution in the eye.
The second point is if the screen is doubled in height so the viewing distance doubles to 8 feet (one "Lechner"), is this a reasonable assumption? Well, a double-height from 65” diagonal screen measures 130” (10.8 feet) diagonally, certainly not feasible for a typical house nor household budget!
Most of us view our TV’s at around 8 feet. A program in HD resolution provides "sufficient spatial quality" for our enjoyment of the program on a UHDTV. Does this mean that there is no benefit to watching in UHD? There is of course, especially if you have larger displays and are viewing at a closer distance—but the differences are subtle. Also, UHD programs can come with improvements such as HDR (High Dynamic Range) and wider color (Rec 2020).
Compression and Production Standards
Are there other factors affecting the received resolution? Yes, video compression. While the compression efficiency of HEVC makes UHD possible to be delivered over ATSC 3.0 (the new digital terrestrial standard) as well as via the internet (OTT), satellite and cable, there are tradeoffs between the available bit rate, video complexity and resolution.
Modern video compression algorithms use adaptive compression techniques that reduce spatial resolution to avoid annoying compression artifacts. When the available bandwidth over the internet drops, OTT distribution servers switch to lower bandwidth renditions that may have less than full UHD resolution.
What about program production? Movies are produced in 4K so UHD can be produced from the cinema master. Live production (mostly sports) has moved up to 1080p60, from which HDTV (either 1080i or 720p) can be extracted as well as upconverted to UHD for distribution.
Some of these live events are also produced using HDR and wide color (the UltraHD Forum Guidelines define UHD as including 1080p60 with HDR and wide color. Big event productions (recently the Olympics from Japan and China) were produced in UHD (either 4K or 8K) along with HDR and wide color, delivered in UHD over cable, satellite, and the internet as well as in HDTV.
In summary, there have been steady improvements in the image processing embedded in TV sets that convert all forms of video formats to be displayed on an UHD TV with sufficient spatial quality. Viewing at home with a modern flat panel UHD TV, both native UHD as well as HDTV programs can be satisfying experiences.
