Blog, Media - Sunday June 29, 2008 - 6 Comments
What Does Quality Online Video Look Like?
Consumers have demonstrated a preference for three basic types of online video experiences over the past few months: Video Snacking, Download-to-Own and Online Television. Each of these three consumer behaviors has a specific value chain associated with it. Video Snacks are hard to directly monetize. Download-to-Own files are hard to protect. But, Online Television is, for all intents and purposes, television using the public Internet as the distribution network. And people who have popular content are enjoying excellent financial results from making that content available online.
You can find examples of Internet Television at hulu.com, abc.com, nbc.com, cbs.com, fox.com. In fact, almost every major television network offers some kind of online viewing experience for their most popular shows. Which begs the question, “What does quality online video look like?” Should it look like Standard Definition Television? Should it look like HDTV? Should it have to meet “broadcast quality” standards as a benchmark?
We have come to the time in the transition from network to networked television where setting some minimum requirements for the online viewing experience would be helpful. I’d like to assemble a group of video professionals, compile a list of requirements and set-up some independent testing groups to play video watchdog for the industry. And, I’d like you to help me get it done!
To start the dialog, here are my suggestions for the subjective attributes of quality online video:
1 - The video has to start very quickly (like within a second of when you press the play button).
2 - Continuous, full motion video that looks sharp at full screen.
3 - Colorspace that matches or exceeds broadcast NTSC television.
4 - Stereo audio with a dynamic range that exceeds broadcast standards.
5 - No buffering after the initial picture comes on, no exceptions.
6 - No drop out, pixilated frames or other artifacts on the screen.
To achieve these subjective goals, we will have to create a set of test criterion that takes several things into consideration:
1 - Encoding, the art and science of master video files and making them available for distribution.
2 - The player software.
3 - The topology of the distribution network.
4 - Speed of the user’s broadband connection.
5 - The quality of the user’s broadband connection.
6 - The quality of the user’s computer.
With all of these variables, it is very difficult to maintain video quality from video publisher to consumer (no matter how you define quality). Mostly because there are so many components along the signal path that video publishers don’t control. But let’s press on.
If we were to start thinking about measuring the quality of an online video viewing experience here are a few things we might measure:
1 - Start Time: As measured by the average time it takes for video to begin playing.
2 - Quantity of Impairments: As measured by the number of impairments over a given length of time.
3 - Average Length of Impairments: As measured by the average duration of stalls or buffering.
4 - Wait Time on Seek: As measured by the average duration of buffering or stalls before the video begins to play from the seek points.
5 - Wait Time on Ad Break / Return: As measured by the average delay duration when programming cuts to an ad, or when an ad ends and returns to regular programming.
6 - Video Quality Delivered: As measured by average video bit rate delivered.
7 - Link Efficiency: As measured by the percentage of a user’s bandwidth consumed.
8 - Encroachment Test: Tiered scoring of the above tests as additional viewers move onto the network.
The list above isn’t complete, but it’s a start.
We also need to set standards that adjust for the type of broadband environment in which the video will be consumed. For example: ADSL at 768 kpbs down and 384 kpbs up or Cable modem at 5 Mbps down by 768 kpbs up. Unless you take the network environment into consideration, the standards will be hard to achieve. We will have to “handicap” our standards to the limits of each network.
So here’s the pitch. Online video is coming into its own. People are watching and, as in industry, we need to define a quality experience the same way that the broadcast networks do. We need to create testing environments and set standards of quality that each distributor can strive to achieve. I think it’s a job for everyone who wants to be involved. If you’re interested in helping out , send me an email. It’s time.
Shelly Palmer is the host of MediaBytes a daily news show featuring news you can use about technology, media & entertainment, Managing Director of Advanced Media Ventures Group LLC and the author of Television Disrupted: The Transition from Network to Networked TV (2006, Focal Press). Shelly is also President of the National Academy of Television Arts & Sciences, NY (the organization that bestows the coveted Emmy® Awards). He is the Vice-Chairman of the National Academy of Media Arts & Sciences an organization dedicated to education and leadership in the areas of technology, media and entertainment. Palmer also oversees the Advanced Media Technology Emmy® Awards which honors outstanding achievements in the science and technology of advanced media. You can read Shelly’s blog here. Shelly can be reached at shelly@palmer.net
Comments
6 Responses to “What Does Quality Online Video Look Like?”Dan Raybun June 30th, 2008 9:05 am
Hi Shelly, what you are asking is for is for the industry to define
a standard, in a world where there are no standards and in some
cases, where standards would hurt content owners. Today, there is
no standard bitrate, aspect ratio, codec, player, etc… and that
won’t change. You can’t get content owners to all agree to use the
same settings and it would not be good for the industry. For some
content owners, encoding at 300Kbps works just fine for their
budget and for the type of content they have. Creating a standard
that for instance says you have to encode at 500Kbps, may not be
right for all content owners. No “standard” should be used in this
case. I think (wish) there was a standard video platform and
player, that would make things a lot easier. But when it comes to
the encoding bitrate, aspect ratio, etc… all of that should be up
to the content owner based on what type of content they have, the
length, who they are trying to reach and the device it is played
back on. As for your comment on the quality, we already have that
from many of the CDNs. In order for any CDN to be in the Microsoft
hosting provider program, they have to use Keynote to test the
quality of their streams, based on the start up time, buffering,
packet loss etc….. and get ranked on that. Most CDNs also use
their Keynote ranking in their SLA. Also, you can’t define what
quality looks like. Neither can I. Only the person watching the
video can. One person may think a 500Kbps stream looks great,
another may not agree. Quality is subjective to the person viewing
the content and they are the only ones who really decide what
“quality” really means.
Rebecca June 30th, 2008 10:25 am
Hi Shelly, You may want to add the desired consumer experience to
your list of parameters — meaning viewers who are “snacking” may
not necessary want to wait for a more quality experience that say a
“download to own” video might display. It’s always about getting
the best bang for the bandwidth.
Kieran July 2nd, 2008 6:59 am
Quality standards for online video is long overdue and is crucial
for the future success of online video as a viable alternative to
TV. As more and more publishers are placing professional,
long-format, content online, standards become more important. Along
with video quality I would like to see standards set for video
players and most importantly video metrics. Video metrics is very
wild west right now with no clear solution on how to track all
aspects of a users interaction with videos.
Mark Taylor July 3rd, 2008 8:49 am
Shelly this sort of discussion is long overdue. We certainly need
an objective way to measure, and more importantly compare, video
experiences over the Internet. Something that takes into account
signal acquisition, encoding, distribution and the player. The
interaction of all those things has a huge impact on quality. I
agree with Dan that much is still subjective to the user and with
Rebecca that different levels of quality are important. But there
is no real way to have an objective quality measure today in order
to do side-by-side comparisons. How do you know if one player is
better than another? One encoder better than another? One CDN
better than another? And combining all those things together makes
it even harder. It amazes me that some publishers never seem to
look at the quality experienced in real life. They seem to accept
demos run on people’s laptops with no actual Internet connection.
In fact I have seen organisations demo their new web site
internally (and it looks great) and then asked them to run it on my
laptop over a wifi network in the same room - didn’t even work. And
the people who really need this sort of data are the advertisers.
They are effectively being asked to assume all video sites (having
the same audience size) are the same. How can they value the
quality of one solution over another across a wide audience base?
They can’t. But there is a huge relationship between the length of
time people watch video and the quality of that video. That needs
to be measurable and comparable. If it were, the best would attract
higher advertising dollars and that in turn would provide an
incentive for everyone to strive for the most appropriate level of
quality for their type of content.
Mark Robertson July 11th, 2008 2:16 am
Shelly, I think it would be nice to test and define ‘quality
experience’ for producers as a guide to “best practices” so to
speak. I would certainly like to see a well researched set of
guidelines for best quality experience. I think even more important
would be a group that could keep this definition, so to speak, up
to date as the parameters will likely evolve with industry growth.
As far as standards, I happen to agree with Dan’s comments. I think
that in addition to the items you mentioned, it would also be nice
to take a look at usability testing and defined parameters for best
user experience in terms UI of player, page, etc… I Look forward
to more…. Thanks
Usability Standards for Web Video (Where the hell are they??) - ReelSEO January 20th, 2009 2:59 am
[...] Palmer, MediaBytes.com. His article, “What does quality
online video look like?”Offers some good suggestions for online
video quality. While the suggestions are largely dedicated [...]