Video – The State of the Web

Video – The State of the Web


RICK VISCOMI: Hello
everyone, and welcome back to “The State of the Web.” My guest is Doug Sillars. He’s a freelance digital nomad
working on the intersection of performance and media. And, today, we’re talking
about the state of video. Let’s get started. [MUSIC PLAYING] Doug, so thanks for being here. Take me back to the days
before the video element. What was it like to
have video on the web? DOUG SILLARS: So before
we had standardization in the browsers, the
only way to get video in like a desktop browser
was to use a Flash plugin, so all the video was
being delivered via Flash. And then when you start
thinking about mobile, you know, we were on really slow networks. It was like 2G, maybe
3G if you were lucky. And it was Motion
JPEG, which was literally just like small,
really grainy images being played at 15 frames
per second that sort of gave the idea that it was a video,
I guess, but it wasn’t pretty. RICK VISCOMI: So what
was the user experience like in terms of,
like, having that kind of jankiness of video? DOUG SILLARS: I guess because– I mean, we’re talking
flip phones, right? This is the era
before the iPhone and smartphones and
big screens, so we weren’t talking high
resolution screens anyways. So I think it never
really caught on because it wasn’t beautiful. But for the early
adopters, it was like, hey, I can get video on my phone. RICK VISCOMI: Mm-hmm. What were some of the
limitations of video? What held it back, back then? DOUG SILLARS: I think a
lot of it was networks. Right, so the video has
to be transmitted, right? I mean, this is the
days when we were all getting DVDs in the mail. Right, that was the way
we were transmitting video to get that sort of
quantity over a network. We just didn’t have the
network bandwidth to do it. And then, of course, we
didn’t have the hardware. Like, the chipsets on our
computers and our phones just didn’t have the oomph
to process that much content. RICK VISCOMI: So fast
forwarding to today, what are the different ways
that video is being used? DOUG SILLARS: So when
we think about the web, there’s lots of different
ways you can do video. You know, you’ve got sort
of the YouTubes, which has a billion people
going, every day, watching billions of hours
of user-generated content. You’ve got the Netflixes
of the world, where you’ve got video on-demand, watching
your favorite movies and TV shows. But what I usually work with
are sort of websites that have video, whether it’s a
background video or maybe a web page that just has a
video like, this is our company, here’s a two minute
video of what we do. And what’s neat
about that is there’s just so many different use cases
for using video on the web, and you sort of have to
think about how you’re going to apply the
different ways of delivering the video for those
different use cases. RICK VISCOMI: My feeling
is that most websites that need to embed video
would probably just prefer to embed it via like a
YouTube embedder, as opposed to self-hosting. What are some of the pros
and cons of third-party versus self-hosted video? DOUG SILLARS: Right. So, like, YouTube is
great because then you’ve got all the experts
at YouTube doing all the encoding
and all the delivery and all of that stuff for you. There are a couple tricks to
that in that one is, like, if you’re embedding a YouTube
video, if somebody flags it on the other platform, it might
stop working on your web page. Another thing that
happens is when you embed a YouTube
player, you’re adding 700 kilobytes to a
megabyte of JavaScript fonts, et cetera, that come
from the YouTube player. So there’s a hit that you take
even before the video starts getting delivered in
terms of tonnage that’s coming down to your
web page, which can add to performance issues. RICK VISCOMI: And
it’s also, who’s paying the bills of that video
being served over the wire? DOUG SILLARS: OK,
so fair enough. So if you’re hosting
that on your own server or you’re putting it up
in your own AWS bucket, you’re paying for
that bandwidth, whereas if it’s up on
YouTube, it’s free. So there is that
trade-off as well. RICK VISCOMI: So what
are some of the features of the modern video
element on the web? DOUG SILLARS: So the
video element is really incredibly full-featured. So it’s amazing what you can do. So you can have a
background video, and you can turn the controls
off so that people can’t play, pause. Or you can add the controls
so the users can zoom forward and back, you know, flip
through the video or stop it. You can autoplay videos,
which is just fabulous. Right, so that’s how
you get the background videos playing on web pages. You can tell the player how
much to download ahead of time. You can add a poster,
so rather than just having the first
frame with video, you can pick something
that denotes, really, what that video is. There are a lot of features. And then, of course, you can
script a lot of things as well. There are a lot of
JavaScript attributes that you can call as well. RICK VISCOMI: And that’s how
you end up with a one megabyte JavaScript player for YouTube? DOUG SILLARS: Well, yeah. RICK VISCOMI: So I want to ask
you also about accessibility. DOUG SILLARS: Yes. RICK VISCOMI: How
can developers ensure that video is able
to be seen and heard and read by all types of users? DOUG SILLARS: Right. So I mean, if you think
about when you watch videos on a lot of social
media platforms, captions are added
automatically. So even if the video is silent,
and so when videos autoplay, to make them autoplay
on mobile, you actually have to mute the video. It won’t play unless
the video is muted. So adding captions
is a great way to let people, even people
who have the ability to hear, to know that there’s
a video playing. And, you know, it
sort of engages you. Like, oh, there’s something
happening in that video. I want to learn more. And maybe they stick and
watch it a little bit more. Maybe then they turn on the
volume if they do have hearing. But when you have
a video tag, you have a source, which points
to the MP4 or the WebM or whatever the format
of the video is, but then you can also
have a track which points to the closed captioning track. And that will play
automatically, and that’s built into the
standard HTML5 video player. So it comes on in every
browser, by default. RICK VISCOMI: What
are some of the things that the video player
can’t do out of the box? DOUG SILLARS: So I think the
biggest thing, and the biggest reason why people will use
a third-party JavaScript library to play video,
is to do streaming video. Streaming video is generally
not supported in the browser. The top streaming
format used today is HLS, which comes from Apple. So HLS works in
Safari, but it doesn’t work in the other browsers. So you would have to add a
JavaScript player to make those videos play. RICK VISCOMI: So just
like images, there are many ways to
encode and serve video? DOUG SILLARS: Right. Yes. RICK VISCOMI: What’s
the state of formats? DOUG SILLARS: So when I look
at what’s out there on the web today, most videos are being
served with the MP4 format, which is H.264 codec. So you have a
format, which is sort of the extension at the end,
and then you can encode it with different codecs. And so the MP4 has to
codecs that are popular, but everything’s
H.264, and that’s supported in every
single browser except for Opera Mini, which
doesn’t play video at all. So like, it’s sort of
the universal format, so if you’re going
to deliver video and you don’t want to encode
lots of different formats like MP4, H.264
is the way to go. RICK VISCOMI: You’re also
writing the media chapter for The Web Almanac. DOUG SILLARS: Yes. RICK VISCOMI: Have you found
anything interesting in there about video? DOUG SILLARS: So there’s a
lot of interesting things. One is– we actually looked,
we talked a little bit about accessibility
and captions. And we, in our
queries, we actually looked for the
number of websites that have the track element,
which points to a VTT, which is the captions
file, and we found a lot of players that
actually have that link. And then when I– so I was really excited
because, like, that’s great for accessibility. And then I started
actually looking at some of the
websites, and the track was commented out in the HTML. RICK VISCOMI: Oh. DOUG SILLARS: Or the
VTT went to a 404 error. So, like, either they’re
copying template code and it’s just not
working or they’re like, yeah, we’ll get to that, and
it’s sort of on the to-do. And we all know what happens
with to-dos on the web. Like it’s just not there. So unfortunately,
from what I’ve found, it’s like the
accessibility, I think that’s a huge loss
because to make the web available to
everyone is super important, and video being such an
engaging part of web pages, I think it’s really
important to make sure that you have
captions for people who can’t hear the video. RICK VISCOMI: Did
you identify who the most popular codecs are? DOUG SILLARS: In terms
of video formats, the most popular formats
are the MP4, HLS streaming, and then probably the third is
WebM, which is a newer format. And so you can actually
deliver two formats. And so like the picture
tag, where you serve a WebP and then fall back
to a JPEG, you can serve the WebM, which has
a newer format, a little bit better compression,
so smaller files. If it doesn’t– if the
browser doesn’t support that, it can fall back to the MP4. So we do see a lot of websites
that do that, you know, WebM first, fall back to
the MP4 for browsers that don’t support that. RICK VISCOMI: You
had mentioned H.264. DOUG SILLARS: Yes. RICK VISCOMI: I’m just curious,
what’s holding back H.265? DOUG SILLARS: Patents. So there’s a lot of IP
and royalty requirements for every single device, and
so it’s not supported just because of the cost. And so when H.265
came out, I mean, H.265 is 25% to 50%
smaller than H.264. But because it’s burdened
by patents and royalties, it just hasn’t gotten the
buy-in that WebM has gotten. And so most people are using
WebM as the newer format. RICK VISCOMI: What types of APIs
are available for developers who are, maybe, self-hosting
video or even embedding, to understand what the
user experience is like? When does a video start
playing, and all these things? DOUG SILLARS: Right. So you create a video. Right? I’ve seen so many web pages
where it’s like someone’s said, put the video on the web page. It’s like, all right,
video source equals, paste. Right? Push it live, and we’re good. And the videos end up
being really large. So if you have an image,
there’s a lot of work out there on image performance. But, you know, if
you mess up an image, you might be a couple megabytes. You know, you probably
could get it smaller. If you put a
full-size video, you could very easily be talking
a hundred megabytes on a web page. And start thinking about what
that’ll do to people’s data plans on mobile, right? I mean, it ends up being
very, very expensive. And so if you start
thinking about sort of the traditional tools for
measuring web performance like WebPageTest, you can
see the video downloading, but you don’t know how fast
it takes to start playing. And so I actually built a
tool called StreamOrNot. And basically, you paste
in the URL of your video, and using DevTools,
you can change the speed of your
network connection, and it will tell you how fast
the video started playing. Now, this is a
really synthetic test because there’s
nothing else going on. There’s no JavaScript. There’s no CSS. There’s no other
images downloading. It’s just that video. But if you’re on a– say, you
set your DevTools to fast 3G, and you start playing the
video, and it takes 14 seconds to the video to
start, nobody sits at the top of the
page of a web page and, like, there’s
an image there. I wonder, maybe
it’ll be a video. Like no one’s going
to hang out that long. So, you know, if you’ve– if
it’s going to take that long, maybe you need to work on some
way to make that video smaller so that it can download faster
and show up on the screen faster. RICK VISCOMI: You’ve told the
story about a sushi restaurant website, had this video. What was the story behind that? DOUG SILLARS: All right, so
looking in the HTTP archives, as we’re doing “The
State of the Web,” I was just looking at web pages
that have large video files. And this web page– so you think about a
restaurant and you’re traveling in a new
city, and you’re like, I want to know if it’s
open, where it is, and hopefully there’s a menu. I’ll deal with PDF if I
have to deal with PDF. That’s fine. This web page has a
video of the interior of the restaurant playing
in the background, and it’s 113 megabytes. RICK VISCOMI: Oh, no. [LAUGHS] DOUG SILLARS: And so it
just constantly plays. And then, due to a caching
issue, when it gets to the end, if you stay on the
page long enough, because it’s like two
minutes long– right, two minutes long is
too long for that– it starts downloading again. [RICK LAUGHING] So you keep the page open, I
had it open for 10 minutes, and I hit really close to
a gigabyte in DevTools. Right, that could
be really ugly. RICK VISCOMI: So first,
you have to decide, do I even need video
on this website– DOUG SILLARS: Right. Right. RICK VISCOMI: –before
you can even get to the, how do I optimize my video? DOUG SILLARS: Right,
so like, do you really need the video there? And then, you know, there’s a
lot of conflicting information about how long a
background video should be, but pretty much the consensus
is like somewhere between 15 seconds to maybe 40
seconds on the long end. So like, two minutes
is way too long. But, yeah, like 100
megabytes on a web page? That’s getting really expensive. You know, I was looking
at roaming costs because I was going to
a different country. And imagine if I was
paying $5 a megabyte– RICK VISCOMI: Oh, my God. DOUG SILLARS: –and, all of
the sudden, like what my cell bill looks like at
the end of the month because I went to
that web page once. Right, sticker shock. RICK VISCOMI: So did you
eat at that restaurant? DOUG SILLARS: I didn’t. So it’s actually, it’s a
sushi restaurant in Brazil. So I actually haven’t seen it. I just found it
on the, you know– RICK VISCOMI: OK, that’s good. DOUG SILLARS: –on
the HTTP archive. RICK VISCOMI: Oh, that’s funny. So, finally, would you
recommend any resources for people who want to
learn more about video? DOUG SILLARS: Sure. So I’ve got a lot of
blog posts on my web page that talk about best
practices on how to re-encode video, maybe
look at different formats, resizing video. And then, of course, once you’ve
resized it and made it smaller, you want to make sure
the quality is good. So I’ve built some
tools that will actually measure the quality, if the
quality is visually pleasing to the end user. Because it’s great if
you make it smaller, but you don’t want it to look
like a motion JPEG from the 2G days and, you know, Nokia
candy-bar phones, right? We want to make sure that
the video still looks good. So we can put all
those resources at the bottom of the
page of the video so that people can look to
see, A, how fast their video is and, B, how to fix their
video so it is faster, and then to look at
the quality afterwards to make sure that it still
looks great and starts quickly. RICK VISCOMI: All right,
Doug, this has been great. Thank you so much for
coming on the show. DOUG SILLARS: Thanks
for having me. RICK VISCOMI: So you can
check out links to everything we talked about in
the description below. Thanks for watching, and
we’ll see you next time. [MUSIC PLAYING]

Leave a Reply

Your email address will not be published. Required fields are marked *