How to Deliver Great Looking HLS Video
to Web and Mobile

Video publishers recognize the importance of a multi-device strategy, but previously no single protocol was compatible with both mobile and Web players. With player-side support for segmented streaming, one format has become hugely important for delivering video across mobile and Web browsers.

HLS Devices

HTTP Live Streaming, or HLS, is a video delivery protocol originally implemented by Apple as part of their iOS operating system. Brightcove Zencoder’s API-driven video encoding solution makes preparing content for HLS easy, but there are still a few things worth knowing that will help video publishers optimize content for their individual situation. This guide provides an overview of important information about HLS video playback in browsers and mobile devices, as well as specific encoding recommendations.

What is HLS?

HLS utilizes segmented H.264 MPEG-2 TS video and M3U8 descriptor files to deliver adaptive bitrate live and on-demand video. An M3U8 file is an index that lets the client know which streams, and which segments, are available at any given time. The device automatically selects the most appropriate stream from the primary manifest file given bandwidth and CPU constraints, downloads the segment, and appends it to the playback buffer.

As the name implies, HLS transmits data over HTTP, which allows for various improvements over traditional streaming protocols like RTP or RTMP, including:

  • Reduced infrastructure costs
  • “Cacheability” at CDNs and other HTTP caching infrastructure
  • Less threat from proxy and firewall restrictions
  • Real-time optimization through client heuristics (adaptive bitrate)
  • Built-in redundancy
  • Simple HTML5 player implementation
Apple Store

Apple App Store has strict requirements for video delivered through the iOS applications — streaming is require for all video content exceeding 10 minutes or 5MB of data transfer within 5 minutes (133 kbps). Apple also requires that developers provide a 64 kbps (or lower) audio stream as a fallback for cellular networks. Applications that don't meet these requirements will be rejected from the App Store. For more information on deploying video to the App Store, see Apple's HHTP Live Streaming Overview documentation online in the iOS Developer Library: http://bit.ly/zgHSJX

Mobile Devices

Generally speaking, newer devices have more processing power and are able to support higher profiles of H.264. With the release of the iPhone 4S (and higher), iOS devices support 3 different profiles of H.264 — Baseline, Main, and High. It’s harder to generalize about Android capabilities because of the sheer number of devices. However, the modern, top-of-the-line Android devices at the very least support what the iPhone 4 does. These profiles and corresponding H.264 levels are used to classify specific client playback capabilities. While these devices do not have a standard High Definition sized screen, some are able to output standard HD and SD resolutions to a television using Apple Digital AV Adapter or Apple VGA Adapter.

IPHONE 1 TO 3GS, IPOD TOUCH 1-3 IPHONE 4, IPAD 1, APPLE TV 1&2, IPOD TOUCH 4&5 & NEWER ANDROIDS* IPHONE 4S+, IPAD 2-4, APPLE TV 3, NEWER ANDROIDS*
Profile Baseline Profile Main Profile High Profile
Level Level 3.0 Level 3.1** Level 4.1***
Audio AAC audio, 1-2 channels AAC audio, 1-2 channels AAC-LC audio, 1-2 channels
Max Framerate 30 fps 30 fps 30 fps*
Video Bitrate Up to 1.5 Mbps Up to 5 Mbps Up to 5 Mbps
Audio Bitrate Up to 160 kbps Up to 160 kbps Up to 160 kbps
Audio Sample Rate 48000 or less 48000 or less 48000 or less
Display Resolution 480x320 iPhone 4/4s: 960x640
iPad 1: 1024x768
iTouch: 1136x640
iPhone 5+: 1136x640
iPad 2: 1024x768
iPad 3/4: 2048x1536

*iPhone 5S and most new top-of-the-line Androids now support High Profile level 4.2, 60fps
** iPod Touch 5 supports Level 4.1 and lower
*** Apple TV 3 supports Level 4.0 and lower

Desktop and OTT

The only browser with native HLS support at this time is Safari (version 4.0+), but a Flash fallback can be used to enable playback in others. Video.js provides HLS support in its Flash component, allowing HLS to be used across almost all browsers. Many applications and OTT devices, such as Quicktime, VLC, Apple TV, Roku, Google TV and XBMC, also support HLS. Depending on the connection speed, these devices support up to High Profile.

Advanced Features

HLS video is constantly evolving, and new features are added over time. Today, HLS supports a range of advanced features, including:

  • AES-128 encryption for content security
  • CEA-608 closed captioning in the transport stream
  • Live streaming and on-demand video
  • Audio tracks can include a still image, like cover art or video screenshots
  • Stream metadata, including timed metadata using the ID3 format

6 Tips for Encoding HLS

Here are 6 tips for getting the most out of HLS encoding. These recommendations are equally applicable to live and on-demand transcoding.

  1. Use HE-AAC audio. Standard AAC (often called AAC-LC) sounds great at bitrates of 96 kbps and above, but at lower bitrates, the compression is noticeable. This is important for HTTP Live Streaming, since the App Store requires a 64 kbps stream for most apps. The HE-AAC (“High Efficiency AAC”) profile is optimized for low bitrate use, and sounds noticeably better than AAC-LC in 64 kbps range which is so important for HLS.
  2. Apple recommends that the audio parameters remain the same throughout to prevent artifacts. If the audio differs from one stream to another, the result can be skips and audible “pops” when switching between audio bitrates. If you need to provide different audio rates to improve the quality of certain streams, at the very least keep a standard sample rate.
  3. Keyframe rate should be an even interval of the segment size. For example, if the segment size is 10 seconds, the keyframe rate should be 2 seconds, 2.5 seconds, 3.33 seconds, 5 seconds, or 10 seconds. Some encoders, including Zencoder, will automatically place keyframes at appropriate places. Use the force_keyframe_rate setting with Zencoder to ensure proper keyframe alignment between streams.
  1. Pay attention to format overhead. The MPEG-TS format uses a lot of unnecessary padding, and with an unoptimized TS muxer, HLS can be significantly larger than their MP4 counterparts - as much as 10%-20% larger overall. Both Zencoder and Apple use highly optimized TS muxers that minimize TS overhead.
    Requested Bitrate Overhead
    Zencoder optimized HLS 364 kbps 4.83%
    Un-optimized HLS 364 kbps 16.34%
  2. Consider constraining bitrate peaks to ensure good streaming performance. We recommend setting a peak bitrate (bitrate_cap) that is 150% of the average bitrate, within a buffer of 3-5 seconds.
  3. The streams value should contain all applicable information. This is especially important when you provide multiple H.264 profiles to include the appropriate codecs value. These settings prevent legacy devices from accidentally downloading segments that they cannot decode. For example, use the values “avc1.42E01F, mp4a.40.2” for Baseline 3.0 profile and “avc1.42E01F, mp4a.40.2” for Main 3.1

Outputs & Playback

HLS video consists of index files (.M3U8 XML manifests) which either reference other M3U8 files or individual video segments (.ts segments, typically 2-10 seconds in length). When using Zencoder, the base_url location specified will contain a .M3U8 manifest after the encoding job has completed.

Zencoder output playbacks

Apple’s HLS file structure representation

You have the option of organizing each set of segments into folders by using an individual base_url. Just make sure that the path value for the playlist portion of the API request reflects the storage location for each individual stream, otherwise your manifest file will direct to an empty location. When the client retrieves the playlist.m3u8 (primary index file), it will decide which alternate stream to play based on platform and network constraints.

EXAMPLE: BASIC HTML5 TEST PLAYER USING VIDEO.JS

      
	<!DOCTYPE html>
	<html>
	<head>
		<title>HTTP Live Streaming Test Player </title>
		<link href="http://vjs.zencdn.net/c/video-js.css" rel="stylesheet" type="test/css">
		<script src="http://vjs.zencdn.net/c/video.js"></script>
	</head>
	<body>
		<video id="example_video_1" class="video-js vjs-default-skin" controls autoplay
		width="640" height="360" data-setup="{}">
			<source src="http://YourServerHere/playlist.m3u8" type="application/x-mpegURL" />
		</video>
	</body>
	</html>
			

Encoding Recommendations

We want your video to look awesome, but because video compression is a subjective art form, we are only able to provide you with a set of recommendations as a starting point. Your application may be geared toward cellular or mobile devices, in which case you may want to include additional streams on the lower end of the spectrum. Alternatively, your application may be aimed at devices connected to a stronger network and can then limit the total number of streams you may need to provide. If your source material contains high amounts of panning, motion, or cuts, an increase to the video_bitrate may be required to meet your desired results.

As with all things video related, let your eyes be the judge of overall quality. Use a test player and the clip_length parameter to output small test streams and tweak settings as you see fit. Don’t forget about the 6 tips listed above and your video is sure to look fantastic on any iOS device.

Resolution Profile Bitrate @16:9 @4:3 Audio Comments
1280x960 High@4.0 4 Mbps 1280x720 1280x960 64kbps HE-AAC
1024x768 Main@3.1 2 Mbps 1024x576 1024x768 56kbps HE-AAC
960x640 Main@3.1 1.5 Mbps 960x540 854x640 56kbps HE-AAC
640x432 Main@3.1 1 Mbps 640x360 576x432 56kbps HE-AAC
480x320 Baseline@3.0 600 kbps 480x272 426x320 56kbps HE-AAC
400x288 Baseline@3.0 400 kbps 400x224 384x288 56kbps HE-AAC
400x288 Baseline@3.0 200 kbps 400x224 384x288 56kbps HE-AAC Decimate framerate
(audio only) (audio only) (audio only) (audio only) (audio only) 56kbps HE-AAC Include still image

Notes

  1. These are just recommendations. Different resolutions and bitrates are perfectly valid, and may actually be preferable in some circumstances. For example, complex content may warrant higher bitrates, and simple content could use lower bitrates.
  2. These 6 resolutions and bitrates provide reasonably good coverage of varying bandwidth. You could certainly do more — Apple recommends 8 — so add or subtract resolutions and profiles as desired.
  3. The first column is the generalized resolution. Depending on the video aspect ratio (4:3 or 16:9), the final video will have a different resolution, which is listed in columns 4 and 5.
  4. Frame rate should be limited to 30 or below. Never force a frame rate change. To ensure smooth playback, just decimate frame rates that are too high. For the lowest stream, decimate frame rates to 15 or below.
  1. For iPhone 1 compatibility, force the use of only 1 H.264 Reference Frame for the Baseline streams.
  2. Use a keyframe interval of 5 or 10, with forced keyframe alignment.
  3. Limit audio sample rates to 44100 or below for widest compatibility.
  4. Consider limiting bitrate peaks for smooth playback. For a good compromise of quality and playback, consider a bitrate cap of 150% of average bitrate, and a buffer of 3-5 seconds.