The State of MPEG-DASH 2016 (part II)

HTML5 and MSE Changing the Game

On the other side of the streaming chessboard, HDS also suffered in 2015 from the deprecation of Flash player in browsers. Although it’s not yet official, several browsers, including Firefox, have temporarily banned Flash for security reasons, and after having started to pause nonessential Flash content in Chrome, Google is working on getting the advertising industry to switch to HTML5. If at some point Google disables the Flash engine embedded in Chrome, it won’t be a surprise. With Flash Player less and less prevalent, the HDS format is steadily losing ground, all the more so since there is no MSE-based HDS player on the market, and because Adobe has ceased all support effort behind OSMF and switched to HLS and DASH in Primetime. Facebook’s recent transition from a Flash-based player to an HTML5-based one is a confirmation of where the market is going: It’s HTML5 everywhere.

This trend clearly jeopardizes all ABR formats other than HLS and DASH. With its Media Source Extensions, HTML5 starts to provide the same kind of universal coverage that Flash provided a few years ago for FLV/ RTMP/HDS, and players written in JavaScript can take advantage of the MSE layer provided by the browser, as well as the Encrypted Media Extensions that allow DRM-protected content using Common Encryption (CENC) to be handled natively by the browser through Content Decryption Modules (CDM). With the W3C WebCrypto API, MSE and EME are shaping up what Netflix called in 2013 the “HTML5 Premium Video Extensions.” The preeminence of this new technology stack over the old ones—at least in browsers—is not debatable anymore; the only question is how quickly others will be deprecated. With MSE providing the high level streaming mechanisms, you can easily write a JavaScript player that will support DASH or HLS across platforms, and that’s a killer feature.

The HTML5 Video Stack Matrix 

DASH in Legacy Browsers

MSE has become the standard media stack in the browsers. It’s now supported on Chrome, Chrome Mobile, Microsoft IE11 on Windows 8.1 and Edge (with HEVC support), Firefox 42+, Safari Yosemite+, and Opera. In the device world, the Chromecast and models powered by Android, FireOS, Windows Phone, or Opera Devices SDK are also MSE-enabled. The two main gaps not covered by MSE are legacy browsers and the iOS world. The legacy browsers, still representing around 30 percent of the market, won’t be updated to support MSE, which means that if you want to support DASH on these platforms, you will need to use a Flash-based fallback component in your player architecture, alongside the MSE-compatible JavaScript code layer. Castlabs and Bitmovin are among the vendors proposing such a Flash-based fallback for their commercial players, but they are not open source. Castlabs once open sourced the dash.as OSMF extension, but in 2015 it focused its efforts on other developments, and no one took over dash.as. Some industry actors, led by Streamroot, are currently working on providing a much needed open source alternative in 2016, a Flash-based component mimicking the MSE API and allowing transparent fallback for HLS or DASH whenever MSE is not supported. It will be a great companion for open source projects.

Axel Delmas, Streamroot’s CTO, summarizes the project intent: “Having two codebases—one for Flash and one for HTML5—is a huge source of frustration and complexity for broadcasters. We developed the beginnings of a Flash MSE polyfill internally nearly a year ago, and have decided to open source the project, given the enthusiastic support of Akamai and other industry players. This is really going to fill a gap in the market and bring us one step closer to putting Flash to rest.”

Growing Player Options

The next step for DASH is to outmatch HLS as the major streaming format. Before 2015, that could hardly have happened, because of a limited player choice and a still-evolving core feature set. The open source DASH player world was previously summarized in MSE-based dash.js, but 2015 has seen the arrival of Google’s Shaka Player for MSE, which was welcomed because of its strong support for live streaming, and also the establishment of Google’s ExoPlayer as the reference DASH player in the Android world. Although all commercial implementations are not based on open source code, as DASH player solutions have blossomed, they’ve led to a significant expansion of DASH in the commercial player sphere.

Alongside the historical Bitmovin Bitdash and Castlabs DASH Everywhere implementations, we have seen all major players add DASH as a supported format, starting with the prominent JW Player, as well as Flowplayer, Kaltura, and THEOplayer. Back in the open source world, we’ve seen video.js integrate dash.js, and more surprisingly a new wave of contributions coming from the broadcasters, with rx-player from Canal+ and DASH PLAY from RTL. This shows the deep interest in DASH within the broadcasting community, beyond the first adopters (Netflix, Hulu, and others). Following Viaplay and other major DASH deployments, it’s an open secret that major broadcasters are working on DASH migration plans for 2016, now that the technology is stable and that the HEVC licensing terms are getting clearer. And given the wide availability of DASH players on almost all platforms, it is now easier for newly launched OTT services to make different technology choices up front, and focus on a DASH-plus-HLS combo rather than using three or four different streaming technologies. If we combine wide DASH player choice with the growing DASH support availability on the major CDNs (in passthrough or repackaging modes) and the extensive offer of DASH-compliant packagers and origin servers, we reach a tipping point where it will be as easy to produce, distribute, and play back DASH as it can be with HLS.

The iOS Conundrum

On iOS, DASH is still locked out. While we can now find several applications on regional App Stores using DASH as streaming format, all of them apply a local repackaging from DASH to HLS. The sole exception is Netflix, which plays ISO Base Media File Format (ISO-BMFF) segments everywhere but on iOS6, thanks to a special arrangement with Apple. On the desktop, Apple authorized Netflix to use FairPlay DRM with DASH streams encrypted using CENC, but other content providers can’t use this combination yet. Thanks to Netflix’s pressure, Apple also introduced MSE to Safari desktop, so we can at least play DASH streams in the clear on this platform now. But on iOS, Safari mobile doesn’t support MSE, and playing DASH natively in compiled apps is still cause for applications being rejected in the App Store, at least for 3G/4G cellular use cases. In theory, if you plan to restrict your app use to Wi-Fi, you can use almost any streaming technology and player component. In 2015 we have seen the first native DASH playback SDKs for iOS: the Viblast Player iOS SDK for live and the DataArt MPEG-DASH Player iOS Application for on-demand. But there is still no obvious sign that the advances made by DASH through MSE on the Safari desktop platform will spread to Safari mobile. HLS will probably stay in the landscape—or at least the M3U8 playlists portion of it—for a long time.

Despite the duality of formats with HLS and DASH, there are already some ways to alleviate the principal burden that affects content providers: having to produce two sets of media files to accommodate the HLS playlists and DASH manifests, or MPDs. Since version four, released in September 2011, the HLS IETF Draft authorizes playing on-demand content using byte range requests that seek in a monolithic MP4 file. It’s working on iOS without MSE being implemented in Safari mobile, and the same MP4 file can also be used in combination with DASH on-demand profile using a similar byte range approach. Today, the remaining part of the problem is live streaming, where HLS still mandates the use of TS segments, since the use of ISO-BMFF segments is impossible in this context. There is pressure from content providers and OTT services to achieve convergence for live streaming, as the current model costs more and lowers the cacheability ratio of the media segments. It will probably be possible to produce live HLS playlists using the same ISO-BMFF segments as the live DASH manifests in just a few months from now. Apple’s incremental evolutions on HLS, combined with MP4 byte ranges and MSE/DASH on Safari desktop, pave the way toward the unified media format the industry is waiting for.

Zooming inside video with SRD 

ISO-BMFF Rules Them All

While very few companies used DASH-TS—mostly in the U.S. cable industry—there was another alternative to ISO-BMFF media encapsulation with the WebM DASH specification proposed by the Google team on the WebM Project wiki. Google joined the DASH Industry Forum (DASH-IF) in 2014 but never proposed its WebM DASH specification for standardization alongside DASH-264 and DASH-265 interoperability points, so the use of this DASH flavor was limited to YouTube.

In the meantime, Microsoft introduced experimental VP9 support in its Edge browser (no WebM support yet) and joined forces with other major industry actors in the Alliance for Open Media to develop the next-generation video format—let’s call it “post-HEVC”—where Google’s VP9 can be one of the starting points with Mozilla’s Daala or Cisco’s Thor. All this movement around free codecs generated a new interest in VP9, as it’s the most advanced among the Alliance for Open Media choices. But VP9 was still stuck in the WebM envelope, until Netflix’s David Ronca and Microsoft’s Kilroy Hughes drafted a proposal for binding Video Partition structured video codecs (such as VP8/9/10) to ISO-BMFF, including the necessary adjustments to be used with CENC and DASH.

This preliminary work, once finished, will make it easy for VP9/VP10 codecs to be introduced alongside a new set of profiles into DASH-IF’s interop guidelines, as these guidelines rely solely on the ISO-BMFF container. The resulting DASH-VPx interop point might then be an interesting alternative to DASH-265 on devices where VP9 decoding is supported in hardware—for instance, the Nvidia Shield Android TV—or software-based browser clients. Here we see another case where the industry is moving toward convergence while preserving space for competition or alternatives in the codec area—and obviously the ISO-BMFF media container standard is one of the strongest forces fostering such a convergence. Let’s examine how DASH’s next evolutions can reinforce this industry move.

Toward the Third Edition of the MPEG-DASH Core Specification

MPEG-DASH’s first edition in 2012 was a major milestone for the streaming industry, as witnessed by the widespread support among encoder/packager solutions on the market today, the second edition was not as widely implemented when it came out in 2014, maybe because it didn’t introduce crucial features on top of the initial version. As a reminder, those new features included media timeline events (to support server-driven interactions and trigger manifest updates), empty periods (to support media blackout signaling), content asset identifiers (to categorize main and advertising contents), and improved ad-insertion support. We can expect suppliers to catch up and implement the upcoming third edition of MPEG-DASH, as it will represent a quantum leap in terms of features and derived user experience. The question is how long it will take to see market solutions implementing the specification, as some components might be difficult to support. In the core scope of features, we’ll find interesting advances, such as an enhanced client-server synchronization logic; improved authentication and authorization mechanisms; the much-awaited Spatial Relationship Description, which will allow the combination of several spatially related videos in a tiled or zoomed immersive combination; a standardized tracking framework for analytics; and a new set of accessibility features to facilitate description and client-side mixing of alternate media tracks for hearing-impaired and blind users.

Manifests aggregation with Playlist Program Description

DASH manifests will also become dynamic through client-side parsing and injection of the query string parameters used in the MPD URL, which might prove useful for adding inline access tokens or custom parameters on media segments. With the companion URI signing specification, DASH will represent a standardized but powerful way to generate and validate access tokens for a given program—something that was difficult in previous technologies, when each CDN required its own access token logic implementation. The Segment Independent SAP Signaling (SISSI) feature will allow fast channel switching and short time to playback through the use of very short segment lengths during channel transition phases, switching as possible to segments of a standard duration.

Additional feature sets, resulting from some of the core experiments initiated by MPEG, will also be able to make their way as extensions to the core specification. The Server and Network Assisted DASH (SAND) will eventually show up as ISO 23009-5, providing a set of QoS parameters to be cross-referenced and shared across the chain by origin servers, network equipment, CDNs, and DASH players in order to bring enhanced ABR decision-making capability to the client. SAND uses the WebSocket protocol to convey the messages through the distribution chain node, which might be challenging for public CDNs to support because WebSocket connections are persistent. The CAPCO core experiment led to a proposal regarding the playback of several chained MPDs inside a dynamic Playlist Program Description (PPD), where a general timeline can be composed from individual MPD timelines or portions of it, mixing live, on-demand, and advertising content, with the content aggregator located anywhere in the chain from the service backend up to the client layer, which sounds like a flexible and efficient way of tackling the problem of dynamic and personalized playlists.

Finally, the upcoming Full Duplex DASH (future ISO 23009-6) will bring standard mechanisms to transport DASH over HTTP/2 and WebSockets. The HTTP/2 flavor will certainly have the most impact on the delivery aspects of DASH, first because it will be natively supported by CDNs on the short term, and also because of its intrinsic capabilities, such as data multiplexing— which allows several segments to be sent in a row over a single connection—and its push model—which can fetch content to the client without having him request it—in contrast to the current model, where only the client makes decisions on bitrates requested. Applied to DASH, this would suppose that the CDN edge understands the segment request flow generated by a DASH playback session and calculates the next segment name to push it as soon as it is available. This is relatively easy to accomplish with SegmentList or Number-based SegmentTemplate fragmentation types, but becomes more complex when Time-based SegmentTemplate fragmentation is used instead. Nevertheless, HTTP/2 without push mode will still be interesting out of the box for header compression and data multiplexing, as well as the ability to combine it into UDP-based transport protocols. The Full Duplex DASH is not an incremental change from the performance perspective; it’s a game changer that will significantly boost the QoS of playback sessions from 720p over ADSL up to 4K over fiber.

DASH-IF Interoperability Points Moving Toward UHD

The DASH Industry Forum acts as a relay to MPEG, providing complementary Interoperability Points (IOP) and best practices for deploying DASH, and specifying DASH profiles that combine precise container/codec/ security options in order to narrow down the scope of what’s possible in the DASH core specification (which is a lot). In June 2013, version two of the Interoperability Guidelines brought to DASH-264 support of HD video up to 1080p and Multichannel Audio. That was a transition to version three, which was a major jump forward in April 2015, with a specific chapter for live services; a developed ad insertion model; and support for H.265/HEVC, CEA6608/708, and stream events. In June 2015, DASH-IF released a companion implementation guidelines document for the Content Protection Information Exchange Format (CPIX), which aims to unify the exchanges between entities in charge of content protection (content provider, encoder, packager, player, DRM license delivery service, etc.). The proposed data model allows secure and standardized key exchanges between those entities, which was not possible before as all options relied on proprietary interfaces. Further 2015 revisions of the Interoperability Guidelines v3 introduced support for ISMSC1 Timed Text (of which EBU TT is a subset) and Dolby AC-4, as well as Adaptation Set switching (which can be used to mix H.264 and H.265 representations in the same MPD), live Key rotation, and MPEG-H 3D Audio.

DASH-IF’s roadmap 

The next big topic for the DASH-IF is the design of new UHD Interoperability Points. This is a challenge, as the overall video ecosystem is still discussing exactly how UHD technologies will combine. Hopefully things will settle down a bit more now that the UHD Alliance announced the Ultra HD Premium certification requirements at CES 2016, summarized as 4K HEVC 10bits/WCG/BT.2020/HDR SMPTE ST2084 EOTF. DASH-IF is working closely with other standardization bodies concerned with DASH and/or UHD (such as DVB, ATSC, SCTE, DECE, or MPEG) in order to ensure that its new UHD IOPs are aligned with other DASH UHD profiles throughout the broadcast industry. Given the complexity and the scope, a few additional iterations will most likely be required before the version four Guidelines, including the new UHD IOPs, are released for community review. Could the expectation for UHD slow the adoption of DASH-IF Guidelines version three? Possibly, but these guidelines condense what is required today for 1080p AVC/HEVC services and the UHD IOPs will not revolutionize with version three, but rather incrementally modify it with specific UHD requirements and recommendations.

There’s no doubt that DASH will be a cornerstone of all major media initiatives, such as the Global Internet Video Ecosystem (GIVE) Project, which wants to push HTML5 video forward as quickly as possible. But starting today, you can deploy DASH in production and take advantage of its technical strengths and vivid ecosystem.

<   BACK: Part I