WebSocket+MSE——HTML5 live broadcast technology analysis
巴扎黑
Release： 2017-06-23 14:45:59
Original
4290 people have browsed it
Author | Liu Bo (also Paiyun Multimedia Development Engineer)
Currently, in order to meet the relatively hot demand for live broadcast on the mobile Web, a series of HTML5 live broadcast technologies have been rapidly developed.
Common live streaming technologies that can be used for HTML5 include HLS, WebSocket and WebRTC. Today I will introduce to you the technical points related to WebSocket and MSE, and finally demonstrate the specific usage through an example.
Article outline
WebSocket protocol introduction
WebSocket Client/Server API introduction
MSE Introduction
fMP4 Introduction
Demo Display
WebSocket
Usual web applications are built around the HTTP request/response model. All HTTP communications are controlled through the client. The client sends a request to the server. After the server receives and processes it, it returns the result to the client, and the client displays the data. Since this mode cannot meet the needs of real-time applications, "server pushed" long-connection technologies such as SSE and Comet have emerged.
WebSocket is a communication protocol based on TCP connection, which can perform full-duplex communication on a single TCP connection. WebSocket was set as a standard RFC 6455 by the IETF in 2011 and supplemented by RFC 7936. The WebSocket API was set as a standard by the W3C.
WebSocket is a protocol created independently on TCP. The concepts in the HTTP protocol are not related to WebSocket. The only correlation is that when using the 101 status code of the HTTP protocol for protocol switching, the TCP port used is 80, which can bypass most firewall restrictions.
WebSocket Handshake
In order to more conveniently deploy the new protocol, HTTP/1.1 introduces the Upgrade mechanism, so that the client and server can use the existing Some HTTP syntax is upgraded to other protocols. This mechanism is described in detail in RFC7230, section 6.7 Upgrade.
To initiate an HTTP/1.1 protocol upgrade, the client must specify these two fields in the request header ▽
If the server agrees to the upgrade, Then you need to respond like this ▽
> HTTP/1.1 101 Switching Protocols
Connection: upgrade
Upgrade: protocol-name[/protocol-version]
[... data defined by new protocol ...]
Copy after login
As you can see, the status code of the HTTP Upgrade response is 101, and the response body can use the data format defined by the new protocol.
The WebSocket handshake takes advantage of this HTTP Upgrade mechanism. Once the handshake is complete, subsequent data transfer is done directly over TCP.
WebSocket JavaScript API
Currently, mainstream browsers provide a WebSocket API interface, which can send messages (text or binary) to the server and receive event-driven response data.
Step1. Check whether the browser supports WebSocket
> if(window.WebSocket) {
// WebSocket代码
}
Copy after login
Step2. Establish a connection
> var ws = new WebSocket('ws://localhost:8327');
Copy after login
Step3. Register callback functions and send and receive data
Register the onopen, onclose, onerror and onmessage callback functions of the WebSocket object respectively.
Send data through ws.send(). Not only strings can be sent here, but also Blob or ArrayBuffer type data can be sent.
If you receive binary data, you need to set the format of the connection object to blob or arraybuffer.
ws.binaryType = 'arraybuffer';
Copy after login
WebSocket Golang API
For the server-side WebSocket library, I recommend using Google’s own http://golang.org/x/net/websocket, which is very convenient Used with net/http. You can also convert WebSocket's handler function into http.Handler through websocket.Handler, so that it can be used with the net/http library.
Then receive data through websocket.Message.Receive and send data through websocket.Message.Send.
The specific code can be found in the Demo section below.
MSE
Before introducing MSE, let's first look at the limitations of HTML5
Limitations for HTML5
Streaming is not supported
DRM and encryption are not supported
Difficult to customize controls, and maintain cross-browser consistency
Coding and encapsulation support differs in different browsers
MSE solves the flow problem of HTML5.
Media Source Extensions (MSE) is a new Web API supported by mainstream browsers such as Chrome, Safari, and Edge. MSE is a W3C standard that allows JavaScript to dynamically construct media streams for
By using MSE, you can dynamically modify media streams without the need for any plug-ins. This allows front-end JavaScript to do more - repackaging, processing, and even transcoding in JavaScript.
Although MSE cannot transmit streams directly to media tags, MSE provides the core technology for building cross-browser players, allowing the browser to push audio and video to media tags through the JavaScript API.
Browser Support
Use caniuse to check whether the browser supports it.
You can further check whether the codec MIME type is supported through MediaSource.isTypeSupported().
fMP4
The more commonly used video encapsulation formats are WebM and fMP4.
WebM and WebP are two sister projects, both sponsored by Google. Since WebM is a container format based on Matroska, it is inherently streaming and is very suitable for use in the field of streaming media.
The following focuses on the fMP4 format.
We all know that MP4 is composed of a series of Boxes. Ordinary MP4 has a nested structure. The client must load an MP4 file from the beginning before it can be played completely, and it cannot start playing from the middle section.
And fMP4 consists of a series of fragments. If the server supports byte-range requests, then these fragments can be independently requested to the client for playback without loading the entire file.
In order to illustrate this point more vividly, below I introduce several commonly used tools for analyzing MP4 files.
gpac, formerly known as mp4box, is a media development framework. There are a large number of media analysis tools under its source code, and you can use testapps;
mp4box.js, is mp4box Javascript version;
bento4, an analysis tool specifically for MP4;
##mp4parser, an online MP4 file analysis tool.
fragment mp4 VS non-fragment mp4The following is a screenshot of a fragment mp4 file analyzed by mp4parser (Online MPEG4 Parser)▽
The following is a screenshot of a non-fragment mp4 file analyzed by mp4parser▽
We can see the top box of the non-fragment mp4 There are very few types, and fragment mp4 is composed of moof+mdat segments. They already contain enough metadata information and data, and you can directly seek to this position and start playing. In other words, fMP4 is a streaming encapsulation format, which is more suitable for streaming on the network without relying on metadata in the file header. Apple announced at the WWDC 2016 conference that it will support fMP4 in HLS of iOS 10, tvOS, and macOS. It can be seen that the prospect of fMP4 is very good. It is worth mentioning that fMP4, CMAF, and ISOBMFF are actually similar things. MSE JavaScript APIAt a high level, MSE provides a
The above is the detailed content of WebSocket+MSE——HTML5 live broadcast technology analysis. For more information, please follow other related articles on the PHP Chinese website!
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
![[Web front-end] Node.js quick start](https://img.php.cn/upload/course/000/000/067/662b5d34ba7c0227.png)
