HTML5: Use Canvas to process Video in real time-H5 Tutorial-php.cn

HTML5: Use Canvas to process Video in real time

高洛峰

Release： 2018-05-28 17:58:12

Original

6056 people have browsed it

Document content

The XHTML document used in this article is as follows.

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<style>
body {
background: black;
color:#CCCCCC;
}
#c2 {
background-image: url(foo.png);
background-repeat: no-repeat;
}
p {
float: left;
border :1px solid #444444;
padding:10px;
margin: 10px;
background:#3B3B3B;
}
</style>
<script type="text/javascript;version=1.8" src="main.js"></script>
</head>

<body onload="processor.doLoad()">
<p>
<video id="video" src="video.ogv" controls="true"/>
</p>
<p>
<canvas id="c1" width="160" height="96"/>
<canvas id="c2" width="160" height="96"/>
</p>
</body>
</html>

Copy after login

The key parts of the above code are as follows:

1. Create two canvas elements with IDs c1 and c2 respectively. c1 is used to display the original video of the current frame, c2 is used to display the video after performing chroma-keying special effects; c2 is preloaded with a static image, which will be used to replace the background color part of the video.

2. The JavaScript code is imported from the main.js file; this script uses the features of JavaScript 1.8, so when importing the script, the version is specified in line 22.

3. When the web page is loaded, the processor.doLoad() method in main.js will run.

JavaScript code

The JS code in main.js contains three methods.

Initialize chroma-key

The doLoad() method is called when the XHTML document is initially loaded. The purpose of this method is to prepare the required variables for the chroma-key processing code and set up an event listener so that we can detect when the user starts playing the video.

doLoad: function() {
this.video = document.getElementById("video");
this.c1 = document.getElementById("c1");
this.ctx1 = this.c1.getContext("2d");
this.c2 = document.getElementById("c2");
this.ctx2 = this.c2.getContext("2d");
let self = this;
this.video.addEventListener("play", function() {
self.width = self.video.videoWidth / 2;
self.height = self.video.videoHeight / 2;
self.timerCallback();
}, false);
},

Copy after login

This code obtains references to the video element and two canvas elements in the XHTML document, and also obtains references to the graphics context of the two canvases. These will be used when we implement the chroma-keying effect.

addEventListener() listens to the video element and is called when the user presses the play button on the video. In order to handle user playback, this code takes the width and height of the video, halves it (we will halve the size of the video when performing the chroma-keying effect), and then calls the timerCallback() method to initiate video capture and visual effects calculations .

Timer callback

The timer callback function is called when the video starts playing (when the "play" event occurs), and is then responsible for its own periodic call to implement keying special effects for each frame of video. .

timerCallback: function() {
if (this.video.paused || this.video.ended) {
return;
}
this.computeFrame();
let self = this;
setTimeout(function () {
self.timerCallback();
}, 0);
},

Copy after login

The callback function first checks whether the video is playing; if not, the callback function does nothing and returns immediately.

Then call the computeFrame() method, which performs chroma-keying special effects on the current video frame.

The last thing the callback function does is call setTimeout() to allow it to be re-called as soon as possible. In a real environment, you might set the call frequency based on the frame rate of the video.

Processing video frame data

The computeFrame() method, as shown below, is actually responsible for grabbing the data of each frame and executing chroma-keying special effects.

computeFrame: function() {
this.ctx1.drawImage(this.video, 0, 0, this.width, this.height);
let frame = this.ctx1.getImageData(0, 0, this.width, this.height);
let l = frame.data.length / 4;

for (let i = 0; i < l; i++) {
let r = frame.data[i * 4 + 0];
let g = frame.data[i * 4 + 1];
let b = frame.data[i * 4 + 2];
if (g > 100 && r > 100 && b < 43)
frame.data[i * 4 + 3] = 0;
}
this.ctx2.putImageData(frame, 0, 0);
return;
}

Copy after login

When it is called, the video element will display the most recent video frame data as follows:

HTML5: Use Canvas to process Video in real time

In line 2, the video frame is Copied into the graphics context of the first canvas ctx1, specify the height and width values as half the frame size we saved earlier. Note that you can draw the current video frame by passing the video element to the drawImage() method of the drawing context. The result is:

HTML5: Use Canvas to process Video in real time

The third line of code obtains a copy of the original image data of the current video frame by calling the getImageData() method of the first canvas context. It provides raw 32-bit pixel image data so we can manipulate it. Line 4 of code calculates the total number of pixels in the image by dividing the total length of the frame image data by 4.

Line 6 of code scans all pixels in a loop, obtains the red, green, and blue values of each pixel, and compares it with the predefined background color. These background colors will use the background image imported in foo.png replace.

For each pixel that is detected as background, its alpha value is replaced with zero, indicating that the pixel is completely transparent. As a result, the background portion of the final image is 100% transparent, so that when it is drawn into the context of the target on line 13, the effect is that the content is superimposed on the static background.

The resulting image looks like this:

HTML5: Use Canvas to process Video in real time