Accessible Audio Descriptions for HTML5 Video

Summary of key points

• The traditional visually impaired audio description requires professional video editing equipment to be encoded and embedded in a separate audio track of a video file. This process is often unrealistic for most content creators.
• HTML5 Video Specification provides the audioTracks object, which theoretically allows the implementation of switch buttons for audio descriptions and controls the audio and video volumes separately. However, browsers currently have limited support for this feature.
• Another solution is to use MediaController, a feature of HTML5 audio and video that allows syncing multiple sources. This feature is currently limited in browser support, but can use existing, widely implemented features to start and keep two media files synchronized simultaneously.
• The Video API provides events such as Play, Pause, End, and Timeupdate to synchronize audio playback with video events. The “timeupdate” event is particularly important for this purpose, triggering on average 3-5 times per second. This approach allows the creation of accessible audio descriptions without the need for dedicated software or separate versions of videos.

A recent client asked me to make an accessible video player and she really wanted one of the features to be audio description. Audio descriptions are aimed at blind people or people with visual impairment, providing additional voice information to describe important visual details. Traditionally, videos with audio descriptions have to be specially made, and the audio is encoded in a separate track of a single video file. This requires quite professional video editing equipment to encode these audio tracks, which makes it difficult for most content creators to achieve. This is what I see online with audio descriptions. For example, BBC iPlayer offers some such content, but the video player cannot control the relative volume and cannot turn off the audio description—you can only watch individual or non-descript versions of the show. HTML5 debut

HTML5 Video Specification does provide audioTracks objects, which makes it possible to implement switch buttons and control audio and video volumes separately. But its browser support is hardly present – ​​at the time of writing, only IE10 supports this feature. Anyway, what my clients want is an audio description in a separate file that can be added to the video without creating a separate version and being easily made without using dedicated software. Of course, it has to run in quite a few browsers. So my next idea is to use MediaController, a feature of HTML5 audio and video that allows you to sync multiple sources. However, browser support for it is equally slim – at the time of writing, only Chrome supports this feature. But you know - even without this support, it is obviously not a problem to start two media files at the same time, just need to be kept in sync. So, can we achieve this using existing, widely implemented features? Video Events

The Video API provides many events that we can hook into, which should synchronize audio playback with video events:

• "Play" event (triggered when video plays).
• "Pause" event (triggered when video is paused).
• "End" event (triggered at the end of the video).
• "timeupdate" event (continuously triggered when the video is played).

The "timeupdate" event is very critical. The frequency it triggers is not specified and it actually varies greatly - but as a rough overall average it is equivalent to 3-5 times per second, which is enough for our purposes. I've seen similar ways to try syncing two video files, but this is not particularly successful because even small differences are obvious. But audio descriptions usually don't need to be synchronized so accurately - either way, a 100 millisecond delay is acceptable - and playing audio files is much less burdening on the browser. So we just need to use existing video events to lock the audio and video playback together:

• Play audio while video is playing.
• Pause the audio when the video is paused.
• At the end of the video, pause the video and audio simultaneously.
• When the time is updated, if the audio time is different from the video time, the audio time is set to match the video time.

After some experiments, I found that the best results can be obtained by comparing the time in seconds, as shown below:

if(Math.ceil(audio.currentTime) != Math.ceil(video.currentTime)) {
  audio.currentTime = video.currentTime;
}

This seems counterintuitive, and at first I thought we needed the exact data we could, but it doesn't seem to be the case. By testing with a text audio copy of the video track (i.e., both the audio and video produce the same sound), it is easy to hear the synchronization is good or bad. Based on this basis, I found that rounding numbers gets better synchronization results than not rounding. So, this is the final script. If the browser supports MediaController, we just need to use it, otherwise we will implement manual synchronization as described below:

var video = document.getElementById('video');
var audio = document.getElementById('audio');

if(typeof(window.MediaController) === 'function') {
  var controller = new MediaController();
  video.controller = controller;
  audio.controller = controller;
} else {
  controller = null;
}

video.volume = 0.8;
audio.volume = 1;

video.addEventListener('play', function() {
  if(!controller && audio.paused) {
    audio.play();
  }
}, false);

video.addEventListener('pause', function() {
  if(!controller && !audio.paused) {
    audio.pause();
  }
}, false);

video.addEventListener('ended', function() {
  if(controller) {
    controller.pause();
  } else {
    video.pause();
    audio.pause();
  }
}, false);

video.addEventListener('timeupdate', function() {
  if(!controller && audio.readyState >= 4) {
    if(Math.ceil(audio.currentTime) != Math.ceil(video.currentTime)) {
      audio.currentTime = video.currentTime;
    }
  }
}, false);

Please note that MediaController is defined by scripts only, but the controller can be defined using the static "mediagroup" attribute:

<video mediagroup="foo"></video> ... <audio mediagroup="foo"> ... </audio>

If we do this, it will work in Chrome without JavaScript. It will sync media sources, but the user cannot control the audio (including not being able to turn it off) because the browser does not know what the audio represents. In this case, it is best to encode the audio into the video, because then it can appear in the object, which the browser can recognize and be able to provide native controls. But since we don't have audioTracks data, this is an irrelevant question! So if the script is not available, the audio will not be played at all. This is the final demo, which can be run in any latest version of Opera, Firefox, Chrome, Safari, or IE9 or later: - Audio Description Demo audioTracks

Of course, this is just a simple proof of concept demonstration - there is no initial functional detection, it only has the basic controls provided by the native "controls" property. For the correct implementation, it requires custom controls to provide (among other things) a button to switch audio and a separate volume slider. The interface should also be accessible via the keyboard, which is not the case in some browsers' native controls. It also needs to handle the buffering correctly - in fact, if you search for the point where the video has preloaded, the audio will continue to play freely until the video loads enough to resynchronize it. I also want to mention that the description itself almost does not meet the professional standards! That's where you can hear me, record and convert using Audacity. But that's it, I think it effectively demonstrates how low the technical barriers of this approach are. I don't have to edit the video, and I made the audio in an hour with free software. As a proof of concept, I think it is very successful – I believe my clients will be very satisfied! FAQs (FAQ) on Accessible Audio Descriptions for HTML5 Videos

What is the importance of accessing audio descriptions in HTML5 videos?

Accessible audio descriptions play a crucial role in making HTML5 videos more inclusive and user-friendly. They provide auditory equivalents of visual information, which is especially beneficial for visually impaired users. These descriptions describe important visual details that cannot be understood by the main track. By adding accessible audio descriptions, content creators can ensure that their videos can be accessed by a wider audience, thereby promoting digital inclusion.

How to add audio description to my HTML5 video?

Adding audio description to HTML5 video includes several steps. First, you need to create a separate audio track that describes the visual elements of the video. This can be done using various audio editing software. Once the audio description track is ready, you can add it to your HTML5 video using the element with the kind attribute set to "descriptions". This will ensure that the audio description track is recognized and played with the video.

Why can't my HTML5 video play?

There may be several reasons why your HTML5 video cannot play. This may be due to problems with the video file itself, such as not encoding correctly. This may also be due to the problem that the web browser or video player does not support the video format. To troubleshoot, try playing videos on different browsers or on different devices. If the problem persists, you may need to check the video file and make sure its format is supported by HTML5.

What commonly used formats are supported in HTML5 videos?

HTML5 video supports several common video formats, including MP4, WebM, and Ogg. The MP4 format is widely supported in all major browsers and devices, making it a popular choice for online video. WebM and Ogg are open source formats and are widely supported, although they may not work properly in all browsers.

How to fix the "HTML5 Video File Not Found" error?

"HTML5 Video File Not Found" error usually occurs when the browser cannot find the video file specified in the source attribute of the <video></video> element. To fix this error, make sure that the file path in the source property is correct and that the video file is in the specified path. If the file path is correct, check whether the video file is in HTML5 and the format supported by the browser.

How to make my HTML5 video responsive?

To make your HTML5 video responsive, you can use CSS to set the video's width to 100% and the height to auto. This will ensure that the video is scaled up or down to fit the width of its container, allowing it to respond to different screen sizes.

Can I add subtitles or narration to my HTML5 video?

Yes, you can add subtitles or narration to your HTML5 video using an element with a kind attribute set to "captions" or "subtitles". You need to create a WebVTT file with subtitles or narrations, and then reference this file in the src attribute of the element.

How to control the playback of my HTML5 videos?

HTML5 provides several built-in video playback controls, including play, pause, volume and full screen. These controls can be enabled by adding the controls attribute to the <video></video> element. Additionally, you can use JavaScript to create custom controls and interactions.

Can I embed HTML5 videos on my website?

Yes, you can use the <video></video> element to embed HTML5 videos on your website. You need to use the src attribute to specify the source of the video file, and you can also add optional attributes such as controls, autoplay, and loop to customize video playback.

What are the benefits of using HTML5 for video playback?

HTML5 provides many benefits for video playback. It supports multiple video formats, provides built-in video playback controls, and allows the addition of accessibility features such as subtitles and audio descriptions. Additionally, HTML5 videos can be responsive, ensuring they look great on all devices and screen sizes. Finally, since HTML5 is the web standard, it is supported by all modern web browsers without additional plug-ins or software.

The above is the detailed content of Accessible Audio Descriptions for HTML5 Video. For more information, please follow other related articles on the PHP Chinese website!