How to capture the sound intensity of an audio or video tag in HTML5

Question

How to capture the sound intensity of an audio or video tag in HTML5

Navigation

#1 by (3 votes)

5

I would like to know if anyone knows how to capture the sound intensity, or some kind of information that could be provided to me if a video has sound or not at a certain moment.

The goal is to have a tag of type <video> in a web application and be able to mark visually when it is ringing or not.

javascript jquery html5 html

asked by oscarvady 20.10.2016 в 12:34

source

1 answer

Change the value of input type text in php Is it possible to know when all the event handlers are finished?

score 3 · Accepted Answer

You can use AudioContext to achieve this, you need to do 4 things:

Create an analyzer node, which allows you to sample the audio piece by piece and know, for example, the signal strength at each point.

Create a processor node, which allows you to process the audio as it plays.

In the processor, and using the analyzer, calculate the volume of the signal for each sample that is processed.

Connect everything together to: Display the current intensity of the audio and send it also to the speaker or output.

Keep in mind that these are quite new APIs and although there is good support it is an option that you should use as Progressive Enhancement, so you should only include it in browsers that support it. It is also very useful in applications made with Electron or Apache Cordoba since they are based on Chromium and it supports these APIs very well.

Here is an example that I could only try in Chrome 53. The audio is not the best but it is what I found that allows to load the audio from here. The example has a red overlay that gets more intense as the power of the audio goes up.

The indicator of the average volume of each audio sample is the variable prom which I also use to indicate the opacity of <div> . You can use this, based on a threshold. For example, if it is greater than 10.0, activate the current audio indicator.

This is compatible with sources getUserMedia and WebRTC so you can use it without problems to indicate that the user is speaking.

Salu2

var audioCtx = new(window.AudioContext || window.webkitAudioContext)();
var overlay = document.querySelector('#overlay');

// creamos un analyzer
var analyser = audioCtx.createAnalyser();
analyser.smoothingTimeConstant = 0.3;
analyser.fftSize = 1024;

// creamos un audio processor
var processor = audioCtx.createScriptProcessor(2048, 1, 1);
// en el evento, verificamos si hay volumen o no 
processor.onaudioprocess = function(audio) {

  // tomar los datos del analyzer
  var array = new Uint8Array(analyser.frequencyBinCount);
  analyser.getByteFrequencyData(array);

  // calcular el volumen promedio
  var prom = 0;
  for (var i = 0; i < array.length; i++) {
    prom += array[i];
  }

  prom = prom / array.length;

  // aqui ponemos la intensidad del volumen como opacidad del div
  overlay.style.opacity = prom/200;
  console.log(prom);

  // pasar los datos hacia el speaker
  var inputBuffer = audio.inputBuffer;
  var outputBuffer = audio.outputBuffer;

  for (var channel = 0; channel < outputBuffer.numberOfChannels; channel++) {
    var inputData = inputBuffer.getChannelData(channel);
    var outputData = outputBuffer.getChannelData(channel);

    for (var sample = 0; sample < inputBuffer.length; sample++) {
      outputData[sample] = inputData[sample];
    }
  }
}

// conectar todo junto
var video = document.querySelector('audio');
var source = audioCtx.createMediaElementSource(video);
source.connect(analyser);
analyser.connect(processor);
processor.connect(audioCtx.destination);

#overlay {
  position: fixed;
  top:0; left: 0; bottom: 0; right: 0;
  background: red;
  opacity: 0;
  z-index: -1;
}

<audio autoplay crossOrigin="anonymous" width="400" controls>
  <source src="http://mdn.github.io/media-source-buffer/viper.mp3" type="audio/mpeg">
</audio>
<div id="overlay"></div>

NOTE: This example is sensitive to the volume of the speaker, but it is a limitation of the example and in order to simplify it and demonstrate how these APIs work. This can be easily solved by using an AudioSpliter and diverting the output to the analyzer without having to attack the output volume.