Tensorflow.js is an open-source library developed by Google for running machine learning models and deep learning neural networks in the browser or node environment.
The tf.data.microphone() function is used to produce an iterator that creates frequency-domain spectrogram Tensors from microphone audio stream with browser’s native FFT.
Note:
- This code is effective only when the device is having a microphone. While running this API, it will ask to permit to open the microphone.
- This API only works through the browser environment.
Syntax:
tf.data.microphone (microphoneConfig)
Parameters: This function accepts a parameter which is illustrated below:
- microphoneConfig: A MicrophoneConfig object contains configurations of reading audio data from the microphone. It is an optional parameter.
This object contains some configuration which is specified below:
- sampleRateHz: Its range lies between 44100 and 48000.
- fftSize: It is a number value must be a power of 2 between 2 to 4 and 2 to 14.
- columnTruncateLength: It is a number value.
- numFramesPerSpectrogram: It is a number value.
- audioTrackConstraints: It is MediaTrackConstraints.
- smoothingTimeConstant: It is a number value.
- includeSpectrogram: It is a boolean value.
- includeWaveform: It is a boolean value.
Return Value: It returns a MicrophoneIterator.
Example 1: After running the below code, it will ask for permission to start the microphone. After giving permission below code will return and will give output.
Javascript
// Importing the tensorflow.js libraryimport * as tf from "@tensorflow/tfjs"// Calling the .data.microphone() function// with its parametersconst mic = await tf.data.microphone({ fftSize: 1024, columnTruncateLength: 32, numFramesPerSpectrogram: 10, sampleRateHz:43000, includeSpectrogram: true, includeWaveform: false});// Capturing the data recorded by microphoneconst audioData = await mic.capture();const spectrogramTensor = audioData.spectrogram;// Printing the data like sampling rate // expected and actualspectrogramTensor.print();const waveformTensor = audioData.waveform;waveformTensor.print();// Stopping the microphonemic.stop(); |
Output: It gives an error because here the sampling rate expected is 43000 but the actual recorded value is 48000.
An error occurred Mismatch in sampling rate: Expected: 43000; Actual: 48000
Example 2: After running the below code, it will ask for permission to start the microphone. After giving permission below code will return and will give output.
Javascript
// Importing the tensorflow.js libraryimport * as tf from "@tensorflow/tfjs"// Initializing the configurations of // reading audio data from microphoneconst x = { fftSize: 1024, columnTruncateLength: 32, numFramesPerSpectrogram: 10, sampleRateHz:48000, includeSpectrogram: true, includeWaveform: false};// Calling the .data.microphone() function// with the parameter specified aboveconst mic = await tf.data.microphone(x);// Capturing the data recorded by microphoneconst audioData = await mic.capture();const spectrogramTensor = audioData.spectrogram;// Creating an iterator that generate frequency-domain// spectrogram Tensors from the microphonespectrogramTensor.print();const waveformTensor = audioData.waveform;// Stopping the microphonemic.stop(); |
Output:
Tensor
[[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[0 ],
[0 ],
[0 ],
...,
[0 ],
[0 ],
[0 ]],
[[-Infinity],
[-Infinity],
[-Infinity],
...,
[-Infinity],
[-Infinity],
[-Infinity]]]
Reference:https://js.tensorflow.org/api/latest/#data.microphone
