In this project, I constructed a real time speech enhancement plugin in MATLAB that reduces the background noise through 2 stages. The first stage is MATLAB’s built in VADnet model which is a neural network that is designed for voice activity detection. It analyzes the audio in order to determine how likely it is that speech is present in a given frame and is used as a preprocessing stage. This speech probability is then used to control the threshold of a noise gate from the dsp toolbox, so that the gate opens when speech is detected and closes when there is no speech in a frame of audio. Additionally, there was smoothing applied to the VAD probability before applying it to the gate in order to prevent audible opening and closing between words.
After the gate stage that is triggered by the VADnet probabilities, the signal is sent into a denoise function that has trained weights stored in the provided file denoisenet.mat. The toggle parameter lets the user choose between using the fully connected and CNN version of this denoiser. This stage is responsible for cleaning up more noise that is mixed with the speech while the gating section is responsible for getting rid of noise when there is no speech. Since the denoiser was trained on spectral analysis up to 8 kHz, the plugin downsamples the audio before it goes into the network in order for it to run in real time. While this works in application, it does take out a significant amount of high frequency content which audibly lowers the quality of the output. In future revisions, it would be useful to train the network on higher frequency band audio so that downsampling isn’t required to use the plugin.
The plugin chain goes from the noisy input audio, VADnet preprocessing, speech probability estimation, noise gate, feed forward/CNN denoising, and then a final dry/wet blend in order to get the denoise strength knob to function. Lower values keep more of the original signal in while high values emphasize the denoised signal more.