Speech enhancement using deep mixture of experts

Abstract

Speech enhancement is at the heart of many applications such as speech com- munication, automatic speech recognition, hearing aids etc. In this work, we consider the speech enhancement under the framework of multiple deep neural network (DNN) system. DNNs have been extensively used in speech enhance- ment due to its ability to capture complex variations in the input data. As a natural extension, researchers have used variants of a network with multi- ple DNNs for speech enhancement. Input data could be clustered to train each DNN or train all the DNNs jointly without any clustering. In this work, we pro- pose clustering methods for training multiple DNN systems and its variants for speech enhancement. One of the proposed works involves grouping phonemes into broad classes and training separate DNN for each class. Such an approach is found to perform better than single DNN based speech enhancement. However, it relies on phoneme information which may not be available for all corpora. Hence, we propose a hard expectation-maximization (EM) based task speci c clustering method, which, automatically determines clusters without relying on the knowledge of speech units. The idea is to redistribute the data points among multiple DNNs such that it enables better speech enhancement. The experimen- tal results show that the hard EM based clustering performs better than the single DNN based speech enhancement and provides results similar to that of the broad phoneme class based approach.