A Two-Stage Phase-Aware Approach for Monaural Multi-Talker Speech Separation

Lu YIN  Junfeng LI  Yonghong YAN  Masato AKAGI  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E103-D   No.7   pp.1732-1743
Publication Date: 2020/07/01
Publicized: 2020/04/20
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019EDP7259
Type of Manuscript: PAPER
Category: Speech and Hearing
Keyword: 
speech separation,  phase recovery,  amplitude estimation,  deep learning,  mask estimation,  

Full Text: PDF>>
Buy this Article



Summary: 
The simultaneous utterances impact the ability of both the hearing-impaired persons and automatic speech recognition systems. Recently, deep neural networks have dramatically improved the speech separation performance. However, most previous works only estimate the speech magnitude and use the mixture phase for speech reconstruction. The use of the mixture phase has become a critical limitation for separation performance. This study proposes a two-stage phase-aware approach for multi-talker speech separation, which integrally recovers the magnitude as well as the phase. For the phase recovery, Multiple Input Spectrogram Inversion (MISI) algorithm is utilized due to its effectiveness and simplicity. The study implements the MISI algorithm based on the mask and gives that the ideal amplitude mask (IAM) is the optimal mask for the mask-based MISI phase recovery, which brings less phase distortion. To compensate for the error of phase recovery and minimize the signal distortion, an advanced mask is proposed for the magnitude estimation. The IAM and the proposed mask are estimated at different stages to recover the phase and the magnitude, respectively. Two frameworks of neural network are evaluated for the magnitude estimation on the second stage, demonstrating the effectiveness and flexibility of the proposed approach. The experimental results demonstrate that the proposed approach significantly minimizes the distortions of the separated speech.