A Covariance-Tying Technique for HMM-Based Speech Synthesis

Keiichiro OURA  Heiga ZEN  Yoshihiko NANKAKU  Akinobu LEE  Keiichi TOKUDA  

IEICE TRANSACTIONS on Information and Systems   Vol.E93-D   No.3   pp.595-601
Publication Date: 2010/03/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E93.D.595
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Speech and Hearing
HMM,  speech synthesis,  decision tree,  context-clustering,  MDL criterion,  embedded device,  

Full Text: PDF(662.9KB)>>
Buy this Article

A technique for reducing the footprints of HMM-based speech synthesis systems by tying all covariance matrices of state distributions is described. HMM-based speech synthesis systems usually leave smaller footprints than unit-selection synthesis systems because they store statistics rather than speech waveforms. However, further reduction is essential to put them on embedded devices, which have limited memory. In accordance with the empirical knowledge that covariance matrices have a smaller impact on the quality of synthesized speech than mean vectors, we propose a technique for clustering mean vectors while tying all covariance matrices. Subjective listening test results showed that the proposed technique can shrink the footprints of an HMM-based speech synthesis system while retaining the quality of the synthesized speech.