An Efficient Image to Sound Mapping Method Preserving Speech Spectral Envelope

Yuya HOSODA  Arata KAWAMURA  Youji IIGUNI  

Publication
IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E103-A   No.3   pp.629-630
Publication Date: 2020/03/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.2019EAL2139
Type of Manuscript: LETTER
Category: Digital Signal Processing
Keyword: 
spectrogram,  spectral envelope,  image to sound,  

Full Text: PDF(210.6KB)>>
Buy this Article




Summary: 
In this paper, we propose an image to sound mapping method. This technique treats an image as a spectrogram and maps it to a sound by taking inverse FFT of the spectrogram. Amplitude spectra of a speech signal are embedded to the spectrogram to give speech intelligibility for the mapped sound. Specifically, we hold amplitude spectra of a speech signal with strong power and embed the image brightness in other frequency bands. Holding amplitude spectra of a speech signal with strong power preserves a speech spectral envelope and improves the speech quality of the mapped sound. The amplitude spectra of the mapped sound with weak power represent the image brightness, and then the image is successfully reconstructed from the mapped sound. Simulation results show that the proposed method achieves sufficient speech quality.