Nonlinear Blind Source Separation by Variational Bayesian Learning

Erkki OJA
Alexander ILIN

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E86-A    No.3    pp.532-541
Publication Date: 2003/03/01
Online ISSN: 
Print ISSN: 0916-8508
Type of Manuscript: Special Section INVITED PAPER (Special Section on Blind Signal Processing: Independent Component Analysis and Signal Separation)
Category: Constant Systems
blind separation,  independent component analysis,  Bayesian learning,  nonlinear dynamics,  

Full Text: PDF>>
Buy this Article

Blind separation of sources from their linear mixtures is a well understood problem. However, if the mixtures are nonlinear, this problem becomes generally very difficult. This is because both the nonlinear mapping and the underlying sources must be learned from the data in a blind manner, and the problem is highly ill-posed without a suitable regularization. In our approach, multilayer perceptrons are used as nonlinear generative models for the data, and variational Bayesian (ensemble) learning is applied for finding the sources. The variational Bayesian technique automatically provides a reasonable regularization of the nonlinear blind separation problem. In this paper, we first consider a static nonlinear mixing model, with a successful application to real-world speech data compression. Then we discuss extraction of sources from nonlinear dynamic processes, and detection of abrupt changes in the process dynamics. In a difficult test problem with chaotic data, our approach clearly outperforms currently available nonlinear prediction and change detection techniques. The proposed methods are computationally demanding, but they can be applied to blind nonlinear problems of higher dimensions than other existing approaches.