Blind separation of circularly distributed sources by neural extended APEX algorithm

doi:10.1016/S0925-2312(00)00161-2

Neurocomputing

Volume 34, Issues 1–4, September 2000, Pages 239-252

https://doi.org/10.1016/S0925-2312(00)00161-2 Get rights and content

Abstract

The aim of this work is to present a generalized Hebbian learning theory for complex-weighted linear feed-forward network endowed with lateral inhibitory connections, and to show how it can be applied to blind separation from complex-valued mixtures. We start by stating an optimization principle for Kung–Diamantaras’ network which leads to a generalized APEX-like learning theory relying on some non-linear functions, whose choice determines network's ability. Then we recall the Sudjianto–Hassoun interpretation of Hebbian learning and show that it drives us to the choice of the right set of non-linear functions allowing the network to achieve blind separation. The proposed approach is finally assessed by numerical simulations.

Introduction

Blind Source Separation (BSS) by neural networks and adaptive algorithms has raised much interest in the scientific community (see for example [11], [13], [14], [15], [16], [17] and references therein). Among other approaches, it has been proven that adding non-linearity to linear Principal Component Analysis (PCA) neural networks makes them able to improve the independence of their outputs so as to allow blind separation of real-valued independent sources [13], [14], [15], [16], [17]. Moreover, attention has been paid to blind separation of complex-valued signals because of the useful applications to telecommunications and array processing and to blind separation of convolutive mixtures (see [1], [2], [6], [8], [18], [20], [25] and references therein). Consequently, as for instantaneous real-valued separation, the generalization of classical PCA techniques to complex-valued data processing seems interesting. Also, some examples of the usefulness of complex principal component analysis and principal subspace decomposition algorithms may be found for instance in [10], [19].

Recently, some attempts have been made in order to extend the best-known PCA algorithms to the complex case. In [3] Chen and Hou presented an heuristic complex version of the well-known APEX algorithm [9]; in [7] De Castro et al. gave a heuristic complex extension of the GHA algorithm [23]. In this work we formally derive a new learning theory as a generalized complex-valued Hebbian learning for a linear feed-forward network with lateral inhibitory connections [9], [21], [22], which relies on non-linear functions that affect network's signal processing ability. Then we discuss the choice of the set of non-linear functions under the theoretical framework proposed by Sudjianto and Hassoun [24] that we extend to the complex case. A particular non-linearity, having the shape of the Rayleigh probability distribution function, allows the neural network to separate out mixed independent complex-valued source signals, as illustrated by computer simulations.

Notation

In the paper we use the following notation: $E_{x} [f (x)]$ denotes mathematical expectation of $f (x)$ with respect to the statistics of the multivariate random process $x$ ; superscript T denotes ordinary transpose while superscript H denotes Hermitian (conjugate) transpose; symbol $∗$ denotes complex conjugation.

Section snippets

Extended APEX algorithm and blind separation of complex sources

Consider the complex-weighted neural network with input $x ∈ C^{p}$ and output $y ∈ C^{m}$ , with m≤p, described by the following relationships: $z_{k} = w_{k}^{H} x, y_{k} =z_{k} + h_{k}^{H} y, k=1,…,m,$ where $z ∈ C^{m}$ , vectors $w_{k} ∈ C^{p}$ represent the network's direct connections, and vectors $h_{k} ∈ C^{p}$ represent the network's inhibitory lateral connections. A schematic of this topology is reported in Fig. 1. The original APEX learning rule for such a network was derived by Kung and Diamantaras by applying Oja's first principal component rule, which is a

Computer simulations

Suppose input $x ∈ C^{4}$ be formed by a linear mixture of four independent signals arranged in a vector $s ∈ C^{4}$ . Signal s₁ is QAM4, s₂ is QAM16, signal s₃ is PSK. Signal s₄ is a Gaussian noise of variance σ²=0.25. The first row of Fig. 2 depicts the independent signals while second row shows the obtained four mixtures. Also, the bar graphs reported in Fig. 3 represent the histograms of the four sources’ modules squashed by function Q_R(u). A close examination of these graphs shows that the QAM4 and PSK

Conclusion

In this work a new adapting rule for linear neural networks as generalization of APEX learning has been presented; it provides a generalization in that it applies to complex-weighted neural networks and involves non-linear functions in the classical Hebbian learning. A particular choice of the non-linearity is discussed by recalling the Sudjianto–Hassoun interpretation of non-classical Hebbian learning extended to the complex case.

References (25)

P Comon
Independent component analysis, a new concept ?
Signal Process.
(1994)
A Hyvärinen et al.
Independent component analysis by general non-linear Hebbian-like rules
Signal Process.
(1998)
J Karhunen et al.
Representation and separation of signals using nonlinear PCA type learning
Neural Networks
(1994)
J Karhunen et al.
Generalizations of PCA, optimization problems, and neural networks
Neural Networks
(1995)
T.D Sanger
Optimal unsupervised learning in a single-layer linear feedforward neural network
Neural Networks
(1989)
J.-F. Cardoso, P. Comon, Independent component analysis, a survey of some algebraic methods, Proceedings of ISCAS, Vol....
J.F Cardoso et al.
Equivariant adaptive source separation
IEEE Trans. Signal Process.
(1996)
Y. Chen, C. Hou, High resolution adaptive bearing estimation using a complex-weighted Neural Network, Proceedings of...
A Cichocki et al.
Adaptive learning algorithm for principal component analysis with partial data
Proc. Cybernet. Systems
(1996)
P Comon et al.
Improved contrast dedicated to blind separation in communications
Proceedings of ICASSP
(1997)

M.C.F De Castro et al.

A complex valued Hebbian learning algorithm

Proceedings of IEEE-IJCNN

(1998)

G. Desodt, D. Muller, Complex ICA applied to the separation of radar signals, Proceedings of EUSIPCO, Vol. I, 1990, pp....

Cited by (33)

Neural learning by geometric integration of reduced 'rigid-body' equations
2004, Journal of Computational and Applied Mathematics
In previous contributions, the second author of this paper presented a new class of algorithms for orthonormal learning of linear neural networks with p inputs and m outputs, based on the equations describing the dynamics of a massive rigid frame on the Stiefel manifold. These algorithms exhibit good numerical stability, strongly binding to the sub-manifold of constraints, and good controllability of the learning dynamics, but are not completely satisfactory from a computational-complexity point of view. In the recent literature, efficient methods of integration on the Stiefel manifold have been proposed by various authors, see for example (Phys. D 156 (2001) 219; Numer. Algorithms 32 (2003) 163; J. Numer. Anal. 21 (2001) 463; Numer. Math. 83 (1999) 599). Inspired by these approaches, in this paper, we propose a new and efficient representation of the mentioned learning equations, and a new way to integrate them. The numerical experiments show how the new formulation leads to significant computational savings especially when p≫m. The effectiveness of the algorithms is substantiated by numerical experiments concerning principal subspace analysis and independent component analysis. These experiments were carried out with both synthetic and real-world data.
Uniqueness of complex and multidimensional independent component analysis
2004, Signal Processing
Citation Excerpt :
Complex S–D theorem Various algorithms for solving the complex BSS problem have been proposed [1,2,7,13,16]. We want to note that many cases where complex BSS is applied can in fact be reduced to using real BSS algorithms.
A complex version of the Darmois–Skitovitch theorem is proved using a multivariate extension of the latter by Ghurye and Olkin. This makes it possible to calculate the indeterminacies of independent component analysis (ICA) with complex variables and coefficients. Furthermore, the multivariate Darmois–Skitovitch theorem is used to show uniqueness of multidimensional ICA, where only groups of sources are mutually independent.
One-unit 'rigid-bodies' learning rule for principal/independent component analysis with application to ECT-NDE signal processing
2004, Neurocomputing
The aim of this research work is to present a detailed theoretical analysis of the one-unit learning rule based on the rigid-bodies learning theory, specialized for first principal/independent component analysis. The adaptation equations are regarded as generators of weight-flows on a structured parameters space; the stationary points of the learning equations are determined and their stability is proven through a suitable Lyapunov function. The neuron is also excited with both synthetic and real-world signals in order to numerically investigate its behavior, and eddy-current-signal processing is carried out as an application of the developed independent component analysis algorithm to non-destructive evaluation of metallic objects.
A feasibility study for electromagnetic pollution monitoring by electromagnetic-source localization via neural independent component analysis
2003, Neurocomputing
The aim of this paper is to present an electromagnetic source localization technique based on independent component analysis (ICA), as a part of a feasibility study for electromagnetic pollution monitoring via neural network techniques. Four ICA algorithms known from the literature, allowing to process complex-valued signals, are used to estimate the propagation parameters of electromagnetic fields emitted by radio-base transmission systems. By properly interpreting the results given by the ICA algorithms it is possible to develop a blind source localization procedure. The experiments are performed on synthetic signals generated according to real-world emission features, and are used to obtain a preliminary evaluation of the expected behavior of the localization system in a real-world context.
Neural independent component analysis by 'maximum-mismatch' learning principle
2003, Neural Networks
The aim of the present paper is to apply Sudjanto-Hassoun theory of Hebbian learning to neural independent component analysis. The basic learning theory is first recalled and expanded in order to make it suitable for a network of non-linear complex-weighted neurons; then its interpretation and application is shown in the context of blind separation of complex-valued sources. Numerical results are given in order to assess the effectiveness of the proposed learning theory and the related separation algorithm on telecommunication signals; a comparison with other existing techniques finally helps assessing the performances and computational requirements of the proposed algorithm.
Overview of independent component analysis technique with an application to synthetic aperture radar (SAR) imagery processing
2003, Neural Networks
We present an overview of independent component analysis, an emerging signal processing technique based on neural networks, with the aim to provide an up-to-date survey of the theoretical streams in this discipline and of the current applications in the engineering area. We also focus on a particular application, dealing with a remote sensing technique based on synthetic aperture radar imagery processing: we briefly review the features and main applications of synthetic aperture radar and show how blind signal processing by neural networks may be advantageously employed to enhance the quality of remote sensing data.

View all citing articles on Scopus

^☆: This research was financially supported by the Italian MURST.

View full text

LettersBlind separation of circularly distributed sources by neural extended APEX algorithm☆

Abstract

Introduction

Section snippets

Extended APEX algorithm and blind separation of complex sources

Computer simulations

Conclusion

Signal Process.

Signal Process.

Neural Networks

Neural Networks

Neural Networks

Equivariant adaptive source separation

IEEE Trans. Signal Process.

Adaptive learning algorithm for principal component analysis with partial data

Proc. Cybernet. Systems

Improved contrast dedicated to blind separation in communications

Proceedings of ICASSP

A complex valued Hebbian learning algorithm

Proceedings of IEEE-IJCNN

Letters
Blind separation of circularly distributed sources by neural extended APEX algorithm☆