Unsupervised learning of binary vectors: A Gaussian scenario

Mauro Copelli and Christian Van den Broeck
Phys. Rev. E 61, 6971 – Published 1 June 2000
PDFExport Citation

Abstract

We study a model of unsupervised learning where the real-valued data vectors are isotropically distributed, except for a single symmetry-breaking binary direction B{1,+1}N, onto which the projections have a Gaussian distribution. We show that a candidate vector J undergoing Gibbs learning in this discrete space, approaches the perfect match J=B exponentially. In addition to the second-order “retarded learning” phase transition for unbiased distributions, we show that first-order transitions can also occur. Extending the known result that the center of mass of the Gibbs ensemble has Bayes-optimal performance, we show that taking the sign of the components of this vector (clipping) leads to the vector with optimal performance in the binary space. These upper bounds are shown generally not to be saturated with the technique of transforming the components of a special continuous vector, except in asymptotic limits and in a special linear case. Simulations are presented which are in excellent agreement with the theoretical results.

  • Received 26 October 1999

DOI:https://doi.org/10.1103/PhysRevE.61.6971

©2000 American Physical Society

Authors & Affiliations

Mauro Copelli*

  • Department of Chemistry and Biochemistry 0340, University of California San Diego, La Jolla, California 92093-0340

Christian Van den Broeck

  • Limburgs Universitair Centrum, B-3590 Diepenbeek, Belgium

  • *Electronic address: mauro@hypatia.ucsd.edu
  • Electronic address: christian.vandenbroeck@luc.ac.be

References (Subscription Required)

Click to Expand
Issue

Vol. 61, Iss. 6 — June 2000

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review E

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×