Retrieval Capabilities of Hierarchical Networks: From Dyson to Hopfield

Elena Agliari, Adriano Barra, Andrea Galluzzi, Francesco Guerra, Daniele Tantari, and Flavia Tavani

Phys. Rev. Lett. 114, 028103 – Published 16 January 2015

Abstract

We consider statistical-mechanics models for spin systems built on hierarchical structures, which provide a simple example of non-mean-field framework. We show that the coupling decay with spin distance can give rise to peculiar features and phase diagrams much richer than their mean-field counterpart. In particular, we consider the Dyson model, mimicking ferromagnetism in lattices, and we prove the existence of a number of metastabilities, beyond the ordered state, which become stable in the thermodynamic limit. Such a feature is retained when the hierarchical structure is coupled with the Hebb rule for learning, hence mimicking the modular architecture of neurons, and gives rise to an associative network able to perform single pattern retrieval as well as multiple-pattern retrieval, depending crucially on the external stimuli and on the rate of interaction decay with distance; however, those emergent multitasking features reduce the network capacity with respect to the mean-field counterpart. The analysis is accomplished through statistical mechanics, Markov chain theory, signal-to-noise ratio technique, and numerical simulations in full consistency. Our results shed light on the biological complexity shown by real networks, and suggest future directions for understanding more realistic models.

Received 9 July 2014

DOI:https://doi.org/10.1103/PhysRevLett.114.028103

Authors & Affiliations

Elena Agliari¹, Adriano Barra¹, Andrea Galluzzi², Francesco Guerra¹, Daniele Tantari², and Flavia Tavani³

¹Dipartimento di Fisica, Sapienza Università di Roma, Piazzale Aldo Moro 2, 00185 Roma, Italy
²Dipartimento di Matematica, Sapienza Università di Roma, Piazzale Aldo Moro 2, 00185 Roma, Italy
³Dipartimento SBAI (Ingegneria), Sapienza Università di Roma, Via Antonio Scarpa 14, 00185 Roma, Italy

Article Text (Subscription Required)

Click to Expand

References (Subscription Required)

Click to Expand

Issue

Vol. 114, Iss. 2 — 16 January 2015

Reuse & Permissions

Access Options

Author publication services for translation and copyediting assistance advertisement

Images

Figure 1
Sketch of the hierarchical topology underlying the model (here $k = 4$ , $N = 2^{k} = 16$ ). Nodes represent spins; the larger the distance between two spins the weaker their coupling. On the right, we highlight how the graph breaks down in two equivalent components (referred to as “left” and “right”) as the weakest link $J (k, k, ρ)$ [see Eq. (2)] is cut.
Reuse & Permissions
Figure 2
Left panel: Schematic representation of the free energy for the DHM, where minima correspond to equilibrium states. At finite size the pure ferromagnetic state (denoted by aligned arrows) is a global minimum, while the mixed state (denoted by misaligned arrows) is a local minimum. In the thermodynamic limit both states are global minima and the system can relax to any of them (although the latter displays a smaller attraction basin). Right panel: Representation of the eigenstates of $W$ for a system with $k = 4$ and $ρ = 0.75$ . Each column represents a different eigenstate, eigenstates pertaining to the same degenerate eigenvalue are highlighted. Different colors represent different entries in the eigenstate, as shown by the color map on the right.
Reuse & Permissions
Figure 3
Left panel: Magnetizations obtained by MC simulations of the DHM for different sizes (main figure) and by the theoretical curves given by Eq. (5) (inset) holding in the thermodynamic limit. In the main plot, the spontaneous switch between serial and mixed state is a finite-size effect. Right panel: Mattis magnetizations obtained by Monte Carlo simulations of the HHM for $k = 12$ and $p = 4$ . Consistently with the analytical picture [Eqs. (13) and (5) give analogous solutions], the simultaneous retrieval of multiple patterns exists and is stable.
Reuse & Permissions
Figure 4
Left panel: Phase diagram for the DHM as derived from the signal-to-noise ratio analysis. The curves separating different phases are obtained by solving numerically the transcendental equation $\tanh [β h_{i} (\vec{σ}, ρ, k)] \approx 1$ as a function of $β$ and $ρ$ . Here we fixed $k = 7$ and we focused on four different configurations (single pattern state, multitasking state, and states where subcommunities made of four and eight spins, respectively, are misaligned with respect to the bulk). Right panel: $p_{crit}^{single}$ (solid line) and $p_{crit}^{multi}$ (dashed line), as a function of $ρ$ and for several choices of $k$ , as explained by the legend. Notice that as $ρ$ increases the capacity of the system (for both single and multiple retrieval) decreases, in fact, the magnitude of couplings decreases exponentially with $ρ$ , hence the overall storage capacity is also reduced [18, 21].
Reuse & Permissions

Physical Review Letters