REPLICAS IN COMPLEX SYSTEMS: APPLICATIONS TO LARGE DEVIATIONS AND NEURAL NETWORKS

In this thesis, we apply methods from replica theory to deal with two contemporary problems in the study of complex systems. In Part I, we discuss the behavior of the rare fluctuations of the observable of choice in most models of spin glasses: the free energy. Due to the quenched disorder, this and other thermodynamic quantities are self-averaging random variables, whose probability distribution can be evaluated within replica theory. In particular, the probability of the free energy fluctuations above its typical value shows an anomalous scaling with the number of degrees of freedom, at variance with the ordinary exponential suppression for fluctuations below. We explain how the introduction of a small magnetic field can remove this anomalous behavior. In Part II, we apply the replica formalism to the problem of linear classification of objects with a geometrical structure, in the context of machine learning. In particular, using combinatorial techniques we evaluate the number of dichotomies (binary classifications) of a set of structured inputs achievable by a linear classifier, as a function of the number of inputs to classify. We prove that this number shows an additional critical point beyond the usual storage capacity for isolated points, at which the number of admissible dichotomies becomes zero in the thermodynamic limit; the associated phase transition present a certain degree of replica symmetry breaking. This behavior is due to a trade-off between the increasing number of points to classify, and the increasing volume excluded by their geometrical structure. This approach goes in the direction of finding bounds on the generalization error of certain simple neural network architectures, more stringent for structured data than the ones known from Statistical Learning Theory.