We studied the mutual information between a stimulus and a system consisting of stochastic, statistically independent elements that respond to a stimulus. Using statistical mechanical methods the properties of the mutual information (MI) in the limit of a large system size $N$ are calculated. For continuous valued stimuli, the MI increases logarithmically with $N$ and is related to the log of the Fisher information of the system. For discrete stimuli the MI saturates exponentially with $N$. We find that the exponent of saturation of the MI is the Chernoff distance between response probabilities that are induced by different stimuli.

• Received 11 January 2001

DOI:https://doi.org/10.1103/PhysRevLett.86.4958