An elementary analysis of the probability that a binomial random variable exceeds its expectation
Introduction
Let be a random variable following a binomial distribution with parameters and , that is, we have for all . Then, apart from maybe extreme cases, it seems very natural that with reasonable probability is at least its expectation or even exceeds it. Surprisingly, and despite the fact that such statements are very important in the machine learning literature, only very recently rigorous proofs of such statements appeared. We refer to Greenberg and Mohri (2014) for a detailed discussion on the previous lack of such results.
Prior to the work of Greenberg and Mohri, apart from general bounds like those in Slud (1977), apparently only a result of Rigollet and Tong (2011) was known. This result is stated as for all in the paper (Lemma 6.4 in Rigollet and Tong (2011)), but the proof shows the stronger statement The main work in the proof is showing another interesting result, namely that for all one has The proof of this result uses a connection between binomial distributions and order statistics of uniform distributions (to be found in Section 7.2 of the second volume of Feller, 1971) and then proceeds by showing the inequality
It is not clear how to extend (1) to . Note that neither (2) nor this equation with the inequality reversed is true for all . Hence the following relatively recent result of Greenberg and Mohri appears to be the first one treating the problem in full generality.
Lemma 1 Greenberg and Mohri, 2014 Let and . Let . Then
This result has found applications not only in machine learning, but also in randomized algorithms, see, e.g., Karppa et al. (2016), Becchetti et al. (2017) and Mitzenmacher and Morgan (2017). While the result is very simple, the proof is not and uses the Camp–Paulson normal approximation to the binomial cumulative distribution function.
Via a different, again non-elementary proof technique, using among others the hazard rate order and the likelihood ratio order of integer-valued distributions, the following result was shown by Pelekis and Ramon (2016).
Lemma 2 Pelekis and Ramon, 2016 Let and . Let . Then
Lemma 2 improves the bound of Lemma 1 when , which in particular requires and . It however never gives a bound better than .
In this work, we show that also truly elementary arguments give interesting results for this problem. We prove in Lemma 8 that for and , we have This bound is not perfectly comparable to the previous, but Fig. 1 indicates that it is often superior. It has the particular advantage that it tends to when and tend to infinity. Our bound does not immediately imply the bound of Greenberg and Mohri (2014), however elementary analyses of a few “small cases” suffice to obtain in Theorem 10 that for all . The strict version of the claim is also valid except when and .
We also show that for , the random variable exceeds its expectation by more than one with probability at least 0.0370, again with better bounds available when and are larger, see Theorem 12. Such a statement was recently needed in the analysis of an evolutionary algorithm (in the proof of Lemma 3 of the extended version of Doerr et al., 2017).
Section snippets
Preliminaries
All of the notation we shall use is standard and should need not much additional explanation. We denote by the positive integers. For intervals of integers, we write . We use the standard definition (and not ).
It is well-known that is monotonically increasing and that is monotonically decreasing in (and that both converge to ). We need two slightly stronger statements in this work (Lemma 3 (a) and (c)). See Fig. 2 for an illustration of
Proofs of our results
We are now ready to prove our results.
Lemma 8 Let and . Let . Let . Then
Proof We compute Here the first inequality stems from the natural stochastic domination relation between binomial distributions with different success probabilities (Lemma 7). The last estimate uses (i) the well-known fact that a
Conclusion
We gave an elementary proof of the fact that a binomial random variable , , exceeds its expectation with probability at least . The main proof idea is (i) that the expectation is very close to the median (which by definition is reached or exceeded with probability at least ) and (ii) that the probabilities for exceeding expectation and median are very close because generally the probability that a binomial random variable attains a particular value is small.
This
Acknowledgments
The author would like to thank Philippe Rigollet (Massachusetts Institute of Technology (MIT)) and Carsten Witt (Danish Technical University (DTU)) for useful discussions and pointers to the literature.
References (13)
- et al.
Tight lower bound on the probability of a binomial exceeding its expectation
Statist. Probab. Lett.
(2014) - et al.
A lower bound on the probability that a binomial random variable is exceeding its mean
Statist. Probab. Lett.
(2016) - et al.
Simple dynamics for plurality consensus
Distrib. Comput.
(2017) - et al.
The (1+) evolutionary algorithm with self-adjusting mutation rate
An Introduction to Probability Theory and Its Applications, Vol. II
(1971)- et al.
Mean, median and mode in binomial distributions
Stat. Neerl.
(1980)
Cited by (37)
A study on the negative binomial distribution motivated by Chvátal's theorem
2024, Statistics and Probability LettersA study on the Poisson, geometric and Pascal distributions motivated by Chvátal's conjecture
2023, Statistics and Probability LettersOn the Chvátal–Janson conjecture
2023, Statistics and Probability LettersCitation Excerpt :Chvátal In addition to be intriguing, Conjecture 1 may have useful applications, since the probability that a binomial random variable exceeds its expected value plays a role in the machine learning framework; see e.g. Doerr (2018), Greenberg and Mohri (2014), Vapnik (1998) and references therein. Such a probability is also connected to an equation given by Ramanujan, as emphasized by Jogdeo and Samuels (1968).
On the moment absolute deviation of order statistics from uniform distribution
2022, Statistics and Probability LettersOn the probability that a binomial variable is at most its expectation
2021, Statistics and Probability LettersWorking principles of binary differential evolution
2020, Theoretical Computer Science