An elementary analysis of the probability that a binomial random variable exceeds its expectation

https://doi.org/10.1016/j.spl.2018.03.016Get rights and content

Highlights

  • An elementary lower bound of the probability that a binomial random variable exceeds its expectation.

  • Stronger than most previous bounds.

  • Yields a still elementary proof for the fact that this probability is at least 1/4 when p is at least 1/n.

Abstract

We give an elementary proof of the fact that a binomial random variable X with parameters n and 0.29np<1 with probability at least 14 strictly exceeds its expectation. We also show that for 1np<11n, X exceeds its expectation by more than one with probability at least 0.0370. Both probabilities approach 12 when np and n(1p) tend to infinity.

Introduction

Let X be a random variable following a binomial distribution with parameters n and p, that is, we have Pr[X=i]=nipi(1p)ni for all i[0..n]. Then, apart from maybe extreme cases, it seems very natural that with reasonable probability X is at least its expectation E[X] or even exceeds it. Surprisingly, and despite the fact that such statements are very important in the machine learning literature, only very recently rigorous proofs of such statements appeared. We refer to Greenberg and Mohri (2014) for a detailed discussion on the previous lack of such results.

Prior to the work of Greenberg and Mohri, apart from general bounds like those in Slud (1977), apparently only a result of Rigollet and Tong (2011) was known. This result is stated as Pr[XE[X]]min{p,14} for all p12 in the paper (Lemma 6.4 in Rigollet and Tong (2011)), but the proof shows the stronger statement Pr[XE[X]]14if p[1n,12]pif p<1n.The main work in the proof is showing another interesting result, namely that for all k[2..n2] one has Pr[Bin(n,kn)k+1]Pr[Bin(n,k1n)k].The proof of this result uses a connection between binomial distributions and order statistics of uniform distributions (to be found in Section 7.2 of the second volume of Feller, 1971) and then proceeds by showing the inequality k0k1ntk1(1t)nkdt(nk)0kntk(1t)nk1dt.

It is not clear how to extend (1) to p>12. Note that neither (2) nor this equation with the inequality reversed is true for all k[n2..n1]. Hence the following relatively recent result of Greenberg and Mohri appears to be the first one treating the problem in full generality.

Lemma 1 Greenberg and Mohri, 2014

Let nN and 1n<p1. Let XBin(n,p). Then Pr[XE[X]]>14.

This result has found applications not only in machine learning, but also in randomized algorithms, see, e.g., Karppa et al. (2016), Becchetti et al. (2017) and Mitzenmacher and Morgan (2017). While the result is very simple, the proof is not and uses the Camp–Paulson normal approximation to the binomial cumulative distribution function.

Via a different, again non-elementary proof technique, using among others the hazard rate order and the likelihood ratio order of integer-valued distributions, the following result was shown by Pelekis and Ramon (2016).

Lemma 2 Pelekis and Ramon, 2016

Let nN and 1np11n. Let XBin(n,p). Then Pr[XE[X]]122np(1p)np(1p)+1+1.

Lemma 2 improves the bound of Lemma 1 when np(1p)>8, which in particular requires n>32 and E[X]=np>8. It however never gives a bound better than 1220.3536.

In this work, we show that also truly elementary arguments give interesting results for this problem. We prove in Lemma 8 that for 1np<1 and knp, we have Pr[X>E[X]]>1212πnk(nk).This bound is not perfectly comparable to the previous, but Fig. 1 indicates that it is often superior. It has the particular advantage that it tends to 12 when np and n(1p) tend to infinity. Our bound does not immediately imply the 14 bound of Greenberg and Mohri (2014), however elementary analyses of a few “small cases” suffice to obtain in Theorem 10 that Pr[X>E[X]]14 for all 0.28771nln(43)1np<1. The strict version Pr[X>E[X]]>14 of the claim is also valid except when n=2 and p=12.

We also show that for 1np<11n, the random variable X exceeds its expectation by more than one with probability at least 0.0370, again with better bounds available when np and n(1p) are larger, see Theorem 12. Such a statement was recently needed in the analysis of an evolutionary algorithm (in the proof of Lemma 3 of the extended version of Doerr et al., 2017).

Section snippets

Preliminaries

All of the notation we shall use is standard and should need not much additional explanation. We denote by N{1,2,} the positive integers. For intervals of integers, we write [a..b]{xZaxb}. We use the standard definition 001 (and not 00=0).

It is well-known that (11x)x is monotonically increasing and that (11x)x1 is monotonically decreasing in x (and that both converge to 1e). We need two slightly stronger statements in this work (Lemma 3 (a) and (c)). See Fig. 2 for an illustration of

Proofs of our results

We are now ready to prove our results.

Lemma 8

Let nN and 1np<1. Let XBin(n,p). Let k=np. Then Pr[X>E[X]]>1212πnk(nk)g(n,k).

Proof

We compute Pr[X>E[X]]=Pr[Xk+1]Pr[Bin(n,kn)k+1]=Pr[Bin(n,kn)k]Pr[Bin(n,kn)=k]>1212πnk(nk).Here the first inequality stems from the natural stochastic domination relation between binomial distributions with different success probabilities (Lemma 7). The last estimate uses (i) the well-known fact that a

Conclusion

We gave an elementary proof of the fact that a binomial random variable XBin(n,p), 0.29np<1, exceeds its expectation with probability at least 14. The main proof idea is (i) that the expectation is very close to the median (which by definition is reached or exceeded with probability at least 12) and (ii) that the probabilities for exceeding expectation and median are very close because generally the probability that a binomial random variable attains a particular value is small.

This

Acknowledgments

The author would like to thank Philippe Rigollet (Massachusetts Institute of Technology (MIT)) and Carsten Witt (Danish Technical University (DTU)) for useful discussions and pointers to the literature.

References (13)

There are more references available in the full text version of this article.

Cited by (37)

  • On the Chvátal–Janson conjecture

    2023, Statistics and Probability Letters
    Citation Excerpt :

    Chvátal In addition to be intriguing, Conjecture 1 may have useful applications, since the probability that a binomial random variable exceeds its expected value plays a role in the machine learning framework; see e.g. Doerr (2018), Greenberg and Mohri (2014), Vapnik (1998) and references therein. Such a probability is also connected to an equation given by Ramanujan, as emphasized by Jogdeo and Samuels (1968).

  • Working principles of binary differential evolution

    2020, Theoretical Computer Science
View all citing articles on Scopus
View full text