Abstract
Varied evidence confirms more strongly than less varied evidence, ceteris paribus. This epistemological Variety of Evidence Thesis enjoys widespread intuitive support. We put forward a novel explication of one notion of varied evidence and the Variety of Evidence Thesis within Bayesian models of scientific inference by appealing to measures of entropy. Our explication of the Variety of Evidence Thesis holds in many of our models which also pronounce on disconfirmatory and discordant evidence. We argue that our models pronounce rightly. Against a backdrop of failures of the Variety of Evidence Thesis, the intuitive case for the Variety of Evidence Thesis emerges strengthened. Our models do however not support the general case for the thesis since our explication of it fails to hold in certain cases. The parameter space of this failure is explored and an explanation for the failure is offered.
Similar content being viewed by others
Notes
This is consistent with the Bovens and Hartmann approach since their analysis “does not apply to unreliable instruments that do not randomize” (Bovens and Hartmann 2003, p. 95).
Variety of Evidence reasoning in climate science has recently been analysed to proceed via different consequences of the hypothesis that temperatures are rising, see Vezér (2017). Consequence considered were patterns in temperature profiles in ice, rock and soil as well as the lengths of mountain glaciers and sizes of tree rings.
Evidence variables which are confirmationally independent regarding the hypothesis variable and their role in confirming the hypothesis with respect to different confirmation measures are investigated in Fitelson (2001). Note that conditional independence with respect to consequence variables (our approach) is not immediately transferable to conditional independence with respect to the hypothesis variable (Fitelson).
Adopting the standard convention that \(0\cdot \log (0):=0\).
See Colombo et al. (2017) for some of the latest on objectivity in science.
For the record, \({\mathcal {E}}\) has a slightly greater variety score than \({\mathcal {E}}'\): \(V({\mathcal {E}})=1.3086>1.2861=V({\mathcal {E}}')\).
We suspect a slip in Claveau’s in his statement of the VET. Claveau’s proofs in the appendix of his paper only deal with evidence which is in agreement with the predicted value. Furthermore, in Claveau (2011, p. 241) he writes “I take evidence to be always evidence for a specific proposition” [emphasis original]. He might have simply forgotten to state this convention in Claveau (2013). Bovens and Hartman and Earman also only consider confirmatory evidence.
The case for treating two observable consequence on equal epistemological footing in (the reconstruction of) scientific inference has recently been defended in Parkkinen (2016).
A rare meeting of Bayes and Popper, see also Sect. 6.2.2 for disconfirmatory evidence.
There are two equivalent formulations, (1) \(P({\bar{c}}|{\bar{h}})\le P(c|h)\) and (2) \(P(c|h)+P(c|{\bar{h}})\ge 1\).
Let us simply assume for a moment that Poincaré was a Bayesian.
In statistical lingo, P(c|h) describes a rate of true positives (sensitivity) while \(P({\bar{c}}|{\bar{h}})\) describes a rate of true negatives (specificity). \(P(c|{\bar{h}})\ge P({\bar{c}}|h)\) may also be read as saying that an error of Type II is more or equally likely than a Type I error.
It is also conjectured that the two conditions \(P(c_1|h)=P(c_2|h)\) and \(P(c_1|{\bar{h}})=P(c_2|{\bar{h}})\) can be weakened into the single condition \(P(c_1|h)/P(c_1|{\bar{h}})=P(c_2|h)/P(c_2|{\bar{h}})\) and all observations for Scenario B continue to hold.
This also dovetails nicely with, as yet, our unpublished results on VET failure in the original topology of Bovens and Hartmann; Landes and Osimani (2018).
It must be admitted that during writing intuitions were also driven by legal reasoning; the hypothesis that a suspect committed the crime deductively entails that the suspect had a motive, an opportunity and the means.
References
Borm, G. F., Lemmers, O., Fransen, J., & Donders, R. (2009). The evidence provided by a single trial is less reliable than its statistical analysis suggests. Journal of Clinical Epidemiology, 62(7), 711–715.
Bovens, L., & Hartmann, S. (2002). Bayesian networks and the problem of unreliable instruments. Philosophy of Science, 69(1), 29–72.
Bovens, L., & Hartmann, S. (2003). Bayesian epistemology. Oxford: Oxford University Press.
Carnap, R. (1962). Logical foundations of probability (2nd ed.). Chicago: University of Chicago Press.
Claveau, F. (2011). Evidential variety as a source of credibility for causal inference: Beyond sharp designs and structural models. Journal of Economic Methodology, 18(3), 233–253.
Claveau, F. (2013). The independence condition in the variety-of-evidence thesis. Philosophy of Science, 80(1), 94–118.
Claveau, F., & Grenier, O. (2018). The variety-of-evidence thesis: A Bayesian exploration of its surprising failures. Synthese. https://doi.org/10.1007/s11229-017-1607-5.
Colombo, M., Gervais, R., & Sprenger, J. (2017). Introduction: Objectivity in science. Synthese, 194(12), 4641–4642.
Crupi, V., Nelson, J., Meder, B., Cevolani, G., & Tentori, K. (2019). Generalized information theory meets human cognition: Introducing a unified framework to model uncertainty and information search. http://philsci-archive.pitt.edu/14436/.
Csiszár, I. (2008). Axiomatic characterizations of information measures. Entropy, 10(3), 261–273.
Dawid, R., Hartmann, S., & Sprenger, J. (2015). The no alternatives argument. British Journal for the Philosophy of Science, 66(1), 213–234.
Earman, J. (1992). Bayes or bust? Cambridge, MA: MIT Press.
Fitelson, B. (1996). Wayne, Horwich, and evidential diversity. Philosophy of Science, 63(4), 652–660.
Fitelson, B. (2001). A bayesian account of independent evidence with applications. Philosophy of Science, 68(3), S123–S140.
Franklin, A., & Howson, C. (1984). Why do scientists prefer to vary their experiments? Studies in History and Philosophy of Science Part A, 15(1), 51–62.
Glymour, C. (1980). Theory and evidence. Princeton, NJ: Princeton University Press.
Hartmann, S., & Bovens, L. (2001). The Variety-of-Evidence Thesis and the reliability of instruments: A Bayesian-network approach. PhilSci-Archive, last modified: 07 Oct 2010 15:10.
Hempel, C. (1966). Philosophy of natural science. Englewood Cliffs, NJ: Prentice Hall.
Horwich, P. (1982). Probability and evidence. Cambridge: Cambridge University Press.
Horwich, P. (1998). Wittgensteinian Bayesianism. In M. Curd & J. A. Cover (Eds.), Philosophy of science: The central issues (pp. 607–624). New York: W. W. Norton & Company.
Howson, C., & Urbach, P. (2006). Scientific reasoning (3rd ed.). Chicago: Open Court.
Hüffmeier, J., Mazei, J., & Schultze, T. (2016). Reconceptualizing replication as a sequence of different studies: A replication typology. Journal of Experimental Social Psychology, 66, 81–92.
Keynes, J. M. (1921). A treatise on probability. New York: MacMillan.
Kuorikoski, J., & Marchionni, C. (2016). Evidential diversity and the triangulation of phenomena. Philosophy of Science, 83(2), 227–247.
Landes, J. (2018). Varied evidence and the elimination of hypotheses. Thought (submitted).
Landes, J., & Osimani, B. (2018). Varieties of error and varieties of evidence. Philosophy of Science (submitted).
Landes, J., Osimani, B., & Poellinger, R. (2018). Epistemology of causal inference in pharmacology. European Journal for Philosophy of Science, 8(1), 3–49. https://doi.org/10.1007/s13194-017-0169-1.
Landes, J., & Williamson, J. (2013). Objective Bayesianism and the maximum entropy principle. Entropy, 15(9), 3528–3591.
Lloyd, E. A. (2015). Model robustness as a confirmatory virtue: The case of climate science. Studies in History and Philosophy of Science Part A, 49, 58–68.
Maxim, L. D., Niebo, R., & Utell, M. J. (2014). Screening tests: A review with examples. Inhalation Toxicology, 26(13), 811–828.
Meehl, P. E. (1990). Appraising and amending theories: The strategy of Lakatosian defense and two principles that warrant it. Psychological Inquiry, 1(2), 108–141.
Myrvold, W. C. (1996). Bayesianism and diverse evidence: A reply to Andrew Wayne. Philosophy of Science, 63(4), 661.
Parkkinen, V.-P. (2016). Robustness and evidence of mechanisms in early experimental atherosclerosis research. Studies in History and Philosophy of Science Part C: Studies in History and Philosophy of Biological and Biomedical Sciences, 60, 44–55.
Pearl, J. (2009). Causality models, reasoning and inference (2nd edn.). Cambridge: Cambridge University Press.
Perović, S., Radovanović, S., Sikimić, V., & Berber, A. (2016). Optimal research team composition: Data envelopment analysis of Fermilab experiments. Scientometrics, 108(1), 83–111.
Perrin, J. (1924). Les Atomes. Libraire: Félix Alcan.
Poincaré, H. (1963). Mathematics and science: Last essays. New York: Dover.
Schupbach, J. N. (2015). Robustness, diversity of evidence, and probabilistic independence (pp. 305–316). Berlin: Springer.
Schupbach, J. N. (2018). Robustness analysis as explanatory reasoning. British Journal for the Philosophy of Science, 69(1), 275–300.
Shannon, C. (1948). A mathematical theory of communication. The Bell System Technical Journal, 27, 379–423.
Steel, D. (1996). Bayesianism and the value of diverse evidence. Philosophy of Science, 63(4), 666–674.
Stegenga, J., & Menon, T. (2017). Robustness and independent evidence. Philosophy of Science, 84(3), 414–435.
Tuomisto, H. (2010). A consistent terminology for quantifying species diversity? Yes, it does exist. Oecologia, 164(4), 853–860.
Vezér, M. A. (2017). Variety-of-evidence reasoning about the distant past. European Journal for Philosophy of Science, 7(2), 257–265.
Wayne, A. (1995). Bayesianism and diverse evidence. Philosophy of Science, 62(1), 111–121.
Acknowledgements
I would like to thank Stephan Hartmann, Barbara Osimani, Roland Poellinger and Christian Wallmann for very helpful comments and discussions. Thanks are also due to George Pólya for teaching me about reasoning by analogy and the value of limiting cases, both of which were most helpful for devising proofs. This work is supported by the European Research Council (Philosophy of Pharmacology: Safety, Statistical standards and Evidence Amalgamation, grant 639276).
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: Proofs of Main Results
We now give the longer proofs. The propositions to be proved are re-stated for ease of reference.
Proposition 1
For bodies of evidence \({\mathcal {E}},{\mathcal {E}}'\) with \(|{\mathcal {E}}|=|{\mathcal {E}}'|\) it holds that
Proof
Using that \(\sum _{C\in {\mathcal {C}}}|C|=|{\mathcal {E}}|\), we find
Hence, \(V_1({\mathcal {E}})\) is a positive-slope affine-linear transformation of \(V({\mathcal {E}})\). Positive-slope affine-linear transformation preserve ordinal comparisons. \(\square \)
Proposition 4
For all finite and not-empty sets\(\Omega \)and all probability functionswon\(\Omega \), \( \vec x:=\langle w(\omega ){:}\,\omega \in \Omega ,w(\omega )\in [0,1] \& \sum _{\omega }w(\omega )=1\rangle \), it holds that\(H(\vec x)<H(\frac{|\Omega |}{|\Omega |+1}\vec x,\frac{1}{|\Omega |+1}).\)
Proof
Using that \(\sum _{i=1}^{|\Omega |}x_i=1\) and that \(H(\vec x)\le -\log (\frac{1}{|\Omega |})=\log (|\Omega |)\) we find
\(\square \)
Theorem 1
In case of Condition A
Proof
To simply notation we let \({\vec {f}}\) denote the conjunction of all items evidence pertaining to the \(D_k\) and obtain:
Hence,
By (4) we have \(P(c_1|h)-P(c_1|{\bar{h}})<0\) and hence the proof is completed by noting that
\(\square \)
Theorem 2
Condition B and the Ceteris Paribus Conditions jointly entail that
Proof
We employ the Ceteris Paribus conditions and simply write P(c|h) and \(P(c|{\bar{h}})\); dropping the subscript of c. To simplify notation we let for \(j,l\in \{0,1\}\)
We begin by calculating the posterior probabilities in turn
recalling that the \(\chi \) were defined in (13). Similarly, we find for \(P_{{\mathcal {E}}'}(h)\) that
Note that the only difference between these two posteriors is that the first posterior contains the term \(P(e_{|C_1|}'|c^j_1)\) while the second posterior contains the term \(P(e_{|C_1|}|c^l_2)\).
The leading factors play no role, the sign of \(P_{{\mathcal {E}}}(h)-P_{{\mathcal {E}}'}(h)\) is thus equal to the sign of
The sign of this expression is equal to the sign of
To simplify notation we let for \(j,l\in \{0,1\}\)
We obtain the more manageable
Spelling this out we obtain
Fortunately, all those terms which do not contain \(\alpha _{00}\) nor \(\alpha _{11}\) (these are precisely those terms with \(P({\bar{c}}|{\bar{h}})P(c|{\bar{h}})P({\bar{c}}|{\bar{h}})P(c|{\bar{h}})\)) cancel out. Furthermore, all terms which contain \(\alpha _{00}^2\) and all terms containing \(\alpha _{11}^2\) cancel out.
For the terms containing \(\alpha _{00}\) and \(\alpha _{11}\) we find
Under the standing assumption that
we note that also these terms cancel out.
What remains is the much more manageable
We now investigate both factors in turn. We begin with the second and find that it is equal to
We note that
After some algebra we find for the negative of the first factor that
Since \(P(c|h)>P(c|{\bar{h}})\), we have \(P({\bar{c}}|{\bar{h}})=1-P(c|{\bar{h}})>1-P(c|h)=P({\bar{c}}|h)\). Thus, \(P({\bar{c}}|{\bar{h}})P(c|h)- P({\bar{c}}|h)P(c|{\bar{h}})>0\).
Using that \(Bf_{C_1}\alpha _{11}/\alpha _{00}=Bf_{{\mathcal {E}}}\), i.e., (10) holds, we find that
\(\square \)
Corollary 3
If\(E_1,\ldots ,E_{|C_1|}\)are the children of\(C_1\), then for all possible measurements\(E_1=e_1,\ldots ,E_{|C_1|}=e_{|C_1|}\)
Proof
The proof is a relatively simple exercise in Bayesian network calculations
Hence,
By (4) we have that \(P(c|{\bar{h}})<P(c|h)\) and that \(P({\bar{c}}|h)<P({\bar{c}}|{\bar{h}})\). Hence, the bracket is negative and it follows that
\(\square \)
Proposition 2
If Condition B, the last two ceteris paribus condition and if \(P(c_1|h)=1=P(c_2|h)\) hold, then
Proof
The proof is a simple exercise in Bayesian network calculations. First, we use (16) and (17) to obtain
Using that \(P(c_1|h)=1=P(c_2|h)\) and hence \(P({\bar{c}}_1|h)=0=P({\bar{c}}_2|h)\) the expressions simplify to
and
The claimed result follows by re-substituting the definitions of \(\chi _{1j}\) and \(\chi _{2l}\). \(\square \)
Proposition 3
If Condition B, the last two ceteris paribus condition,\(P(c_1|h)=1=P(c_2|h)\) and if \(P(e_{|C_1|}|c_1)= P(e_{|C_1|}'|c_2)>P(e_{|C_1|}'|{\bar{c}}_2)\)hold, then
where the\(\alpha \)and\(\beta \)are parameters independent of\(|C_1|\)which are defined in (19).
Proof
Under the assumption that \(P(e_{|C_1|}|c_1)= P(e_{|C_1|}'|c_2)\) the denominators are equal. Furthermore, the terms with \(j=l=1\) cancel out. We hence find
With (subscripts represent the values for j and l)
this becomes equal to
Note that the \(\alpha \) and the \(\beta \) parameters do not change value with varying \(|C_1|\) and are hence treated as constants.
Using that \(\alpha _{1,0}\cdot \beta _{1,0}<0\) and \(\prod _{n=1}^{|C_1|-1}P(e_n|{\bar{c}}_1)>0\) we find that
\(\square \)
Appendix 2: High Arity Variables
We now address the claim that the so-far established technical results, also hold for models which employ higher arity hypothesis and/or consequence variables.
Denote by \(h^2,h^3,\ldots \) the values of H different from h and by \(c^2,c^3,\ldots \) the values of consequence variable C different from c. \(h^0\) is the (possibly infinite) disjunction of the \(h^i\) with \(i\ge 2\). \(c^0\) is the (possibly infinite) disjunction of the \(c^k\) with \(k\ge 2\). To simplify notation we let \(P(c_j^0|h^i):=1-P(c_j|h^i)\).
The ceteris paribus conditions are that for every consequence variables \(C_j\) and every evidence variable E pertaining to the consequence variable \(C_j\) it holds that
This formalises the thought that all values of the hypothesis variable H different from h are equal. Furthermore, the conditional probability of the evidence does not depend on the particular value \(c^k\). We can hence define our Bayes factors in the usual way unequivocally as \(P(e|c)/P(e|{\bar{c}})\).
Proposition 5
If (20) and (21) hold, then Corollarys 1and 2hold for higher arities, too.
Proof
Using the ceteris paribus condition for the second and third equality, we find
To complete the proof it suffices make the following observations:
- 1.
Whenever the term ‘\(P(e|{\bar{c}}_j)\cdot P({\bar{c}}_j|{\bar{h}})\cdot P({\bar{h}})\)’ appears in a proof for the basic model it is replaced by the term ‘\(\sum _{i=2}\sum _{k=2}P(h^i)\cdot P(c^k_j|h^i)\cdot P(e|c^k_j)\)’. Since both terms are equal, no new difficulties arise.
- 2.
The terms of the form ‘\(P({\bar{c}}_j|{\bar{h}})\)’ for \(i\ge 2\) are replaced by terms of the form \(\sum _{k\ge 2}P(c_j^k|h^i)\)’. The latter is equal to \(P(c_j^0|h^i)\). By the first ceteris paribus assumption for greater arities, these terms are equal. \(\square \)
Rights and permissions
About this article
Cite this article
Landes, J. Variety of Evidence. Erkenn 85, 183–223 (2020). https://doi.org/10.1007/s10670-018-0024-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10670-018-0024-6