Reply paperResponse paper to “The likelihood of encapsulating all uncertainty”: The relevance of additional information for the LR☆
Introduction
In this short paper we will use the opportunity to react to the various response papers that have been published to the papers in the Virtual Special Issue “Measuring and Reporting the Precision of Forensic Likelihood Ratios”. While we largely agree with Dawid [3] and Biedermann et al. [4], we think it is worthwhile to comment on Martire et al. [2].
Martire et al. state that the assignment of probability “is a mental operation subject to all the frailties of human memory, perception and judgment”. While this is undoubtedly true, this is not related to the issue of whether or not to accept probability to be subjective or not. The issue is not about “The presentation of impression, beliefs or ‘guesses’” versus “objectively true results”, or about guessing numbers and claiming they can't be wrong. It is, in our opinion, about the philosophical interpretation of the relevant probabilities: as subjective probabilities, or as frequencies. Before we proceed, let us emphasize that we agree with the authors on many counts and we elaborate on these topics first.
We completely agree with Martire et al. [2] that an expert assessment of the weight of evidence should be done in a way which is transparent for the recipient. We have argued that conceptually, a likelihood ratio (LR) is a single number [1]. This does not, however, imply that the LR is the only thing which should be communicated to a court by the forensic expert, and we welcome the opportunity to clarify this. For an LR to be meaningful, the hypotheses should be clearly stated, as should be the evidential data that have been taken into account. In order for the results to be open for criticism and challenge, the statistical/probabilistic model that has been used to calculate the LR, the assumptions that have been made, the necessary parameter estimates etc., should also be made available (directly or upon request). The forensic experts should explain their choice of statistical model and the assumptions that they made so that the appropriateness of the model can be judged. Different experts may apply different models, and consequently arrive at different likelihood ratios, even if they considered identical evidence. A likelihood ratio value alone cannot be examined, and therefore the road towards it must also be made explicit.
We also agree with Martire et al. [2] that biases must be avoided in the evaluation process, and that it can certainly be useful to have evidence evaluated by multiple qualified experts. We agree that there are various pitfalls and difficulties and that it is important, as always in science, to be critical and not to forget common sense.
Section snippets
Subjective probability
An evaluation of a likelihood ratio necessitates the evaluation of two probabilities. In general we think that these probabilities can only be meaningfully interpreted as subjective probabilities. There are several reasons why we take this position, the main one is that we need to be able to deal with probabilities associated with single events that can only with great difficulty be imagined as coming from a repeated experiment. We believe we have no choice but to accept the consequences of the
Martire et al. example: different experts with different interpretations
Martire et al. give an example where two experts have access to the same data, yet use it differently [2]. By construction, they happen to arrive at the same likelihood ratio, but the change in LR due the introduction of new data is different for both experts. The example is meant to illustrate that different experts may obtain a likelihood ratio via different interpretations of the evidence, and that the weight of evidence obtained may be altered differently if new data are disclosed to both
Treatment of new data
Now let us consider what could happen when more data are made available. These data may invalidate the equivalence between E1 and E2 by containing a record of a person from the UK who uses ‘color’ or a record from someone who uses ‘colour’ but who is not from the UK. In the example, this does not happen, and 50 persons from the UK using ‘colour’ as well as 50 others, not from the UK and using ‘color’ are added. Martire et al. show that in that case, expert A updates his LR to 67 and expert B
Interpretation of the frequency distribution of the shared characteristic
In the example in [1] where an LR is associated with a shared characteristic, we worked out the situation where one starts with a (purely subjective) β(1, 1) prior. Obtaining m samples with, and n without the characteristic updates our distribution to a β(m + 1, n + 1) distribution. The expectation and variance of this distribution determine our LR, which is equal to (m + n + 3) / (m + 2).
One may wonder what would be the meaning for the forensic scientist – and possibly a court – of the distribution β(m + 1,
Conclusion
We can see several things from this example. The first one is that in expectation it pays to obtain more information in the sense that we expect an LR further away from one. This is because, if we have more data, we expect the frequency of the characteristic to remain the same but it will be based on more data and hence the probability distribution modelling the frequency will be more concentrated around its mean and have a smaller variance. Of course the LR only increases in expectation, we
References (4)
- et al.
The LR does not exist
Sci. Justice
(2016) - et al.
On the likelihood of “encapsulating all uncertainty”
Sci. Justice
(2017)
Cited by (5)
Formal description of the probabilistic models
2020, Forensic Practitioner's Guide to the Interpretation of Complex DNA ProfilesInformation economics in the criminal standard of proof
2022, Law, Probability and RiskThe Limits of Bayesian Thinking in Court
2020, Topics in Cognitive ScienceForensic Practitioner's Guide to the Interpretation of Complex DNA Profiles
2020, Forensic Practitioner's Guide to the Interpretation of Complex DNA ProfilesAn epistemic interpretation of the posterior likelihood ratio distribution
2020, Law, Probability and Risk
- ☆
This paper is part of the Virtual Special Issue entitled: Measuring and Reporting the Precision of Forensic Likelihood Ratios, [http://www.sciencedirect.com/science/journal/13550306/vsi], Guest Edited by G. S. Morrison.