False samples are not the same as blind controls

Informal efforts to “test” a laboratory corrupt the data stream, where integrity is crucial.

Sir

As director of the laboratory that analysed the samples for the National Lynx Survey sponsored by the United States Department of Agriculture's Forest Service (USFS), I am responding to the damaging yet unsubstantiated conclusions in your Opinion article “Lynch mob turns on lynx researchers”¹ and News article “Fur flies over lynx survey's suspect samples”² about sampling for Canada lynx (Lynx canadensis).

My laboratory developed the protocol to distinguish among species of North American felids based on DNA analysis of hairs using known individuals across the geographical range of the target species, and comprehensively validated it using blind control tests both within our lab and at another lab. Our protocol was peer-reviewed and published³.

In practice, positive and negative controls are used with every sample set analysed. We have never misidentified a species when the identity of the sample was known, and we consistently detect lynx in areas where they are known to exist. In contrast to the implications of your Opinion article, therefore, the laboratory procedure to distinguish lynx based on hair samples is fully diagnostic and validated.

In 1999, the USFS implemented the National Lynx Survey to evaluate systematically the distribution of lynx on federal lands across 12 states; species identification based on DNA analysis was the final step. I was principal investigator in charge of DNA analysis, and Kevin McKelvey from the Rocky Mountain Research Station (USFS) was the principal investigator for the nationwide survey protocol. Field personnel (several hundred permanent and temporary federal and state employees) collected hairs from over 13,000 rub pads⁴ and sent them to the lab in labelled vials with accompanying data sheets. Because integrity of the data stream is particularly crucial in a study of such enormous scope and scale, the protocol was explicit, with comprehensive written instructions for all aspects of gathering, labelling and submitting samples.

For a field worker to arbitrarily decide “to test the lab” by labelling a hair from elsewhere as if it were a field-collected sample corrupts the integrity of the data and does not constitute a blind control. By analogy, a medical field worker who surreptitiously contaminates a blood sample to “test the rigour” of the serological protocol during a wide-scale survey to assay a disease is not conducting a blind control but is fabricating data that could lead to false conclusions about disease distribution.

I do not know the field personnel who submitted false samples and cannot assume anything about their motivation. But the vast majority of those participating in this unprecedented large-scale survey followed scientific protocol. The few who misrepresented data without notifying those who analyse and interpret the data did not, as your Opinion article suggested, merely implement blind controls.

Nobody contacted either me or the USFS research scientist in charge of the survey protocol about any concerns or told us about the plan to misrepresent data. In fact, we learned about the 1999 false samples only after the start of an internal USFS investigation.

I agree with your Opinion article that “Clean data are needed to prevent years of court battles over use of the forests”¹. In this case, the methodology was peer-reviewed, published, explicit, complete and followed by all except those few who intentionally mislabelled samples. The only potential smudge on the 'cleanliness' of data in the National Lynx Survey at this point comes from submission of samples not in compliance with scientific protocol.

Our Letter published on page 520 of this issue⁵ is not part of the National Lynx survey; all 599 samples used in ref. 5 were collected from freezers and other known sources.