White House Report Concludes That Bite-Mark Analysis Is Junk Science

The President’s Council of Advisors on Science and Technology has concluded that forensic bite-mark evidence is not scientifically valid and is unlikely ever to be validated, according to a draft report obtained by The Intercept. The report, titled “Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods,” is marked as a “predecisional” draft created August 26 that is not to be quoted or distributed, though the title page suggests the report will be made public sometime this month.

The report reviews a handful of common forensic practices, so called feature-comparison disciplines, or pattern-matching practices — bite-mark analysis, fingerprint and firearm analysis, shoe tread analysis, and DNA mixture analysis — each of which involves an “expert” looking at a piece of evidence and eyeballing whether it matches a particular image, person, or object. The report discusses whether each practice has been scientifically validated, what it would take to do so, and how each practice should be used in the courtroom — if at all.

In the case of bite-mark evidence, the report is especially critical. “PCAST finds that bitemark analysis does not meet the scientific standards for foundational validity, and is far from meeting such standards,” it reads. “To the contrary, available scientific evidence strongly suggests that examiners cannot consistently agree on whether an injury is a human bitemark and cannot identify the source of [a] bitemark with reasonable accuracy.”

Bite-mark analysis is conducted by forensic dentists and relies on two foundational premises: first, that human dentition is unique — as unique as DNA ­— and second, that human skin (or another malleable substrate) is a suitable medium on which to record such an impression. The problem is that neither premise has been proved. Nonetheless, bite-mark analysis has been used in criminal cases to match individuals to alleged bites ­since the 1950s, when Texas’s highest criminal court cleared the way for its use (in that case, a dentist claimed that a bite mark left in a piece of cheese found at the scene of a grocery burglary matched the teeth of a particular man).

Although bite-mark matching is nowhere near as ubiquitous as, say, fingerprint analysis in the criminal justice system, it has left a string of victims in its wake. To date, 25 wrongful arrests or convictions have been linked to bite-mark analysis and additional cases are still pending in the courts, according to the Innocence Project. In June, Bill Richards was freed from prison after spending nearly 23 years behind bars for the murder of his wife in California based on the erroneous testimony of a forensic dentist who said a wound found on his wife Pamela’s hand was a match to Richards’s supposedly unusual dentition.

Despite their poor track record, many forensic dentists ­— most associated with the American Board of Forensic Odontology, the discipline’s certifying body ­— have remained rabidly defensive of their practice and the notion that it is a righteous undertaking that helps put criminals behind bars. An integral part of their defense has been to lash out at critics ­—even those among their own ranks. The group unsuccessfully tried to expel Dr. C. Michael Bowers from the American Academy of Forensic Sciences ­— the nation’s preeminent umbrella organization for forensic practitioners ­— on a trumped-up ethics charge. And they have attacked both professionally and personally Dr. Mary Bush and her husband, Peter, who have conducted significant research demonstrating the practice’s lack of a scientific underpinning. Notably, the PCAST report gives a distinct nod to the Bushes’ work in determining that there is no actual science behind bite-mark analysis.

While the report acknowledges the dentists’ concern that excluding bite-mark evidence “could hamper efforts to convict criminals in some cases,” it rejects the idea that this alone is sufficient to warrant the practice. If the dentists’ concerns are valid, “the correct solution, from a scientific perspective, would not be to admit expert testimony based on invalid and unreliable methods, but rather to attempt to develop scientifically valid methods,” it reads. “However, PCAST considers the prospects of developing bitemark analysis into a scientifically valid method to be low. We advise against devoting significant resources to such efforts.”

Pattern Matching, Subjectivity, and Junk Science

President Obama created the advisory council in 2009 as a means of gathering scientific input that could impact policy decisions. Although the practice of presidents naming scientific advisers dates back to Franklin D. Roosevelt’s administration, the timing of the creation of Obama’s council was particularly notable, coming just roughly two months after the release of a groundbreaking report from the National Academy of Sciences National Research Council, which cast a long shadow over a host of pattern-matching forensic disciplines that have been used for decades in criminal cases. That report ­— “Strengthening Forensic Science in the United States: A Path Forward” ­— noted a lack of scientific underpinning in nearly every forensic practice, except for DNA, and called for research to be done to shore up the disciplines.

In the wake of the NAS report, the government has taken action ­— a National Commission on Forensic Science was created, and a number of scientific working groups have undertaken the arduous task of coordinating the development of standards and guidelines for various forensic disciplines. PCAST was asked by Obama in September 2015 to see what additional efforts might be undertaken to aid the effort to instill science into forensics. Upon review, PCAST concluded “that the most useful contribution it could make would be to add clarity on the scientific meaning of ‘reliable principles and methods’ and ‘scientific validity’ in the context of certain forensic disciplines.”

Scientific validity and reliability mandate that each method be subjected to empirical testing in order to determine meaningful error rates. “Without appropriate estimates of accuracy, an examiner’s statement that two samples are similar ­— or even indistinguishable ­— is scientifically meaningless: it has no probative value and considerable potential for prejudicial impact,” reads the report. “Nothing ­— not training, personal experience nor professional practices ­— can substitute for adequate empirical demonstration of accuracy.”

To be fair, bite-mark analysis is not the only practice criticized by PCAST. Fingerprint matching has only recently been put to rigorous testing (it is “distressing” that it has taken so long to do so, the report notes), and two such studies produced high error rates ­— in one, an error rate of 1 in 18 was found, in the other 1 in 306. The report recommends that jurors be informed of the studies and their results.

Where firearm analysis is concerned, the group notes early studies indicate that examiners can, “under some circumstances,” connect spent ammunition to the gun from which it was fired, but also reports that just one appropriate study has so far been conducted and that other “frequently cited studies seriously underestimate the false positive rate.” Importantly, for both fingerprints and firearms analysis, the group writes that transforming the practice from a subjective one to an objective one is preferable, by automating the disciplines through “machine learning” image analysis.

Regarding footwear analysis ­— matching a shoe tread pattern left at a crime scene to a particular shoe based on “randomly acquired characteristics,” such as cuts and gouges on a shoe’s sole ­— the PCAST finds that there are no studies that demonstrate the practice is based on actual science. “Such conclusions are unsupported by any meaningful evidence or estimates of their accuracy and thus are not scientifically valid.”

And while PCAST finds that single-source and simple mixture DNA analysis rests firmly on a scientific foundation, the analysis of complex mixtures requires more validation. Complex mixtures involve DNA samples with two or more unknown sources and result in a profile that “superimposes multiple individual DNA profiles.” In such cases, an analyst has to visually inspect the results and make a subjective determination about whether a particular suspect is present in the sample. And that determination can become clouded by outside knowledge of the particular case at hand, the report notes. The creation of “probabilistic genotyping software” and the testing of such software would go a long way to curing the problem. (The PCAST report notes that there are problems associated with other forensic practices it has not reviewed ­— singling out cases of alleged arson and prosecutions based on shaken baby syndrome ­— that also “require urgent attention.”)

Predictably, reaction to the draft report by the National District Attorneys Association (NDAA) was swift. On September 2 ­— the day after the PCAST reportedly voted to approve it ­— the group issued a press release blasting the PCAST — and, seemingly missing the point the PCAST is trying to make. “The forensic disciplines that the PCAST authors attack are (and have been) reliably used every day by investigators, prosecutors, and defense attorneys across the United States to aid in both exonerating the innocent and convicting the guilty,” the group wrote. “The PCAST position regarding the use of forensic science is scientifically irresponsible. Adopting any of their recommendations would have a devastating effect on the ability of law enforcement, prosecutors and the defense bar to fully investigate their cases, exclude innocent suspects, implicate the guilty and achieve true justice at trial.”

NDAA president Mike Ramos, the elected district attorney in San Bernardino County, California, said that the NDAA will “continue to serve the citizens of this great nation” in part by defending the criminal justice system “against those who would seek to undermine” it.

Notably, it was under Ramos’s leadership that Bill Richards was wrongfully convicted of his wife’s murder; his office tried Richards three times before finally getting a conviction, and did so only after the questionable bite-mark evidence was introduced. After the California Supreme Court in late May threw out that conviction, which it determined was based on junk science, Ramos’s office made it clear to defense attorneys that they would seek to try Richards again. Ultimately, Ramos’s office dismissed the case, and Richards was finally freed.

“As a man who lost his life behind the false science of bite marks, I feel the proposed action of eliminating its use in the courtroom is long overdue,” Richards wrote in an email to The Intercept. “The desire to … convict appears to blind experts who don’t consider that some defendants are innocent. No subjective ‘science’ should be used in any courtroom. The witch hunt mentality takes over and they find matches that are not there, sending innocent people to prison.”