Sunday, August 3, 2014

A Review of Math on Trial by Leila Schneps and Coralie Colmez

In my previous essay, I pontificated on the importance of mathematics and suggested several possible underlying deficiencies in mathematics education that has led to a general public (particularly in America, though it remains true throughout the world) which is woefully unprepared to engage with the mathematical challenges we all face in our day to day lives.

Today, I return to this topic in the form of a review of Math on Trial: How Numbers get Used and Abused in the Courtroom by Leila Schneps and Coralie Colmez, a mother-daughter team of mathematicians and members of the Bayes and the Law Research Consortium, an international organization of mathematicians dedicated to the construction of a set of criteria for the proper use of probabilities in courts of law in order to avoid miscarriages of justice.  Their book is certainly related to that quest, as it is essentially a catalogue of miscarriages of justice committed in the name of mathematics by people who failed to understand the nuance of the mathematics they were erroneously using.

Each of the book’s ten chapters presents a different case study intended to help the reader explore some mathematical error which has affected legal proceedings.  The cases range from the historic to the current and cover both criminal trials and civil affairs.

Students of mathematics who read this book will find little mathematical knowledge they do not already possess.  Indeed, many of the mathematical lessons are so simple (mathematicians might say elementary or obvious) as noting that it is improper to multiply non-independent probabilities.  For instance, that exact error comes up in the book’s first chapter, in which Sally Clark was accused (and later convicted) of murdering her two children.  Her defense was simple: the children, tragically, were victims of cot death, not of murder.  There was no medical evidence to the contrary, but a pediatrician, Roy Meadow, calculated that the odds of a single family experiencing two cot deaths was 1 in 73 million.  Thus it was argued that Sally Clark’s probability of innocence was 1 in 73 million.  However, as the book points out, this is a gross misuse of statistics.  He obtained the figure by squaring the odds (about 1 in 8500) of cot death under similar circumstances.  It seems reasonable.  We know that to determine the odds of two separate events, we multiply the probabilities together.  The odds of one cot death are 1 in 8500, so the odds of two are (1/8500)^2, or just under 1 in 73 million.  This elementary error, however, assumes the probabilities are independent.  Unfortunately, a family who suffers one cot death is in fact more likely to suffer another.  This could be due to environmental or genetic influences, but certainly does not point to murder.  Similarly, even if the calculation of the probability were correct, its interpretation was incorrect.  The value of 1 in 73 million was not the probability of innocence, but a calculation of the number of people for whom those conditions would be true.  They got the math wrong, and Sally Clark spent several years in prison, wrongfully accused of the worst crime, before the mistake was corrected.

However obvious the mathematical lessons might be to students of mathematics, the lessons on law will likely be eye-opening.  I suspect there are many mathematicians who remain unaware of how large a problem institutional misunderstanding of mathematics has become in the judicial system, and Schneps and Colmez provide a succinct primer.

Similarly, students of law (or those generally interested in criminal trials) may be very well aware of the cases mentioned in the book.  I suspect even most members of the general public are at least peripherally aware of cases such as the Amanda Knox murder trial or the Alfred Dreyfus affair of the 1890s (if you’ve forgotten the name of the latter, your memory might be jogged by recalling the famous open letter published in a French newspaper in 1898 by Emile Zola entitled “J’Accuse…!”).  However, these people who maintain a knowledge of the law might not be expected to have great depth of understanding in mathematics.

For both groups of people, as well as for those who would seek to expand their knowledge of both fields of inquiry simultaneously, this book is highly recommended.  While its depth of analysis in both mathematics and law is minimal (no reader will ever become an expert on the basis of this short book’s treatments), it provides an important introductory text.  It would behoove members of the legal profession in particular to heed its warnings about misuse of mathematics in the courtroom, because lives really do hang in the balance.

In the book’s concluding chapter, the authors mention the argument by Lawrence Tribe (in his article, “Trial by Mathematics: Precision and Ritual in the Legal Process”) that mathematical argumentation actually does not belong in trials at all.  However sympathetic they seem to his argument (which essentially hinges on the notion that juries are ill-equipped to handle mathematical arguments and should instead be expected to employ a more heuristic approach to determining guilt), they correctly point out that the advent of DNA forensics has rendered this argument moot.  Probabilistic arguments will and must now appear before juries if DNA is to remain in use as a forensic tool.  Since dispensing with DNA seems neither likely nor a good idea, we will continue to use mathematics.  Therefore, it is argued, a greater mathematical literacy amongst both legal professionals and the general public (from which juries are selected) is necessary, as is the development (as is the Bayes and the Law Research Consortium’s goal) of a set of criteria for the allowable use of probabilistic arguments in trials.

I, for one, find the latter to be a worthy goal indeed, but do not at all feel sympathetic to Tribe’s argument in the first place.  I remain of a mind that mathematics in the courtroom is not only necessitated by the advent of more advanced forensic techniques, but should be generally encouraged as providing one more set of tools for the determination of truth.  Tribe certainly is correct in his argument that this presents unique challenges, but I believe these are challenges we must face head-on with greater access to high-quality education in mathematics and the statistical sciences.  It would not serve justice to ignore an entire branch of evidence simply because it is thought too confusing for jurors.

Of course it is true that many jurors will lack the mathematical background to properly evaluate some of these arguments.  However, the same can be said of any particular branch of expert testimony.  Jurors are expected to become educated--in a relatively short time--on matters of fingerprint identification, handwriting analysis, genetics, psychology, and any number of complex disciplines whose experts may be qualified to speak with authority on the evidence in any particular trial.  Greater education amongst the public in any number of fields is to be desired, but the more important and more immediate solution is immensely greater education within the legal profession so that prosecutors and defense attorneys can confidently analyze each others’ arguments and know which experts to call when the edges of their own mathematical abilities are probed.  It becomes the duty of these attorneys and their expert witnesses to educate the jury.

Still, despite some minor disagreements, the book holds great value.  Students of mathematics should read it to better understand their field’s application, and students of law should be required to read it to better understand how their colleagues have made devastating (if often subtle) mathematical errors.  And of course, members of the general public should read it simply to remain informed about world events and the precarious nature of human liberty when courts of law fail to understand their own evidence.

Some readers will find certain errors, inconsistencies, or disagreements.  The errors are unfortunate but fail to impact the overall message of the book and so might be forgiven.  Readers may also disagree with the authors’ interpretation of some events, especially since legal analysis is minimal throughout the book.  This is to be expected.  While I do not necessarily disagree with the authors on any particular point, even if I disagreed with their interpretation of all ten case studies, the important points of mathematical error within those cases would remain unchanged, and so the book’s value would remain unaltered.


The book is available for purchase through Amazon here, or at your favorite bookstore.

No comments: