03 July 2014

Cot death, Sally Clark and the prosecutor's fallacy

Sally Clark was an English solicitor who was prosecuted for the murder of two of her infant children. Her defence was that the deaths were both “cot death”, also known as “Sudden Infant Death Syndrome” or SIDS.

She was initially convicted. Much weight was given to the evidence of a medical expert, Sir Roy Meadow. He was a paediatrician and an expert in child abuse. He wasn’t a statistician. Using statistics from reports about SIDS, he estimated the chance of a cot death was 1:8543, and therefore the chance of a second was about 1:73,000,000. He arrived at this latter figure by multiplying 8543 by itself, squaring it.

Her conviction was upheld at a first appeal. However, at a second appeal she was acquitted, not because the statistics and calculations were incorrect (which they were), but because there was evidence of bacterial infection in the second child, information which was previously withheld from the defence. Sadly, the story does not have a happy ending; Sally Clark was found dead from acute alcoholic poisoning about four years after her acquittal.

I followed the first trial in the newspapers. I remember being distinctly disturbed by what I read; an intuition if you like, that there was something wrong, even though I understood that intuition is often a very poor guide to the results of mathematical calculations. It has taken me to now to understand what was going on.

I was reminded of the case of Sally Clark by reading Math on Trial by the mother and daughter Leila Schneps and Coralie Colmez. They are both mathematicians, and their book concentrates on the uses and abuses of maths in court cases. As such, it concentrates on the mathematics used in trials; the mathematics are often statistics and probability, and at times I found it really hard going.

SIDS


SIDS is a description, not a cause. It is applied to cases of natural death in infants, where no definite cause has been found. The causes of SIDS are usually felt to be a combination of genetic and environmental factors, even if these are either unknown or poorly understood. It hardly needs saying, that even if the cause or causes or SIDS are not known, this does not mean that there are no causes.

At the time of her first trial there were known associations with SIDS, such as smoking, maternal age and unemployment. As ever, an association is nothing more than that, it is not causation. Further, it was found that SIDS was rare in Hong Kong; the explanation was that there infants sleep on their backs, rather than face down. The subsequent “Back to Sleep” is said to have reduced the SIDS rate by about 50%. Again, the sleeping position is an association, not a causation. (The mechanism may relate to breathing patterns and brain wave activity which differ depending on the infants’ sleeping position.)

Murder is the unlawful killing of a human with malice aforethought (that is, intent). Usually, the prosecution must actively show and prove the means, method and opportunity of the perpetrator.

The Trial


Essentially, the prosecution argument was: either the deaths were due to SIDS, or murder. We will show, they implied, that SIDS is so mathematically unlikely, that the infants must have been murdered. That is, we will prove her guilt, not by proving her guilt, but by proving that she could not be innocent. This is a reversal of the usual ‘innocent until proven guilty’ rule. This distinction seems not to have been apparent at the time.

Professor Sir Roy Meadow was a paediatrician, famous for his descriptions of Munchausen’s Syndrome by Proxy, and child abuse. The original Munchausen’s Syndrome was described by Richard Asher; he found that there were patients who would self-harm to gain attention and satisfy their inner desires. Sir Roy extended this to parents (usually mothers) who harmed their children to satisfy their (maternal) needs. In child abuse, he is said to have coined “Meadow’s Law”, which is that “one death is a tragedy, two are suspicious, and three are murder”. This is almost a restatement of the quotation attributed to Joseph Stalin, that “one death is a tragedy, a million is a statistic”.

Sir Roy produced the 1:8543 incidence from a report; he did not accept the rather different figures from a similar report. This incidence relates to people in Sally Clark’s socio-economic group, and who aren’t smokers, unemployed or below 27 years of age. Boys are also more likely to die of SIDS than girls, compounding the overall possibilities. The real overall incidence of SIDS at that time seems to be 1:1300. “Seems” because the Back to Sleep campaign may have had an influence, and because if a registered cause of death is given as SIDS, and later information indicates a different diagnosis, the Registrar-General’s records will be changed. A subsequent search of the records may therefore not be complete, and reports and statistics on SIDS are often compiled retrospectively.

Sir Roy squared the 1:8543 figure to get 1;73,000,000. By itself this arithmetic is correct. But the (hidden) assumption is that two cot deaths are entirely unrelated, and this is incorrect. They are neither independent or “non-conditional”,  rather they are two events in which common factors (genetic and environmental) cannot be excluded Technically, two such events are described as a conditional probability. The mathematics underlying this is really very difficult for a non-expert.

Neither Sir Roy or the prosecution produced any statistics about double murders of infants.  The correct use of statistics in this case would be a comparison of the probabilities of double murder and the probabilities of double SIDS, not the presentation of only one “side” of the argument. There are other possibilities, such as one murder and one SIDS.

This misuse of statistics is known as the prosecutor’s fallacy, and is an example of Berkson’s paradox. This paradox arises when conditional probability is mistaken for unconditional probability.

Determining how often a second infant dies of SIDS is not easy. The best evidence I can find suggests that there is a dependency of between 5 and 10 in the case of a second death; that is, there is such an increased chance of a second death from SIDS. Intuitively, I would expect that a second death would be more common after a first, even if I could not quantify it. I would base this intuition on similar genetic and environmental conditions. Further, the possibility of a second murder does seem to be much less common after a first, and much less common than a double SIDS. In part, this may be because the murderer is likely to be imprisoned.

The figures of “5” and “10” have been produced from data relating to the incidence of SIDS. They have not been produced by “pure” mathematics, that is where there is a known correlation between two things, this has been factored in.

Evidence of the frequency of double murder was not given at the first trial; if it had, and a proper comparison of the rates of double murder and SIDS had been made available, it is very unlikely that she would have been convicted on statistical evidence alone. The medical evidence was not conclusive, showing no definite evidence of abuse or the lack of abuse. The infection that the second infant had was not disclosed; ultimately this proved vital.

If you are like me, you are probably finding all this statistical stuff confusing. Let us go through the logic and statistics step-by-step. I have rounded the figures to make the calculations more obvious.

The 1:8543 ratio (hereafter 1:8000) refers to a subset of people whose babies had died from SIDS. About 400,000 babies are born in the UK every year. Thus, if we take 1:8000 and divide this into 400,000 we find that 50 babies die of SIDS in a year. But, it is known that around 200 babies die of SIDS annually. We can only use the figure of 1:8000 if we know what subset of 400,000 it refers to—and we don’t. Overall, 200 babies in a population of 400,000 is 1:2000. So, you might say that Sally Clark’s subset had only a quarter of cases of SIDS compared to the general population. This isn’t relevant.

Around 20 to 30 infants were known to have been murdered annually. This represents 1:20,000 to (about) 1:13300. Thus, an infant is more likely to die from SIDS than to be murdered by a factor between 7 and 10.

Sir Roy squared 1:8000 to get 1:64,000,000 (actually 1:8543 to get 1:73 million). If we square 1:2000 we get 4,000,000. Likewise from 1:20,000 we get 1:400,000,000; and from 1:13300 we get about 177,000,000. Now, which is the most likely scenario?

So, even if we treat the two events as totally independent, which they aren’t, we can still show that SIDS is more likely than murder. We should realise that the events aren’t independent, but we cannot anyway easily quantify this. We might say that the dependency between two cases of SIDS and two murders are likely to be similar; but this is an unsupported supposition.


Repercussions


This picture, via Reuters, shows Sally Clark at the time of her release from prison; it speaks for itself: