Type I and Type II Errors - Making Mistakes in the Justice System

[Anonymous]

An explanation of the law system of putting someone on trial, in our Occidental society, with parallels to the statistical method of Type I and Type II errors.

= = = = = = = = = = = =

Type I and Type II Errors - Making Mistakes in the Justice System

Ever wonder how someone in America can be arrested if they really are presumed innocent, why a defendant is found not guilty instead of innocent, or why Americans put up with a justice system which sometimes allows criminals to go free on technicalities? These questions can be understood by understanding the similarity of the American justice system to hypothesis testing in statistics and the two types of errors it can produce. (This discussion assumes that the reader has at least been introduced to the normal distribution and its use in hypothesis testing. Also please note that the American justice system is used for convenience. Others are similar in nature such as the British system which inspired the American system)

True, the trial process does not use numerical values as does hypothesis testing in statistics, but both the justice system and statistical versions of hypothesis testing  share at least three common elements (other than a lot of jargon that sounds like double talk):

   1. The alternative hypothesis - This is the reason a criminal is arrested. Obviously the police don't think the arrested person is innocent or they wouldn't arrest him. In statistics the alternative hypothesis is the hypothesis the researchers wish to evaluate.
   2. The null hypothesis - In the criminal justice system this is the presumption of innocence. In both the judicial system and statistics the null hypothesis indicates that the suspect or treatment didn't do anything. The null is the logical opposite of the the alternative. For example "not white" is the logical opposite of white. Colors such as red, blue and green as well as black all qualify as "not white".
   3. A standard of judgment - In the justice system and statistics there is no possibility of absolute proof and so a standard has to be set for rejecting the null hypothesis. In the justice system the standard is "a reasonable doubt". The null hypothesis has to be rejected beyond a reasonable doubt. In statistics the standard is the maximum acceptable probability that the effect is due to random variability in the data rather than the cause being investigated. This standard is often set at 5% which is called the alpha level.

It only takes one good piece of evidence to send a hypothesis down in flames but an endless amount to prove it. If the null is rejected then logically the alternative hypothesis is accepted. This is why both the justice system and statistics concentrate on disproving or rejecting the null hypothesis rather than proving the alternative. It's much easier to do. If a jury rejects the presumption of innocence, the defendant is pronounced guilty.

Unfortunately, neither the legal system or statistical testing are perfect. A jury sometimes makes an error and an innocent person goes to jail. Statisticians, being highly imaginative, call this a type I error. Civilians call it a travesty.

In the justice system, failure to reject the presumption of innocence gives the defendant a not guilty verdict. This means only that the standard for rejecting innocence was not met. It does not mean the person really is innocent. It would take an endless amount of evidence to actually prove the null hypothesis of innocence.

Sometimes, guilty people are set free. Statisticians have given this error the highly imaginative name, type II error.

Americans find type II errors disturbing but not as horrifying as type I errors. A type I error means that not only has an innocent person been sent to jail but the truly guilty person has also gone free. In a sense, a type I error is twice as bad as a type II error. Needless to say, the American justice system puts a lot of emphasis on avoiding type I errors.  This emphasis on avoiding type I errors, however, is not true in all cases where hypothesis testing is done.

[attachment=0:3jv0lt98]chartlaw.png[/attachment:3jv0lt98]

It would be nice to completely eliminate both error types but it can't be done. In fact, reducing one causes the other to become higher. This happens because the appearance of guilt and innocence are not clear cut. Innocent people can appear to be guilty and guilty people appear to be innocent.

The normal distribution as shown in figure 1 can be used to represent the variability in the appearance of guilt for the population of  innocent suspects. In a homicide investigation, for example, just about anyone who is a friend, relative, associate, or was in the immediate vicinity of the crime will initially be considered a suspect.

figure 1. Possible Suspects Who Are Innocent

 People represented by the left hand tail would be either criminals in jail who couldn't have committed the crime or super upstanding citizens who would virtually never be accused in the first place. Those represented by the right tail would be the "usual suspects", innocent people who appear to be guilty such as hot heads, career criminals, or bad dressers.

If the standard of judgment were positioned as shown in figure 2 then all the innocent people represented by the red area would be judged guilty if they were unlucky enough to be arrested and tried for the crime (type I errors). Since the normal distribution extends to infinity, type I errors would never be zero even if the standard of judgment were moved to the far right. The only way to prevent all type I errors would be to arrest no one. Unfortunately this would drive the number of unpunished criminals or type II errors through the roof.

figure 2. Innocent Suspects Who Are Arrested and Tried for a Crime

Figure 3 shows what happens not only to innocent suspects but also guilty ones when they are arrested and tried for crimes. In this case, the criminals are clearly guilty and face certain punishment if arrested.

figure 3. What Happens to Innocent Suspects or Clearly Guilty Criminals if They Are Arrested and Tried


figure 4. What Happens to Innocent Suspects or Not So Clearly Guilty Criminals if They Are Arrested and Tried.

If the police bungle the investigation and arrest an innocent suspect, there is still a chance that the innocent person could go to jail. Also, since the normal distribution extends to infinity in both positive and negative directions there is a very slight chance that a guilty person could be found on the left had side of the standard of judgment. This means he would incorrectly be set free.  

Unfortunately, justice is often not as straightforward as illustrated in figure 3. Figure 4 shows the more typical case in which the real criminals are not so clearly guilty. Notice that the means of the two distributions are much closer together. As before, if bungling police officers arrest an innocent suspect there's a small chance that the wrong person will be convicted. However, there is now also a significant chance that a guilty person will be set free. This is represented by the yellow/green area under the curve on the left and is a type II error.

If the standard of judgment is moved to the left by making it less strict the number of type II errors or criminals going free will be reduced. This could be done by throwing out the reasonable doubt standard and instructing the jury to find the defendant guilty if they think it's possible that she did the crime. However, this would make the type I errors unacceptably high. While fixing the justice system by moving the standard of judgment has great appeal in the end there is no free lunch.

The need for unbiased, highly trained, top quality police investigators with state of the art equipment should be obvious.  There is no possibility of having a type I error if the police never arrest the wrong person. Improving the accuracy of police investigations is expensive and complex but achievable in today's world with tools such as DNA testing. The famous trial of  O. J.  Simpson would either have ended in a guilty verdict or not have occurred at all if the Los Angeles Police officers investigating the crime had been beyond reproach.