Home | About Sanderson Smith | Writings and Reflections | Algebra 2 | AP Statistics | Statistics/Finance | Forum

STATISTICAL ERRORS (TYPE I, TYPE II, POWER)

Data, data everywhere, but not a thought to think.

-Jesse Shera

 

The most recent Advanced Placement Statistics Outline of Topics includes the concepts of type I and type II errors, and power. The purpose of this paper is to provide simple examples of these topics.

 

Assume that two samples of people have the indicated ethnic distributions. The sample sizes are 25 and 20, respectively.

	African-American	Native-American	Caucasian	Oriental
SAMPLE #1	3	1	15	6
SAMPLE #2	10	4	3	3

Sometimes a visual display of data is helpful. Here are dot plots for each sample.

SAMPLE #1
African-American	* * *
Native-American	*
Caucasian	* * * * * * * * * * * * * * *
Oriental	* * * * * *

SAMPLE #2
African-American	* * * * * * * * * *
Native-American	* * * *
Caucasian	* * *
Oriental	* * *

Here is our challenge: One of the two samples is randomly chosen, and then one individual is randomly picked from the chosen sample. Based on our observation of the individual, we must make a conjecture as to which sample the chosen individual belongs.

Here is a null hypothesis, H_o.

H_o: The individual came from SAMPLE #1

Based on our observation of the randomly chosen individual, we must decide to either accept or reject H_o. A rejection of H_o is, of course, equivalent to asserting that the individual came from SAMPLE #2.

There are two types of errors that can be made during this process.

Type I error: H_o is rejected when it is true.

Type II error: H_o is accepted when it is false.

Here are four possible tests relating to H_o. A test consists of randomly choosing an individual and then accepting or rejecting H_o based on an observation of the individual. As defined, each test can produce a correct conclusion or an incorrect conclusion.

Test #1: Accept Ho if the randomly chosen individual is Caucasian.

Test #2: Accept Ho is the randomly chosen individual is Caucasian or Oriental.

Test #3: Accept Ho if the randomly chosen individual is not Native-American.

Test #4: Accept Ho if the randomly chosen individual is not African-American.

Let's examine Test #1. A Type I error can only occur when H_o is true. Hence, the probability of a Type I error with Test #1 is 10/25 = 40%. A Type II error can only occur when H_o is false. In this situation, if H_o is false, then the selected individual came from Sample #2. The probability of a Type II error is 3/20 = 15%. Here is a probability summary for Test #1.

	Sample is #1	Sample is #2
Accept Ho	60% (Correct decision)	15% (Type II error)
Reject Ho	40% (Type I error)	85% (Correct Decision)

The power of a test is the probability that an incorrect null hypothesis is rejected. In this situation, the power is 85%, which is calculated by the formula

1 - probability(Type II error).

In this situation, if the randomly chosen individual is from SAMPLE #2, one can expect this test to correctly conclude that the individual is not from SAMPLE #1 in 85 out of every 100 trials.

Here are corresponding probabilities for all of the indicated tests. (You can easily check these out!)

	Prob. (Type I error)	Prob. (Type II error)	Power of the test
Test #1	40%	15%	85%
Test #2	16%	30%	70%
Test #3	4%	80%	20%
Test #4	12%	50%	59%

There are, of course, other tests that could be used. Of the four tests examined, Test #3 produces the smallest Type I error, but yields a whopping 80% Type II error. Strategy #1 has the smallest Type II error, but also the largest Type I error.

As suggested by the examples above, decreasing the chance of one type of error frequently increases the chance for the other error type. In real-life situations, one can decrease the probability of both error types by collecting more data or having more information available. However, one must frequently decide which error type should be minimized. Here are two simple examples:

Example #1:
In the legal world, a null hypothesis might be "This person is innocent." A Type I error would be judging the person guilty when he is innocent. A Type II error would involve declaring the person innocent when he is guilty. If one accepts the thought that it is better to release a guilty person than to convict an innocent one, then it would be important to minimize the chances of a Type I error.

Example #2:
In the world of medicine, a null hypothesis might be "This drug will cure an illness." A Type I error would be concluding that the drug does not work when it actually does. A Type II error would conclude that the drug does work when it actually doesn't. One could argue that a Type II error should be minimized here if one agrees that spending time and money on a useless drug would replace what might be some other effective treatment.

Here are three important facts that can help minimize the confusion that sometimes results when working with these error types:

1. A Type I error can only occur if a null hypothesis,H_o, is true.

2. A Type II error can only occur if a null hypothesis,H_o, is false.

3. The power of a test is 1 - probability(Type II error).

It is usually a practical impossibility to work with an entire population. A statistician takes samples and "generalizes" his/her results to reach a conclusion about a population. This does involve producing a statement (null hypothesis) which may or may not be true. As noted above, there are two types of statistical errors that can be made. The main purpose of this paper is simply to introduce the two error types and, provide some simple examples illustrating them. It is hoped that these examples will be helpful to teachers and students as they prepare for the Advanced Placement Statistics Examination.

 

Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write.

-H.G. Wells, 1866-1946

RETURN TO WRITING HOME PAGE