Notes for March 18 and 20

The topic for the next few weeks is hypothesis testing. The main idea is that experiments must be conducted to test the validity of an idea, which is called a hypothesis. There are always two hypotheses available, the null hypothesis H0 (pronounced "H zero" or "H nought") and the alternate hypothesis HA (pronounced "H A"). The standard is to assume the null hypothesis is true, which says that nothing special is happening, which in most cases means that two things we can measure should be equal or close to it. The alternate hypothesis says the two measurements are different. We can have one tailed high tests, where we want "large" positive test statistics. In one tailed low tests, only negative test statistics with "large" absolute value will do. In two tailed test, "large" absolute value for positive or negative numbers will work. We will only accept the alternate hypothesis if the experiment produces impressive results given our particular criteria for that test.


The basics of hypothesis testing are similar to the ideals of the English legal system, which is also the system used in United States courts, that a defendant is presumed innocent until proven guilty. There are different levels of proof of guilt in different trials, whether it is beyond a reasonable doubt or the less rigorous standard of preponderance of evidence.

In a case involving an alleged crime, there is the reality of what the defendant did and the result of the trial. If the defendant did the illegal act, then being found guilty is the correct result under the law. If the reality is that the defendant didn't do the act, the correct result would be a not guilty verdict.

The reasonable doubt standard is put in place in theory to make sending an innocent person to jail unlikely, and this is called a Type I error. The best known Type I result in legal history is Jesus Christ.

It is also possible that a person who did a crime will be found not guilty. This is called a Type II error. When I ask my students for an example of Type II error, O.J. Simpson's name still rings out the loudest.


In hypothesis testing, there is the reality and the result of the experiment. If H0 is true, the two things measured are equal or pretty close to equal. If HA is true, they are significantly different.

If the experiment produces a test statistic that is beyond the threshold we set for it, and "beyond" could mean lower if it is a one-tailed low test, or higher if it is a one-tailed high test, or either lower or higher in a two-tailed test, then we reject H0. If the test statistic fails to get "beyond" the threshold, we fail to reject H0.

Rejecting a true null hypothesis is a Type I error. Rejecting a false alternate hypothesis is a Type II error.

In class, we discussed Sir Ronald Fisher and his hypothesis testing of the lady who said she could tell the difference in taste between tea poured into milk or milk poured into tea.

Here are the things that have to be done to make such an experiment work.

#1 Define the null hypothesis. In modern experiments, the null hypothesis is always defined as an equation. In a proportion test, the equation will be concerning p, the true probability of success. In the lady tasting tea, we would assume if nothing special is happening, then she is just guessing whether the tea or milk was poured first, and the probability of being correct on any given trial is 50% or .5. We write this as follows.

H0: p = .5

#2 Pick a threshold. The trials we are going to perform are taste tests where the lady cannot see the tea-milk mixture being poured. We have to decide on how high a test statistic we will consider impressive. The three standard choices are 90% confidence, 95% confidence or 99% confidence. For experiments in the medical field, where the decision is whether or not to bring a new drug to market, the 99% confidence level is common. For an experiment like this, where the result is not truly earth shattering, we might decide to us the 95% confidence threshold.

The experiment will produce a z-score the thresholds for high z-scores are as follows:

90% threshold: z = 1.28
95% threshold: z = 1.645
99% threshold: z = 2.325



#3 Decide on the number of trials in an experiment. There is a tug-of-war in deciding the number of trials. More trials produces numbers we can be more confident in, but more trials is also more expensive and more time consuming. In the case of the lady tasting tea, we don't want to keep her drinking tea mixtures for hours.

Different books set different standards for the minimum number of trials based on np and nq. Some say both np > 5 and nq > 5. Others say both numbers should be greater than 10, yet others say 15. The standard that np >= 10 and nq >= 10 can be connected to the standard that says n > np + 3*sqrt(pqn) > np - 3*sqrt(pqn) > 0 by a little algebraic manipulation.

np - 3*sqrt(pqn) > 0 [add the square root to both sides]
np > 3*sqrt(pqn) [square both sides]
n^2*p^2 > 9pqn [divide both sides by np]
np > 9q

Since q must be less than 1, but can be as close to 1 as we want, set it equal to 1 and the inequality becomes

np > 9, which we can change to np >= 10.

For example, If let's look at the different possible positive z-score results if the lady were given ten trials, which would be enough if we used the lowest standard of np >= 5 and nq >= 5.

10 correct out of 10: z = 3.16227... ~= 3.16, which is above the 99% threshold.
(look-up table: .9992)

9 correct out of 10: z = 2.52982... ~= 2.53, which is above the 99% threshold.
(look-up table: .9943)

8 correct out of 10: z = 1.89737... ~= 1.90, which is above the 95% threshold, but not the 99%.
(look-up table: .9713)

7 correct out of 10: z = 1.26491... ~= 1.26, which is below the 90% threshold.
(look-up table: .8962)

If we set the bar at the 90% threshold, she could impress us by getting 8 right out of 10 or better. Likewise, at the 95% threshold, 8 of 10 will be beyond the threshold and the result would make us reject H0. At the 99% threshold, she would have to get 9 of 10 or 10 of 10 to make a z-score that breaks the threshold.

#4 Interpreting the test statistic. Let's say for the sake of argument that the lady got 8 of 10 correct. (There is a book about 20th Century statistics entitled The Lady Tasting Tea, where a witness to the experiment says he can't recall how many times she was tested, but the lady got a perfect score.) If we set the threshold at 90% confidence or 95% confidence, we would be impressed by the z score of 1.90 and we would reject H0, which says that we don't think she is "just guessing", but actually has the talent she says she has. If we set the value at 99% confidence, we would fail to reject H0.

Here's the thing. We could be wrong. If we reject H0 incorrectly, it means she was just guessing and she was very lucky during this test, a Type I error. A z-score of 1.90 corresponds to a probability of 97.13%, which is called the p-value in hypothesis testing. She can pass the test by being better than 97.13% of lucky guessers. If she is a lucky guesser, she would fool anyone who had set the threshold at 90% or 95%.

If we fail to reject H0, which we would do if we set the threshold at 99%, this could also be an error, but this time it would be a Type II error. Under this scenario, she got 8 of 10 but she should do better usually. It's more difficult math to figure out how good she actually is and how unlucky she had to be to get only 8 of 10. The probability of a Type I error is called alpha, and it is determined by the threshold. The probability of a Type II error is called beta, and it usually explained in greater detail in the class after the introduction to statistics.

A low tailed test

Let's say we want a low error rate. Unlike the lady tasting tea who needed a lot of right answers to impress us, now we need a low score to get a result that will make us reject the null hypothesis.

Now our z-score thresholds are
90% threshold: z = -1.28
95% threshold: z = -1.645
99% threshold: z = -2.325

Let's say we want to be convinced our error rate is less than 10% and we want to be convinced to the 95% confidence level.

H0: p = 0.10
HA: p < 0.10
n = 50

10% of 50 is 5, so we should check to see what happens at f = 4, 3, 2, 1 and 0 errors. Typing this into the calculator will look like

(f/50 - .1)/sqrt(.1*.9/50)

f = 4 gives a rounded z-score of -.47
(look-up table: .3192 fail to reject H0)

f = 3 gives a rounded z-score of -.94
(look-up table: .1736 fail to reject H0)
 f = 2 gives a rounded z-score of -1.41
(look-up table: .0793 fail to reject H0 at 95% confidence, but reject at 90%)

f = 1 gives a rounded z-score of -1.89
(look-up table: .0294 reject H0 at 95% confidence, but fail reject at 99%)

f = 0 gives a rounded z-score of -2.36
(look-up table: .0091 reject H0 at any confidence level we use 90%, 95% or 99%)

Notes for February 11 and 13

Reverse look-up of z scores: Percentiles to z-scores, z-scores to raw scores

So far, we only use the percentile look-up table if we know a data set is normally distributed. In these cases, we are given the average mux and the standard deviation sigmax and find a z-score z(x) for a raw score x using this formula.

z(x) = (x - mux)/sigmax

We can then use this z-score to look up a percentile on the orange look-up table.  

Example: For men, the average height is 70.5 inches (5' 10.5") and the standard deviation is 2.8 inches. What is the z-score for a man 6'0" tall (72 inches)?

z(72) = (72 -70.5)/2.8 = 1.5/2.8 = 0.535714..., which rounded to the nearest hundredth is 0.54. Using our lookup table in row 0.5 and column 0.04, we get .7054, which means a man 6'0" is as tall or taller than 70.54% of the male population. Subtracting from 100%, we can also say that 29.46% of the male population is 6'0" or taller.

What if instead we wanted to find the cut-off height for the 75th percentile of men. What we have to find is when the table has two consecutive numbers that look like this

... .74xx .75xx ...

What that would mean is that between those two z-score will be the cutoff for the 75th percentile. Looking on the positive side of the chart we find

0.6 ... .7486 .7517 ...

In the columns 0.06 and 0.07. This means the 75th percentile lies in between 0.66 and 0.67. Since .7486 is .0014 below .7500 and .7517 is .0017 above, it's fair to say the 75th percentile is about half way in between, and 0.665 is a good approximation.

Now that we have a z-score, we use this formula to find the raw score x.

x = mux + z(x) × sigmax

In this case we get

x = 70.5" + 0.665 × 2.8 = 70.5" + 1.862" = 72.362" = 6" 0.4"

This says the 75th percentile of men's heights is at just below six feet and one half inch tall.

What about the percentage of men between 5'11" and 6'2". Here were find the percentages for both hieghts, then subtract the smaller percentage from the larger.

6'2" = 74", and z(74) = (74-70.5)/2.8 = 3.5/2.8 = 1.25

z = 1.25 corresponds to 0.8944

5'11" = 71" and z(71) = (71-70.5)/2.8 = 0.5/2.8 = 0.17857... = 0.18

z = 0.18 corresponds to 0.5714

0.8944 - .5714 = 0.323, which says about 32.3% of men fall between exactly 5'11" and exactly 6'2" tall.

The Central Limit Theorem 

 If we have a normally distributed population and take a random sample, z(x-bar) is different from z(x) by changing the standard deviation. The simplest way to compute z(x-bar) is to take the z-score the normal way then multiply by the square root of n, the size of the sample. For example, if 10 men averaged to a height of 6'0", they would have a different z-score and correspond to a different proportion of the population.

 z(72) = (72 -70.5)/2.8 × sqrt(10) = 1.694..., which rounds to 1.69. Looking up in row 1.6 and column 0.09, we get .9545.  This says a sample of 10 men averaging 6'0" tall is taller than 95.45% of all samples of ten men, and only 4.55% of all samples of ten men are shorter than that.

This rule is called the Central Limit Theorem. We will be using it to try to get estimates of the average of a population using the average of a sample that is assumed to be representative.

Notes for 4 February 2014

Two different versions of standard deviation of a numerical data set, sx and sigmax

The two formulas here are standard deviations for a numerical data set. When we have a sample, we use sx. When the data set is a population, we use sigmax. We get two different values because the denominator in the first case is n-1 and N in the latter case. The reason for this is degrees of freedom.

The idea of degrees of freedom is to count how much information you need to get an answer. For example, if a football game ends in regulation we know that

final score = score in 1st + score in 2nd + score in 3rd + score in 4th

There are five pieces of information, but if you have any four of these numbers, you can find the fifth. Here we would say the degrees of freedom are 4, which is 5-1. For example, if we know the Seahawks scored 43 points total in the Super Bowl, and the scored 8 points in the first quarter, 14 points in the second quarter and 14 points in the third, we can figure out how much they scored in the 4th quarter without being told.

score in 4th = final score - score in 1st - score in 2nd - score in 3rd = 43 - 8 - 14 - 14 = 7

This is a situation where the degrees of freedom are n-1, just like with the standard deviation of the sample. The idea is that if we know the average and somehow we get all the scores except for one, we can find the last score by multiplying the average by n then subtracting all the scores we know to find the one score we don't know.

A new way to measure data: z-scores

 
We can compare two sets of data against one another using the averages (x-bar or mux) and the standard deviation (sx or sigmax) with a formula known as the z-score. We subtract the average from the raw score x. If x is above average, we will get a positive number, if x is below average, we will get a negative number. (if x is exactly at teh average, we'll get zero.) We then divide by the standard deviation to get the z-score. This tells us how many standard deviations we are away from average, either high or low.

Here is an example using the American and National League final standings last year. The data set we will check out is the number of wins. I will treat both leagues as samples of all of Major League Baseball and the reason there is a difference in the average wins is because of inter-league play.



American League:
x-bar = 81.3
sx = 13.7

National League:
x-bar = 80.7
sx = 11.1

The AL had slightly more wins, but their standard deviation is higher because the data is more spread out, largely because the Houston Astros were so terrible. In any data set, we can use z-scores to find outliers.

z is greater than 3: The score is very unusually high
z is greater than 2: The score is unusually high
z is less than -3: The score is very unusually low
z is less than -2: The score is unusually low

The Astros had 51 wins, so their z-score is (51-81.3)/13.7 = -2.21. They are the only team in either league that can be considered an outlier, either high or low. 

We can also compare teams in the different leagues. For example, both the As and the Braves had 96 wins, but because they are in different leagues they won't have the same z-score.

z-score for As: (96-81.3)/13.7 = 1.07
z-score for Braves: (96-80.7)/11.1 = 1.38

What this would say is that it was more impressive for the Braves to win 96 that for the As. A big part of this is that the Athletics played 19 games against the Astros, winning 15 and losing 4. The Braves never played the Astros, and that alone meant they got the same number of wins against a tougher set of opponents, which accounts for their higher z-score.

Z-scores in normally distributed sets

Not every set can be assumed to be normally distributed. Usually, we will be told a set is normally distributed and be given both mux and sigmax. If we have such a set, we can take a raw score and turn it into a percentile using the look-up table we got on the orange hand-out on Tuesday. Let's take an example.

It is assumed that IQ scores are normally distributed, where the average is 100 and the standard deviation is 15. This would say an IQ of 110 has a z-score of (110-100)/15 = 0.67. Because of our assumption, we can use the look-up table to find the percentile for an IQ of 110.

1. Use the Positive z-score side
2. Look in the row next to the 0.6 label in bold
3. Look in the column labeled 0.07.

The value at that position on the table is .7486. What this means is that 74.86% of the population has an IQ of 110 or less. If we subtract 74.86% from 100%, we get 25.14%. Rounding to the nearest percent, this means the 110 IQ is at about the cutoff for the 75th percentile.

We will discuss this in greater detail on Tuesday, February 11.

Confidence intervals for proportions.

In Excel, we can find the z-scores that give us the end points of the confidence intervals as follows.

90% confidence interval.  The middle 90% is between 5% and 95%. The Excel formulas are

=norminv(.05, 0, 1)

= norminv(.95, 0, 1)

Rounded to four places after the decimal, we get -1.645 and 1.645.

95% confidence interval.  The middle 95% is between 2.5% and 97.5%. The Excel formulas are

=norminv(.025, 0, 1)

= norminv(.975, 0, 1)

Rounded to four places after the decimal, we get -1.960 and 1.960.


99% confidence interval.  The middle 90% is between 0.5% and 99.5%. The Excel formulas are

=norminv(.005, 0, 1)

= norminv(.995, 0, 1)

Rounded to four places after the decimal, we get -2.576 and 2.576.
In our first sample of 180 m&ms, there were 15 red m&ms. This says p_hat(red) = 15/180 ~= 8.3% when rounded to the nearest tenth of a percent.

The standard deviation for this proportion sp_hat(red) = sqrt(.083(1-.083)/180) or .020600514, which I will round to 2.1%

Here are the confidence intervals for this sample.

90% confidence interval.

0.083 + 1.645(.021) = .117545 about 11.8%
0.083 - 1.645(.021) = .048455 about 4.8%

Given this sample, we are 90% confident the true proportion of red m&ms in the population is between 4.8% and 11.8%


95% confidence interval.

0.083 + 1.960(.021) = .12416 about 12.4%
0.083 - 1.960(.021) = .04184 about 4.2%

Given this sample, we are 95% confident the true proportion of red m&ms in the population is between 4.2% and 12.4%



99% confidence interval. 

0.083 + 2.576(.021) = .137096 about 13.7%
0.083 - 2.576(.021) = .028904 about 2.8%

Given this sample, we are 99% confident the true proportion of red m&ms in the population is between 2.8% and 13.7%


As we increase the confidence, the interval gets larger.

As the sample size n gets larger, sqrt(p_hat*q_hat/n) will tend to get smaller, so larger sample sizes will give us smaller intervals, assuming p_hat doesn't change much.

Answers to quiz from 9/11

(10 points) On any single section of the SAT, mux = 500 and sx = 100. Find the z-scores for the following raw SAT scores and use the look-up table to find what proportion of the people taking the SAT will get a score that is less than or equal to the given score. Z-scores should be rounded to two places after the decimal and the proportions from the table are written to four decimal places.

SAT score: 570
z-score = .7
proportion at that z-score or less = .7580

SAT score: 750
z-score = 2.5
proportion at that z-score or less = .9938

SAT score: 395
z-score = -1.05
proportion at that z-score or less = .1469

SAT score: 625
z-score = 1.25
proportion at that z-score or less = .8944

SAT score: 490
z-score = -0.1
proportion at that z-score or less = .4602


(6 points) For the following movie studios find the average number of theaters on the opening weekend for the movies they made that were in the top 25 grossing movies of 2009 and find the median number of theaters of those sets of films. Round the answers to the nearest tenth.


Fox
Average number of theaters 3771.5
Median number of theaters 3898
Numbers on list: 3452, 3700, 4099, 4099, 4096, 3183


Sony
Average number of theaters 3298.5
Median number of theaters 3274
Numbers on list: 3404, 3144, 3527, 3119

WB
Average number of theaters 3572
Median number of theaters 3530
Numbers on list: 4325, 3269, 3110, 3626, 3530
(1 point each)
What movie had the biggest opening weekend?
The Twilight Saga: New Moon

What month had the most opening days on the list?
May (6)

What movie on the list opened earliest in the year?
Paul Blart: Mall Cop, Jan. 16

What movie on the list opened latest in the year?
Sherlock Holmes, Dec. 25

Practice problems for 6/30

The idea behind z-scores is that we can compare data from completely different numerical data sets by changing the scale to be the distance away from the average, with the new yardstick being the standard deviation. We can take z-scores and the first two pages of the notes to see how common a particular z-score is.

Example: let mux = 63.6 and sigmax = 2.5, which are the average and standard deviation for heights in inches of females.

1) What is the z-score for 67 inches? What percentage of the female population is greater than 67 inches tall?

2) What is the z-score for 64 inches? What percentage of the female population is less than 64 inches tall?


Going the other direction, we have the third page of the notes, which gives z-scores to three decimals that correspond to percentiles in the population. To find the raw score that corresponds to that percentile, use the formulas shown at the left.

3) To the nearest tenth of an inch, what is the height that corresponds to the 96th percentile in U.S. women's heights?

4) To the nearest tenth of an inch, what is the height that corresponds to the 24th percentile in U.S. women's heights?


Answer in the comments.

Preview of class for 6/29

Today, we will be discussing distributions of numerical variables. Some variables are evenly distributed through all the values, other skew to have more high values than low, some are the other way around with more low than high.


A lot of data sets have what is called normal distribution, where the most common values are the ones closest to the average, with values much higher or much lower than average being much rarer, and the farther from average those values are, the rarer they become. This curve, known as the bell-shaped curve or the normal curve or the Gaussian curve represents the normal distribution, where the high point shows the density of the values that are near average, and the lower levels at both ends represent the scarcity of values farther away from average.


One of the reasons to find the standard deviation of a set of numbers is to calculate the z-score of a raw value x. This tells us how many standard deviations a value is away from the average. Negative z-scores are for numbers below average and positive z-scores are for values above average. z(x) = 0 only when x = x-bar, the value is exactly at average. The first four pages in the class notes let us change z-scores into proportions, and using these numbers we can talk about the probability of finding values greater than some value x, or the probability of finding values between two values, call them x1 and x2

Class notes for 5/6: Hypothesis testing

The topic for the next few weeks is hypothesis testing. The main idea is that experiments must be conducted to test the validity of an idea, which is called a hypothesis. There are always two hypotheses available, the null hypothesis H0 (pronounced "H zero" or "H nought") and the alternate hypothesis HA (pronounced "H A"). The standard is to assume the null hypothesis is true, which says that nothing special is happening, which in most cases means that two things we can measure should be equal or close to it. The alternate hypothesis says the two measurements are different. We can have one tailed high tests, where we want "large" positive test statistics. In one tailed low tests, only negative test statistics with "large" absolute value will do. In two tailed test, "large" absolute value for positive or negative numbers will work. We will only accept the alternate hypothesis if the experiment produces impressive results given our particular criteria for that test.


The basics of hypothesis testing are similar to the ideals of the English legal system, which is also the system used in United States courts, that a defendant is presumed innocent until proven guilty. There are different levels of proof of guilt in different trials, whether it is beyond a reasonable doubt or the less rigorous standard of preponderance of evidence.

In a case involving an alleged crime, there is the reality of what the defendant did and the result of the trial. If the defendant did the illegal act, then being found guilty is the correct result under the law. If the reality is that the defendant didn't do the act, the correct result would be a not guilty verdict.

The reasonable doubt standard is put in place in theory to make sending an innocent person to jail unlikely, and this is called a Type I error. The best known Type I result in legal history is Jesus Christ.

It is also possible that a person who did a crime will be found not guilty. This is called a Type II error. When I ask my students for an example of Type II error, O.J. Simpson's name still rings out the loudest.


In hypothesis testing, there is the reality and the result of the experiment. If H0 is true, the two things measured are equal or pretty close to equal. If HA is true, they are significantly different.

If the experiment produces a test statistic that is beyond the threshold we set for it, and "beyond" could mean lower if it is a one-tailed low test, or higher if it is a one-tailed high test, or either lower or higher in a two-tailed test, then we reject H0. If the test statistic fails to get "beyond" the threshold, we fail to reject H0.

Rejecting a true null hypothesis is a Type I error. Rejecting a false alternate hypothesis is a Type II error.

In class, we discussed Sir Ronald Fisher and his hypothesis testing of the lady who said she could tell the difference in taste between tea poured into milk or milk poured into tea.

Here are the things that have to be done to make such an experiment work.

#1 Define the null hypothesis. In modern experiments, the null hypothesis is always defined as an equation. In a proportion test, the equation will be concerning p, the true probability of success. In the lady tasting tea, we would assume if nothing special is happening, then she is just guessing whether the tea or milk was poured first, and the probability of being correct on any given trial is 50% or .5. We write this as follows.

H0: p = .5

#2 Pick a threshold. The trials we are going to perform are taste tests where the lady cannot see the tea-milk mixture being poured. We have to decide on how high a test statistic we will consider impressive. The three standard choices are 90% confidence, 95% confidence or 99% confidence. For experiments in the medical field, where the decision is whether or not to bring a new drug to market, the 99% confidence level is common. For an experiment like this, where the result is not truly earth shattering, we might decide to us the 95% confidence threshold.

The experiment will produce a z-score the thresholds for high z-scores are as follows:

90% threshold: z = 1.28
95% threshold: z = 1.645
99% threshold: z = 2.325



#3 Decide on the number of trials in an experiment. There is a tug-of-war in deciding the number of trials. More trials produces numbers we can be more confident in, but more trials is also more expensive and more time consuming. In the case of the lady tasting tea, we don't want to keep her drinking tea mixtures for hours.

Different books set different standards for the minimum number of trials based on np and nq. Some say both np > 5 and nq > 5. Others say both numbers should be greater than 10, yet others say 15. The standard that np >= 10 and nq >= 10 can be connected to the standard that says n > np + 3*sqrt(pqn) > np - 3*sqrt(pqn) > 0 by a little algebraic manipulation.

np - 3*sqrt(pqn) > 0 [add the square root to both sides]
np > 3*sqrt(pqn) [square both sides]
n^2*p^2 > 9pqn [divide both sides by np]
np > 9q

Since q must be less than 1, but can be as close to 1 as we want, set it equal to 1 and the inequality becomes

np > 9, which we can change to np >= 10.

For example, If let's look at the different possible positive z-score results if the lady were given ten trials, which would be enough if we used the lowest standard of np >= 5 and nq >= 5.

10 correct out of 10: z = 3.16227... ~= 3.16, which is above the 99% threshold.
9 correct out of 10: z = 2.52982... ~= 2.52, which is above the 99% threshold.
8 correct out of 10: z = 1.89737... ~= 1.90, which is above the 95% threshold, but not the 99%.
7 correct out of 10: z = 1.26491... ~= 1.26, which is below the 90% threshold.

If we set the bar at the 90% threshold, she could impress us by getting 8 right out of 10 or better. Likewise, at the 95% threshold, 8 of 10 will be beyond the threshold and the result would make us reject H0. At the 99% threshold, she would have to get 9 of 10 or 10 of 10 to make a z-score that breaks the threshold.

#4 Interpreting the test statistic. Let's say for the sake of argument that the lady got 8 of 10 correct. (There is a book about 20th Century statistics entitled The Lady Tasting Tea, where a witness to the experiment says he can't recall how many times she was tested, but the lady got a perfect score.) If we set the threshold at 90% confidence or 95% confidence, we would be impressed by the z score of 1.90 and we would reject H0, which says that we don't think she is "just guessing", but actually has the talent she says she has. If we set the value at 99% confidence, we would fail to reject H0.

Here's the thing. We could be wrong. If we reject H0 incorrectly, it means she was just guessing and she was very lucky during this test, a Type I error. A z-score of 1.90 corresponds to a probability of 97.13%, which is called the p-value in hypothesis testing. She can pass the test by being better than 97.13% of lucky guessers. If she is a lucky guesser, she would fool anyone who had set the threshold at 90% or 95%.

If we fail to reject H0, which we would do if we set the threshold at 99%, this could also be an error, but this time it would be a Type II error. Under this scenario, she got 8 of 10 but she should do better usually. It's more difficult math to figure out how good she actually is and how unlucky she had to be to get only 8 of 10. The probability of a Type I error is called alpha, and it is determined by the threshold. The probability of a Type II error is called beta, and it usually explained in greater detail in the class after the introduction to statistics.

Practice problems for population percentiles

Based on data from the National Health Survey, the average height in inches for women in the United States is 63.6 inches and the standard deviation is 2.5 inches. These are mux and sigmax, respectively. If we ask how many women are exactly some height, like 5'0" (60 inches) for example, the answer is assumed to be 0%. But if a woman says she is 5'0" tall, and she has measured herself correctly, she could be anywhere between 59.5 and 60.5 inches tall, which is to say 4' 11.5" and 5' 0.5". Let's find the z-scores for 59.5 and 60.5.

z(59.5) = (59.5-63.6)/2.5 = -1.64
z(59.5) = (59.5-63.6)/2.5 = -1.24

Using the Negative z-score table, we look up the percentages for each of these z-scores.

-1.64 -> .0505
-1.24 -> .1075

subtracting the percentages, we get .1075 - .0505 = .0570, which says 5.7% of American women list their height as 5'0".

Questions

1. What percentage of American women list their height as 5'1"?
2. What percentage of American women list their height as 5'2"?
3. What percentage of American women list their height as 5'3"?
4. What percentage of American women list their height as 5'4"?
5. What percentage of American women list their height as 5'5"?
6. What percentage of American women list their height as 5'6"?
7. What percentage of American women list their height as 5'7"?
8. What percentage of American women list their height as 5'8"?
9. What percentage of American women list their height as 5'9"?

Answers in the comments.

Class notes for 3/11

The positive and negative z-score tables connect z-scores to proportions, numbers between 0 and 1 written to four decimal places of accuracy. For example, z = 1.23 corresponds to .8907, which means that 89.07% of data in a normally distributed set should have a z-score of 1.23 or less, while (100-89.07)% = 10.93% of data will have a z-score of 1.23 or more. Moreover, the normal curve is symmetric around z = 0. This means when z = -1.23, that is the cut-off point between the low 10.93% of the data and the high 89.07% of the data, just the opposite of the percentages that correspond to z = 1.23.



Critical values, or Confidence Level Multipliers (CLMxx%)

Sometimes, instead of being interested in the highest n% or the lowest, we will need to deal with the middle n% for what is known as a confidence interval. In the bottom right hand corner of the Positive z Scores table (first page of yellow handout), there is a small table labeled Common Critical Values. On tests and homework, I will call these Confidence Level Multipliers, or CLMxx%, where xx% is the percentage of confidence associated with the z-scores. The table gives the z-scores I will be calling CLM90%, CLM95% and CLM99%.

CLM90%: The end points are -1.645 and +1.645
CLM95%: The end points are -1.96 and +1.96
CLM99%: The end points are -2.575 and +2.575

For example, what this means for the case of CLM90% is that the low 5% of the data is below z=-1.645, the middle 90% is between z=-1.645 and z=+1.645, and the high 5% is above z=+1.645.


Student's t-scores

Confidence intervals are used over and over again in statistics, most especially in trying to find out what value the parameter of the population has, when all we can effectively gather is a statistic from a sample. For numerical data, we aren't allowed to use the normal distribution table for this process, because the standard deviation sx of a sample isn't a very precise estimator of sigmax of the underlying population. To deal with this extra level of uncertainty, a statistician named William Gossett came up with the t-score distribution, also known as Student's t-score because Gossett published all his work under the pseudonym Student. He used this fake name for publishing to get around a ban on publishing in journals established by his superiors at the Guinness Brewing Company where he worked.

The critical t-score values are published on table A-3. The values depend on the Degrees of Freedom, which in the case of a single sample set of data is equal to n-1. For every degree of freedom, we could have another positive and negative t-score table two pages long, just like the z-score table, but that would take up way too much room, so statistics textbooks have reverted instead to publishing just the highlights. There are five columns on the table, each column labeled with a number from "Area in One Tail" and "Area in Two Tails". Let's look at the degrees of freedom row for the number 13.

1 tail___0.005______0.01_____0.025______0.05______0.10
2 tails__0.01_______0.02_____0.05_______0.10______0.20

13_______3.012_____2.650_____2.160_____1.771_____1.333

What this means is that if we have a sample of size 14, then the degrees of freedom are 13 and we can use these numbers to find the cut-off points for certain percentages. The formula for t-scores looks like the formula for z-scores, t = (x - x-bar)/sigmax, but we use the different look-up table to decide what these numbers mean. For example, the second column in row 13 is the number 2.650. This means that in a sample of 14, a Student's t-score of -2.650 is the cutoff for the bottom 1%, the t-score of +2.650 is the cutoff for the top 1% and the middle 98% is between t-scores of -2.650 and +2.650.

As the degrees of freedom get larger, the numbers in the columns get smaller. The last row has the label Large and reads as follows.

Large____2.576_____2.326_____1.960_____1.645_____1.282

These values exactly correspond to the z-score table. As the data set size gets larger, the differences between the z-distribution and the t-distribution shrink down to nothing.

When can we use t-scores?

Because we don't know sigmax, we are prohibited from using the z-score tables. But there are cases when we shouldn't use the t-score tables either. Here is the decision method.

Step 1: Is n at least 30? If yes, we are good. If no, go to Step 2.

Step 2: Is the sample normally distributed or can we assume the underlying data set is normally distributed? If yes, we can continue. If no, we would only be able to use non-parametric statistical techniques, which are not covered in this course.

For example, the cotinine data sets we have on the handout sheet have one set that looks normally distributed, the smokers data, and two that do not look normally distributed, the exposed and unexposed non-smoker data. Because n=40 for all the sets, we can use the t-score method because we answered yes to Step 1. If the data sets had less than 30 subjects, we would not be able to use the non-smoker data because we would have answered no to the questions from both Step 1 and Step 2.


The formula for a confidence interval for the mean of a population given the mean of a sample.

The confidence interval for the mean of a population given the mean of the sample is a formula that gives us two endpoints as follows

x-bar - CLMxx%*sx/sqrt(n) < mux < x-bar + CLMxx%*sx/sqrt(n)

Let's take an example. We have the heights of males from Data Set #1. The statistics from that set are as follows.

n = 18
x-bar = 71.17
sx = 2.57

The sample size is 18, which is less than 30, so since we answer no in Step 1, we have to move on to Step 2. Here we can answer yes, because we can assume that human height is a normally distributed set. Let's now move on to finding the 95% confidence interval for the average male height given this sample.

Since n = 18, degrees of freedom = 17, and the CLM95% = 2.110. This is because the area in two tails of 5% is the same as the middle region having 95%. Here is our formula with these numbers plugged in.

71.17 - 2.110*2.57/sqrt(18) < mux < style="font-style: italic;">mux < 72.4

From census information, we know the average height of males in the United States is 69.5 inches, so this interval does not contain the true value. This semester, both the data sets had averages for male heights well above average, largely because of how many football and baseball players are enrolled in the classes, as well as a few other tall males who are not on the sports teams. This is a good example that a confidence interval is NOT a promise of a correct answer, and that statistical methods include confidence intervals exactly for this reason. (note: If we changed the interval to the 99% confidence interval, our confidence level multiplier would be 2.898 instead of 2.110, and the 99% confidence interval would contain the correct answer.)

Class notes for 3/9

Not all data sets are normally distributed, but it has been proven that the set of averages of subsets of a fixed size of a data set are normally distributed around the average of the whole data set. In the language we have used in class, this means that if we take a sample and get an average x-bar, it should be relatively close to the average of the population mux. This is called the Central Limit Theorem.


It's a little hard to see in the picture here, but the equation reads sigmax-bar = sigmax/sqrt(n). As n gets larger, sigmax-bar gets smaller.


To find z(x-bar), we subtract mux from x-bar and divide by sigmax-bar. On a calculator it's easiest to type in (x-bar-mux)/sigmax*sqrt(n).

Here is an example of the difference between z(x) and z(x-bar). We know that for IQ scores, the data set is normally distributed, mux = 100 and sigmax = 15. This means an IQ of 115, has a z-score of (115-100)/15 = 1. The z-score of 1 corresponds to the proportion .8413, which says that the percentage of people with IQs below 115 is 84.13% and the percentage with IQs above 115 is (100-84.13)% = 15.87%.

The Central Limit Theorem z-score answers a different question. What if we have a group of 8 people whose average IQ is 115. How often does that happen? Now, the formula changes to (115-100)/15*sqrt(8) = 2.828..., which rounds to 2.83. The proportion that corresponds to a z-score of 2.83 is .9977, which means that about 99.77% of all groups of eight people have average IQs under 115, while only (100-99.77)% = 0.23% of groups of eight have average IQs at 115 or over.

A standard usage of the Central Limit Theorem is to take a data set and see if the result is unusual or not. This is done by the following procedure.

1. Choose an outlying value, either a z-score or the percentage that corresponds to it.
2. Take a data sample from a population where you know the average and standard deviation already. If the Central Limit Theorem z-score gives us a value beyond the outlying value, we flag the sample we took as an outlying sample.

Practice problems for standard deviation

The data for IQ test scores is normally distributed. The average IQ is 100 and the standard deviation is 15.

1. What are the z-scores for the following IQ scores?

a. 90
b. 112
c. 120

2. Using the answers from part 1 and the Positive and Negative z-score tables.

a. What percentage of the population has an IQ less than 90?

b. What percentage of the population has an IQ higher than 120?

c. What percentage of the population has an IQ between 112 and 120?

3. Percentiles

a. Find the z-score that corresponds to the 63rd percentile. Find the IQ that corresponds to the 63rd percentile, rounded to the nearest whole number.

b. Find the z-score that corresponds the 13th percentile. Find the IQ that corresponds to the 13th percentile, rounded to the nearest whole number.

Answers in the comments.

Class notes for 3/4

If we have a data set that is normally distributed, we turn raw scores, what we usually call the x values into z-scores. The z-score is to answer the question "How many standard deviations away from the average is the raw score x?"
In any set of numerical data, we can find both the average (x-bar in a sample or mux in a population) and the standard deviation (sx in a sample or sigmax in a population). The standard deviation will always be a non-negative number, and it can only be zero if the data set is really boring and all the values are exactly the same, a very rare situation if the data is random.

Not every data set is normally distributed. The signs of a normally distributed set include, but are not limited to

a) the average and the median are very close to equal
b) most of the data is near the average, with only a few values either far above average or far below average

For example, the data concerning cotinine levels for smokers, exposed non-smokers and unexposed non-smokers are very different from one another. The smoking data is very near the normal distribution, with x-bar at 172.475 and the median at 170. The two non-smoking data sets have a much bigger split between the average and the median. With the exposed, x-bar is 60.575, while the median is 1.5, while the unexposed have an x-bar of 16.35 and a median of 0.

With normally distributed data, the z-scores can be confidently mapped to percentages, using the positive and negative z-score tables, the first two pages of the yellow sheets.

Raw score to z-score (formula) and z-score to percentage (table lookup)

Example #1: What percentage of U.S. women are 5 feet tall or less? The average height of women in the United States is 63.6 inches (5'3.6") and the standard deviation is 2.5 inches. The z-score for 5 feet, or 60 inches, is (60-63.6)/2.5 = -1.44. We now look that number up on the negative z-score table, where the row is the -1.4 row and the column is the .04 column. (This is akin to stem and leaf plots where the ones place and the tenths place are the stem, while the hundredths place is the leaf.)

___...___.03____.04____.05
...
-1.4____.0764 _.0749 _.0735 ...


So the z-score -1.44 corresponds to .0749, which can also be written as 7.49%. That means that 7.49% of women in the U.S. are under five feet tall. Conversely, the percentage of women above five feet tall is (100 - 7.49)% = 92.51%. The "under" number is always the value you will find on the yellow sheet, while the over number will equal (100 - table value)%. Because the normal distribution is symmetrical, 92.51% is the percentage that corresponds to a z-score of 1.44, the opposite of -1.44 on the other side of zero.

Example #2: What percentage of U.S. women are 6 feet tall or more? The z-score for 6 feet, or 72 inches, is (72-63.6)/2.5 = 3.36. We now look that number up on the positive z-score table, where the row is the 3.3 row and the column is the .06 column.

___...___.05____.06____.07
...
3.3_____.9996 _.9996 _.9996 ...


The z-score 3.36 corresponds to .9996, which can also be written as 99.96%. That means that 99.96% of women in the U.S. are under six feet tall. Conversely, percentage of women above six feet tall is (100 - 99.96)% = 0.04%. About 4 in every 10,000 women in the U.S. is over six feet tall.

Example #3: What percentage of U.S. women are between 5 and 6 feet tall? Using the information from examples 1 and 2, we get .9996 - .0749 = .9247, or 92.47% of women in the U.S. are between five feet and six feet tall.

Percentage to z-score (backwards table lookup)

Here's a different question. What is the z-score that corresponds to the 53rd percentile? What this means is we want the z-score that corresponds as closely as possible to .5300 on our look-up table. Since 53% is more than 50%, we will look on the positive z-scores side.

The percentage for z = 0.07 is .5279, while the percentage for z = 0.08 is .5319. The first one is .0021 below .5300 and the second is .0019 above, but we are going to add a third option, the average between them. The average of 0.07 and 0.08 is 0.075. The average of .5279 and .5319 is .5299, which is much closer than the other values, since it is only .0001 below .5300. For that reason, we will choose .0075 as the z-score than corresponds most closely to the 53rd percentile.

z-scores to raw scores

If we know that the z-score for the 53rd percentile is 0.075, what height for U.S. women corresponds to that z-score? We use the formulas below.


This would say in our case that the height that corresponds to 53rd percentile is x = 63.6 + 0.075*2.5 = 63.7875 inches. Only a tiny bit taller, still under 64 inches or 5'4", the 53rd percentile is very close to the 50th percentile, not surprisingly.

We will continue this discussion on Monday.