Testing a hypothesis about the average of an underlying population (mux) based on the average of a sample (x-bar)
Let's assume we have information about an average from a population. For example, it is regularly assumed that human body temperature is 98.6° Fahrenheit or that the average IQ is 100. Another way to assume information would like our test to see if 2015 in Oakland is warmer than average, comparing it to the average temperature for 1999 to 2014. Here are the steps to take to create a null hypothesis H0 and an alternate hypothesis HA, determine a confidence level at which we will reject H0, and use a sample's statistics to see if the data warrants the rejection of the null hypothesis or it fails to meet that standard.
Setting the two hypotheses: The null hypothesis is always an equality and the alternate hypothesis an inequality. There are three kinds of inequalities.
One tailed high: Consider a drug that is supposed to increase muscle mass. The only kind of result that will impress us is one that shows that increase. This would set the two hypotheses as
H0: mux = constant
HA: mux > constant
One tailed low: Instead let's say we have a drug that is supposed to reduce cholesterol. We want to see results where the average goes down.
H0: mux = constant
HA: mux < constant
Two tailed: In class, we looked at data that seems to indicate human body temperature is not 98.6° Fahrenheit. When this claim was made, it was not made clear if the temperature was now higher or lower and in this case, any significant difference would be a surprising result.
H0: mux = constant
HA: mux != constant (The equal sign with the slash isn't available in this text editor. The symbols '!=' are used in some computer languages to mean inequality.)
Setting the confidence level: We may have some leeway as to whether our confidence level is 90%, 95% or 99%. In most experiments I've seen published about scientific statements, the 99% confidence level is standard.
Using the numbers from the experiment: We will need x-bar, sx and n to produce our test statistic t, shown in the equation on the left. We will also use n to get the degrees of freedom, which in this case is n-1.
Example: In class we had a set where n = 36, so degrees of freedom would be 35.
Threshold for one-tailed high test in this situation: Our test stat t would have to be greater than 2.438.
Threshold for one-tailed low test in this situation: Our test stat t would have to be less than -2.438.
Threshold for two-tailed test in this situation: Our test stat t would have to be greater than 2.724 -OR- less than -2.724.
When we plugged in the values 97.96 for the sample average and 0.69 for the sample standard deviation we got (97.96 - 98.6)/0.69 * sqrt(36) = -5.49. This number is well beyond or low threshold and we would reject the null hypothesis. The technical statement in this case would be
"We are 99% confident from the evidence our our sample that the average human body temperature is not 98.6° Fahrenheit." Notice that we cannot say what the true value is from this, though most samples with fairly large n place the true number now at around 98.2° Fahrenheit. We cannot be certain if the temperature has changed over time or if the means of measurement have become more accurate.
Notes on Bayesian probabilities.
Showing posts with label hypothesis testing. Show all posts
Showing posts with label hypothesis testing. Show all posts
Thursday, October 15, 2015
Tuesday, April 22, 2014
Notes for April 22 and April 24
Confidence interval for standard deviation of population
The confidence interval formula for sigmax is completely different than the formula for mux, and uses values from a new table, Table A-4, known as the chi squared table. Chi is pronounced like the chi in chiropractor, "kai", not "chee" or "chai".
Note that we don't have to test for normal distribution of the underlying data.
Let's give an example. If we have a set where n = 13 and sx = 3.21, we use n-1 as our degrees of freedom. Here is the line from Table A-4 for the row that corresponds to 12.
_____0.995___0.99___0.975___0.95___0.90___0.10____0.05___0.025____0.01___0.005
_12__3.074__3.571___4.404__5.226__6.304__18.549__21.026__23.337__26.217__28.299
The values of chi²Right and chi²Left are taken from the table as follows.
90% confidence: The right value is from the 0.05 column, the left value s from the 0.95 column. (Note that 0.95 - 0.05 = 0.90 or 90%)
95% confidence: The right value is from the 0.025 column, the left value s from the 0.975 column. (Note that 0.975 - 0.025 = 0.95 or 95%)
99% confidence: The right value is from the 0.005 column, the left value s from the 0.995 column. (Note that 0.995 - 0.005 = 0.99 or 99%)
In this instance, here are the confidence intervals for each of the standard percentages of confidence.
90% confidence: 3.21*sqrt(12/21.026) < sigmax < 3.21*sqrt(12/5.226)
2.425 < sigmax < 4.864
95% confidence: 3.21*sqrt(12/23.337) < sigmax < 3.21*sqrt(12/4.404)
2.302 < sigmax < 5.299
99% confidence: 3.21*sqrt(12/28.299) < sigmax < 3.21*sqrt(12/3.074)
2.090 < sigmax < 6.342
Like confidence intervals for other parameters like proportion and average, the 99% is the largest, the 95% is nested inside the 99% and the 90% is nested inside both the others. Unlike other confidence intervals, we get the endpoints by multiplying our statistic sx by two positive numbers, one less than 1 and the other greater than 1. The other thing that is unlike the early confidence intervals is that our statistic is not in the exact center. for example 3.21 - 2.425 = .785, while 5.226 - 3.21 = 2016, so there is a greater distance to the high end of the interval than there is to the low end.
Goodness of fit
If we want to know if a coin is fair we can do a two tailed z-score test of the proportion of heads to tails, where the null hypothesis is that both should be 50%. For example, if I flip a coin 100 times and get 52 heads and 48 tails, that isn't very far off from 50-50. The test statistic would be
z = (.52-.5)/sqrt(.5*.5/100) = .4
A z-score of .4 corresponds to a proportion of .6554. Because this is a two-tailed test, we use the higher number (either heads or tail percentage) and the percentage has to be fairly high.
Two-tailed 90% confidence: over .9500
Two-tailed 95% confidence: over .9750
Two-tailed 99% confidence: over .9950
If instead we got 60 heads and 40 tails, the z-score would produce a much higher proportion.
z = (.60-.5)/sqrt(.5*.5/100) =2.0
A z-score of 2.0 corresponds to a proportion of .9772. If we wanted proof to 90% confidence or 95% confidence, we would say this isn't a fair coin, rejecting the null hypothesis. If we want the proof to 99% confidence, the coin would have to be even more unfair, at least 62 to 38 in 100 flips.
We can't use this method to figure out if a six-sided die is fair, because we need to test that all six possibilities are coming up an equal number of times. For example, if we rolled the die 60 times, we would expect every number to show up exactly 10 times each. In the real world, that's not likely to happen, but we should expect something close. Here is the result of an experiment done with the random number generator in Excel.
number::: 1 2 3 4 5 6
expected: 10 10 10 10 10 10
observed: 14 9 10 10 7 10
Our test statistic will be the sum of (Observed - Expected)²/Expected. Here are the six values.
1st: (14-10)²/10 = 16/10 = 1.6
2nd: (9-10)²/10 = 1/10 = 0.1
3rd: (10-10)²/10 = 0/10 = 0.0
4th: (10-10)²/10 = 0/10 = 0.0
5th: (7-10)²/10 = 9/10 =0.9
6th: (10-10)²/10 = 0/10 = 0.0
The sum is 2.6. This is our test statistic. The degrees of freedom is the number of categories - 1, which in this case is 5. Look on the right side of chi square table for the thresholds in row d.f. = 5.
90% threshold: The 0.10 column, which has 9.236 in row 5
95% threshold: The 0.05 column, which has 11.071 in row 5
99% threshold: The 0.01 column, which has 15.086 in row 5
Our test statistic is much lower than even the 90% confidence level, so we fail to reject the null hypothesis. In other words, while this test wasn't perfect, it was much too close to what we expected to doubt the die was unfair.
Independence of contingency tables
You may remember the idea of dependent probability, where p(A, given B) would not be equal to p(A). It is possible to make a contingency table where p(A, given B) = p(A) for any A and B, where one is a row and the other is a column. Here is an example of a team whose road record and home record are exactly the same.
___H || A|| Total
W |16||16|| 32
L |10||10|| 20
|26||26|| 52 = grand total
We see that p(Wins) = p(Wins GIVEN Home) = p(Wins GIVEN Away). This hypothetical win-loss record is at the same proportion whether on the road or at home.
When we look at a contingency table, it most likely won't be exactly independent, but we have a new test using chi square table once again and using a test statistic that has the same formula as goodness of fit, the sum of (Observed - Expected)²/Expected. This time, the table we are given is the Observed and we must create the Expected using the formula
(row total) * (column total) = grand total.
Here is an example.
___H || A|| Total
W |22||18|| 40
L | 8||12|| 20
|30||30|| 60 = grand total
Now the home and road records aren't identical. We create the expected by keeping the row and column totals and blanking out the values inside.
___H || A|| Total
W |__||__|| 40
L |__||__|| 20
|30||30|| 60 = grand total
In the box for Home Wins, upper left hand corner we put
40 * 30/60 = 20. (Usually this is a decimal number, but I designed it to be a whole number to make this example easier.
___H || A|| Total
W |20||__|| 40
L |__||__|| 20
|30||30|| 60 = grand total
We could do the (row total) * (column total) = grand total method three more times to fill in the rest, but we don't have to. We can simply subtract the number in the box we just filled in to get the rest of the top row and the left column.
___H || A|| Total
W |20||20|| 40
L |10||__|| 20
|30||30|| 60 = grand total
Now it's easy to fill in the last box as well.
___H || A|| Total
W |20||20|| 40
L |10||10|| 20
|30||30|| 60 = grand total
Now we get the four values of (Observed - Expected)²/Expected. Here is the Observed contingency table once again.
___H || A|| Total
W |22||18|| 40
L | 8||12|| 20
|30||30|| 60 = grand total
Top left: (22-20)²/20 = 4/20 = 0.2
Top right: (18-20)²/20 = 4/20 = 0.2
Bottom left: (8-10)²/10 = 4/10 = 0.4
Bottom right: (12-10)²/10 = 4/10 = 0.4
Sum = 1.2
The degrees of freedom for a contingency table is (# of rows - 1) * (# of columns - 1), which in this case is (2-1)*(2-1) = 1*1 = 1. (Notice that we only had to fill in one box in the contingency table and the rest could be found by subtraction.) When degrees of freedom = 1, our thresholds are as follows.
90% confidence level (column 0.10): 2.706
95% confidence level (column 0.05): 3.841
99% confidence level (column 0.01): 6.635
Our test statistic of 1.2 does not let us reject the null hypothesis. This means that while the home and road records are different, we do not consider them to be statistically significantly different.
Restriction of range and correlation
If we take two matched sets of data, we can create a trendline, yp = ax + b, which is also know as the predictor line or the line of regression or the line of least squares. This is the illustration of a set of miles per gallon highway on the x axis and weight on the y axis. Not surprisingly, lighter cars get better gas mileage in general and heavier cars get less miles per gallon. Here is the set of 18 matched pairs, written as (mpg_highway, weight).
(24, 3930)
(24, 3985)
(26, 3995)
(26, 4020)
(27, 3515)
(27, 3175)
(27, 3225)
(28, 3220)
(29, 3115)
(29, 3450)
(30, 3525)
(30, 3245)
(30, 3115)
(31, 2795)
(32, 3235)
(34, 2500)
(37, 2440)
(37, 2290)
The correlation coefficient R² isn't as perfect 1.000, but according to the table we use to check how strong the correlation is, For 18 points the 95% confidence level threshold is 0.2190 and the 99% is 0.3841. So a value of R² = 0.7643 for these two variables shows they have very strong correlation and that isn't surprising. Strong negative correlation here (we can tell it's negative because the line slopes downward) means lighter cars generally get better gas mileage than heavier cars. That makes sense.
The statistical concept that is new here is restriction of range. This says that there is a tendency if you only look at data where the x values are limited, the R² value will go down. Let's say we only look at the first eight cars on the list, the ones that get under 30 mpg highway. We see that the trend is still downward and the slope is steeper. The thing our rule says is that
R² will be less usually, and it is less here. 0.60711 instead of 0.7643. For 8 points, this level of correlation surpasses the 95% threshold of 0.4998, but does not surpass the 99% threshold of 0.6956.
We see the same tendency come true when we look only at the cars getting 30 mpg highway or better. Here there are ten data points and the R² value is 0.61952. The thresholds for 10 pairs are 0.3994 and 0.5852 for 95% and 99% confidence, respectively. By our measure, this set has better correlation than the heavier cars do, but not as good as the set of all 18 cars.
The Monty Hall Problem (or for younger people, The Game Show Problem)
Way back in the day, there was a game show called Let's Make a Deal and the host was Monty Hall. (As a survey in class showed, only one student was aware of this, watching it in re-runs on the cable channel The Game Show Network.) There were many different games played in a half hour using many rules, but one of the famous ones is called The Monty Hall Problem. We can call it The Game Show Problem or more descriptively One Brand New Car and Two Goats.
The rules of the game are as follows. There are three closed doors and the contestant must choose one. Behind two of the doors there are bad prizes and behind the last door there is a great prize. Usually but not always, the bad prizes were goats. Usually but not always, the great prize was a brand new car.
(Note from a different perspective: If you have a place to raise them, goats are excellent producers of meat and milk. The most famous cheese of Greece, known as Feta Cheese, is almost always made from goat's milk. A new car, on the other hand, usually means higher insurance rate and even though it's a prize, the winner has to pay the sales tax.)
Back to the game. After the contestant chooses a door, not knowing if the prize is good or bad, the game show host (Monty Hall) shows that there's a goat behind some unchosen door and asks the contestant if he (or she) wants to switch to another door. (In the three door game, switching means over to the only other door available.)
The math question here is this. Does it make sense to stick with your original door or switch?
Explaining the simplest situation, the three door, one car, two goat version: Okay, on the contestant's first choice, there is a 1/3 chance of getting the car and 2/3 probability of getting a goat. If the contestant doesn't switch, the chance of winning is 1/3.
If the contestant switches, we have to look at two possible situations.
1. The contestant picked the car in the first place. There is a 1/3 chance of this and if the contestant switches, there are no other cars so the contest will lose.
2. The contest picked a goat in the first place. There was 2/3 chance this would happen. If your door has a goat and Monty shows you the second goat, the door you would switch to must have the car, so if the contestant switches, the chance of winning is 2/3.
Generalizing the problem. Let's call the number of doors D, the number of bad prizes B and the number of good prizes G, where G + B = D.
Change the rules so that there can be three doors or more with at least two bad prizes. (If you have a bad prize, the host still needs to show a bad prize.)
If you don't switch, the odds are G/D.
After doing some algebra, we see that the are G/D * (D-1)/(D-2). The second fraction is of the form big/little, so it must be more than 1. What this means is that no matter how many goats (at least two or the game doesn't work) and how many cars (at least one or you can only pick goats), it always makes sense to switch.
Monday, March 31, 2014
Notes for March 25 and 27
The first hypothesis tests we studied were checking to see if an
experimental sample produced a value that was significantly different
from some known value produced either by math or by earlier experiments.
For example, in the lady tasting tea, since she has two choices each time a mixture is given to her, the math would say that her chance of getting it right by just guessing is 50% or H0: p = .5. In testing psychic abilities, there are five different symbols on the cards, so random guessing should get the right answer 1 out of 5 times, or 20%, so H0: p = .2.
In a test for average human body temperature, the assumption of 98.6 degrees Fahrenheit being the average came from an experiment performed in the 19th Century.
We can also do tests by taking samples from two different populations. The null hypothesis, as always, is an equality, the assumption that the parameters from the two different populations are the same. As always, we need convincing evidence that the difference is significant to reject the null hypothesis, and we can choose just how convincing that evidence must be by setting the confidence level, which is usually either 90% or 95% or 99%.
Two proportions from two populations

Like with the one proportion test, the test statistic is a z-score. We have the proportions from the two samples, p-hat1 = f1/n1 and p-hat2 = f2/n2, but we also need to create the pooled proportion p-bar = (f1 + f2)/(n1 + n2).
Here's an example from the polling data from last year.
Question: Was John McCain's popularity in 2008 Iowa significantly different from his popularity in Pennsylvania?
Let's assume we don't know either way, so it will be a two tailed test. Polling data traditionally uses the 95% confidence level, so that means the z-score will have to be either greater than or equal to 1.96 or less than or equal to -1.96 for us to reject the null hypothesis. Here are our numbers, with Iowa as the first data set.
f1 = 263
n1 = 658
p-hat1 = .400
f2 = 283
n2 = 657
p-hat2 = .430
p-bar = (263+283)/(658+657) = .415 (q-bar = .585)
Type this into your calculator.
(.400-.430)/sqrt(.415x.585/658+.415x.585/657[enter]
The answer is -1.103..., which rounds to -1.10. This would say the difference we see in the two samples is not enough to convince us of a significant difference in popularity for McCain between the two states, so we would fail to reject the null hypothesis. In the actual election, McCain had 45.2% of the vote in Pennsylvania and 44.8% of the vote in Iowa, which are fairly close to equal.
Student's t-scores
Confidence intervals are used over and over again in statistics, most especially in trying to find out what value the parameter of the population has, when all we can effectively gather is a statistic from a sample. For numerical data, we aren't allowed to use the normal distribution table for this process, because the standard deviation sx of a sample isn't a very precise estimator of sigmax of the underlying population. To deal with this extra level of uncertainty, a statistician named William Gossett came up with the t-score distribution, also known as Student's t-score because Gossett published all his work under the pseudonym Student. He used this fake name for publishing to get around a ban on publishing in journals established by his superiors at the Guinness Brewing Company where he worked.
The critical t-score values are published on table A-3. The values depend on the Degrees of Freedom, which in the case of a single sample set of data is equal to n-1. For every degree of freedom, we could have another positive and negative t-score table two pages long, just like the z-score table, but that would take up way too much room, so statistics textbooks have reverted instead to publishing just the highlights. There are five columns on the table, each column labeled with a number from "Area in One Tail" and "Area in Two Tails". Let's look at the degrees of freedom row for the number 13.
1 tail___0.005______0.01_____0.025______0.05______0.10
2 tails__0.01_______0.02_____0.05_______0.10______0.20
13_______3.012_____2.650_____2.160_____1.771_____1.333
What this means is that if we have a sample of size 14, then the degrees of freedom are 13 and we can use these numbers to find the cut-off points for certain percentages. The formula for t-scores looks like the formula for z-scores, where z = (x - mux)/sigmax and t = z = (x - x-bar)/sx. Because we don't know sigmax, we use the t-score table. For example, the second column in row 13 is the number 2.650. This means that in a sample of 14, a Student's t-score of -2.650 is the cutoff for the bottom 1%, the t-score of +2.650 is the cutoff for the top 1% and the middle 98% is between t-scores of -2.650 and +2.650.
Using the t-score table
Let's say we have a t-score of 2.53 and n = 25, which means the degrees of freedom are 25-1 = 24. Here is the line of the t-score table that corresponds to d.f. = 24.
1 tail___0.005______0.01_____0.025______0.05______0.10
2 tails__0.01_______0.02_____0.05_______0.10______0.20
24_______2.797_____2.492_____2.064_____1.711_____1.318
What does this mean for our t-score of 2.53. If it was a z-score, the look-up table would give us an answer to four digits, .9943, which is a score that would be beyond the 99% confidence threshold for one tail (.9943 > .9500) but not beyond the confidence interval for 99% confidence and one tail because those thresholds are .9950 high and .0050 low. On the t-score table, all we can say is 2.53 is between 2.797 and 2.492, the closest scores on our line. In an two tailed test, it is beyond the 0.02 threshold (which would be 98% confidence, a number we don't use much) but not beyond the 99% threshold. In a one tailed (high) test our t-score is between 0.005 and 0.01, which means it passes the 99% threshold. Unlike the z-score table, t-scores only work with positive values, so if we get a negative t-score test, we follow these rules.
1. You have a negative t-score and the test is two tailed. Take the absolute value of the t-score and work with it.
2. You have a negative t-score and the test is one tailed low. Again, the absolute value will work.
3. You have a positive t-score and the test is one tailed low. This would be a problem, since only a negative t-score is useful in a one-tailed low test. You should fail to reject H0.
In the example below, we have yet another choice which always lets a one-tailed test be a one-tailed high test.
Two averages from two populations

In the tests to see if the average of some numerical value is significantly different when comparing two populations, we need the averages, standard deviations and sizes of both populations. The score we use is a t-score and the degrees of freedom is the smaller of the two sample sizes minus 1.
Question: Do female Laney students sleep a number of hours each night different from male Laney students?
This uses data sets from a previous class. Here are the numbers for the students who submitted data, with the males listed as group #1. Again, let's assume a two-tailed test, since we don't have any information going in which should be greater, and let's do this test to 90% level of confidence.
With a test like this, we can arbitrary choose which set is the first set and which is the second. Let's do it so x-bar1 > x-bar2. This way, our t-score will be positive, which is what the table expects.
H0: mu1 = mu2 (average hours of sleep are the same for males and females at Laney)
x-bar1 = 7.54
s1 = 1.47
n1 = 12
x-bar2 = 7.31
s2 = .94
n2 = 26
The degrees of freedom will be 12-1=11, and 10% in two tails gives us the thresholds of +/-1.796. Here is what to type into the calculator.
(7.54-7.31)/sqrt(1.47^2/26+.94^2/12)[enter]
0.4971...
2 tails__0.01_______0.02_____0.05_______0.10______0.20
11_______3.106_____2.718_____2.201_____1.796_____1.383
This number is less than every threshold, and so does not impress us enough to make us reject the null hypothesis. It's possible that larger samples would give us numbers that would show a difference, which if true would mean this example produced a Type II error, but we have no proof of that.
For example, in the lady tasting tea, since she has two choices each time a mixture is given to her, the math would say that her chance of getting it right by just guessing is 50% or H0: p = .5. In testing psychic abilities, there are five different symbols on the cards, so random guessing should get the right answer 1 out of 5 times, or 20%, so H0: p = .2.
In a test for average human body temperature, the assumption of 98.6 degrees Fahrenheit being the average came from an experiment performed in the 19th Century.
We can also do tests by taking samples from two different populations. The null hypothesis, as always, is an equality, the assumption that the parameters from the two different populations are the same. As always, we need convincing evidence that the difference is significant to reject the null hypothesis, and we can choose just how convincing that evidence must be by setting the confidence level, which is usually either 90% or 95% or 99%.
Two proportions from two populations

Like with the one proportion test, the test statistic is a z-score. We have the proportions from the two samples, p-hat1 = f1/n1 and p-hat2 = f2/n2, but we also need to create the pooled proportion p-bar = (f1 + f2)/(n1 + n2).
Here's an example from the polling data from last year.
Question: Was John McCain's popularity in 2008 Iowa significantly different from his popularity in Pennsylvania?
Let's assume we don't know either way, so it will be a two tailed test. Polling data traditionally uses the 95% confidence level, so that means the z-score will have to be either greater than or equal to 1.96 or less than or equal to -1.96 for us to reject the null hypothesis. Here are our numbers, with Iowa as the first data set.
f1 = 263
n1 = 658
p-hat1 = .400
f2 = 283
n2 = 657
p-hat2 = .430
p-bar = (263+283)/(658+657) = .415 (q-bar = .585)
Type this into your calculator.
(.400-.430)/sqrt(.415x.585/658+.415x.585/657[enter]
The answer is -1.103..., which rounds to -1.10. This would say the difference we see in the two samples is not enough to convince us of a significant difference in popularity for McCain between the two states, so we would fail to reject the null hypothesis. In the actual election, McCain had 45.2% of the vote in Pennsylvania and 44.8% of the vote in Iowa, which are fairly close to equal.
Student's t-scores
Confidence intervals are used over and over again in statistics, most especially in trying to find out what value the parameter of the population has, when all we can effectively gather is a statistic from a sample. For numerical data, we aren't allowed to use the normal distribution table for this process, because the standard deviation sx of a sample isn't a very precise estimator of sigmax of the underlying population. To deal with this extra level of uncertainty, a statistician named William Gossett came up with the t-score distribution, also known as Student's t-score because Gossett published all his work under the pseudonym Student. He used this fake name for publishing to get around a ban on publishing in journals established by his superiors at the Guinness Brewing Company where he worked.
The critical t-score values are published on table A-3. The values depend on the Degrees of Freedom, which in the case of a single sample set of data is equal to n-1. For every degree of freedom, we could have another positive and negative t-score table two pages long, just like the z-score table, but that would take up way too much room, so statistics textbooks have reverted instead to publishing just the highlights. There are five columns on the table, each column labeled with a number from "Area in One Tail" and "Area in Two Tails". Let's look at the degrees of freedom row for the number 13.
1 tail___0.005______0.01_____0.025______0.05______0.10
2 tails__0.01_______0.02_____0.05_______0.10______0.20
13_______3.012_____2.650_____2.160_____1.771_____1.333
What this means is that if we have a sample of size 14, then the degrees of freedom are 13 and we can use these numbers to find the cut-off points for certain percentages. The formula for t-scores looks like the formula for z-scores, where z = (x - mux)/sigmax and t = z = (x - x-bar)/sx. Because we don't know sigmax, we use the t-score table. For example, the second column in row 13 is the number 2.650. This means that in a sample of 14, a Student's t-score of -2.650 is the cutoff for the bottom 1%, the t-score of +2.650 is the cutoff for the top 1% and the middle 98% is between t-scores of -2.650 and +2.650.
Using the t-score table
Let's say we have a t-score of 2.53 and n = 25, which means the degrees of freedom are 25-1 = 24. Here is the line of the t-score table that corresponds to d.f. = 24.
1 tail___0.005______0.01_____0.025______0.05______0.10
2 tails__0.01_______0.02_____0.05_______0.10______0.20
24_______2.797_____2.492_____2.064_____1.711_____1.318
What does this mean for our t-score of 2.53. If it was a z-score, the look-up table would give us an answer to four digits, .9943, which is a score that would be beyond the 99% confidence threshold for one tail (.9943 > .9500) but not beyond the confidence interval for 99% confidence and one tail because those thresholds are .9950 high and .0050 low. On the t-score table, all we can say is 2.53 is between 2.797 and 2.492, the closest scores on our line. In an two tailed test, it is beyond the 0.02 threshold (which would be 98% confidence, a number we don't use much) but not beyond the 99% threshold. In a one tailed (high) test our t-score is between 0.005 and 0.01, which means it passes the 99% threshold. Unlike the z-score table, t-scores only work with positive values, so if we get a negative t-score test, we follow these rules.
1. You have a negative t-score and the test is two tailed. Take the absolute value of the t-score and work with it.
2. You have a negative t-score and the test is one tailed low. Again, the absolute value will work.
3. You have a positive t-score and the test is one tailed low. This would be a problem, since only a negative t-score is useful in a one-tailed low test. You should fail to reject H0.
In the example below, we have yet another choice which always lets a one-tailed test be a one-tailed high test.
Two averages from two populations

In the tests to see if the average of some numerical value is significantly different when comparing two populations, we need the averages, standard deviations and sizes of both populations. The score we use is a t-score and the degrees of freedom is the smaller of the two sample sizes minus 1.
Question: Do female Laney students sleep a number of hours each night different from male Laney students?
This uses data sets from a previous class. Here are the numbers for the students who submitted data, with the males listed as group #1. Again, let's assume a two-tailed test, since we don't have any information going in which should be greater, and let's do this test to 90% level of confidence.
With a test like this, we can arbitrary choose which set is the first set and which is the second. Let's do it so x-bar1 > x-bar2. This way, our t-score will be positive, which is what the table expects.
H0: mu1 = mu2 (average hours of sleep are the same for males and females at Laney)
x-bar1 = 7.54
s1 = 1.47
n1 = 12
x-bar2 = 7.31
s2 = .94
n2 = 26
The degrees of freedom will be 12-1=11, and 10% in two tails gives us the thresholds of +/-1.796. Here is what to type into the calculator.
(7.54-7.31)/sqrt(1.47^2/26+.94^2/12)[enter]
0.4971...
2 tails__0.01_______0.02_____0.05_______0.10______0.20
11_______3.106_____2.718_____2.201_____1.796_____1.383
This number is less than every threshold, and so does not impress us enough to make us reject the null hypothesis. It's possible that larger samples would give us numbers that would show a difference, which if true would mean this example produced a Type II error, but we have no proof of that.
Wednesday, March 26, 2014
Notes for March 18 and 20
The topic for the next few weeks is hypothesis testing. The main idea
is that experiments must be conducted to test the validity of an idea,
which is called a hypothesis. There are always two hypotheses
available, the null hypothesis H0 (pronounced "H zero" or "H nought") and the alternate hypothesis HA (pronounced "H A").
The standard is to assume the null hypothesis is true, which says that
nothing special is happening, which in most cases means that two things
we can measure should be equal or close to it. The alternate
hypothesis says the two measurements are different. We can have one
tailed high tests, where we want "large" positive test statistics. In
one tailed low tests, only negative test statistics with "large"
absolute value will do. In two tailed test, "large" absolute value for
positive or negative numbers will work. We will only accept the
alternate hypothesis if the experiment produces impressive results given
our particular criteria for that test.

The basics of hypothesis testing are similar to the ideals of the English legal system, which is also the system used in United States courts, that a defendant is presumed innocent until proven guilty. There are different levels of proof of guilt in different trials, whether it is beyond a reasonable doubt or the less rigorous standard of preponderance of evidence.
In a case involving an alleged crime, there is the reality of what the defendant did and the result of the trial. If the defendant did the illegal act, then being found guilty is the correct result under the law. If the reality is that the defendant didn't do the act, the correct result would be a not guilty verdict.
The reasonable doubt standard is put in place in theory to make sending an innocent person to jail unlikely, and this is called a Type I error. The best known Type I result in legal history is Jesus Christ.
It is also possible that a person who did a crime will be found not guilty. This is called a Type II error. When I ask my students for an example of Type II error, O.J. Simpson's name still rings out the loudest.

In hypothesis testing, there is the reality and the result of the experiment. If H0 is true, the two things measured are equal or pretty close to equal. If HA is true, they are significantly different.
If the experiment produces a test statistic that is beyond the threshold we set for it, and "beyond" could mean lower if it is a one-tailed low test, or higher if it is a one-tailed high test, or either lower or higher in a two-tailed test, then we reject H0. If the test statistic fails to get "beyond" the threshold, we fail to reject H0.
Rejecting a true null hypothesis is a Type I error. Rejecting a false alternate hypothesis is a Type II error.
In class, we discussed Sir Ronald Fisher and his hypothesis testing of
the lady who said she could tell the difference in taste between tea
poured into milk or milk poured into tea.
Here are the things that have to be done to make such an experiment work.
#1 Define the null hypothesis. In modern experiments, the null hypothesis is always defined as an equation. In a proportion test, the equation will be concerning p, the true probability of success. In the lady tasting tea, we would assume if nothing special is happening, then she is just guessing whether the tea or milk was poured first, and the probability of being correct on any given trial is 50% or .5. We write this as follows.
H0: p = .5
#2 Pick a threshold. The trials we are going to perform are taste tests where the lady cannot see the tea-milk mixture being poured. We have to decide on how high a test statistic we will consider impressive. The three standard choices are 90% confidence, 95% confidence or 99% confidence. For experiments in the medical field, where the decision is whether or not to bring a new drug to market, the 99% confidence level is common. For an experiment like this, where the result is not truly earth shattering, we might decide to us the 95% confidence threshold.
The experiment will produce a z-score the thresholds for high z-scores are as follows:
90% threshold: z = 1.28
95% threshold: z = 1.645
99% threshold: z = 2.325
#3 Decide on the number of trials in an experiment. There is a tug-of-war in deciding the number of trials. More trials produces numbers we can be more confident in, but more trials is also more expensive and more time consuming. In the case of the lady tasting tea, we don't want to keep her drinking tea mixtures for hours.
Different books set different standards for the minimum number of trials based on np and nq. Some say both np > 5 and nq > 5. Others say both numbers should be greater than 10, yet others say 15. The standard that np >= 10 and nq >= 10 can be connected to the standard that says n > np + 3*sqrt(pqn) > np - 3*sqrt(pqn) > 0 by a little algebraic manipulation.
np - 3*sqrt(pqn) > 0 [add the square root to both sides]
np > 3*sqrt(pqn) [square both sides]
n^2*p^2 > 9pqn [divide both sides by np]
np > 9q
Since q must be less than 1, but can be as close to 1 as we want, set it equal to 1 and the inequality becomes
np > 9, which we can change to np >= 10.
For example, If let's look at the different possible positive z-score results if the lady were given ten trials, which would be enough if we used the lowest standard of np >= 5 and nq >= 5.
10 correct out of 10: z = 3.16227... ~= 3.16, which is above the 99% threshold.
(look-up table: .9992)
9 correct out of 10: z = 2.52982... ~= 2.53, which is above the 99% threshold.
(look-up table: .9943)
8 correct out of 10: z = 1.89737... ~= 1.90, which is above the 95% threshold, but not the 99%.
(look-up table: .9713)
7 correct out of 10: z = 1.26491... ~= 1.26, which is below the 90% threshold.
(look-up table: .8962)
If we set the bar at the 90% threshold, she could impress us by getting 8 right out of 10 or better. Likewise, at the 95% threshold, 8 of 10 will be beyond the threshold and the result would make us reject H0. At the 99% threshold, she would have to get 9 of 10 or 10 of 10 to make a z-score that breaks the threshold.
#4 Interpreting the test statistic. Let's say for the sake of argument that the lady got 8 of 10 correct. (There is a book about 20th Century statistics entitled The Lady Tasting Tea, where a witness to the experiment says he can't recall how many times she was tested, but the lady got a perfect score.) If we set the threshold at 90% confidence or 95% confidence, we would be impressed by the z score of 1.90 and we would reject H0, which says that we don't think she is "just guessing", but actually has the talent she says she has. If we set the value at 99% confidence, we would fail to reject H0.
Here's the thing. We could be wrong. If we reject H0 incorrectly, it means she was just guessing and she was very lucky during this test, a Type I error. A z-score of 1.90 corresponds to a probability of 97.13%, which is called the p-value in hypothesis testing. She can pass the test by being better than 97.13% of lucky guessers. If she is a lucky guesser, she would fool anyone who had set the threshold at 90% or 95%.
If we fail to reject H0, which we would do if we set the threshold at 99%, this could also be an error, but this time it would be a Type II error. Under this scenario, she got 8 of 10 but she should do better usually. It's more difficult math to figure out how good she actually is and how unlucky she had to be to get only 8 of 10. The probability of a Type I error is called alpha, and it is determined by the threshold. The probability of a Type II error is called beta, and it usually explained in greater detail in the class after the introduction to statistics.
A low tailed test
Let's say we want a low error rate. Unlike the lady tasting tea who needed a lot of right answers to impress us, now we need a low score to get a result that will make us reject the null hypothesis.
Now our z-score thresholds are
90% threshold: z = -1.28
95% threshold: z = -1.645
99% threshold: z = -2.325
Let's say we want to be convinced our error rate is less than 10% and we want to be convinced to the 95% confidence level.
H0: p = 0.10
HA: p < 0.10
n = 50
10% of 50 is 5, so we should check to see what happens at f = 4, 3, 2, 1 and 0 errors. Typing this into the calculator will look like
(f/50 - .1)/sqrt(.1*.9/50)
f = 4 gives a rounded z-score of -.47
(look-up table: .3192 fail to reject H0)
f = 3 gives a rounded z-score of -.94
(look-up table: .1736 fail to reject H0)
f = 2 gives a rounded z-score of -1.41
(look-up table: .0793 fail to reject H0 at 95% confidence, but reject at 90%)
f = 1 gives a rounded z-score of -1.89
(look-up table: .0294 reject H0 at 95% confidence, but fail reject at 99%)
f = 0 gives a rounded z-score of -2.36
(look-up table: .0091 reject H0 at any confidence level we use 90%, 95% or 99%)

The basics of hypothesis testing are similar to the ideals of the English legal system, which is also the system used in United States courts, that a defendant is presumed innocent until proven guilty. There are different levels of proof of guilt in different trials, whether it is beyond a reasonable doubt or the less rigorous standard of preponderance of evidence.
In a case involving an alleged crime, there is the reality of what the defendant did and the result of the trial. If the defendant did the illegal act, then being found guilty is the correct result under the law. If the reality is that the defendant didn't do the act, the correct result would be a not guilty verdict.
The reasonable doubt standard is put in place in theory to make sending an innocent person to jail unlikely, and this is called a Type I error. The best known Type I result in legal history is Jesus Christ.
It is also possible that a person who did a crime will be found not guilty. This is called a Type II error. When I ask my students for an example of Type II error, O.J. Simpson's name still rings out the loudest.

In hypothesis testing, there is the reality and the result of the experiment. If H0 is true, the two things measured are equal or pretty close to equal. If HA is true, they are significantly different.
If the experiment produces a test statistic that is beyond the threshold we set for it, and "beyond" could mean lower if it is a one-tailed low test, or higher if it is a one-tailed high test, or either lower or higher in a two-tailed test, then we reject H0. If the test statistic fails to get "beyond" the threshold, we fail to reject H0.
Rejecting a true null hypothesis is a Type I error. Rejecting a false alternate hypothesis is a Type II error.
In class, we discussed Sir Ronald Fisher and his hypothesis testing of
the lady who said she could tell the difference in taste between tea
poured into milk or milk poured into tea.Here are the things that have to be done to make such an experiment work.
#1 Define the null hypothesis. In modern experiments, the null hypothesis is always defined as an equation. In a proportion test, the equation will be concerning p, the true probability of success. In the lady tasting tea, we would assume if nothing special is happening, then she is just guessing whether the tea or milk was poured first, and the probability of being correct on any given trial is 50% or .5. We write this as follows.
H0: p = .5
#2 Pick a threshold. The trials we are going to perform are taste tests where the lady cannot see the tea-milk mixture being poured. We have to decide on how high a test statistic we will consider impressive. The three standard choices are 90% confidence, 95% confidence or 99% confidence. For experiments in the medical field, where the decision is whether or not to bring a new drug to market, the 99% confidence level is common. For an experiment like this, where the result is not truly earth shattering, we might decide to us the 95% confidence threshold.
The experiment will produce a z-score the thresholds for high z-scores are as follows:
90% threshold: z = 1.28
95% threshold: z = 1.645
99% threshold: z = 2.325
#3 Decide on the number of trials in an experiment. There is a tug-of-war in deciding the number of trials. More trials produces numbers we can be more confident in, but more trials is also more expensive and more time consuming. In the case of the lady tasting tea, we don't want to keep her drinking tea mixtures for hours.
Different books set different standards for the minimum number of trials based on np and nq. Some say both np > 5 and nq > 5. Others say both numbers should be greater than 10, yet others say 15. The standard that np >= 10 and nq >= 10 can be connected to the standard that says n > np + 3*sqrt(pqn) > np - 3*sqrt(pqn) > 0 by a little algebraic manipulation.
np - 3*sqrt(pqn) > 0 [add the square root to both sides]
np > 3*sqrt(pqn) [square both sides]
n^2*p^2 > 9pqn [divide both sides by np]
np > 9q
Since q must be less than 1, but can be as close to 1 as we want, set it equal to 1 and the inequality becomes
np > 9, which we can change to np >= 10.
For example, If let's look at the different possible positive z-score results if the lady were given ten trials, which would be enough if we used the lowest standard of np >= 5 and nq >= 5.
10 correct out of 10: z = 3.16227... ~= 3.16, which is above the 99% threshold.
(look-up table: .9992)
9 correct out of 10: z = 2.52982... ~= 2.53, which is above the 99% threshold.
(look-up table: .9943)
8 correct out of 10: z = 1.89737... ~= 1.90, which is above the 95% threshold, but not the 99%.
(look-up table: .9713)
7 correct out of 10: z = 1.26491... ~= 1.26, which is below the 90% threshold.
(look-up table: .8962)
If we set the bar at the 90% threshold, she could impress us by getting 8 right out of 10 or better. Likewise, at the 95% threshold, 8 of 10 will be beyond the threshold and the result would make us reject H0. At the 99% threshold, she would have to get 9 of 10 or 10 of 10 to make a z-score that breaks the threshold.
#4 Interpreting the test statistic. Let's say for the sake of argument that the lady got 8 of 10 correct. (There is a book about 20th Century statistics entitled The Lady Tasting Tea, where a witness to the experiment says he can't recall how many times she was tested, but the lady got a perfect score.) If we set the threshold at 90% confidence or 95% confidence, we would be impressed by the z score of 1.90 and we would reject H0, which says that we don't think she is "just guessing", but actually has the talent she says she has. If we set the value at 99% confidence, we would fail to reject H0.
Here's the thing. We could be wrong. If we reject H0 incorrectly, it means she was just guessing and she was very lucky during this test, a Type I error. A z-score of 1.90 corresponds to a probability of 97.13%, which is called the p-value in hypothesis testing. She can pass the test by being better than 97.13% of lucky guessers. If she is a lucky guesser, she would fool anyone who had set the threshold at 90% or 95%.
If we fail to reject H0, which we would do if we set the threshold at 99%, this could also be an error, but this time it would be a Type II error. Under this scenario, she got 8 of 10 but she should do better usually. It's more difficult math to figure out how good she actually is and how unlucky she had to be to get only 8 of 10. The probability of a Type I error is called alpha, and it is determined by the threshold. The probability of a Type II error is called beta, and it usually explained in greater detail in the class after the introduction to statistics.
A low tailed test
Let's say we want a low error rate. Unlike the lady tasting tea who needed a lot of right answers to impress us, now we need a low score to get a result that will make us reject the null hypothesis.
Now our z-score thresholds are
90% threshold: z = -1.28
95% threshold: z = -1.645
99% threshold: z = -2.325
Let's say we want to be convinced our error rate is less than 10% and we want to be convinced to the 95% confidence level.
H0: p = 0.10
HA: p < 0.10
n = 50
10% of 50 is 5, so we should check to see what happens at f = 4, 3, 2, 1 and 0 errors. Typing this into the calculator will look like
(f/50 - .1)/sqrt(.1*.9/50)
f = 4 gives a rounded z-score of -.47
(look-up table: .3192 fail to reject H0)
f = 3 gives a rounded z-score of -.94
(look-up table: .1736 fail to reject H0)
f = 2 gives a rounded z-score of -1.41
(look-up table: .0793 fail to reject H0 at 95% confidence, but reject at 90%)
f = 1 gives a rounded z-score of -1.89
(look-up table: .0294 reject H0 at 95% confidence, but fail reject at 99%)
f = 0 gives a rounded z-score of -2.36
(look-up table: .0091 reject H0 at any confidence level we use 90%, 95% or 99%)
Labels:
hypothesis testing,
p-values,
proportions,
test statistic,
z-scores
Monday, October 25, 2010
More on hypothesis testing
True false questions about hypothesis testing.
The basic facts about hypothesis testing.

Practice problems
In testing for psychic powers, researchers use a deck with five different shapes, as shown in the picture on the left. If the deck is re-shuffled every time, the probability of guessing correctly by pure chance is 1/5 or p = .2 written in decimal. The test would be one tailed high, and we use the z-score table, so the threshold for 95% confidence is = 1.645 and the threshold for 99% confidence is z = 2.325.
Questions:
1. If a subject gets 3 out of 10 correct in a psychic test, are we 95% confident the subject shows psychic powers?
2. If a subject gets 4 out of 10 correct in a psychic test, are we 95% confident the subject shows psychic powers? Are we 99% confident?
3. If a subject gets 5 out of 10 correct in a psychic test, are we 95% confident the subject shows psychic powers? Are we 99% confident?
4. If a subject gets 30 out of 100 correct in a psychic test, are we 95% confident the subject shows psychic powers? Are we 99% confident?
5. If n = 100 and p = .2 in a high one tailed test, find the minimum number of correct answers for rejecting H0 to the 95% confidence level and the 90% confidence level.
Answers in the first comment.
Bonus questions
We have a sample with n = 40, x-bar = 172.5 and sx = 119.5.
1. What is the one tailed low threshold for 95% confidence?
2. What is the one tailed high threshold for 99% confidence?
3. If H0 is mux = 200, are we 95% confident we can reject this for HA: mux < 200?
4. If H0 is mux = 100, are we 99% confident we can reject this for HA: mux > 100?
Answers in second comment.
The basic facts about hypothesis testing.

Practice problems
In testing for psychic powers, researchers use a deck with five different shapes, as shown in the picture on the left. If the deck is re-shuffled every time, the probability of guessing correctly by pure chance is 1/5 or p = .2 written in decimal. The test would be one tailed high, and we use the z-score table, so the threshold for 95% confidence is = 1.645 and the threshold for 99% confidence is z = 2.325.
Questions:
1. If a subject gets 3 out of 10 correct in a psychic test, are we 95% confident the subject shows psychic powers?
2. If a subject gets 4 out of 10 correct in a psychic test, are we 95% confident the subject shows psychic powers? Are we 99% confident?
3. If a subject gets 5 out of 10 correct in a psychic test, are we 95% confident the subject shows psychic powers? Are we 99% confident?
4. If a subject gets 30 out of 100 correct in a psychic test, are we 95% confident the subject shows psychic powers? Are we 99% confident?
5. If n = 100 and p = .2 in a high one tailed test, find the minimum number of correct answers for rejecting H0 to the 95% confidence level and the 90% confidence level.
Answers in the first comment.
Bonus questions
We have a sample with n = 40, x-bar = 172.5 and sx = 119.5.
1. What is the one tailed low threshold for 95% confidence?
2. What is the one tailed high threshold for 99% confidence?
3. If H0 is mux = 200, are we 95% confident we can reject this for HA: mux < 200?
4. If H0 is mux = 100, are we 99% confident we can reject this for HA: mux > 100?
Answers in second comment.
Monday, July 20, 2009
Test results and errors
We do the hypothesis testing because we cannot truly know what reality is, only the test result. If we reject the null hypothesis Ho, we did so because of strong evidence. If there is an error, it is a Type I error. If we set the error threshold at 90% confidence, we expect such errors about 10% of the time. If it is set at 95% confidence, then Type I errors should happen about 5% of the time and at 99% confidence, Type I errors should only happen about 1% of the time.
If we fail to reject H0, the only type of error we can make is called Type II error. The probability of such errors is trickier to compute and we will not work on this problem during this class.
If we fail to reject H0, the only type of error we can make is called Type II error. The probability of such errors is trickier to compute and we will not work on this problem during this class.
Thursday, May 14, 2009
Class notes for 5/13: Two population tests for proportions and averages
The first hypothesis tests we studied were checking to see if an experimental sample produced a value that was significantly different from some known value produced either by math or by earlier experiments.
For example, in the lady tasting tea, since she has two choices each time a mixture is given to her, the math would say that her chance of getting it right by just guessing is 50% or H0: p = .5. In testing psychic abilities, there are five different symbols on the cards, so random guessing should get the right answer 1 out of 5 times, or 20%, so H0: p = .2.
In a test for average human body temperature, the assumption of 98.6 degrees Fahrenheit being the average came from an experiment performed in the 19th Century.
We can also do tests by taking samples from two different populations. The null hypothesis, as always, is an equality, the assumption that the parameters from the two different populations are the same. As always, we need convincing evidence that the difference is significant to reject the null hypothesis, and we can choose just how convincing that evidence must be by setting the confidence level, which is usually either 90% or 95% or 99%.
Two proportions from two populations

Like with the one proportion test, the test statistic is a z-score. We have the proportions from the two samples, p-hat1 = f1/n1 and p-hat2 = f2/n2, but we also need to create the pooled proportion p-bar = (f1 + f2)/(n1 + n2).
Here's an example from the polling data from last year.
Question: Was John McCain's popularity in Iowa significantly different from his popularity in Pennsylvania?
Let's assume we don't know either way, so it will be a two tailed test. Polling data traditionally uses the 95% confidence level, so that means the z-score will have to be either greater than or equal to 1.96 or less than or equal to -1.96 for us to reject the null hypothesis. Here are our numbers, with Iowa as the first data set.
f1 = 263
n1 = 658
p-hat1 = .400
f2 = 283
n2 = 657
p-hat2 = .430
p-bar = (263+283)/(658+657) = .415 (q-bar = .585)
Type this into your calculator.
(.400-.430)/sqrt(.415x.585/658+.415x.585/657[enter]
The answer is -1.103..., which rounds to -1.10. This would say the difference we see in the two samples is not enough to convince us of a significant difference in popularity for McCain between the two states, so we would fail to reject the null hypothesis. In the actual election, McCain had 45.2% of the vote in Pennsylvania and 44.8% of the vote in Iowa, which are fairly close to equal.
Two averages from two populations

In the tests to see if the average of some numerical value is significantly different when comparing two populations, we need the averages, standard deviations and sizes of both populations. The score we use is a t-score and the degrees of freedom is the smaller of the two sample sizes minus 1.
Question: Do female Laney students sleep more hours each night than male Laney students?
We will take our data from the larger of the two class surveys, Data Set #2. Here are the numbers for the students who submitted data, with the females listed as group #1. Again, let's assume a two-tailed test, since we don't have any information going in which should be greater, and let's do this test to 90% level of confidence.
H0: mu1 = mu2 (average hours of sleep are the same for males and females at Laney)
x-bar1 = 7.31
s1 = .94
n1 = 26
x-bar2 = 7.54
s2 = 1.47
n2 = 12
The degrees of freedom will be 12-1=11, and 10% in two tails gives us the thresholds of +/-1.796. Here is what to type into the calculator.
(7.31-7.54)/sqrt(.94^2/26+1.47^2/12)[enter]
-0.4971...
This number is between the thresholds, and so does not impress us enough to make us reject the null hypothesis. It's possible that larger samples would give us numbers that would show a difference, which if true would mean this example produced a Type II error, but we have no proof of that.
For example, in the lady tasting tea, since she has two choices each time a mixture is given to her, the math would say that her chance of getting it right by just guessing is 50% or H0: p = .5. In testing psychic abilities, there are five different symbols on the cards, so random guessing should get the right answer 1 out of 5 times, or 20%, so H0: p = .2.
In a test for average human body temperature, the assumption of 98.6 degrees Fahrenheit being the average came from an experiment performed in the 19th Century.
We can also do tests by taking samples from two different populations. The null hypothesis, as always, is an equality, the assumption that the parameters from the two different populations are the same. As always, we need convincing evidence that the difference is significant to reject the null hypothesis, and we can choose just how convincing that evidence must be by setting the confidence level, which is usually either 90% or 95% or 99%.
Two proportions from two populations

Like with the one proportion test, the test statistic is a z-score. We have the proportions from the two samples, p-hat1 = f1/n1 and p-hat2 = f2/n2, but we also need to create the pooled proportion p-bar = (f1 + f2)/(n1 + n2).
Here's an example from the polling data from last year.
Question: Was John McCain's popularity in Iowa significantly different from his popularity in Pennsylvania?
Let's assume we don't know either way, so it will be a two tailed test. Polling data traditionally uses the 95% confidence level, so that means the z-score will have to be either greater than or equal to 1.96 or less than or equal to -1.96 for us to reject the null hypothesis. Here are our numbers, with Iowa as the first data set.
f1 = 263
n1 = 658
p-hat1 = .400
f2 = 283
n2 = 657
p-hat2 = .430
p-bar = (263+283)/(658+657) = .415 (q-bar = .585)
Type this into your calculator.
(.400-.430)/sqrt(.415x.585/658+.415x.585/657[enter]
The answer is -1.103..., which rounds to -1.10. This would say the difference we see in the two samples is not enough to convince us of a significant difference in popularity for McCain between the two states, so we would fail to reject the null hypothesis. In the actual election, McCain had 45.2% of the vote in Pennsylvania and 44.8% of the vote in Iowa, which are fairly close to equal.
Two averages from two populations

In the tests to see if the average of some numerical value is significantly different when comparing two populations, we need the averages, standard deviations and sizes of both populations. The score we use is a t-score and the degrees of freedom is the smaller of the two sample sizes minus 1.
Question: Do female Laney students sleep more hours each night than male Laney students?
We will take our data from the larger of the two class surveys, Data Set #2. Here are the numbers for the students who submitted data, with the females listed as group #1. Again, let's assume a two-tailed test, since we don't have any information going in which should be greater, and let's do this test to 90% level of confidence.
H0: mu1 = mu2 (average hours of sleep are the same for males and females at Laney)
x-bar1 = 7.31
s1 = .94
n1 = 26
x-bar2 = 7.54
s2 = 1.47
n2 = 12
The degrees of freedom will be 12-1=11, and 10% in two tails gives us the thresholds of +/-1.796. Here is what to type into the calculator.
(7.31-7.54)/sqrt(.94^2/26+1.47^2/12)[enter]
-0.4971...
This number is between the thresholds, and so does not impress us enough to make us reject the null hypothesis. It's possible that larger samples would give us numbers that would show a difference, which if true would mean this example produced a Type II error, but we have no proof of that.
Tuesday, May 12, 2009
Class notes for 5/11: Hypothesis testing for the mean of a population
t-scores and p values
If we have a z-score between -3.5 and +3.5, Table A-2 lets us find the p value associated with that z-score accurate to four decimal places. For example, if z = 1.71, the p value is .9564, which is to say that z-score is higher than 95.64% of data in a normally distributed set.
The t-score table is smaller, and to read a t-score correctly, we also need n, the size of the sample, because that gives us the Degrees of Freedom, which is n-1 in this case.
If n=10, then d.f. = 9, and the t-score table reads as follows.
___________________________Area in One Tail_____________
_______0.005______0.01______0.025______0.05______0.10__________df=9___3.250_____2.821______2.262_____1.833_____1.383__________
If t = 1.71, that value isn't on our table, but because it lies between the values associated with 0.05 and 0.10, that means that score is in the top 10% of scores, but not in the top 5%.
If instead n=30 and d.f. = 29, here are the t-score values.
___________________________Area in One Tail_____________
_______0.005______0.01______0.025______0.05______0.10__________df=29__2.756_____2.462______2.045_____1.699_____1.311__________
Now a t-score of 1.71 lies between 2.045 and 1.699, which means it is in the top 5%, but not the top 2.5%.
Like the z-score table, the t-score table is symmetric about the value t=0. If d.f.=29, t=-1.71 is a score in the bottom 5%, but not the bottom 2.5%.
Hypothesis testing for the mean of a population

Hypothesis testing for the mean of a population assumes we know the population mean from some previously obtained information. Perhaps that mean has changed over time or the previous information wasn't correct to begin with, but the null hypothesis assumes we know that mean, which we call mux. If we take a sample from the population, we will get the values x-bar, sx and n, and using those values and mux, we can get the t-score.
Just like with the hypothesis test for a proportion, the test can be one-tailed high, one-tailed low or two-tailed.
For example, if we were testing people who had studied using a special method and we were checking scores on a standardized test, we would only be impressed if the average went up, so a one-tailed high test would be appropriate.
If the experiment was dealing with a cholesterol drug, we would want to see a lower average reading, and a one-tailed low test would be used.
If we assume the average duration of a pop song on the charts today is the same as the duration of pop songs in the seventies, we can't assume beforehand if the new readings will be higher or lower, and would be surprised if the new average were significantly different in either direction, so a two-tailed test would be appropriate.
Here is some data we went over in class. In most textbooks, the 'normal' human body temperature is listed at 98.6 degrees Fahrenheit, based on the work of Dr. Carl Wunderlich back in the 19th Century. If we do a test, it should be a two-tailed test, since we would be surprised if the normal temperature is significantly higher or significantly lower than this. Since this is a medical experiment, let's use the 99% level of confidence.
The size of the sample was n=103, which means the degrees of freedom are 102. Our table doesn't have a listing for d.f.=102, and the next lowest available value is d.f.=100. Here are the table values for that row of Table A-3.
___________________________Area in Two Tails____________
_________0.01______0.02_______0.05______0.10______0.20__________df=100__2.626_____2.364______1.984_____1.660_____1.290__________
With a two tailed test at the 99% confidence level, this means we want the 0.01 column. The "middle" 99% of data lies between t-scores of -2.616 and 2.616. If the t-score lies in that range, we will fail to reject H0. If it is greater or equal to 2.616 or less than or equal to -2.616, we will reject H0.
The values from this study found that x-bar = 98.2 degrees and the standard deviation sx was 0.62. Plugging into our t-score equation from above, we get
t = (98.2-98.6)/0.62*sqrt(103) = -6.547671977... ~= -6.548.
We don't get an exact p value for a number so far away from zero, but if we look at outlier z-score table, we can roughly approximate that this p value is somewhere around 1 in 1,000,000,000. We can say with 99% confidence that the average body temperature is not 98.6 degrees, but probably close to the sample average of 98.2 degrees. Our p value shows we could qualify for even greater confidence with our statement, but very rarely do tests ask for more than 99% confidence, and changing the criteria after the fact is not recommended. Still, publishing this incredibly tiny p value will convince people who can read a statistical report that the evidence is very strong indeed.
This test also changed the idea of what should constitute a fever. Instead of one temperature of 100.4 degrees Fahrenheit being the absolute gauge, the temperature will fluctuate depending on the time of day, as do the normal temperature readings.
If we have a z-score between -3.5 and +3.5, Table A-2 lets us find the p value associated with that z-score accurate to four decimal places. For example, if z = 1.71, the p value is .9564, which is to say that z-score is higher than 95.64% of data in a normally distributed set.
The t-score table is smaller, and to read a t-score correctly, we also need n, the size of the sample, because that gives us the Degrees of Freedom, which is n-1 in this case.
If n=10, then d.f. = 9, and the t-score table reads as follows.
___________________________Area in One Tail_____________
_______0.005______0.01______0.025______0.05______0.10__________df=9___3.250_____2.821______2.262_____1.833_____1.383__________
If t = 1.71, that value isn't on our table, but because it lies between the values associated with 0.05 and 0.10, that means that score is in the top 10% of scores, but not in the top 5%.
If instead n=30 and d.f. = 29, here are the t-score values.
___________________________Area in One Tail_____________
_______0.005______0.01______0.025______0.05______0.10__________df=29__2.756_____2.462______2.045_____1.699_____1.311__________
Now a t-score of 1.71 lies between 2.045 and 1.699, which means it is in the top 5%, but not the top 2.5%.
Like the z-score table, the t-score table is symmetric about the value t=0. If d.f.=29, t=-1.71 is a score in the bottom 5%, but not the bottom 2.5%.
Hypothesis testing for the mean of a population

Hypothesis testing for the mean of a population assumes we know the population mean from some previously obtained information. Perhaps that mean has changed over time or the previous information wasn't correct to begin with, but the null hypothesis assumes we know that mean, which we call mux. If we take a sample from the population, we will get the values x-bar, sx and n, and using those values and mux, we can get the t-score.
Just like with the hypothesis test for a proportion, the test can be one-tailed high, one-tailed low or two-tailed.
For example, if we were testing people who had studied using a special method and we were checking scores on a standardized test, we would only be impressed if the average went up, so a one-tailed high test would be appropriate.
If the experiment was dealing with a cholesterol drug, we would want to see a lower average reading, and a one-tailed low test would be used.
If we assume the average duration of a pop song on the charts today is the same as the duration of pop songs in the seventies, we can't assume beforehand if the new readings will be higher or lower, and would be surprised if the new average were significantly different in either direction, so a two-tailed test would be appropriate.
Here is some data we went over in class. In most textbooks, the 'normal' human body temperature is listed at 98.6 degrees Fahrenheit, based on the work of Dr. Carl Wunderlich back in the 19th Century. If we do a test, it should be a two-tailed test, since we would be surprised if the normal temperature is significantly higher or significantly lower than this. Since this is a medical experiment, let's use the 99% level of confidence.
The size of the sample was n=103, which means the degrees of freedom are 102. Our table doesn't have a listing for d.f.=102, and the next lowest available value is d.f.=100. Here are the table values for that row of Table A-3.
___________________________Area in Two Tails____________
_________0.01______0.02_______0.05______0.10______0.20__________df=100__2.626_____2.364______1.984_____1.660_____1.290__________
With a two tailed test at the 99% confidence level, this means we want the 0.01 column. The "middle" 99% of data lies between t-scores of -2.616 and 2.616. If the t-score lies in that range, we will fail to reject H0. If it is greater or equal to 2.616 or less than or equal to -2.616, we will reject H0.
The values from this study found that x-bar = 98.2 degrees and the standard deviation sx was 0.62. Plugging into our t-score equation from above, we get
t = (98.2-98.6)/0.62*sqrt(103) = -6.547671977... ~= -6.548.
We don't get an exact p value for a number so far away from zero, but if we look at outlier z-score table, we can roughly approximate that this p value is somewhere around 1 in 1,000,000,000. We can say with 99% confidence that the average body temperature is not 98.6 degrees, but probably close to the sample average of 98.2 degrees. Our p value shows we could qualify for even greater confidence with our statement, but very rarely do tests ask for more than 99% confidence, and changing the criteria after the fact is not recommended. Still, publishing this incredibly tiny p value will convince people who can read a statistical report that the evidence is very strong indeed.
This test also changed the idea of what should constitute a fever. Instead of one temperature of 100.4 degrees Fahrenheit being the absolute gauge, the temperature will fluctuate depending on the time of day, as do the normal temperature readings.
Labels:
hypothesis testing,
p-values,
Student's t-scores
Subscribe to:
Posts (Atom)






