Notes for 9/5/12

Here is a link to the topics for this Wednesday, September 5.

We will also discuss ordered and unordered categorical data and the problems with Excel using some of these ideas.

Class notes for 5/18: Regression and correlation

When we have a data set, sometimes we collect more than one variable of information about the units. For example, in our class survey, among the numerical variables were the height in inches, the GPA, the opinion about the difficulty of the class, age and average hours of sleep per night.

A question about two variables is if they are related to one another in some simple way. One simple way is correlation, which can be positive or negative. Here is a general definition of each.

Positive correlation between two numerical variables, call them x and y, means that the high values of x tend to be paired with the high values of y, the middle values of x tend to be paired with the middle values of y and the low values of x tend to be paired with the low values of y.

The variables x and y show negative correlation if that the high values of x tend to be paired with the low values of y, the middle values of x tend to be paired with the middle values of y and the low values of x tend to be paired with the high values of y.

If we pick two variables at random, we do not expect to see correlation. We can write this as a null hypothesis, where the test statistic is rx,y, the correlation coefficient. The sign of low correlation is rx,y = 0. The values of rx,y are always between -1, which means perfect negative correlation, and +1, which means perfect positive correlation.

The seventh page of the yellow sheets gives us threshold numbers for the 99% confidence level and 95% confidence level for correlation given the number of points n. For instance, when n = 5, the thresholds are .878 for 95% confidence and .959 for 99% confidence. This splits up the numbers from -1 to 1 into five regions.

-1 <= rx,y <= -.959: Very strong negative correlation
-.959 < rx,y <= -.878: Strong negative correlation
-.878 < rx,y < .878: The correlation is not particularly strong, regardless of positive or negative.
.878 <= rx,y < .959: strong positive correlation
.959 <= rx,y <= 1: very strong positive correlation

Just like with any hypothesis test, we should decide the confidence level before testing. This is a two-tailed test, because whether correlation is positive or negative, the relationships between number sets can often give us vital scientific information.

There is an important warning: Correlation is not causation. Just because two number sets have a relation, it doesn't mean that x causes y or y causes x. Sometimes there is a hidden third factor that is the cause of both of the things we are looking at. Sometimes, it's random chance and there is no causative agent at all.


Here is a set of five points, listed as (x,y) in each case.

(1,1)
(2,2)
(3,4)
(4,4)
(6,5)

As we can see, the points are ordered from low to high in both coordinates, so we expect some correlation. If we input the points into our calculator, we get a value for r (which is the same as rx,y) of .933338696..., which is strong positive correlation, but not very strong positive correlation. Assuming the 95% confidence level is good enough for us, we can use the a and b variables from out calculator to give us the equation of the line

yp = .797x + .649

This is called the predictor line (that's where the p comes from) or the line of regression or the line of least squares. Any such line for a given data set has two important criteria it meets. It passes through the centroid (x-bar, y-bar), the center point of all the data, and it minimizes the sum of the absolute values of the residuals, which is |y - yp| for all points.

Let's find the absolute values of the residuals for each of the five points, using the rounded values of a and b.

Point (1,1): |1 - .797*1 - .649| = 0.446
Point (2,2): |2 - .797*2 - .649| = 0.243
Point (3,4): |4 - .797*3 - .649| = 0.96
Point (4,4): |4 - .797*4 - .649| = 0.163
Point (6,5): |5 - .797*6 - .649| = 0.431

As we can see, the point (3,4) is farthest from the line, while the point (4,4) is the closest. The centroid (3.2, 3.2) is exactly on the line if you use the un-rounded values of a and b, and even using the rounded values, the centroid only misses the line by .0006.

In class, we used five points, but the last point was (5,6) instead of (6,5). This changes the numbers. rx,y goes up to .973328527..., which is above the 99% confidence threshold. The formula for the new predictor line is

yp = 1.2x - .2

Where we see the difference in these two different examples is in the residuals.

Point (1,1): |1 - 1.2*1 + .2| = 0
Point (2,2): |2 - 1.2*2 + .2| = 0.2
Point (3,4): |4 - 1.2*3 + .2| = 0.6
Point (4,4): |4 - 1.2*4 + .2| = 0.6
Point (5,6): |6 - 1.2*5 + .2| = 0.2

The closest point is now exactly on the line, which is a rarity, but even the farthest away point is only .6 units away, closer than the farthest away on the line with the lower correlation coefficient.

As we get more points in our data set, we lower our threshold that shows correlation strength. This way, a few points that are outliers do not completely ruin the chances of the data showing correlation, though sometimes strong outliers can mess up the data set and the correlation coefficient gets so close to zero that we cannot reject the null hypothesis that the two variables are not simply related.

Class notes for 5/13: Two population tests for proportions and averages

The first hypothesis tests we studied were checking to see if an experimental sample produced a value that was significantly different from some known value produced either by math or by earlier experiments.

For example, in the lady tasting tea, since she has two choices each time a mixture is given to her, the math would say that her chance of getting it right by just guessing is 50% or H0: p = .5. In testing psychic abilities, there are five different symbols on the cards, so random guessing should get the right answer 1 out of 5 times, or 20%, so H0: p = .2.

In a test for average human body temperature, the assumption of 98.6 degrees Fahrenheit being the average came from an experiment performed in the 19th Century.

We can also do tests by taking samples from two different populations. The null hypothesis, as always, is an equality, the assumption that the parameters from the two different populations are the same. As always, we need convincing evidence that the difference is significant to reject the null hypothesis, and we can choose just how convincing that evidence must be by setting the confidence level, which is usually either 90% or 95% or 99%.

Two proportions from two populations


Like with the one proportion test, the test statistic is a z-score. We have the proportions from the two samples, p-hat1 = f1/n1 and p-hat2 = f2/n2, but we also need to create the pooled proportion p-bar = (f1 + f2)/(n1 + n2).

Here's an example from the polling data from last year.

Question: Was John McCain's popularity in Iowa significantly different from his popularity in Pennsylvania?

Let's assume we don't know either way, so it will be a two tailed test. Polling data traditionally uses the 95% confidence level, so that means the z-score will have to be either greater than or equal to 1.96 or less than or equal to -1.96 for us to reject the null hypothesis. Here are our numbers, with Iowa as the first data set.

f1 = 263
n1 = 658
p-hat1 = .400

f2 = 283
n2 = 657
p-hat2 = .430

p-bar = (263+283)/(658+657) = .415 (q-bar = .585)

Type this into your calculator.

(.400-.430)/sqrt(.415x.585/658+.415x.585/657[enter]

The answer is -1.103..., which rounds to -1.10. This would say the difference we see in the two samples is not enough to convince us of a significant difference in popularity for McCain between the two states, so we would fail to reject the null hypothesis. In the actual election, McCain had 45.2% of the vote in Pennsylvania and 44.8% of the vote in Iowa, which are fairly close to equal.


Two averages from two populations


In the tests to see if the average of some numerical value is significantly different when comparing two populations, we need the averages, standard deviations and sizes of both populations. The score we use is a t-score and the degrees of freedom is the smaller of the two sample sizes minus 1.

Question: Do female Laney students sleep more hours each night than male Laney students?

We will take our data from the larger of the two class surveys, Data Set #2. Here are the numbers for the students who submitted data, with the females listed as group #1. Again, let's assume a two-tailed test, since we don't have any information going in which should be greater, and let's do this test to 90% level of confidence.

H0: mu1 = mu2 (average hours of sleep are the same for males and females at Laney)

x-bar1 = 7.31
s1 = .94
n1 = 26

x-bar2 = 7.54
s2 = 1.47
n2 = 12

The degrees of freedom will be 12-1=11, and 10% in two tails gives us the thresholds of +/-1.796. Here is what to type into the calculator.

(7.31-7.54)/sqrt(.94^2/26+1.47^2/12)[enter]

-0.4971...

This number is between the thresholds, and so does not impress us enough to make us reject the null hypothesis. It's possible that larger samples would give us numbers that would show a difference, which if true would mean this example produced a Type II error, but we have no proof of that.

Class notes for 5/11: p-values and Confidence Levels

The test statistics we will use for hypothesis testing will differ, sometimes using z-scores, other times t-scores and yet other times the chi-square table. What all these tests have in common is that they correspond to probabilities, called p-values.

The idea is that if we have a test with 90% confidence, we will only reject H0 if an event happens 10% of the time or less.

Likewise, 95% confidence means we reject H0 when the probability of the event is less than or equal to 5%.

99% confidence means we reject H0 with tests that happen 1% of the time or less.



This table shows what p-value corresponds to the threshold where we reject the null hypothesis, which technically is the same as accepting the alternate hypothesis. When it's one tailed high, the p-value threshold is easy, .9 for 90% confidence, .95 for 95% confidence and .99 for 99% confidence. For one tailed low, the pattern is that the p-value equals 100% minus the confidence level. For the two tailed test, the two tails have to add up to 100% minus the confidence level, so the 90% confidence level threshold is at the p-values of 5% (.05) and 95% (.95).

Remember, just because a test convinced us to reject the null hypothesis doesn't mean the null hypothesis is false. It could still be true, and we would be making a Type I error.

If we use the 90% confidence level, we should make Type I errors about 10% of the time.

At the 95% confidence level, the probability of Type I errors is 5%.

At 99% confidence level, the probability of Type I errors is 1%.

The lower likelihood of errors is why most tests done on medical data is done at the 99% confidence level.

The p-value is often published to show just how well the test did. Maybe you only asked to prove something to 90% confidence on a one-tailed high test, but the p-value is .9978. This shows people that read the findings that it would be strong enough data to reject H0 at even higher confidence levels.

Class notes for 5/6: Hypothesis testing

The topic for the next few weeks is hypothesis testing. The main idea is that experiments must be conducted to test the validity of an idea, which is called a hypothesis. There are always two hypotheses available, the null hypothesis H0 (pronounced "H zero" or "H nought") and the alternate hypothesis HA (pronounced "H A"). The standard is to assume the null hypothesis is true, which says that nothing special is happening, which in most cases means that two things we can measure should be equal or close to it. The alternate hypothesis says the two measurements are different. We can have one tailed high tests, where we want "large" positive test statistics. In one tailed low tests, only negative test statistics with "large" absolute value will do. In two tailed test, "large" absolute value for positive or negative numbers will work. We will only accept the alternate hypothesis if the experiment produces impressive results given our particular criteria for that test.


The basics of hypothesis testing are similar to the ideals of the English legal system, which is also the system used in United States courts, that a defendant is presumed innocent until proven guilty. There are different levels of proof of guilt in different trials, whether it is beyond a reasonable doubt or the less rigorous standard of preponderance of evidence.

In a case involving an alleged crime, there is the reality of what the defendant did and the result of the trial. If the defendant did the illegal act, then being found guilty is the correct result under the law. If the reality is that the defendant didn't do the act, the correct result would be a not guilty verdict.

The reasonable doubt standard is put in place in theory to make sending an innocent person to jail unlikely, and this is called a Type I error. The best known Type I result in legal history is Jesus Christ.

It is also possible that a person who did a crime will be found not guilty. This is called a Type II error. When I ask my students for an example of Type II error, O.J. Simpson's name still rings out the loudest.


In hypothesis testing, there is the reality and the result of the experiment. If H0 is true, the two things measured are equal or pretty close to equal. If HA is true, they are significantly different.

If the experiment produces a test statistic that is beyond the threshold we set for it, and "beyond" could mean lower if it is a one-tailed low test, or higher if it is a one-tailed high test, or either lower or higher in a two-tailed test, then we reject H0. If the test statistic fails to get "beyond" the threshold, we fail to reject H0.

Rejecting a true null hypothesis is a Type I error. Rejecting a false alternate hypothesis is a Type II error.

In class, we discussed Sir Ronald Fisher and his hypothesis testing of the lady who said she could tell the difference in taste between tea poured into milk or milk poured into tea.

Here are the things that have to be done to make such an experiment work.

#1 Define the null hypothesis. In modern experiments, the null hypothesis is always defined as an equation. In a proportion test, the equation will be concerning p, the true probability of success. In the lady tasting tea, we would assume if nothing special is happening, then she is just guessing whether the tea or milk was poured first, and the probability of being correct on any given trial is 50% or .5. We write this as follows.

H0: p = .5

#2 Pick a threshold. The trials we are going to perform are taste tests where the lady cannot see the tea-milk mixture being poured. We have to decide on how high a test statistic we will consider impressive. The three standard choices are 90% confidence, 95% confidence or 99% confidence. For experiments in the medical field, where the decision is whether or not to bring a new drug to market, the 99% confidence level is common. For an experiment like this, where the result is not truly earth shattering, we might decide to us the 95% confidence threshold.

The experiment will produce a z-score the thresholds for high z-scores are as follows:

90% threshold: z = 1.28
95% threshold: z = 1.645
99% threshold: z = 2.325



#3 Decide on the number of trials in an experiment. There is a tug-of-war in deciding the number of trials. More trials produces numbers we can be more confident in, but more trials is also more expensive and more time consuming. In the case of the lady tasting tea, we don't want to keep her drinking tea mixtures for hours.

Different books set different standards for the minimum number of trials based on np and nq. Some say both np > 5 and nq > 5. Others say both numbers should be greater than 10, yet others say 15. The standard that np >= 10 and nq >= 10 can be connected to the standard that says n > np + 3*sqrt(pqn) > np - 3*sqrt(pqn) > 0 by a little algebraic manipulation.

np - 3*sqrt(pqn) > 0 [add the square root to both sides]
np > 3*sqrt(pqn) [square both sides]
n^2*p^2 > 9pqn [divide both sides by np]
np > 9q

Since q must be less than 1, but can be as close to 1 as we want, set it equal to 1 and the inequality becomes

np > 9, which we can change to np >= 10.

For example, If let's look at the different possible positive z-score results if the lady were given ten trials, which would be enough if we used the lowest standard of np >= 5 and nq >= 5.

10 correct out of 10: z = 3.16227... ~= 3.16, which is above the 99% threshold.
9 correct out of 10: z = 2.52982... ~= 2.52, which is above the 99% threshold.
8 correct out of 10: z = 1.89737... ~= 1.90, which is above the 95% threshold, but not the 99%.
7 correct out of 10: z = 1.26491... ~= 1.26, which is below the 90% threshold.

If we set the bar at the 90% threshold, she could impress us by getting 8 right out of 10 or better. Likewise, at the 95% threshold, 8 of 10 will be beyond the threshold and the result would make us reject H0. At the 99% threshold, she would have to get 9 of 10 or 10 of 10 to make a z-score that breaks the threshold.

#4 Interpreting the test statistic. Let's say for the sake of argument that the lady got 8 of 10 correct. (There is a book about 20th Century statistics entitled The Lady Tasting Tea, where a witness to the experiment says he can't recall how many times she was tested, but the lady got a perfect score.) If we set the threshold at 90% confidence or 95% confidence, we would be impressed by the z score of 1.90 and we would reject H0, which says that we don't think she is "just guessing", but actually has the talent she says she has. If we set the value at 99% confidence, we would fail to reject H0.

Here's the thing. We could be wrong. If we reject H0 incorrectly, it means she was just guessing and she was very lucky during this test, a Type I error. A z-score of 1.90 corresponds to a probability of 97.13%, which is called the p-value in hypothesis testing. She can pass the test by being better than 97.13% of lucky guessers. If she is a lucky guesser, she would fool anyone who had set the threshold at 90% or 95%.

If we fail to reject H0, which we would do if we set the threshold at 99%, this could also be an error, but this time it would be a Type II error. Under this scenario, she got 8 of 10 but she should do better usually. It's more difficult math to figure out how good she actually is and how unlucky she had to be to get only 8 of 10. The probability of a Type I error is called alpha, and it is determined by the threshold. The probability of a Type II error is called beta, and it usually explained in greater detail in the class after the introduction to statistics.

Class notes for 5/4, part 3: Bayesian probability and double testing

If a trait is very rare, only a very accurate test gives us useful information. For example, if a trait shows up in only 1 in 10,000 people but the test for the trait has an error rate of 1 in 1,000, we should expect about 10 false positives for every true positive. Here is the completed table for that situation.

________don't___have____row total

test + ____9,999__999______10,998__

test - _9,989,001___1_____9,989,002__

col.___9,999,000_1,000______10,000,000 grand total

In a situation such as this, testing positive twice could give us useful information, as testing positive once has an error rate of about 90.9%. We have to assume the errors are random and not deterministic. For example, if a test for a chemical compound in opium also catches a similar compound found in poppy seed bagels, testing twice won't get rid of the errors. Assuming just random errors here is what we do.

Step 1: The top row of the first contingency table is the column total/grand total row of the second contingency table. What this does is takes the numbers from the people who tested positive the first time and makes them the totals for those who will be tested twice.

________don't___have____row total

test + __________________________

test - __________________________

col._______9,999__999_____10,998 grand total

Step 2: Multiply error rate by have column total to find the number who have that test negative. Round to the nearest whole number. (We didn't have to round before, but now we do.)
999*1/1000 = .999 ~= 1, this means test positive and have is 998.

________don't___have____row total

test + ___________998____________

test - _____________1____________

col._______9,999__999_____10,998 grand total

Step 3: Multiply error rate by don't have column total to find the errors. 9,999*1/1000 = 9.999 ~= 10. That means the test negative in that column is 9,999 - 10 = 9,989.

________don't___have____row total

test + _______10__998____________

test - _____9,989____1____________

col._______9,999__999_____10,998 grand total

Step 4: row totals

________don't___have____row total

test + _______10__998_____1,008___

test - _____9,989____1_____9,990___

col._______9,999__999_____10,998 grand total

Step 5: Find the error rate for testing positive twice. 10/1,008 = .0099... or about 1%.

Of the ten million people tested, we would send letters to 1,008 telling them they tested positive twice. Of those people, ten don't have the trait and are getting false information, but 998 are getting the right information. In the first test, there was someone with the trait who tested negative, and the same is true in the second test, so there are two people with the trait who did not get two positive test results. While this isn't a perfect situation, it's much better than the over 90% error rate we got for positive tests the first time through.

Class notes for 5/4, part 2: Bayesian probability

Earlier in the term, we created contingency tables from reading data sets and filling in the positions of the table, then finding the row totals, column totals and the grand total. We then learned about conditional probability, where we find that p(left, given female) might not equal p(left, given male) or p(left). If these probabilities are not equal, we call them dependent, because it depends on if we are looking a the whole population or some specific sub-population. If they are all equal, the probabilities are independent.

In Bayesian probability, we will be building a contingency table "backwards". Instead of filling in each value in the table then finding row totals, column totals and grand total, we will start with a trait in the population and a test for that trait. We will make a 2x2 contingency table, where the columns deal with having the trait on not and the rows refer to testing positive of testing negative. If the test has an error rate, as is often the case, some people are going to get incorrect information. What we will see is that the overall error rate can sometimes be quite different for the error rate for those who test positive and the error rate for those who test negative.

Let's say there is a genetic trait in the population that shows up in 25% of subjects, which we will write as 1 in 4. The test for the trait has a 2% error rate, so it is 1 in 50.

Step 1: The grand total is the product of the denominators of the fractions.

In our case, 4*50 = 200


________don't___have____row total

test + __________________________

test - __________________________

col._____________________200 grand total

Step 2: Multiply the grand total by the trait proportion to find the column totals.

Since 25% of the population has the trait, 25% of 200 = 50 subjects have the trait in our idealized sample. By subtraction, 150 don't have the trait.


________don't___have____row total

test + __________________________

test - __________________________

col._____150_____50______200 grand total

Step 3: Fill in the "have the trait" column by multiplying the error rate by the column total to fill in the mistaken position, and fill in the rest by subtracting.

In our case, the error rate is 1 in 50. This means for the people who have the trait, 1 person will test negative, while the other 49 will correctly test positive.

________don't___have____row total

test + ___________49______________

test - ____________1______________

col._____150_____50______200 grand total

Step 4: Fill in the "don't have the trait" column using the same method.

3 of the 150 will get the wrong information, which in their case will be a positive test. The other 147 will get the right information, a negative test result.

________don't___have____row total

test + ____3______49______________

test - ___147______1_______________

col._____150_____50______200 grand total

Step 5: Fill in the row totals.

________don't___have____row total

test + ____3______49______52______

test - ___147______1______148______

col._____150_____50______200 grand total

The error numbers are marked in bold and blue for the next step.

Step 6: Find the error rates given test positive and test negative.
p(error) = (3+1)/200 = 1/50 = .02, which was the advertised error rate.
p(error, given test positive) = 3/52 ~= .058, much higher than .02
p(error, given test negative) = 1/148 ~= .0068, much lower than .02

Unless the trait shows up in 50% of the population, we expect to get differences between the error rates for test positive and test negative. Whichever is the smaller part of the population should see a higher error rate. Just how significant the differences are in the error rates between the two test groups depends on the size of the error rate to the trait rate. A 99% accurate test sounds good, but if the trait is very rare, we might well get more false positives than true positives.

Class notes for 5/4, part 1: more on expected value

The expected value of a two outcome game (win or lose) can be written as EV= p*(profit+risk)/risk. By dividing by risk, the game's expected value can be thought of as a percentage of money returned to you on average every time you play. Again, this is an average, so the average outcome doesn't have to be achieved. In many games, it never is. Expected value is really about the long run.

Flipping a fair coin, if you call "heads" every time, you should win about 50% of the time, and so EV = .5(1+1)/1 = 100%, so calling heads is a way to make this a fair game with a fair coin.

If you mix up your calls, it's also a fair game.

If you go with "rock" every time in rock/papers/scissors, your opponent will catch on soon enough and go with "paper" every time, and you will lose money in the long run.

Rock/scissors/paper is a fair game only if you can mix up your calls using some random method, or at least a method hard for your opponent to determine. The best method is 1/3 rock, 1/3 scissors and 1/3 paper, and the expected value is 100%, or that you will neither lose nor win, but break even.

Let's go back to roulette. We saw that playing a single number or playing either black or red produce the same expected value for the player.

Single number for player
p = 1/38
profit = 35
risk = 1
EV = 1/38*(35+1)/1 = 36/38 ~= 94.7%

Red (or Black) for player
p = 18/38
profit = 1
risk = 1
EV = 18/38*(1+1)/1 = 36/38 ~= 94.7%

For the casino, the probability of winning is 1 minus the probability for the player. The risk and profit numbers are switched from the players' values.

Single number for casino
p = 37/38
profit = 1
risk = 35
EV = 37/38*(35+1)/35 = (36*37)/(35*38) ~= 100.15%

Red (or Black) for player
p = 20/38
profit = 1
risk = 1
EV = 20/38*(1+1)/1 = 40/38 ~= 105.3%

When profit = risk, which is the same as saying the classic parimutuel odds are 1:1 or the modern parimutuel odds are +100 (which is the same as -100, though rarely written that way), the average of the two expected values will be 100%. You expect to lose about 5.3 cents every game and the casino expects to win about 5.3 cents.

When profit does not equal risk, we get different percentage advantages and disadvantages. Because the casino must risk 35 bets and the player only 1 bet when the player picks a single number at roulette, the expected value for the casino is still positive, but relatively small at .15 of a cent per game. In reality, the casino rarely spins the wheel with only one bettor playing, so the casino is not truly risking only its own money on a single spin, but can use the losses of some players to help offset any possible winner. Even if that weren't the case, a game with an expected value greater than 100% means a winner in the long run, and that is the business model the casino operate on, and very successfully as anyone can see.

Class Notes for 4/27 and 4/29

Dependence and Independence
The first line of Leo Tolstoy's Anna Karenina is "All happy families are alike; each unhappy family is unhappy in its own way." In statistics, all independent trials are alike, in that the probability of a particular outcome of one trial does not effect the outcome of later trials, nor was it effected by earlier trials. With dependent probability, the outcome of one trial is effected by the outcome of previous trials, but how that effects things is not always the same.

For example, if we talk about a 70% free throw shooter taking two shots, does missing the first shot effect the probability of missing the second shot? Let's look at this simple problem three different ways.

Predicting the future mathematically by carefully studying the past: Let's say I called this person a 70% free throw shooter because so far in the season, she has made 7 of 10 shots from the line. If she misses, she has now made 7 of 11 shots from the line, and she is now a 63.6% shooter. Should we factor that in to be more precise? Some mathematical models would say yes.

Using this method, missing a shot would make her percentage worse and making one would make it better, no matter how many shots she had taken. But if she had made 70 of 100 so far in the season, one miss would make her 70 of 101, which would lower her percentage to 69.3%, a much smaller effect than it has if she had made 7 of 10. If instead we were looking at her entire career instead a single season, perhaps she has made 700 of 1000, and missing one would mean she was 700 of 1001, which changes her percentage to 69.9%. When discussing free throw percentage, announcers on TV usually round to the nearest percent, so the first example of 7 of 10 to 7 of 11 would be a drop from 70% to 64%, the second example would be a drop from 70% to 69%, and in the third example, 69.9% would round to 70% and the change would be too small to notice.

Factors effecting success and failure: The free throw shooter goes to the line for two shots and misses the first. Is there a physical reason? We might treat free throw shooting as we would picking a random number from 1 through 100, where we count any number from 1 through 70 as a success and 71 through 100 as failures, but shooting free throws includes the human factor. Maybe she missed because she is nervous or distracted. Maybe it's late in the game and she is tired or injured, changing her technique. If any of these are the case, it might make more sense for us to downgrade the probability of making the next shot, though exactly how much it should be downgraded is no longer some simple formula like turning a fraction into a percentage.

Compensating for failure: Again, let's add in the human factor in this problem. She misses the first free throw, and her coach notices her technique looks inconsistent. "Elbow up!" the coach shouts from the sidelines, and the shooter hears the coach and readjusts her technique to match the way she shoots in practice. Should this change bring her back to being a 70% shooter, or even upgrade her chance of success? That is uncertain, but the failure on the first shot and the diagnosis of at least one reason for the failure could effect the odds, and that effect means the second shot should not be considered independent of the first.

The dependent probabilities in a 52 card deck

One of the simplest mathematical models of dependency is sampling without replacement, which is the way most card games or lotteries or the game of Bingo works. You have a set of outcomes which get effectively randomized and a trial is performed, meaning a card is taken from the deck or a ping pong ball is removed from the hopper or a bingo marker is removed from the spinner. Once removed, the number of possible outcomes has been reduced by one and probabilities for success and failure of certain outcomes change.

Looking for an ace: There are 52 cards is a standard deck and 4 of them are aces. If I draw a card from a randomized deck, the chances are 4/52 = 1/13 ~= 7.7% that the card will be an ace. What are the chances the second card is an ace?

That depends on the first card.

Probability that the second card is an ace, given the first card is an ace is 3/51 = 1/17 ~= 5.9%.

Probability that the second card is an ace, given the first card is not an ace is 4/51 ~= 7.8%.

Unlike the mathematical model of free throw shooting where we re-calculate the probabilities by adding the most recent make or miss into the precentage, which means a miss brings the odds down and a make brings the odds up, not getting an ace makes the odds a little better next time, and getting an ace makes the odds worse.


This is the formula for the dependent probability model of sampling without replacement is given at the left. The two numbers in parentheses are a binomial coefficient, the numbers you get when you use nCr on your calculator, which I pronounce "n choose r" in class. The pairs of numbers that look like a base and an exponent, except that the exponent is underlined, is the convention developed by Donald Knuth at Stanford for writing the numbers that you get on your calculator using the nPr function, which I pronounce "n fall r", referring to the name "the falling factorial". If we think about a deck of cards, the lowercase letters refer to the size of the hand n, where r is the number of successful trials (r for right) and w is the number of unsuccessful trials (w for wrong), and r+w=n. The uppercase letters refer to the size of the deck, where T is the size of the deck, G is the number of cards we consider success if we draw them and B is the number of cards we consider a failed trial if we draw them. The letter T stands for Total, G for Good and B for Bad. Again, we have an equation, G+B=T.

Example: If we want consider drawing a heart a success and anything else a failure, what is the probability of drawing three hearts and two non hearts in a five card hand from a well-shuffled 52 card deck.

Here are the six numbers we need.
n = 5
r = 3
w = 2
T = 52
G = 13
B = 39

On a TI-30XIIs, here are the keys you would press.

5[prb][right]3×13[prb]3×39[prb]2÷52[prb]5[enter]

The calculator will read as follows.

5 nCr 3*13 nPr 3*39 nPr 2/52 nPr 5
0.081542617

This means the probability of exactly three hearts and two cards of some other suit is about 8.15%.

The Expected Value (EV) of a two outcome game

Let us assume we have a game that has only two outcomes, winning and losing. Let us further assume that two players have decided to wager on this game, both putting money into a pooled amount and the winner taking all at the end.


If we look at the game from the point of view of one of the players, we need to know the probability of winning p, how much that player put in, which we call Risk and how much the opponent put in, called Profit. The expected value EV equals the probability of victory p times the sum of Profit and Risk divided by Risk.

Different books use different formulas for this. Some do not divide by Risk. By dividing, the number we get is a percentage of return, and a game of flipping coins for $1 a game is equivalent to a game of flipping coins for $100 a game. Some subtract 1 from this formula. This just changes the most important number in identifying results from 1 to 0.

If EV = 1, we consider this a "fair game". For every $1 risked on this game, the expected value is that you will have that dollar returned to you, breaking even. Notice if we are flipping coins, that event never happens on any single play. Either the player makes a dollar profit or a dollar loss, but expected value is about the long run.

If EV > 1, the game is advantageous to the player. If EV < 1, the game is disadvantageous to the player.

In the game of roulette, there are 38 slots where the ball can land, and for simplicity's sake we will assume each has an equal chance of showing up, so p = 1/38. For every $1 you risk, you can make a profit of $35 if you correctly guess in the exact slot where the ball will land. To find the expected value using the TI-30xIIs, you should type in this.

1÷38×(35+1)÷1[enter]

The calculator will read as follows.

1/38*(35+1)/1
0.947368421

What this number means is that for every dollar risked on the spin of a roulette wheel, you should expect about 94.7 cents returned to you in change. In other words, about 5.3 cents is lost from every dollar you bet on every spin of the wheel.

Another way to play the game is to bet red or black. Of the 38 compartments, 18 are red and 18 are black and 2 are green. The probability of victory on betting one of the two major colors is 18/38 = 9/19 ~= 0.473684211. The profit and risk are now both $1. Here's what to type on the TI-30xIIs.


16÷38×(1+1)÷1[enter]

The calculator will read as follows.

16/38*(1+1)/1
0.947368421

The game has changed, both in probability and amount of profit compared to risk, but from the player's point of view, the expected value is precisely the same and still in favor of the casino.

No matter what the levels of profit and risk are, we can find a probability p that will make the expected value equal to 1, and that is p = Risk/(Profit+Risk). If the probability is increased with the profit and risk remaining unchanged, the game becomes advantageous. If is is decreased, the game becomes disadvantageous.

Modern and Classic Parimutuel odds

Profit and risk are listed either in classic form like 3-1 or 2-7 (or sometimes with colons 3:1 or 2:7), where profit is the first number and risk is the second.

In online betting sites, the numbers are given as numbers with absolute value greater than 100, with either a + or - in front of them. +250 means 250 is the profit and 100 is the risk, while -250 means 10o is the profit and 250 is the risk. The fourth page of the yellow sheet explains this in greater detail and shows how to switch back and forth between the two systems.

Practice problems

1. With a well-shuffled 52 card deck, find the probability of getting exactly r hearts in a five card hand when
a) r = 0
b) r = 1
c) r = 2
d) r = 3 (already solved above)
e) r = 4
f) r = 5

2. Find the break-even p when Profit and Risk are as given. Round to three places after the decimal point.

a) Modern parimutuel = +150
b) Modern parimutuel = -110
c) Classic parimutuel = 5:3
d) Classic parimutuel = 5:11

Answers in the comments.

Class notes for 4/22

Random and deterministic

It's common for people to use the word random in a casual manner, but in the field of the philosophy of science, discussion of whether something can be considered random or not is a subject of intense debate. The opposite of random is deterministic, which is to say that when we perform a task, we understand the possible outcomes thoroughly. For example, putting a key in a lock is a deterministic act. Will the door open? Not necessarily. It might be the wrong key. The key might be correct, but it might have worn out over time. A brand new key in a lock that has been used might have edges that are too sharp. The lock could malfunction. Maybe the person didn't turn the key in the right direction. So deterministic does not mean, "If you do a, then b will also happen." It can be more complicated than that. But in a completely deterministic act, we have an expected outcome, and even when it fails, we have explanations of why it fails.

The problem of determinism versus randomness is not a yes/no situation. We have some acts we consider random, like flipping a coin or rolling a die or choosing a card from a deck. If done under certain circumstances, even these can be deterministic. If the deck is removed from the pack and unshuffled, the top card will be the way the cards were sorted at the factory, and so it is completely deterministic. Some card tricks are done with decks that aren't actually randomly shuffled or the magician has ways of forcing the participant to pick a certain card, so picking a card is deterministic, or picking the card is random but returning it to the deck is deterministic, and so it can easily be found by being out of position. If we drop a coin or a die only a short distance, it might not bounce much, so the result is strongly determined by the original state of the die. Much of the randomness of things like coin flips or dice rolls or lottery balls being removed from a hopper have to do with physics problems that could be considered deterministic if we understood all the variables, but the equations are so difficult that solving them completely is beyond even the most sophisticated computer simulations.

This brings us to random numbers and computers. A computer is completely deterministic. It cannot truly produce a random number, though every computer and even many calculators, including the TI-30XIIs, have random number generators. Here, the computer takes some input unknown to the user, puts it through a function also unknown to the user and produces an output. This is called pseudorandom, and debate over these methods continue to this day. One of the fathers of computer science, the great Hungarian mathematician John Von Neumann, was quoted as saying, "Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin." A computer has nothing but arithmetical methods available. The tests done to see if a method produces output that is sufficiently random have to do with a distributions of a large set of pseudorandom output.


The probability of r successes in n independent trials, where p is the probability of success of any given trial

The simplest kind of problem in multiple trials is to assume independence between the trials, that the probability of success remains constant over all the trials, and we call that probability p. A basic question in these kinds of experiments is to ask what is the probability of getting a specific number of successes, calling that number r, in a specific number of trials, a number we will call n. If you have a TI-83 or TI-84, there is a function in the distribution menu (blue button then DISTR) called binompdf, that takes as its input n, p and r in that order. On the TI-30XIIs, we have to type in a formula.

Example #1: If we flip a fair coin, p = .5, ten times, what is the probability of exactly 6 heads and 4 tails?

TI-83 or TI-84: Go to the distribution menu, select binompdf and type in the following values

binompdf(10,.5,6) = 0.205078125

On the TI-30xIIs, here is what to type in

10[prb][right arrow]6x.5^6x.5^4[enter]

It will read
10 nCr 6 * .5^6 * .5^4
0.205078125



A different question would be what are the chances of at most r successes in n trials. Again, the TI-83 and TI-84 have a single function solution, this one called binomcdf(n, p, r). On the TI-30XIIs, we either have to calculate all the separate values, from the probability of 0 successes up through the probability of r successes then add these up, or if this is too much work, there is a method using normal distribution to approximate binomial distribution.

If you have Java on your computer, you can go to this website to see how the numbers across a row of Pascal's Triangle look like the bell-shaped curve. We can use z-scores and the lookup table to get values that will approximate these probabilities, and the approxomations get better as the numbers get bigger.

Example #2: If we flip a fair coin, p = .5, ten times, what is the probability of at most 6 heads?

TI-83 or TI-84: Go to the distribution menu, select binomcdf and type in the following values

binompcf(10,.5,6) = 0.828125

On the TI-30XIIs:

This entails a lot of work, but not an impossible amount. We need to find
p(exactly 0 heads) + p(exactly 1 head) + p(exactly 2 heads) + p(exactly 3 heads) + p(exactly 4 heads) + p(exactly 5 heads) + p(exactly 6 heads) = 0.828125

We have the normal approximation method, but we will see when n = 10, it's not very good.

n = 10, p = .5, r = 6

z = (6 + .5 - .5*10)/sqrt(.5*.5*10) = .3

Lookup table: .3 -> .6179

The approximate value at around 62% is very far from the true value at nearly 83%.

Different books have different cut-off points, but at a minimum, using the normal approximation to binomial should only be used when both np > 5 and nq > 5. In this case, both values equal 5, so this would not be a good candidate for using this method.

Example #3: If we flip a fair coin, p = .5, one hundred times, what is the probability of at most 60 heads?

TI-83 or TI-84: Go to the distribution menu, select binomcdf and type in the following values

binompcf(100,.5,60) = 0.982399899891... ~= .9824

TI-30XIIs: It's too many things to add up, so the approximation method is our only hope. Both np > 5 and nq > 5, so we can move forward.

(60 + .5 - .5*100)/sqrt(.5 * .5 * 100) = 2.1

Lookup table: 2.1 -> .9821

When n=100, the approximation is not perfect, but it is very close, unlike the results when n=10.


Another question that could be asked is the probability of getting at least r successes in n trials. "At least r" is the complement of "At most r-1", which means the two probabilities will add up to one.

Example #4: If a 70% free throw shooter takes ten shots, p = .7, what is the probability she makes 8 shots or more?

On the TI-83 or TI-84:

1 - binomcdf(10, .7, 7) = .3827827864..., which rounds to .3828 to four places.

On the TI-30XIIs

10[prb][right arrow]8x.7^8x.3^2[enter]

It will read
10 nCr 8 * .7^8 * .3^2
0.233474441

10[prb][right arrow]9x.7^9x.3^1[enter]

It will read
10 nCr 9 * .7^9 * .3^1
0.121060821

10[prb][right arrow]10x.7^10x.3^0[enter]

It will read
10 nCr 10 * .7^10 * .3^0
0.028247525

If we round these numbers to five places, add them up and round the answer to four places, it should agree with the answer above.

.23347 + .12106 + .02825 = .38278, which rounds to .3828, which agrees with the above answer.

Example #4: If a 70% free throw shooter takes 100 shots, p = .7, what is the probability she makes 75 shots or more?

On the TI-83 or TI-84:

1 - binomcdf(100, .7, 74) = .163130104..., which rounds to .1631 to four places.

On the TI-30XIIs: The problem is adding up too many parts, but np = 70 and nq = 30, both of which are more than 5, so let's go to the approximation method.

z = (75 - .5 - 70)/sqrt(.7*.3*100) = .98198..., which rounds to .98

Lookup table: .98 -> .8365, and 1-.8365 = .1635, which again is pretty close to the actual answer.

Class Notes for 4/20

We dealt with probability in a single instance earlier in the class when we had the relative frequencies of the values of categorical variables. Relative frequencies, listed in a population as p and in a sample as p-hat, are numbers between 0 and 1. If we take all the relative probabilities of all the values of a variable, they will add up to 1, or something very close to 1 depending on rounding error.

We will now talk about probability in multiple event experiments, like flipping ten coins or rolling five dice or drawing a hand of four cards from a 52 card deck. The first important split in the types of multiple event experiments is between independent and dependent events.

Events are independent if the probability of a later event does not change based on the result of an earlier event. For example, if I flip a coin that I can assume is fair, there is a 50% chance of heads and a 50% chance of tails every time I flip it. If by chance, the coin comes up heads ten times in a row, even though earlier testing had shown it to be a 50%-50% chance each time, the eleventh flip is still 50%-50%. Unusually long runs of all heads or all tails are rare, but they are not impossible. Flipping coins and rolling dice are typical examples of independent random events.

Events are dependent if the probability of a later event changes based on the result of an earlier event. The typical example of this is drawing cards from a shuffled deck. If the deck has 52 cards and 4 aces, the probability of drawing an ace from the deck is 4/52 = 1/13 ~= .0769...

If the card has been drawn, what is the probability of the second card being an ace? That depends on what the first card is. If the first card is an ace, there are only 3 left in the deck, which now has 51 cards, so the probability is 3/51 = 1/17 ~= .0588..., which is a lower probability than getting an ace the first card.

If the first card wasn't an ace, the odds are 4/51, ~= .0784..., a slightly higher probability than drawing an ace the first time.

If I say someone is a 70% free throw shooter, is every free throw attempt independent of what happened before? Often, we set up such an experiment assuming independence just to make our work simpler, but the human factor is involved, so in reality it's very likely to be dependent. Some people get frustrated after a few misses and will do worse. Others will learn from the mistakes of a few misses and figure out what they are doing wrong and make improvements. A player might be having a bad day for some reason, or might instead have excellent concentration or just really good luck that day. But again, these kinds of experiments are often set up as though each free throw trial is independent of what came before.

Let's look at flipping coins. A list of all possible events is called the event space. Here are some examples of event spaces.

Event space for flipping one coin
Heads (H)
Tails (T)

ways to get one head = 1
ways to get no heads = 1

Event space for flipping two coins
HH
HT
TH
TT

ways to get two heads = 1
ways to get one head = 2
ways to get no heads = 1

Event space for flipping three coins
HHH
HHT
HTH
HTT
THH
THT
TTH
TTT

ways to get three heads = 1
ways to get two heads = 3
ways to get one head = 3
ways to get no heads = 1


The list of numbers of ways to get r successes in n trials is often written in the pattern of the picture shown here, and this pattern is called Pascal's Triangle, at least in most of the world. The Italians call it Tartaglia's Triangle and the Chinese call it Yanghui's Triangle. None of these people actually invented it or claimed to have invented it. It's been around since before the time of Christ, and it has been studied all around the world.

While it is very common to see it presented in the form here as an equilateral triangle, it can also be presented where the first numbers in each row are lined up straight as follows

1
1 1
1 2 1
1 3 3 1
1 4 6 4 1
... etc.

It is standard to start counting the top row as row 0, and the left most column as column 0. For example, the 6 we see in the middle of the last row I typed in is row 4, column 2. Instead of having a copy of Pascal's Triangle around, our calculators have these numbers available. On Texas Instruments calculators, the function is under the probability menu. On the TI-30XIIs, the way to get that 6 is to type

4 [prb][right arrow]2[enter]

The calculator will read

4 nCr 2
6

All scientific calculators should have this function available, but all of them are slightly different. The TI-89 writes it as nCr(4,2) and Casio calculators write it as 4 C 2. I will pronounce it "4 choose 2", and when I type on the blog, I will type C(4,2). When I write it on the board or on tests, I will put a 4 on top of a 2 and surround both numbers with a large parentheses. These numbers are called the binomial coefficients.


The formula for finding the probability for exactly r successes in n independent trials where the probability of success on any single trial is p is shown here. In some books, they don't use the letter q, instead replacing it with (1-p). Likewise, sometimes w is replaced with (n-r). I use the extra letters and include the relationships between them. The letters r and w stand for right and wrong. The letter p and q are standard in probability texts for the probability of a success or a failure.

Let's do an example. You are given a four question multiple choice test, each question having five possible answers. The test is given in a language you do not read, so all you can do is guess. Each question is independent from the others, meaning that if C is the right answer to the first question, it's also possibly the answer to the second. The probability p of a correct guess is 1 chance in 5, or .2, The probability of failure q is 1-.2 = .8, and of course p + q = 1.

Probability of no correct answers = C(4,0)*.2^0*.8^4 = .4096
Probability of exactly one correct answer = C(4,1)*.2^1*.8^3 = .4096
Probability of exactly two correct answers = C(4,2)*.2^2*.8^2 = .1536
Probability of exactly three correct answers = C(4,3)*.2^3*.8^1 = .0256
Probability of four correct answers = C(4,4)*.2^4*.8^0 = .0016

The expected value of correct answers is n*p, so in this case it's 4*.2 = .8, which isn't possible. You can't get a fraction of correct answers on a multiple choice test. The expected value in this case says that over the long run, a test like this should average .8 right answers out of four. As we can see, the most likely thing to happen is actually a tie for first, where getting either no answers right or one answer right both have a probability of about 41%. If you need to get three answers right to pass the test, the odds are less than 3% to get either three or four right, and the odds of getting everything right by chance is a very slim 16 chances in 10,000.

If you have a TI-83 or TI-84, there is a function under the distribution menu called binompdf(n,p,r). All you have to is enter the function, then the three values in the order given, separated by commas.

The function for three right in four trials with probability .2 at each trial is binompdf(4, .2, 3), which as we see above is .0256.


Practice problem.
The test is changed. There are now five multiple choice questions and four choices for each, but it is still given in a language you do not read.

Round the probabilities to four places after the decimal.

1. What is the expected value?

2. What is the probability of no correct answers?

3. What is the probability of exactly one correct answer?

4. What is the probability of exactly two correct answers?

5. What is the probability of exactly three correct answers?

6. What is the probability of exactly four correct answers?

7. What is the probability of five correct answers?

Answers in the comments.

Class notes for 4/1

We have learned how to find a confidence interval for a proportion in a population. Unlike numerical data, where the confidence level multipliers, known in this class as CLMxx%, are taken from the t-score tables, the CLMxx% for proportions are from the z-score table.

CLM90% = 1.645
CLM95% = 1.96
CLM99% = 2.575

The most common use of confidence intervals for proportions are in opinion polls, though when the people in the media talk about "margin of error", they rarely say that the margin of error is associated with a confidence level, which in opinion polls is always 95%. For example, in early 2008 the opinion polls before the New Hampshire election showed Barack Obama with a lead in a multi-candidate race in the Democratic primary, but the primary was won by Hillary Clinton. There were four candidates who polled well over 1% of the voters, Clinton, Obama, John Edwards and Bill Richardson. Clinton's final total was not within the 95% confidence interval set for her by the poll results, but the other three were in the 95% confidence intervals set for them. These things happen. The idea of the 95% confidence interval is that is leaves open the possibility that it will get the numbers wrong about 1 time in every 20.

Final opinion poll (true result in parentheses)
n = 500
Obama: 39% +/- 4.3% (36.5%, inside the confidence interval)
Clinton: 34% +/- 4.2% (39.1%, outside the confidence interval)
Edwards: 15% +/- 3.1% (16.9%, inside the confidence interval)
Richardson: 4% +/- 1.7% (4.6%, inside the confidence interval)

Obama did worse than predicted, everyone else did better, especially Hillary, whose result was well outside her 95% confidence interval. Again, remember that the 95% confidence interval is not a promise. It says it will be right about 19 times out of 20, but it never knows when that 1 time in 20 that it will be wrong will happen. If Bill Richardson outperformed expectations and gotten 9% of the vote, it would still have been a mistake by the polls, but no one would have paid much attention, because it wouldn't have changed the outcome of who finished first.

Of all the websites, TV shows and newspapers that report on opinion polls, the only one that consistently explains them correctly is The New York Times, which has a standard sidebar it puts next to opinion poll results.


Finding n when the margin of error is given

Opinion polling companies always report the margin of error, but the public does not completely understand it, largely because the media does not explain it. In general, the lower the margin of error the better, but the simplest way to guarantee a low margin of error, given that you can't change the industry standard confidence level of 95%, is to increase the sample size. While this is simple, it's also expensive. If the polling company in New Hampshire had wanted all the margins of error to be no more than 3.0%, it could have used the formula above, with MoE95% = .03 and p-hat = 39%, the best estimate for the leader, who they assumed was Barack Obama.

n >= 1.96^2*.39*.61/.03^2 = 1,015.46...

This says a sample of 1,016 likely voters would have produced a margin of error for each candidate of no more than 3.0%. If we did not have the previous information that the highest percentage expected was about 39%, we would have had to assume someone might be close to 50%, which would increase the needed sample size.

n >= 1.96^2*.5*.5/.03^2 = 1,067.11...

In this case, n would have to be 1,068 to guarantee the margin of error of 3.0% or less.


Confidence of victory

The margin of error is the industry standard, but the people who use these numbers, most especially the news media, really don't understand them very well. Here is a different method to produce a more useful piece of mathematical information which this author has developed, called the confidence of victory method.

Confidence of victory should only be used if the top two vote getters combined are getting 90% or more of the respondents to the opinion poll. So in the New Hampshire primary, we could not use this method. In the final poll taken in New Hampshire before the general election, these were the results.

n = 700
Obama 51%
McCain 44%

Since they add up to 95% of the respondents, we can use the confidence of victory method. What we do is effectively ignore the 5% who are either voting for third party candidates, are preferring none of the candidates or are still telling pollsters they are undecided. We figure out how many people in the poll said they prefer Obama and how many prefer McCain by multiplying the percents by the size of the poll.

f(Obama) = 700 * .51 = 357
f(McCain) = 700 * .44 = 308

new n = 357+308 = 665

p-hat(Obama) = 357/665 ~ 53.7%
p-hat(McCain) = 308/665 ~ 46.3%

sp-hat = sqrt(.537*.463/665) ~ 1.93%

z(Obama) = (53.7 - 50)%/1.93% ~ 1.91

This says Obama's percentage is about 1.91 standard deviations above 50%. The percentage he will get in the actual election may be higher or lower than what we see here. We assume there's about a half a chance he will do better than the final opinion poll, and a half a chance he will do worse. What the public actually cares about is whether he wins or loses. What the confidence of victory method does is find the percentage that corresponds to the z-score. That number is the confidence level we have that the true percentages from the population polled will show that the leader in the poll will be the winner of the election. In this example, z=1.91 corresponds to .9719 on our positive z-score table. Because the confidence of victory method is sensitive to small changes, we should round to the nearest percent and use this sentence to describe the results.

If the election were held when the poll was taken, we are 97% confident that Obama will hold on to the lead shown in the poll and win the election in New Hampshire.

In the actual election, Obama outpolled McCain 55% to 44%, which is to say he did better than expected. Confidence of victory is not concerned with the margin of victory, just whether the favored candidate in the polls wins the actual elections.

In 2008, the final polls in the 50 states and Washington D.C. had two states too close to call, Missouri and Indiana. Both elections were very close, called late in the evening, Missouri for McCain and Indiana for Obama. In the other 49 contests where confidence of victory claimed an advantage for one side or the other, 48 contests were won by the person leading in the most recent poll, which is to say the confidence of victory method was vindicated about 98% of the time. The only state where the confidence of victory method did not get the right result was North Carolina. McCain had a 60% confidence of victory in North Carolina, but Obama actually won the state.

In 2004, there were two states that the confidence of victory method got wrong, Ohio and Florida which looked to be favoring John Kerry in the final polls. Of course, 2004 was a much closer election, and either of those states could have turned the tide. 2008 was an electoral college landslide. Even if McCain had won North Carolina, he still would have lost the election.

Practice problems

Here are some final poll numbers from the 2008 election where the totals favoring either Obama or McCain add up to over 90%. Use the confidence of victory method.

Colorado
n = 600
Obama 52%
McCain 45%

Arizona
n = 600
Obama 46%
McCain 50%

Answers in the comments.

Class notes for 3/30

One of the most common uses of statistics is in opinion polling, which is most popular in election years. TV, radio and newspapers are nearly constantly reporting on the results of polls, new ones being released every day. The numbers for the candidates, or at least how much of a lead one candidate has over the major competitor, is reported up front, and sometimes at the end of the report the margin of error will be given. Almost never is the margin of error given with a confidence level attached. The one major exception to this oversight is The New York Times, which does explain the confidence level in a sidebar, that confidence level always being 95% in opinion polls.

As we see in the sidebar above, the true percentage of the population p is expected to be inside an interval that surrounds p-hat, the percentage of our sample. The Confidence Level Multipliers are taken from the z-score table instead of the t-score table. They are given in the lower right hand corner of the Positive z Score table (Table A-2), where they are labeled Common Critical Values.

CLM90% = 1.645
CLM95% = 1.96
CLM99% = 2.575

Let's take some polling data from last year's election and find the margins of error.

Final poll from Florida - 2008
n = 678
p-hat(Obama) = 49%
p-hat(McCain) = 48%
p-hat(undecided or other candidates) = 3%

Margin of error for Obama = 1.96*sqrt(.49*.51/678) = 0.037629145... ~ 3.7%
Margin of error for McCain = 1.96*sqrt(.48*.52/678) = 0.037606552... ~ 3.7%

The margin of error is typically rounded to the nearest tenth of a percent, and unless there is a lot of undecided or support for other candidates, it is very common in a two person race that the margin or error for each candidate will round to the same number.

The correct sentence to explain the following data would be as follows: If the election were held the day the poll was taken, we are 95% confident that Obama would get between 45.2% to 52.7% of the vote, while McCain would garner between 44.2% to 51.7% of the vote.

The final election results in Florida, rounded to the nearest thousand, were as follows.

Obama 4,282, 000 51.0%
McCain 4,045,000 48.2%
Other 63,000 0.8%

Both candidates were inside the 95% confidence intervals stated in the final poll. The other candidate vote is significantly lower than the 3% of the final poll, but that included undecided voters, who may have chosen one candidate over another, or not voted at all.

Final poll from North Dakota - 2008
n = 500
p-hat(Obama) = 46%
p-hat(McCain) = 47%
p-hat(undecided or other candidates) = 7%

Margin of error for Obama = 1.96*sqrt(.46*.54/500) = 0.043686461... ~ 4.4%
Margin of error for McCain = 1.96*sqrt(.47*.53/500) = 0.043747973... ~ 4.4%

Again, the two margins of error round to the same tenth of a percent.

The correct sentence to explain the following data would be as follows: If the election were held the day the poll was taken, we are 95% confident that Obama would get between 41.6% to 50.4% of the vote, while McCain would garner between 42.6% to 51.4% of the vote.

The final election results in North Dakota, rounded to the nearest thousand, were as follows.

Obama 141,000 44.6%
McCain 169,000 53.5%
Other 6,000 1.9%

While Obama's result was inside the 95% confidence interval stated in the final poll, McCain did better than expected. 95% confidence means there will be mistakes about 5% of the time, which is about 1 chance in 20 of being wrong. If the polling company had used the 99% confidence interval, and in polling data no one ever does, the numbers would have been as follows.

Margin of error for Obama = 2.575*sqrt(.46*.54/500) = 0.057394203... ~ 5.7%
Margin of error for McCain = 2.575*sqrt(.47*.53/500) = 0.057475015... ~ 5.7%

McCain's 53.5% is still above the high end of the 99% confidence interval.

In class, we did samples of m&m's. Here are the totals for the second class, with both samples put together.

n= 1700
p-hat(red) = 221/1700 = 13.0%
p-hat(blue) = 382/1700 ~ 22.5%

sp-hat(red) = sqrt(.13*.87/1700) = 0.008156556...
sp-hat(blue) = sqrt(.225*.775/1700) = 0.010127859...

Here are the confidence intervals for the percentage of red m&m's in the current world population of milk chocolate m&m's, or at least those manufactured in Hackettstown, New Jersey

90% confidence interval: 13% +/- 1.645*sqrt(.13*.87/1700) = 13% +/- 1.3% = 11.7% to 14.3%
95% confidence interval: 13% +/- 1.96*sqrt(.13*.87/1700) = 13% +/- 1.6% = 11.4% to 14.6%
99% confidence interval: 13% +/- 2.575*sqrt(.13*.87/1700) = 13% +/- 2.1% = 10.9% to 15.1%

Practice problems:

Find the confidence intervals for the true percentage of blue milk chocolate m&m's, for 90% confidence, 95% confidence and 99% confidence. Do the 99% confidence intervals for blue and red overlap?

Answers in the comments.