1. Find the frequencies and relative frequencies for the ethnicity variable. Round the relative frequencies by the +/- 0.1% rule.
2. Using just the men's heights, find the following statistics from the sample. Round non-integer answers to the nearest tenth.
n =
x-bar =
mode =
median =
maximum value =
minimum value =
mid-range =
Answer in the comments.
Tuesday, June 23, 2009
summer survey data
Here is the data from the summer survey. There has been some sorting, so the subject numbers will be different from the numbers on your list, but the general data is the same.
The columns are
Subject #
Gender
Ethnicity
Height in inches
Age
L/R handedness
Difficulty of class
GPA
Hours of sleep
Major
Math Opinion
_1 F African_ 64 30-39___ L 5 3.10 6___ BF___ B
_2 F AfroAm__ 66 20-29___ R 4 2.00 ____ CESM_ E
_3 F AfroAm__ 60 20-29___ R 2 3.19 6.5_ HE___ C
_4 F AfroAm__ 66 30-39___ R 4 3.00 4.5_ BF___ C
_5 F AfroAm__ 74 30-39___ R 3 2.90 8.5_ AH___ C
_6 F Asian___ 61 19&under R 5 4.00 7___ Und__ B
_7 F Asian___ 62 19&under R 4 3.16 7___ BF___ B
_8 F Asian___ __ 20-29___ R 3 3.59 7.5_ CESM_ C
_9 F Asian___ 66 20-29___ R 3 ____ ____ BF___ C
10 F Asian___ 65 20-29___ R 2 3.70 5___ HE___ B
11 F Asian___ 60 30-39___ R 4 4.00 8___ SS___ D
12 F Asian___ 63 40-49___ R 4 ____ 8___ Und__ B
13 F AsianAm_ 60 19&under L 3 2.66 10__ CESM_ D
14 F AsianAm_ 62 20-29___ L 5 2.89 6___ Other D
15 F AsianAm_ 66 40-49___ R 5 3.85 7___ HE___ E
16 F EuroAm__ 69 20-29___ R 3 4.00 8.5_ HE___ D
17 F Hispanic 66 20-29___ R 4 3.50 7___ SS___ A
18 F Other___ 66 20-29___ R 4 3.50 6.5_ Other B
19 M African_ 68 30-39___ R 3 3.78 5.5_ HE___ E
20 M AfroAm__ 72 19&under R 3 2.45 10.5 CESM_ B
21 M AfroAm__ 71 20-29___ R 3 2.36 7___ Other C
22 M AfroAm__ 78 50&over_ R 4 3.10 7___ AH___ E
23 M Asian___ 72 19&under R 3 2.50 7___ BF___ B
24 M Asian___ 69 19&under R 2 3.52 7___ CESM_ E
25 M Asian___ 70 20-29___ R 4 2.85 7___ Other C
26 M Asian___ __ 20-29___ R 3 3.00 7.5_ BF___ C
27 M Asian___ __ 20-29___ R 3 ____ 7___ BF___ D
28 M AsianAm_ 66 19&under R 3 3.00 6___ Und__ C
29 M EuroAm__ 76 19&under R 3 3.60 7___ Und__ C
30 M EuroAm__ 71 50&over_ L 1 3.76 5___ CESM_ E
31 M European 69 20-29___ R 4 ____ 9___ SS___ B
32 M Hispanic 63 19&under R 2 3.40 8___ AH___ E
33 M Hispanic 77 19&under R 1 ____ 5___ CESM_ E
34 M Hispanic 66 20-29___ R 3 2.67 6.5_ HE___ C
35 M Hispanic 68 30-39___ R 3 3.50 7.5_ Other E
36 M Other___ 70 19&under R 5 3.83 9___ Und__ E
37 M Other___ 65 20-29___ R 5 3.87 4___ Other C
38 M Other___ 71 20-29___ R 5 3.00 6.5_ Und__ C
n= 38 38_____ 35 38_____ 38 38 33_ 36__ 38___ 38
The columns are
Subject #
Gender
Ethnicity
Height in inches
Age
L/R handedness
Difficulty of class
GPA
Hours of sleep
Major
Math Opinion
_1 F African_ 64 30-39___ L 5 3.10 6___ BF___ B
_2 F AfroAm__ 66 20-29___ R 4 2.00 ____ CESM_ E
_3 F AfroAm__ 60 20-29___ R 2 3.19 6.5_ HE___ C
_4 F AfroAm__ 66 30-39___ R 4 3.00 4.5_ BF___ C
_5 F AfroAm__ 74 30-39___ R 3 2.90 8.5_ AH___ C
_6 F Asian___ 61 19&under R 5 4.00 7___ Und__ B
_7 F Asian___ 62 19&under R 4 3.16 7___ BF___ B
_8 F Asian___ __ 20-29___ R 3 3.59 7.5_ CESM_ C
_9 F Asian___ 66 20-29___ R 3 ____ ____ BF___ C
10 F Asian___ 65 20-29___ R 2 3.70 5___ HE___ B
11 F Asian___ 60 30-39___ R 4 4.00 8___ SS___ D
12 F Asian___ 63 40-49___ R 4 ____ 8___ Und__ B
13 F AsianAm_ 60 19&under L 3 2.66 10__ CESM_ D
14 F AsianAm_ 62 20-29___ L 5 2.89 6___ Other D
15 F AsianAm_ 66 40-49___ R 5 3.85 7___ HE___ E
16 F EuroAm__ 69 20-29___ R 3 4.00 8.5_ HE___ D
17 F Hispanic 66 20-29___ R 4 3.50 7___ SS___ A
18 F Other___ 66 20-29___ R 4 3.50 6.5_ Other B
19 M African_ 68 30-39___ R 3 3.78 5.5_ HE___ E
20 M AfroAm__ 72 19&under R 3 2.45 10.5 CESM_ B
21 M AfroAm__ 71 20-29___ R 3 2.36 7___ Other C
22 M AfroAm__ 78 50&over_ R 4 3.10 7___ AH___ E
23 M Asian___ 72 19&under R 3 2.50 7___ BF___ B
24 M Asian___ 69 19&under R 2 3.52 7___ CESM_ E
25 M Asian___ 70 20-29___ R 4 2.85 7___ Other C
26 M Asian___ __ 20-29___ R 3 3.00 7.5_ BF___ C
27 M Asian___ __ 20-29___ R 3 ____ 7___ BF___ D
28 M AsianAm_ 66 19&under R 3 3.00 6___ Und__ C
29 M EuroAm__ 76 19&under R 3 3.60 7___ Und__ C
30 M EuroAm__ 71 50&over_ L 1 3.76 5___ CESM_ E
31 M European 69 20-29___ R 4 ____ 9___ SS___ B
32 M Hispanic 63 19&under R 2 3.40 8___ AH___ E
33 M Hispanic 77 19&under R 1 ____ 5___ CESM_ E
34 M Hispanic 66 20-29___ R 3 2.67 6.5_ HE___ C
35 M Hispanic 68 30-39___ R 3 3.50 7.5_ Other E
36 M Other___ 70 19&under R 5 3.83 9___ Und__ E
37 M Other___ 65 20-29___ R 5 3.87 4___ Other C
38 M Other___ 71 20-29___ R 5 3.00 6.5_ Und__ C
n= 38 38_____ 35 38_____ 38 38 33_ 36__ 38___ 38
Notes for 6/22
Much of statistics deals with data sets. A data set can either be a population, which means it contains everyone in a particular group we are interested in, or it can be a sample, which means a subset of a population. For instance, if everyone in class shows up on the day of a quiz, I could consider that the population of students in the class and the scores on the quiz could be the variable we are collecting. On the other hand, the class could be considered a sample of all students at Laney, or all students at Laney taking statistics, or all students in classes that start at 12:15. The decision on whether it is a sample or a population in many cases can be considered arbitrary, which means that someone made a decision. Arbiter means judge, and some arbitrary decisions are based on simple personal preference while other arbitrary decisions may be based on pre-set rules.

In statistics, there are some symbols that are reserved, which means a particular letter cannot be used to mean just anything. The first two such letters are N and n, which mean the size of a population and the size of a sample, respectively. The first kind of data we have dealt with is categorical data, where the answers to the questions are not numerical. How often a particular answer shows up in the population is called the frequency, denoted by F in a population or f in a sample. It would be nice if capital letters always meant population and lowercase letters always meant sample, but that is not the case. We also have relative frequency, which is p in a population and p-hat in a sample.
Note: the text editor for the blog doesn't allow for fancy marks on letters or even Greek letters, so in some cases when typing, the symbols will have to be replaced with words like p-hat or x-bar, which is the way they are pronounced.
There is also the situation of subscripts and superscripts. On the blog, it is possible to show subscripts, like x3, but if we want to square a number, the text editor doesn't allow making a small number that floats above the line, so instead we will use x^2. The symbol "^" is also used on your calculator to indicate raising a number to a power.
Frequencies are always whole numbers, either positive integers or zero. Relative frequencies are proportions, numbers between 0 and 1, inclusively. We could write these as fractions, but we often use decimals or percents, which leads to rounding.
Rounding proportions
In gneral, people like to use percents when talking about proportions because 23% looks like a whole number, though it really isn't. 23% is the same as 23/100 or 0.23. When dealing with large proportions, which is to say proportions over 1%, percent will be the standard. How far we round the number will depend on the sum of all relative frequencies.
For example, if we have four categories and each category has a relative frequency of 1/4, we could write 25% for each. If we add up all relative frequencies and we don't round, the sum will be exactly 1 or 100%.
25%+25%+25%+25= 100%
If instead we have eight categories and each has a relative frequency of 1/8, rounding to the nearest percent gives us 13%. (Without rounding 1/8 = .125, so we would round up.)
13%+13%+13%+13%+13%+13%+13%+13%=104%
If the sum is more than one tenth of one percent away from 100%, we need to round to more places. In this case 1/8 = 12.5% exactly so if we write the proportions to the nearest tenth of a percent, we get
12.5%+12.5%+12.5%+12.5%+12.5%+12.5%+12.5%+12.5%=100%
With 1/3 = .333......, we don't get so lucky.
33%+33%+33%=99% (Close, but not 100%.)
33.3%+33.3%+33.3%=99.9% (Still not exactly right.)
33.33%+33.33%+33.33%=99.99% (Still not exactly right, and it never will be.)
Since in some cases, rounding will always produce some error, we make the arbitrary decision that being within one tenth of one percent is close enough, which means between 99.9% and 100.1%, inclusive. So with in this case, we should round to the nearest tenth of a percent.
Scales based on powers of 10: The most famous scale base on powers of ten in percentage, which really means "per 100". It is much more common to see "53% of the people agree with the president's plan" than ".53 of the people..." or "53 out of every 100 people...". Technically, all those phrases are saying the same thing, but percentage is the most popular.
One of the places where decimals are used for proportions is in the sports pages. A batting average in baseball (hits/at bats) is given as a percent to three decimal place, and likewise winning proportions (win/total games) are written as .xxx. If a batter has 27 hits in 92 at-bats, the batting average 27/92 = .293478261... is shortened to .293 and pronounced "two ninety three". Likewise, a team who has won 17 games and lost 5 will have a winning proportion of 17/22 = .77272727... = .773, and often stated as "team has a winning percentage of seven seventy three." Technically, this is a mistake, because "percentage" means out or 100. The correct word from the dictionary, which no one ever uses, is "permillage", which means out of 1,000. The team in question would have a winning percentage of 77/100, and a winning permillage of 773/1000.
In both of the cases from the sports pages, the greater number of place after the decimal is used to break ties. For example, a team with 14 wins and 4 losses is at .778, which is better than 17 wins and 5 losses, while 20 wins and 6 losses is at .769, so is slightly worse.
To get a number based on a power of 10 scale, you take the proportion and multiply by the power of ten, so it is either p*scale or p-hat*scale, depending on population or sample. Besides greater precision for breaking ties, sometimes we need greater precision because the proportions are so small.
When I ask a class what is the legal limit for blood alcohol while driving, invariably someone will say "point oh eight" and most people will agree. But .08 is wrong. .08 = 8%, and the correct answer is .08% = .0008. I don't blame the students. The number is badly represented and it is an easy mistake to make. Let's take a look at the number on other scales of 10.
.08 out of 100 is the same as
.8 out of 1,000 0r
8 out of 10,000 or
80 out of 100,000
80 parts out of 100,000 is a tiny proportion. To give an idea, ounce of pure alcohol mixed into ten gallons of blood would give you 78 parts out of 100,000, and most people have between a half gallon and a gallon and a half of blood in their body, between 4 and 12 pints. The amount of alcohol in a person's blood stream that is over the legal limit is about the same amount of alcohol as found in a capful of mouthwash used after brushing your teeth.
We will be dealing with much smaller proportions later in the class, where there are things that can be hazardous to your health at ranges measure in parts per billion, but for now, we will look at the per 100,000 scale for another type of statistic, measurements of mortality rates.
Here are the number of homicides in some local cities in 2007.
Oakland: 124 homicides
Richmond: 28 homicides
San Francisco: 98 homicides
Clearly, comparing these numbers is misleading, because we know these cities have very different numbers of citizens, so the standard way to measure these statistics is the per 100,000 population scale, which we find by the formula
f/n* scale
which in this case is
(# of homicides)/(city population) * 100,000
Oakland's population in 2007 is estimated at 415,000, Richmond at 106,000 and San Francisco at 825,000, so the murder rates on this standard scale are as follows
Oakland: 124/415000 * 100000 = 29.9
Richmond: 28/106000 * 100000 = 26.4
San Francisco: 98/825000 * 100000 = 11.9
So even though more people were murdered in San Francisco than in Richmond in 2007, the murder rate in Richmond was over twice as high, because Richmond has barely 1/8 of the population of San Francisco. (note: The trends for the three cities this decade are going in different directions. Oakland's murder rate is on the rise, while Richmond's is falling and San Francisco's has stayed about the same.)
Calculating proportions (probabilities): There are times when we will need to find new proportions from information previously calculated, either adding and subtracting old numbers or multiplying or dividing. It's best to use the fractional forms of the data when available, then round the answers after using the exact numbers instead of using answers that might have been rounded. Every time you use a rounded answer in a calculation, there is a change to increase the rounding error even more.

In statistics, there are some symbols that are reserved, which means a particular letter cannot be used to mean just anything. The first two such letters are N and n, which mean the size of a population and the size of a sample, respectively. The first kind of data we have dealt with is categorical data, where the answers to the questions are not numerical. How often a particular answer shows up in the population is called the frequency, denoted by F in a population or f in a sample. It would be nice if capital letters always meant population and lowercase letters always meant sample, but that is not the case. We also have relative frequency, which is p in a population and p-hat in a sample.
Note: the text editor for the blog doesn't allow for fancy marks on letters or even Greek letters, so in some cases when typing, the symbols will have to be replaced with words like p-hat or x-bar, which is the way they are pronounced.There is also the situation of subscripts and superscripts. On the blog, it is possible to show subscripts, like x3, but if we want to square a number, the text editor doesn't allow making a small number that floats above the line, so instead we will use x^2. The symbol "^" is also used on your calculator to indicate raising a number to a power.
Frequencies are always whole numbers, either positive integers or zero. Relative frequencies are proportions, numbers between 0 and 1, inclusively. We could write these as fractions, but we often use decimals or percents, which leads to rounding.
Rounding proportions
In gneral, people like to use percents when talking about proportions because 23% looks like a whole number, though it really isn't. 23% is the same as 23/100 or 0.23. When dealing with large proportions, which is to say proportions over 1%, percent will be the standard. How far we round the number will depend on the sum of all relative frequencies.
For example, if we have four categories and each category has a relative frequency of 1/4, we could write 25% for each. If we add up all relative frequencies and we don't round, the sum will be exactly 1 or 100%.
25%+25%+25%+25= 100%
If instead we have eight categories and each has a relative frequency of 1/8, rounding to the nearest percent gives us 13%. (Without rounding 1/8 = .125, so we would round up.)
13%+13%+13%+13%+13%+13%+13%+13%=104%
If the sum is more than one tenth of one percent away from 100%, we need to round to more places. In this case 1/8 = 12.5% exactly so if we write the proportions to the nearest tenth of a percent, we get
12.5%+12.5%+12.5%+12.5%+12.5%+12.5%+12.5%+12.5%=100%
With 1/3 = .333......, we don't get so lucky.
33%+33%+33%=99% (Close, but not 100%.)
33.3%+33.3%+33.3%=99.9% (Still not exactly right.)
33.33%+33.33%+33.33%=99.99% (Still not exactly right, and it never will be.)
Since in some cases, rounding will always produce some error, we make the arbitrary decision that being within one tenth of one percent is close enough, which means between 99.9% and 100.1%, inclusive. So with in this case, we should round to the nearest tenth of a percent.
Scales based on powers of 10: The most famous scale base on powers of ten in percentage, which really means "per 100". It is much more common to see "53% of the people agree with the president's plan" than ".53 of the people..." or "53 out of every 100 people...". Technically, all those phrases are saying the same thing, but percentage is the most popular.
One of the places where decimals are used for proportions is in the sports pages. A batting average in baseball (hits/at bats) is given as a percent to three decimal place, and likewise winning proportions (win/total games) are written as .xxx. If a batter has 27 hits in 92 at-bats, the batting average 27/92 = .293478261... is shortened to .293 and pronounced "two ninety three". Likewise, a team who has won 17 games and lost 5 will have a winning proportion of 17/22 = .77272727... = .773, and often stated as "team has a winning percentage of seven seventy three." Technically, this is a mistake, because "percentage" means out or 100. The correct word from the dictionary, which no one ever uses, is "permillage", which means out of 1,000. The team in question would have a winning percentage of 77/100, and a winning permillage of 773/1000.
In both of the cases from the sports pages, the greater number of place after the decimal is used to break ties. For example, a team with 14 wins and 4 losses is at .778, which is better than 17 wins and 5 losses, while 20 wins and 6 losses is at .769, so is slightly worse.
To get a number based on a power of 10 scale, you take the proportion and multiply by the power of ten, so it is either p*scale or p-hat*scale, depending on population or sample. Besides greater precision for breaking ties, sometimes we need greater precision because the proportions are so small.
When I ask a class what is the legal limit for blood alcohol while driving, invariably someone will say "point oh eight" and most people will agree. But .08 is wrong. .08 = 8%, and the correct answer is .08% = .0008. I don't blame the students. The number is badly represented and it is an easy mistake to make. Let's take a look at the number on other scales of 10.
.08 out of 100 is the same as
.8 out of 1,000 0r
8 out of 10,000 or
80 out of 100,000
80 parts out of 100,000 is a tiny proportion. To give an idea, ounce of pure alcohol mixed into ten gallons of blood would give you 78 parts out of 100,000, and most people have between a half gallon and a gallon and a half of blood in their body, between 4 and 12 pints. The amount of alcohol in a person's blood stream that is over the legal limit is about the same amount of alcohol as found in a capful of mouthwash used after brushing your teeth.
We will be dealing with much smaller proportions later in the class, where there are things that can be hazardous to your health at ranges measure in parts per billion, but for now, we will look at the per 100,000 scale for another type of statistic, measurements of mortality rates.
Here are the number of homicides in some local cities in 2007.
Oakland: 124 homicides
Richmond: 28 homicides
San Francisco: 98 homicides
Clearly, comparing these numbers is misleading, because we know these cities have very different numbers of citizens, so the standard way to measure these statistics is the per 100,000 population scale, which we find by the formula
f/n* scale
which in this case is
(# of homicides)/(city population) * 100,000
Oakland's population in 2007 is estimated at 415,000, Richmond at 106,000 and San Francisco at 825,000, so the murder rates on this standard scale are as follows
Oakland: 124/415000 * 100000 = 29.9
Richmond: 28/106000 * 100000 = 26.4
San Francisco: 98/825000 * 100000 = 11.9
So even though more people were murdered in San Francisco than in Richmond in 2007, the murder rate in Richmond was over twice as high, because Richmond has barely 1/8 of the population of San Francisco. (note: The trends for the three cities this decade are going in different directions. Oakland's murder rate is on the rise, while Richmond's is falling and San Francisco's has stayed about the same.)
Calculating proportions (probabilities): There are times when we will need to find new proportions from information previously calculated, either adding and subtracting old numbers or multiplying or dividing. It's best to use the fractional forms of the data when available, then round the answers after using the exact numbers instead of using answers that might have been rounded. Every time you use a rounded answer in a calculation, there is a change to increase the rounding error even more.
Labels:
data sets,
population,
reserved symbols,
rounding,
sample
Monday, June 22, 2009
Syllabus Summer 2009
Math 13: Introduction to Statistics Summer 2009 – Laney College
Instructor: Matthew Hubbard Text: none
Email: mhubbard@peralta.edu, profhubbard@gmail.com
website: budgetstats.blogspot.com
Office hours:
T-Th 9:25 to 9:55 am in G-201 (math lab)
M-W 3:15-3:45 pm in G-201 (math lab)
Wednesday 6-8 (math lab)
Scientific calculator required (TI-30X IIs or TI-83 recommended)
Important academic schedule dates
Last date to add, if class is not full: Sat., June 27Last date to drop class without a “W”: Thurs., July 2
Last date to drop class with a “W”: Wed., July 15
Holidays and professional development days that effect the Summer schedule:
NoneMidterm and Finals schedule:
Thurs., July 2 Midterm 1 (2 hours)Thurs., July 16 Midterm 2 (2 hours)
Thurs., July 30 (3 hours - comprehensive)
Grading Policy
Homework to be turned in: Assigned every Tuesday and Thursday, due the next class period(late homework accepted AT THE BEGINNING of class period after next, 2 points off)
Quizzes: Tuesdays and Thursdays weeks without midterms – no make-up quizzes
If arranged beforehand, make-up midterms can be given, but must be taken before the next class meeting.
The two lowest scores from homework and the lowest score from quizzes will be removed from consideration before grading.
Grading system
Quizzes * 25%Midterm 1 * 25%
Midterm 2 * 25%
Homework 20%
Final 30%
The lowest grade from Quizzes and the two Midterms will be dropped from the total.
Anyone getting a higher percentage score on the final than the weighted average of all grades combined will get the final percentage instead on the final grade, provided that student has not missed more than two homework assignments.
Academic honesty
Your homework, exams and quizzes must be your own work. Anyone caught cheating on these assignments will be punished, where the punishment can be as severe as failing the class or being put on college wide academic probation.Class rules
Cell phones and beepers turned off, no headphones or text messaging during classNo food or drink in class, except for sealable bottles. All empty bottles should be put in the recycling bins after class is over.
You will need your own calculator and handout sheets for tests and quizzes. Do not expect to be able to borrow these from someone else.
Student Learning Outcomes
1. Describe numerical and categorical data using statistical terminology and notation.
2. Analyze and explain relationships between variables in a sample or a population.
3. Make inferences about populations based on data obtained from samples.
4. Given a particular statistical or probabilistic context, determine whether or not a particular analytical methodology is appropriate and explain why.
Thursday, May 28, 2009
correlation practice
Here are three sets of seven numbers each. The x values are the number of points scored by Kobe Bryant in the playoff games against the Houston Rockets, the y values are the points scored by the Lakers in those same games and the z values are the difference between the Lakers' score and the Rockets' score in each game.
x: __32__40__33__35__26__32__33
y: __92_111_108__87_118__80__89
z: __-8_+13_+14_-12_+30_-15_+19
1. Find the correlation coefficients for all three pairs of number sets, rx,y, rx,z and ry,z.
2. What are the cut-off values for 95% confidence and 99% confidence for correlation?
3. Which of the pairs of sets has the highest absolute correlation and what confidence level does that correlation exceed?
Round all answers to three digits. Answers in the comments.
x: __32__40__33__35__26__32__33
y: __92_111_108__87_118__80__89
z: __-8_+13_+14_-12_+30_-15_+19
1. Find the correlation coefficients for all three pairs of number sets, rx,y, rx,z and ry,z.
2. What are the cut-off values for 95% confidence and 99% confidence for correlation?
3. Which of the pairs of sets has the highest absolute correlation and what confidence level does that correlation exceed?
Round all answers to three digits. Answers in the comments.
Wednesday, May 20, 2009
Topic for final exam
The final exam will be given at two times.
May 22: 8 a.m. to 10 a.m.
May 29: 10 a.m. to noon.
The final is comprehensive. You will need your yellow sheets, a calculator, scratch paper and a pencil.
The test will be four or five pages long. The amount from each part of the class will be
25%-30% from first exam
25%-30% from second exam
40-50% from after the second exam
On the list of topics below, any topic with an asterisk (*) means that though it might have been introduced before the first or second exam, it gets used throughout the class, so I don't necessarily count it as in the percentage of problem promised for each section.
Any topic in bold means you are expected to know how to get the answer without a formula or instructions being provided. In many cases, this means knowing how to use your calculator properly.
First exam
==========
frequency tables
stem and leaf plots
five number summary
box and whiskers
IQR and outliers for box and whiskers
Mean*, median*, mode, mid-range
parameter* and statistic*
population* and sample*
categorical data*
numerical data*
Bar charts
Pie charts
Line charts
Ogives
dotplots
percentage increase and decrease
contingency tables*
degrees of freedom*
conditional probability*
frequency and relative frequency
inclusion-exclusion
complementary event
order of operations*
Second exam through April 8
==========
standard deviation*
confidence intervals and margin of error
t-scores and z-scores*
raw scores, z-scores and percentages*
common critical values
Central Limit Theorem
Confidence of victory
After second exam
=======
Binomial coefficients and falling factorial
expected value of correct results
dependent and independent probabilities
Classic and modern parimutuel
expected value of a game
exactly r correct out of n trials
Bayesian probabilities
Hypothesis testing:
null hypothesis, alternative hypothesis, type I error, type II error
test statistic
threshold for xx% confidence (one-tailed high, one-tailed low, two-tailed)
one sample testing
two samples testing
Correlation (rx,y and the equation of the line yp = ax + b)
If you have any specific questions or want to make time to talk to me before the final, send me an e-mail and we can make an appointment.
May 22: 8 a.m. to 10 a.m.
May 29: 10 a.m. to noon.
The final is comprehensive. You will need your yellow sheets, a calculator, scratch paper and a pencil.
The test will be four or five pages long. The amount from each part of the class will be
25%-30% from first exam
25%-30% from second exam
40-50% from after the second exam
On the list of topics below, any topic with an asterisk (*) means that though it might have been introduced before the first or second exam, it gets used throughout the class, so I don't necessarily count it as in the percentage of problem promised for each section.
Any topic in bold means you are expected to know how to get the answer without a formula or instructions being provided. In many cases, this means knowing how to use your calculator properly.
First exam
==========
frequency tables
stem and leaf plots
five number summary
box and whiskers
IQR and outliers for box and whiskers
Mean*, median*, mode, mid-range
parameter* and statistic*
population* and sample*
categorical data*
numerical data*
Bar charts
Pie charts
Line charts
Ogives
dotplots
percentage increase and decrease
contingency tables*
degrees of freedom*
conditional probability*
frequency and relative frequency
inclusion-exclusion
complementary event
order of operations*
Second exam through April 8
==========
standard deviation*
confidence intervals and margin of error
t-scores and z-scores*
raw scores, z-scores and percentages*
common critical values
Central Limit Theorem
Confidence of victory
After second exam
=======
Binomial coefficients and falling factorial
expected value of correct results
dependent and independent probabilities
Classic and modern parimutuel
expected value of a game
exactly r correct out of n trials
Bayesian probabilities
Hypothesis testing:
null hypothesis, alternative hypothesis, type I error, type II error
test statistic
threshold for xx% confidence (one-tailed high, one-tailed low, two-tailed)
one sample testing
two samples testing
Correlation (rx,y and the equation of the line yp = ax + b)
If you have any specific questions or want to make time to talk to me before the final, send me an e-mail and we can make an appointment.
Monday, May 18, 2009
Class notes for 5/18: Regression and correlation
When we have a data set, sometimes we collect more than one variable of information about the units. For example, in our class survey, among the numerical variables were the height in inches, the GPA, the opinion about the difficulty of the class, age and average hours of sleep per night.
A question about two variables is if they are related to one another in some simple way. One simple way is correlation, which can be positive or negative. Here is a general definition of each.
Positive correlation between two numerical variables, call them x and y, means that the high values of x tend to be paired with the high values of y, the middle values of x tend to be paired with the middle values of y and the low values of x tend to be paired with the low values of y.
The variables x and y show negative correlation if that the high values of x tend to be paired with the low values of y, the middle values of x tend to be paired with the middle values of y and the low values of x tend to be paired with the high values of y.
If we pick two variables at random, we do not expect to see correlation. We can write this as a null hypothesis, where the test statistic is rx,y, the correlation coefficient. The sign of low correlation is rx,y = 0. The values of rx,y are always between -1, which means perfect negative correlation, and +1, which means perfect positive correlation.
The seventh page of the yellow sheets gives us threshold numbers for the 99% confidence level and 95% confidence level for correlation given the number of points n. For instance, when n = 5, the thresholds are .878 for 95% confidence and .959 for 99% confidence. This splits up the numbers from -1 to 1 into five regions.
-1 <= rx,y <= -.959: Very strong negative correlation
-.959 < rx,y <= -.878: Strong negative correlation
-.878 < rx,y < .878: The correlation is not particularly strong, regardless of positive or negative.
.878 <= rx,y < .959: strong positive correlation
.959 <= rx,y <= 1: very strong positive correlation
Just like with any hypothesis test, we should decide the confidence level before testing. This is a two-tailed test, because whether correlation is positive or negative, the relationships between number sets can often give us vital scientific information.
There is an important warning: Correlation is not causation. Just because two number sets have a relation, it doesn't mean that x causes y or y causes x. Sometimes there is a hidden third factor that is the cause of both of the things we are looking at. Sometimes, it's random chance and there is no causative agent at all.

Here is a set of five points, listed as (x,y) in each case.
(1,1)
(2,2)
(3,4)
(4,4)
(6,5)
As we can see, the points are ordered from low to high in both coordinates, so we expect some correlation. If we input the points into our calculator, we get a value for r (which is the same as rx,y) of .933338696..., which is strong positive correlation, but not very strong positive correlation. Assuming the 95% confidence level is good enough for us, we can use the a and b variables from out calculator to give us the equation of the line
yp = .797x + .649
This is called the predictor line (that's where the p comes from) or the line of regression or the line of least squares. Any such line for a given data set has two important criteria it meets. It passes through the centroid (x-bar, y-bar), the center point of all the data, and it minimizes the sum of the absolute values of the residuals, which is |y - yp| for all points.
Let's find the absolute values of the residuals for each of the five points, using the rounded values of a and b.
Point (1,1): |1 - .797*1 - .649| = 0.446
Point (2,2): |2 - .797*2 - .649| = 0.243
Point (3,4): |4 - .797*3 - .649| = 0.96
Point (4,4): |4 - .797*4 - .649| = 0.163
Point (6,5): |5 - .797*6 - .649| = 0.431
As we can see, the point (3,4) is farthest from the line, while the point (4,4) is the closest. The centroid (3.2, 3.2) is exactly on the line if you use the un-rounded values of a and b, and even using the rounded values, the centroid only misses the line by .0006.
In class, we used five points, but the last point was (5,6) instead of (6,5). This changes the numbers. rx,y goes up to .973328527..., which is above the 99% confidence threshold. The formula for the new predictor line is
yp = 1.2x - .2
Where we see the difference in these two different examples is in the residuals.
Point (1,1): |1 - 1.2*1 + .2| = 0
Point (2,2): |2 - 1.2*2 + .2| = 0.2
Point (3,4): |4 - 1.2*3 + .2| = 0.6
Point (4,4): |4 - 1.2*4 + .2| = 0.6
Point (5,6): |6 - 1.2*5 + .2| = 0.2
The closest point is now exactly on the line, which is a rarity, but even the farthest away point is only .6 units away, closer than the farthest away on the line with the lower correlation coefficient.
As we get more points in our data set, we lower our threshold that shows correlation strength. This way, a few points that are outliers do not completely ruin the chances of the data showing correlation, though sometimes strong outliers can mess up the data set and the correlation coefficient gets so close to zero that we cannot reject the null hypothesis that the two variables are not simply related.
A question about two variables is if they are related to one another in some simple way. One simple way is correlation, which can be positive or negative. Here is a general definition of each.
Positive correlation between two numerical variables, call them x and y, means that the high values of x tend to be paired with the high values of y, the middle values of x tend to be paired with the middle values of y and the low values of x tend to be paired with the low values of y.
The variables x and y show negative correlation if that the high values of x tend to be paired with the low values of y, the middle values of x tend to be paired with the middle values of y and the low values of x tend to be paired with the high values of y.
If we pick two variables at random, we do not expect to see correlation. We can write this as a null hypothesis, where the test statistic is rx,y, the correlation coefficient. The sign of low correlation is rx,y = 0. The values of rx,y are always between -1, which means perfect negative correlation, and +1, which means perfect positive correlation.
The seventh page of the yellow sheets gives us threshold numbers for the 99% confidence level and 95% confidence level for correlation given the number of points n. For instance, when n = 5, the thresholds are .878 for 95% confidence and .959 for 99% confidence. This splits up the numbers from -1 to 1 into five regions.
-1 <= rx,y <= -.959: Very strong negative correlation
-.959 < rx,y <= -.878: Strong negative correlation
-.878 < rx,y < .878: The correlation is not particularly strong, regardless of positive or negative.
.878 <= rx,y < .959: strong positive correlation
.959 <= rx,y <= 1: very strong positive correlation
Just like with any hypothesis test, we should decide the confidence level before testing. This is a two-tailed test, because whether correlation is positive or negative, the relationships between number sets can often give us vital scientific information.
There is an important warning: Correlation is not causation. Just because two number sets have a relation, it doesn't mean that x causes y or y causes x. Sometimes there is a hidden third factor that is the cause of both of the things we are looking at. Sometimes, it's random chance and there is no causative agent at all.

Here is a set of five points, listed as (x,y) in each case.
(1,1)
(2,2)
(3,4)
(4,4)
(6,5)
As we can see, the points are ordered from low to high in both coordinates, so we expect some correlation. If we input the points into our calculator, we get a value for r (which is the same as rx,y) of .933338696..., which is strong positive correlation, but not very strong positive correlation. Assuming the 95% confidence level is good enough for us, we can use the a and b variables from out calculator to give us the equation of the line
yp = .797x + .649
This is called the predictor line (that's where the p comes from) or the line of regression or the line of least squares. Any such line for a given data set has two important criteria it meets. It passes through the centroid (x-bar, y-bar), the center point of all the data, and it minimizes the sum of the absolute values of the residuals, which is |y - yp| for all points.
Let's find the absolute values of the residuals for each of the five points, using the rounded values of a and b.
Point (1,1): |1 - .797*1 - .649| = 0.446
Point (2,2): |2 - .797*2 - .649| = 0.243
Point (3,4): |4 - .797*3 - .649| = 0.96
Point (4,4): |4 - .797*4 - .649| = 0.163
Point (6,5): |5 - .797*6 - .649| = 0.431
As we can see, the point (3,4) is farthest from the line, while the point (4,4) is the closest. The centroid (3.2, 3.2) is exactly on the line if you use the un-rounded values of a and b, and even using the rounded values, the centroid only misses the line by .0006.
In class, we used five points, but the last point was (5,6) instead of (6,5). This changes the numbers. rx,y goes up to .973328527..., which is above the 99% confidence threshold. The formula for the new predictor line is
yp = 1.2x - .2
Where we see the difference in these two different examples is in the residuals.
Point (1,1): |1 - 1.2*1 + .2| = 0
Point (2,2): |2 - 1.2*2 + .2| = 0.2
Point (3,4): |4 - 1.2*3 + .2| = 0.6
Point (4,4): |4 - 1.2*4 + .2| = 0.6
Point (5,6): |6 - 1.2*5 + .2| = 0.2
The closest point is now exactly on the line, which is a rarity, but even the farthest away point is only .6 units away, closer than the farthest away on the line with the lower correlation coefficient.
As we get more points in our data set, we lower our threshold that shows correlation strength. This way, a few points that are outliers do not completely ruin the chances of the data showing correlation, though sometimes strong outliers can mess up the data set and the correlation coefficient gets so close to zero that we cannot reject the null hypothesis that the two variables are not simply related.
Labels:
class notes,
correlation,
predictor line,
regression
Subscribe to:
Posts (Atom)
