Showing posts with label Bayesian probability. Show all posts
Showing posts with label Bayesian probability. Show all posts
Tuesday, October 20, 2015
Links to posts on Bayesian probability
An easier method for probabilities of false positives and false negatives
Here are the previous posts about Bayesian probability, which in our case means we want to understand the different between the probabilities of false positive tests and false negative tests.
In class we used a contingency table method to find the probabilities of false negatives and false positives. This is useful if we want to work with doing a second sample, as we did when the false positive probability was so high. If we just want to find probabilities on a single test, here is an easier method.
e = error rate and a = accuracy rate, where e + a = 1
p = proportion of trait in the population and q = 1 - p
probability of a false positive = eq/(eq + pa)
probability of a false negative = ep/(ep + qa)
Sunday, May 11, 2014
Notes for May 6th and 8th
Bayesian probability
If a trait is very rare, only a very accurate test gives us useful information. For example, if a trait shows up in only 1 in 10,000 people but the test for the trait has an error rate of 1 in 1,000, we should expect about 10 false positives for every true positive. Here is the completed table for that situation.
________don't___have____row total
test + ____9,999__999______10,998__
test - _9,989,001___1_____9,989,002__
col.___9,999,000_1,000______10,000,000 grand total
In a situation such as this, testing positive twice could give us useful information, as testing positive once has an error rate of about 90.9%. We have to assume the errors are random and not deterministic. For example, if a test for a chemical compound in opium also catches a similar compound found in poppy seed bagels, testing twice won't get rid of the errors. Assuming just random errors here is what we do.
Step 1: The top row of the first contingency table is the column total/grand total row of the second contingency table. What this does is takes the numbers from the people who tested positive the first time and makes them the totals for those who will be tested twice.
________don't___have____row total
test + __________________________
test - __________________________
col._______9,999__999_____10,998 grand total
Step 2: Multiply error rate by have column total to find the number who have that test negative. Round to the nearest whole number. (We didn't have to round before, but now we do.)
999*1/1000 = .999 ~= 1, this means test positive and have is 998.
________don't___have____row total
test + ___________998____________
test - _____________1____________
col._______9,999__999_____10,998 grand total
Step 3: Multiply error rate by don't have column total to find the errors. 9,999*1/1000 = 9.999 ~= 10. That means the test negative in that column is 9,999 - 10 = 9,989.
________don't___have____row total
test + _______10__998____________
test - _____9,989____1____________
col._______9,999__999_____10,998 grand total
Step 4: row totals
________don't___have____row total
test + _______10__998_____1,008___
test - _____9,989____1_____9,990___
col._______9,999__999_____10,998 grand total
Step 5: Find the error rate for testing positive twice. 10/1,008 = .0099... or about 1%.
Of the ten million people tested, we would send letters to 1,008 telling them they tested positive twice. Of those people, ten don't have the trait and are getting false information, but 998 are getting the right information. In the first test, there was someone with the trait who tested negative, and the same is true in the second test, so there are two people with the trait who did not get two positive test results. While this isn't a perfect situation, it's much better than the over 90% error rate we got for positive tests the first time through.
Relative frequency charts and ogives
Relative frequencies are also known as proportions and can sometimes be considered probabilities. If the proportions correspond to ordered categories, a line chart can be a clear way to present the data.
Here is the data for the Los Angeles Angels scoring by inning, given as percentages. The first number is the percentage scored in that inning and the second number in brackets is the percentage of entire runs scored in a game up through that inning.
1st: 13.5% [13.5%]
2nd: 10.0% [23.5%]
3rd: 10.4% [33.9%]
4th: 11.0% [45.0%]
5th: 11.7% [56.6%]
6th: 11.7% [68.3%]
7th: 8.8% [77.1%]
8th: 12.3% [89.4%]
9th: 9.4% [98.8%]
extra innings: 1.2% [100.0%]
The Angels are fairly consistent, except for the big bump in the first inning and the drop-off in the 7th. Unsurprisingly, they score very few extra inning runs, though some teams like the Giants score four times as many.
In a line graph, we have two possible options. The first is the line in blue, which shows the production inning by inning. The red line shows the cumulative numbers and that graph is called the ogive, pronounced "oh-jive". If they were completely consistent, the ogive would be a completely straight line, but instead we see the slight bends in the red line when run production increased and decreases per inning.
If a trait is very rare, only a very accurate test gives us useful information. For example, if a trait shows up in only 1 in 10,000 people but the test for the trait has an error rate of 1 in 1,000, we should expect about 10 false positives for every true positive. Here is the completed table for that situation.
________don't___have____row total
test + ____9,999__999______10,998__
test - _9,989,001___1_____9,989,002__
col.___9,999,000_1,000______10,000,000 grand total
In a situation such as this, testing positive twice could give us useful information, as testing positive once has an error rate of about 90.9%. We have to assume the errors are random and not deterministic. For example, if a test for a chemical compound in opium also catches a similar compound found in poppy seed bagels, testing twice won't get rid of the errors. Assuming just random errors here is what we do.
Step 1: The top row of the first contingency table is the column total/grand total row of the second contingency table. What this does is takes the numbers from the people who tested positive the first time and makes them the totals for those who will be tested twice.
________don't___have____row total
test + __________________________
test - __________________________
col._______9,999__999_____10,998 grand total
Step 2: Multiply error rate by have column total to find the number who have that test negative. Round to the nearest whole number. (We didn't have to round before, but now we do.)
999*1/1000 = .999 ~= 1, this means test positive and have is 998.
________don't___have____row total
test + ___________998____________
test - _____________1____________
col._______9,999__999_____10,998 grand total
Step 3: Multiply error rate by don't have column total to find the errors. 9,999*1/1000 = 9.999 ~= 10. That means the test negative in that column is 9,999 - 10 = 9,989.
________don't___have____row total
test + _______10__998____________
test - _____9,989____1____________
col._______9,999__999_____10,998 grand total
Step 4: row totals
________don't___have____row total
test + _______10__998_____1,008___
test - _____9,989____1_____9,990___
col._______9,999__999_____10,998 grand total
Step 5: Find the error rate for testing positive twice. 10/1,008 = .0099... or about 1%.
Of the ten million people tested, we would send letters to 1,008 telling them they tested positive twice. Of those people, ten don't have the trait and are getting false information, but 998 are getting the right information. In the first test, there was someone with the trait who tested negative, and the same is true in the second test, so there are two people with the trait who did not get two positive test results. While this isn't a perfect situation, it's much better than the over 90% error rate we got for positive tests the first time through.
Relative frequency charts and ogives
Relative frequencies are also known as proportions and can sometimes be considered probabilities. If the proportions correspond to ordered categories, a line chart can be a clear way to present the data.
Here is the data for the Los Angeles Angels scoring by inning, given as percentages. The first number is the percentage scored in that inning and the second number in brackets is the percentage of entire runs scored in a game up through that inning.
1st: 13.5% [13.5%]
2nd: 10.0% [23.5%]
3rd: 10.4% [33.9%]
4th: 11.0% [45.0%]
5th: 11.7% [56.6%]
6th: 11.7% [68.3%]
7th: 8.8% [77.1%]
8th: 12.3% [89.4%]
9th: 9.4% [98.8%]
extra innings: 1.2% [100.0%]
The Angels are fairly consistent, except for the big bump in the first inning and the drop-off in the 7th. Unsurprisingly, they score very few extra inning runs, though some teams like the Giants score four times as many.
In a line graph, we have two possible options. The first is the line in blue, which shows the production inning by inning. The red line shows the cumulative numbers and that graph is called the ogive, pronounced "oh-jive". If they were completely consistent, the ogive would be a completely straight line, but instead we see the slight bends in the red line when run production increased and decreases per inning.
Tuesday, July 7, 2009
Notes on Bayesian probability
You can find notes on Bayesian probability in three posts from last term you can find through this link. Here are a few more practice problems, with answers in the comments.
A) A trait shows up in 20% of the population and the test has a 2% error rate. Find p(error given test positive) and p(error, given test negative).
B) A trait shows up in 10% of the population and the test has a 1% error rate. Find p(error given test positive) and p(error, given test negative).
A) A trait shows up in 20% of the population and the test has a 2% error rate. Find p(error given test positive) and p(error, given test negative).
B) A trait shows up in 10% of the population and the test has a 1% error rate. Find p(error given test positive) and p(error, given test negative).
Tuesday, May 5, 2009
Class notes for 5/4, part 3: Bayesian probability and double testing
If a trait is very rare, only a very accurate test gives us useful information. For example, if a trait shows up in only 1 in 10,000 people but the test for the trait has an error rate of 1 in 1,000, we should expect about 10 false positives for every true positive. Here is the completed table for that situation.
________don't___have____row total
test + ____9,999__999______10,998__
test - _9,989,001___1_____9,989,002__
col.___9,999,000_1,000______10,000,000 grand total
In a situation such as this, testing positive twice could give us useful information, as testing positive once has an error rate of about 90.9%. We have to assume the errors are random and not deterministic. For example, if a test for a chemical compound in opium also catches a similar compound found in poppy seed bagels, testing twice won't get rid of the errors. Assuming just random errors here is what we do.
Step 1: The top row of the first contingency table is the column total/grand total row of the second contingency table. What this does is takes the numbers from the people who tested positive the first time and makes them the totals for those who will be tested twice.
________don't___have____row total
test + __________________________
test - __________________________
col._______9,999__999_____10,998 grand total
Step 2: Multiply error rate by have column total to find the number who have that test negative. Round to the nearest whole number. (We didn't have to round before, but now we do.)
999*1/1000 = .999 ~= 1, this means test positive and have is 998.
________don't___have____row total
test + ___________998____________
test - _____________1____________
col._______9,999__999_____10,998 grand total
Step 3: Multiply error rate by don't have column total to find the errors. 9,999*1/1000 = 9.999 ~= 10. That means the test negative in that column is 9,999 - 10 = 9,989.
________don't___have____row total
test + _______10__998____________
test - _____9,989____1____________
col._______9,999__999_____10,998 grand total
Step 4: row totals
________don't___have____row total
test + _______10__998_____1,008___
test - _____9,989____1_____9,990___
col._______9,999__999_____10,998 grand total
Step 5: Find the error rate for testing positive twice. 10/1,008 = .0099... or about 1%.
Of the ten million people tested, we would send letters to 1,008 telling them they tested positive twice. Of those people, ten don't have the trait and are getting false information, but 998 are getting the right information. In the first test, there was someone with the trait who tested negative, and the same is true in the second test, so there are two people with the trait who did not get two positive test results. While this isn't a perfect situation, it's much better than the over 90% error rate we got for positive tests the first time through.
________don't___have____row total
test + ____9,999__999______10,998__
test - _9,989,001___1_____9,989,002__
col.___9,999,000_1,000______10,000,000 grand total
In a situation such as this, testing positive twice could give us useful information, as testing positive once has an error rate of about 90.9%. We have to assume the errors are random and not deterministic. For example, if a test for a chemical compound in opium also catches a similar compound found in poppy seed bagels, testing twice won't get rid of the errors. Assuming just random errors here is what we do.
Step 1: The top row of the first contingency table is the column total/grand total row of the second contingency table. What this does is takes the numbers from the people who tested positive the first time and makes them the totals for those who will be tested twice.
________don't___have____row total
test + __________________________
test - __________________________
col._______9,999__999_____10,998 grand total
Step 2: Multiply error rate by have column total to find the number who have that test negative. Round to the nearest whole number. (We didn't have to round before, but now we do.)
999*1/1000 = .999 ~= 1, this means test positive and have is 998.
________don't___have____row total
test + ___________998____________
test - _____________1____________
col._______9,999__999_____10,998 grand total
Step 3: Multiply error rate by don't have column total to find the errors. 9,999*1/1000 = 9.999 ~= 10. That means the test negative in that column is 9,999 - 10 = 9,989.
________don't___have____row total
test + _______10__998____________
test - _____9,989____1____________
col._______9,999__999_____10,998 grand total
Step 4: row totals
________don't___have____row total
test + _______10__998_____1,008___
test - _____9,989____1_____9,990___
col._______9,999__999_____10,998 grand total
Step 5: Find the error rate for testing positive twice. 10/1,008 = .0099... or about 1%.
Of the ten million people tested, we would send letters to 1,008 telling them they tested positive twice. Of those people, ten don't have the trait and are getting false information, but 998 are getting the right information. In the first test, there was someone with the trait who tested negative, and the same is true in the second test, so there are two people with the trait who did not get two positive test results. While this isn't a perfect situation, it's much better than the over 90% error rate we got for positive tests the first time through.
Class notes for 5/4, part 2: Bayesian probability
Earlier in the term, we created contingency tables from reading data sets and filling in the positions of the table, then finding the row totals, column totals and the grand total. We then learned about conditional probability, where we find that p(left, given female) might not equal p(left, given male) or p(left). If these probabilities are not equal, we call them dependent, because it depends on if we are looking a the whole population or some specific sub-population. If they are all equal, the probabilities are independent.
In Bayesian probability, we will be building a contingency table "backwards". Instead of filling in each value in the table then finding row totals, column totals and grand total, we will start with a trait in the population and a test for that trait. We will make a 2x2 contingency table, where the columns deal with having the trait on not and the rows refer to testing positive of testing negative. If the test has an error rate, as is often the case, some people are going to get incorrect information. What we will see is that the overall error rate can sometimes be quite different for the error rate for those who test positive and the error rate for those who test negative.
Let's say there is a genetic trait in the population that shows up in 25% of subjects, which we will write as 1 in 4. The test for the trait has a 2% error rate, so it is 1 in 50.
Step 1: The grand total is the product of the denominators of the fractions.
In our case, 4*50 = 200
________don't___have____row total
test + __________________________
test - __________________________
col._____________________200 grand total
Step 2: Multiply the grand total by the trait proportion to find the column totals.
Since 25% of the population has the trait, 25% of 200 = 50 subjects have the trait in our idealized sample. By subtraction, 150 don't have the trait.
________don't___have____row total
test + __________________________
test - __________________________
col._____150_____50______200 grand total
Step 3: Fill in the "have the trait" column by multiplying the error rate by the column total to fill in the mistaken position, and fill in the rest by subtracting.
In our case, the error rate is 1 in 50. This means for the people who have the trait, 1 person will test negative, while the other 49 will correctly test positive.
________don't___have____row total
test + ___________49______________
test - ____________1______________
col._____150_____50______200 grand total
Step 4: Fill in the "don't have the trait" column using the same method.
3 of the 150 will get the wrong information, which in their case will be a positive test. The other 147 will get the right information, a negative test result.
________don't___have____row total
test + ____3______49______________
test - ___147______1_______________
col._____150_____50______200 grand total
Step 5: Fill in the row totals.
________don't___have____row total
test + ____3______49______52______
test - ___147______1______148______
col._____150_____50______200 grand total
The error numbers are marked in bold and blue for the next step.
Step 6: Find the error rates given test positive and test negative.
p(error) = (3+1)/200 = 1/50 = .02, which was the advertised error rate.
p(error, given test positive) = 3/52 ~= .058, much higher than .02
p(error, given test negative) = 1/148 ~= .0068, much lower than .02
Unless the trait shows up in 50% of the population, we expect to get differences between the error rates for test positive and test negative. Whichever is the smaller part of the population should see a higher error rate. Just how significant the differences are in the error rates between the two test groups depends on the size of the error rate to the trait rate. A 99% accurate test sounds good, but if the trait is very rare, we might well get more false positives than true positives.
In Bayesian probability, we will be building a contingency table "backwards". Instead of filling in each value in the table then finding row totals, column totals and grand total, we will start with a trait in the population and a test for that trait. We will make a 2x2 contingency table, where the columns deal with having the trait on not and the rows refer to testing positive of testing negative. If the test has an error rate, as is often the case, some people are going to get incorrect information. What we will see is that the overall error rate can sometimes be quite different for the error rate for those who test positive and the error rate for those who test negative.
Let's say there is a genetic trait in the population that shows up in 25% of subjects, which we will write as 1 in 4. The test for the trait has a 2% error rate, so it is 1 in 50.
Step 1: The grand total is the product of the denominators of the fractions.
In our case, 4*50 = 200
________don't___have____row total
test + __________________________
test - __________________________
col._____________________200 grand total
Step 2: Multiply the grand total by the trait proportion to find the column totals.
Since 25% of the population has the trait, 25% of 200 = 50 subjects have the trait in our idealized sample. By subtraction, 150 don't have the trait.
________don't___have____row total
test + __________________________
test - __________________________
col._____150_____50______200 grand total
Step 3: Fill in the "have the trait" column by multiplying the error rate by the column total to fill in the mistaken position, and fill in the rest by subtracting.
In our case, the error rate is 1 in 50. This means for the people who have the trait, 1 person will test negative, while the other 49 will correctly test positive.
________don't___have____row total
test + ___________49______________
test - ____________1______________
col._____150_____50______200 grand total
Step 4: Fill in the "don't have the trait" column using the same method.
3 of the 150 will get the wrong information, which in their case will be a positive test. The other 147 will get the right information, a negative test result.
________don't___have____row total
test + ____3______49______________
test - ___147______1_______________
col._____150_____50______200 grand total
Step 5: Fill in the row totals.
________don't___have____row total
test + ____3______49______52______
test - ___147______1______148______
col._____150_____50______200 grand total
The error numbers are marked in bold and blue for the next step.
Step 6: Find the error rates given test positive and test negative.
p(error) = (3+1)/200 = 1/50 = .02, which was the advertised error rate.
p(error, given test positive) = 3/52 ~= .058, much higher than .02
p(error, given test negative) = 1/148 ~= .0068, much lower than .02
Unless the trait shows up in 50% of the population, we expect to get differences between the error rates for test positive and test negative. Whichever is the smaller part of the population should see a higher error rate. Just how significant the differences are in the error rates between the two test groups depends on the size of the error rate to the trait rate. A 99% accurate test sounds good, but if the trait is very rare, we might well get more false positives than true positives.
Monday, May 4, 2009
Practice homework for 5/4
Change the following parimutuel payoffs from modern to classic or vice versa.
a) -125
b) +144
c) 7:5
d) 4:3
Fill in the Bayesian contingency table for a trait that shows up in 1 in 50 people in the population and a test for the trait that has an error rate of 1 in 200. Find the following probabilities.
p(error, given test -)
p(error, given test +)
Answers in the comments.
a) -125
b) +144
c) 7:5
d) 4:3
Fill in the Bayesian contingency table for a trait that shows up in 1 in 50 people in the population and a test for the trait that has an error rate of 1 in 200. Find the following probabilities.
p(error, given test -)
p(error, given test +)
Answers in the comments.
Subscribe to:
Posts (Atom)

