Tech Tip #5 Statistics Without Math
In preparation for the analysis of the paintball breakage tests we have to cover the subject of statistics. Many people feel that statistics are unreliable and we have all heard that "you can lie with statistics" but good statistics tells us a lot. For those who are not mathematically inclined, don't worry there are no formulas in this Tech Tip!
We will examine how to measure the accuracy of a paintball marker even though it might not be very apparent. To do this we have to start with a typical shot grouping as you can see in #1 below.
The grouping in #1 is obviously not perfect and they never are, the shots are distributed in a wide area. If you wanted to measure the accuracy of this marker you could use a circular shape around the group as we see in #2 above. We all know that if you measure the diameter of this circular shape it gives you a number that you can use for accuracy. But wait, you say that there is that one "flier" at the top, should you include that? You might say "well that must have been a bad ball so we should leave it out". If we redraw the circle as in #3 our accuracy magically goes up!! Now we have a dilemma, which shape to use and what to include but it doesn't stop there.
If we look at figures #4 and #5 below we see that the same size circular shape can be misleading.
Figure #4 is obviously a tighter more accurate group with one flier, while #5 is worse overall but still fits in the same circle. So now we understand the dilemma, how do we measure these distributions accurately? Scientists face this all the time and fortunately there is a simple way to use another shape to more accurately tell us what we have. It takes a few more steps but they are not that hard.
The first step in statistically analyzing the distribution is to use more circles, as we see in #6 below, to give us some zones to count hits in.
Next in #7 we count the hits in each zone and stack them up. If a hit touches the line we will count it as being in the next zone. In #8 we mirror over the first three columns to the right side so our shape will be easier to see. This is the same data just copied to the other side for clarity.
Now we have a nice stack of hits and can remove the circles as we have done in #9 below.
In #10 we fit our new shape to the stack, look familiar? Sure it is, it's our old friend the Bell Curve!! If you ever suffered through high school math you must have run into this guy. I remember being "graded on a curve" aughh! So now in #11 we have a new shape, the Bell Curve, which has to be fitted to the stack by adjusting it's height and width etc. Programs like Excel do this for you today and just give you the results.
So now that we have our Bell Curve shape how the heck to we measure this weird thing? See #12 below!
Lets start at the top of the Bell Curve and slide down one side. As we just come off the top, the curve gets steeper and steeper. As we get about half way down the side the curve starts to flatten out. It is at this point, where the curve transitions from getting steeper to getting shallower, that we take our measurement.
We simply measure the width of the curve at that point. This measurement is called, are you ready, a standard deviation! Get it? A "standard way" to measure deviations! Just what we were looking for, yea! Much more sophisticated than a circle, way better than guessing and we got here without math. This measurement is also called one sigma and has it's own special symbol.
So what does the measure of standard deviation tell you? It means that 68% of the shots were inside this section of the curve. The dark section of #13 represents 68% of all the area under the Bell Curve. It tells you that this marker will put about 2/3rds of its shots in a circle one standard deviation wide. This is how the military measures their accuracy, this is how polls are determined etc. It allows for fliers but doesn't give them a lot of weight.
Now we are ready to compare two shot groups and see what comes out. Look at #14 and #15 below, which one has better accuracy?
In the two examples above, #14 didn't put that many right in the center but shows a little tighter overall pattern. #15 has a bunch dead center but a wider spread. Two shooters could easily argue that each had the better accuracy than the other. Note that they both fit in the same size circle so that doesn't help.
Below we see the results of a fit to each of the shot groups above.
We see the fruits of our labor as an obvious difference between the standard deviations of the two groups. If we were to take the group and input all the x and y coordinates for each shot into a program like Excel it would just kick out the standard deviation and we would be done. Doing it the long way gives a solid understanding of how we got the measurement.
The one thing we haven't covered is the error factor. In all statistics you get a result, plus or minus X amount. This is the amount that your result could be off if you did the same test again. You wouldn't expect it to be exactly the same but you wouldn't expect it to be way off either. We won't go into all of that here but you now know how to build the curve so errors will make some sense. In order to build an accurate Bell Curve you need a BUNCH of points! If you only had say 10 shots there would be no way you could fit the curve with any accuracy. If you had 1000 shots your curve could be fit very tightly. The moral of the story here is you can't determine accuracy from 5-10 shots. This is why I wanted 40 samples for the paintball test, otherwise the error bars would be so wide they would be meaningless. Sort of like firing one shot, hitting the middle of the target and claiming your marker is perfectly accurate!!
Statistics has shown us the light and made our job easier with less argument. We have learned that a circle is not the best shape to determine accuracy, the Bell Curve part of math really will help you with fun stuff like paintball and most importantly all of this is comprehensible to most anyone. I myself only have a high school math education so I might have missed some details. If I have, some smart people here on AO will be sure to catch it and correct me. If you want to impress on your parents that paintball is more than just a game, tell them in the forum tonight we were discussing a one sigma deviation of a standard distribution based on data taken from two shot groups. Mention you are worried that your error bars were getting out of control due to a lack of input but your Bell Curve should look better after more runs. That might net you some bucks for paint!
On another note Tom Kaye's Tech Tips will be a monthly feature in Action Pursuit Magazine starting in two months. We will be extending our search for truth, justice and the American Way to the rest of the paintball world!
Tom Kaye
In preparation for the analysis of the paintball breakage tests we have to cover the subject of statistics. Many people feel that statistics are unreliable and we have all heard that "you can lie with statistics" but good statistics tells us a lot. For those who are not mathematically inclined, don't worry there are no formulas in this Tech Tip!
We will examine how to measure the accuracy of a paintball marker even though it might not be very apparent. To do this we have to start with a typical shot grouping as you can see in #1 below.
The grouping in #1 is obviously not perfect and they never are, the shots are distributed in a wide area. If you wanted to measure the accuracy of this marker you could use a circular shape around the group as we see in #2 above. We all know that if you measure the diameter of this circular shape it gives you a number that you can use for accuracy. But wait, you say that there is that one "flier" at the top, should you include that? You might say "well that must have been a bad ball so we should leave it out". If we redraw the circle as in #3 our accuracy magically goes up!! Now we have a dilemma, which shape to use and what to include but it doesn't stop there.
If we look at figures #4 and #5 below we see that the same size circular shape can be misleading.
Figure #4 is obviously a tighter more accurate group with one flier, while #5 is worse overall but still fits in the same circle. So now we understand the dilemma, how do we measure these distributions accurately? Scientists face this all the time and fortunately there is a simple way to use another shape to more accurately tell us what we have. It takes a few more steps but they are not that hard.
The first step in statistically analyzing the distribution is to use more circles, as we see in #6 below, to give us some zones to count hits in.
Next in #7 we count the hits in each zone and stack them up. If a hit touches the line we will count it as being in the next zone. In #8 we mirror over the first three columns to the right side so our shape will be easier to see. This is the same data just copied to the other side for clarity.
Now we have a nice stack of hits and can remove the circles as we have done in #9 below.
In #10 we fit our new shape to the stack, look familiar? Sure it is, it's our old friend the Bell Curve!! If you ever suffered through high school math you must have run into this guy. I remember being "graded on a curve" aughh! So now in #11 we have a new shape, the Bell Curve, which has to be fitted to the stack by adjusting it's height and width etc. Programs like Excel do this for you today and just give you the results.
So now that we have our Bell Curve shape how the heck to we measure this weird thing? See #12 below!
Lets start at the top of the Bell Curve and slide down one side. As we just come off the top, the curve gets steeper and steeper. As we get about half way down the side the curve starts to flatten out. It is at this point, where the curve transitions from getting steeper to getting shallower, that we take our measurement.
We simply measure the width of the curve at that point. This measurement is called, are you ready, a standard deviation! Get it? A "standard way" to measure deviations! Just what we were looking for, yea! Much more sophisticated than a circle, way better than guessing and we got here without math. This measurement is also called one sigma and has it's own special symbol.
So what does the measure of standard deviation tell you? It means that 68% of the shots were inside this section of the curve. The dark section of #13 represents 68% of all the area under the Bell Curve. It tells you that this marker will put about 2/3rds of its shots in a circle one standard deviation wide. This is how the military measures their accuracy, this is how polls are determined etc. It allows for fliers but doesn't give them a lot of weight.
Now we are ready to compare two shot groups and see what comes out. Look at #14 and #15 below, which one has better accuracy?
In the two examples above, #14 didn't put that many right in the center but shows a little tighter overall pattern. #15 has a bunch dead center but a wider spread. Two shooters could easily argue that each had the better accuracy than the other. Note that they both fit in the same size circle so that doesn't help.
Below we see the results of a fit to each of the shot groups above.
We see the fruits of our labor as an obvious difference between the standard deviations of the two groups. If we were to take the group and input all the x and y coordinates for each shot into a program like Excel it would just kick out the standard deviation and we would be done. Doing it the long way gives a solid understanding of how we got the measurement.
The one thing we haven't covered is the error factor. In all statistics you get a result, plus or minus X amount. This is the amount that your result could be off if you did the same test again. You wouldn't expect it to be exactly the same but you wouldn't expect it to be way off either. We won't go into all of that here but you now know how to build the curve so errors will make some sense. In order to build an accurate Bell Curve you need a BUNCH of points! If you only had say 10 shots there would be no way you could fit the curve with any accuracy. If you had 1000 shots your curve could be fit very tightly. The moral of the story here is you can't determine accuracy from 5-10 shots. This is why I wanted 40 samples for the paintball test, otherwise the error bars would be so wide they would be meaningless. Sort of like firing one shot, hitting the middle of the target and claiming your marker is perfectly accurate!!
Statistics has shown us the light and made our job easier with less argument. We have learned that a circle is not the best shape to determine accuracy, the Bell Curve part of math really will help you with fun stuff like paintball and most importantly all of this is comprehensible to most anyone. I myself only have a high school math education so I might have missed some details. If I have, some smart people here on AO will be sure to catch it and correct me. If you want to impress on your parents that paintball is more than just a game, tell them in the forum tonight we were discussing a one sigma deviation of a standard distribution based on data taken from two shot groups. Mention you are worried that your error bars were getting out of control due to a lack of input but your Bell Curve should look better after more runs. That might net you some bucks for paint!
On another note Tom Kaye's Tech Tips will be a monthly feature in Action Pursuit Magazine starting in two months. We will be extending our search for truth, justice and the American Way to the rest of the paintball world!
Tom Kaye
Comment