Paintball research respository

[FatMan]

After reading and posting to Deep Blue for a while, it seems to be it would be really nice to have a repository of paintball research results. Maybe this could be a classic thread with restricted postings or maybe an area in AO separate from the forums, or maybe on another site altogether.

What I'd like to see is not ramblings or pontificating, but experimental data. Results from experiments, simulations, etc. I'm not sure who all would be willing to share their results, but I know we have HEARD about a lot of tests. It would be really nice if these were listed in one place where we could find them and read and reread them as we wanted to.

For example:
- there's the bounce test data that AGD and others have produced.
- there's some accuracy tests posted on that we could post a link to
- there are all sorts of mythical tests AGD has run and talked about, but maybe we could get them to record them in a little more detail

From time to time us Deep Bluers are inspired to do more of this. Maybe such a repository would prompt more people to do that and post it.

What do you guys think?

FatMan

[bjjb99]

I like the idea. In addition to results, one could post testing procedures that were used so other folks could repeat the experiments (or modify them for different markers, paint, etc.). I also see a section on the actual design of experiments... folks collaborating to develop rigorous testing procedures following scientific methodologies.

Now that I've used a bunch of big words, I'm gonna take a nap.

BJJB

[AGD]

Not a bad idea. We should post the data in two places. One in the thread it's being discussed and a second in an independent thread that I can lock. As time goes on I can merge all the data threads into one.

AGD

[FatMan]

That's the idea!

Originally posted by AGD
Not a bad idea. We should post the data in two places. One in the thread it's being discussed and a second in an independent thread that I can lock. As time goes on I can merge all the data threads into one.

AGD

Yeah, that's what I had in mind. Folks can post their data in Deep Blue as a regular thread, and AGD can move selected items to a locked thread for posterity - only the data any any peritent updates, not all the discussion.

Also, as bjjb99 pointed out, good postings should have their testing procedures spelled out as well as both raw and cooked data.

FatMan

[FatMan]

An example

As an example, I've been reading the thread about barrel bore size. I'm thinking about doing the following experiment:

Mount my Mag in a bench mount (cphillip has one that should do). Select three different paints: NelSplat (kindof big), Powerball (kindof medium), and Proball Platinum (kindof small). For each paint I do the following:

1) bounce test.
2) see which insert from my Freak set seems to be the best. - have cphillip do the same (independently)
3) starting with the smallest Freak insert, fire 10 rounds at a clean target. Measure velocity of each shot. Afterwards, measure the center of the group and record the distance of each shot from the center.
4) repeat with all 8 inserts.
5) if I'm not bored silly by then, repeat the whole thing with the different length barrel tip (between Phil and I we have a 10" and 12").

Two tips times 8 inserts times 3 paints times 10 rounds is 480 data points.

Probably the biggest problem will be breaks. Of course I want to record breaks, but then I need to get the barrel back to clean before I continue. If I'm willing to assume two Freaks backs are the same (probably OK since the inserts control the ID) I should be able to remove a dirty one and clean it while using the other. I also want to be sure to fire a few rounds through a recently cleaned barrel to "age" the cleanliness.

Then we get to cook the data and see what we get. I'm thinking average velocity, standard deviation of velocity, group size, average distance from center, standard deviation of distance from center, number of breaks.

Boy, will I be disappointed if it all comes back that it makes no difference at all! I mean, after gathering all that data I'd like to see SOMETHING!

What do you think? Am I missing something? Anyone out there want to repeat the experiment on different markers (Angle, Cocker, etc) or with a different barrel system (like a set of Boomies)?

Once thing I'm unclear about. I plan to leave the pressure set the same for all tests - chrono'd close to 300fps. One thing I'd like to see is how the barrel afects velocity - and thus efficiency. But is that fair in assessing accuracy? Should I do two sets of tests - one for velocity where I leave the pressure set and one for accuracy where I re-chrono to 300fps for each test? Makes sense, but its a lot more work!

Comments?

FatMan

[than205]

I would think you would be more interested in the data aquired from the maintained fps than preset fps. While any data acquired regarding bore size changes are interesting, if you don't adjust (correct) the velocity you won't be getting any real world data. Meaning, the average player whether they have five barrels or one barrel will adjust velocity as needed. Besides what barrel would you use for your benchmark? (crown point? hehehe)

My concern for the repository would setting guidelines for the non-scientific to contribute. Not everyone has a grasp of the scientific process.

[AGD]

Fot this to make sense you would have to publish your protocall first and have it approved by the group. For instance firing 10 shots from each size is not statistically significant, you need about 50 or more. You should also process your data in statistical format and give a confidence value in how different two groups of velocities are .

AGD

[Redkey]

testing

It would probably be a good idea to have some type of a peer review process for the results before they are published... or better yet, before the tests are run.

I'd hate to be the guy who shoots 500 shots recording every detail about them only to find out he missed some important issue that makes his results meaningless.

How many people would actually be able to perform testing like this in a controlled and repeatable manner? From what I've read AGD has a test setup and I guess warpig has something going as well.

As for Fatmans tests...
I would also include the weight of the individual paintballs

You should probably establish a standardized method of cleaning the barrel.

photos of the test setup would be a good idea too. Actually, if you take photos of the paint marks on the target, image analysis can be used to measure the relative locations of the shots to each other and the center of the target.

How will you control the aimpoint of the marker? If you're going to include the average distance from center you'll need to make sure the gun is aimed the exact same location every time.

bjjb99... are you talking about designing experimental procedures or design of experiments (DOE) which is a statistical method of looking for interactions between variables without actually running all the tests to examine every possible combination of inputs.

Anyhow... I'm interested in this and would be willing to help people with their test procedures and statistical analysis work.

[bjjb99]

FatMan wrote:
>
> What do you think? Am I missing
> something? Anyone out there want to
> repeat the experiment on different
> markers (Angle, Cocker, etc) or with a
> different barrel system (like a set of
> Boomies)?

I agree with Tom regarding the number of shots fired. You really want much more than ten per insert. We use 32 elements as the bare minimum for statistical sample sets; usually the number is a hundred or more.

I assumed you were planning on adjusting the marker's velocity so that each insert chronographed the same, at least for the accuracy tests. For consistancy testing, you may not want to adjust the velocity at all... just set the mean velocity at 300 fps (or 290, 280, whatever) for the insert that best fits the paint and see how that velocity changes when different diameter inserts are used.

You will also want to measure the meteorlogical conditions during the test. A few data points (before, twice during, and at the end) should be enough for temperature, relative humidity, and barometric pressure. More would be needed for wind speed and direction. Of course, the ideal situation would be testing indoors.

For each insert, you could do the talcum powder visualization test that I mentioned in the barrel diameter thread; lightly coat the inside of the Freak insert with powder and fire a paintball through it... then examine the streaks left in the powder where the ball contacted the barrel. You can photograph a few for each insert diameter to determine if the ball behaves differently as a function of ball-barrel match.


Redkey wrote:
>
> bjjb99... are you talking about
> designing experimental procedures or
> design of experiments (DOE) which is a
> statistical method of looking for
> interactions between variables without
> actually running all the tests to
> examine every possible combination of
> inputs.

I was thinking more along the lines of the actual designing of experimental procedures. I haven't done much, if any, DoE work, though I can see where it would really come in handy when the number of experimental variables becomes large.

BJJB

[FatMan]

Thanks for the comments!

Great comments all!

I think we should try to give feedback to potential experimenters - just like you guys just gave to me. There would be extra incentive to repeat experiments with the same setup. At the same time, this isn't going to be the most scientific testing in the world. I mean, for the most part we're a bunch of paintball guys who are trying to put some reality into some of the myths of paintball. We don't all have perfect lab facillities or a budget for 5000 rounds of test paint.

I guess what I'm saying is probably any data can be criticized to some extent and we should not be TOO hard nosed about it. To the extent we feel some data is useful we should include it - while pushing for better testing procedures. It seems that right now we have nothing. If we have a bunch of data, then there's no point in adding more unless the data is somehow better (by virtue of the procedure or statistical method).

I think we should encourage folks to propose their procedure first, and then present their data, and then based on response decide what should be archived. Rather than make a bunch or rules, let's see what people come up with.

Anyway, as for specific comments:

I agree on the number of shots thing. I hadn't thought about numbers so much yet.

I like the standard cleaning procedure (I was hinting around at that) and weighing the balls seems a reasonable thing. I was thinking of photos also.

As for the aimpoint - I was planning to compute the center of the group for each test and then measure deviation from the center - I'm more interested in consistency than absolute aim. That was also why I wasn't so concerned with velocity, but if velocity varies too much it could have an unfair effect on apparent consistency.

Environmental conditions I was planning to record also. I don't have an indoor range, but I might be able to put in some wind breaks to reduce the effect of wind gusts.

As for the powder stuff, I think that's cool, but it would be a real pain to do THAT several hundred times - so I'm thinking I'll leave that for someone else to do. I guess if everyone things we MUST do that we could consider it ...

Wow, this is really cool. Now I'm really fired up to do some of this. Not sure how quickly I can get everything together to get started, but as soon as I can ...

Now let's see who ELSE wants to get some experiments going. Bjjb, weren't you going to be doing some stuff?

FatMan

[FatMan]

Proposal

So, I'd like to propose that a copy of AGD's pressure curve graph at:

http://www.automags.org/forums/showthread.php?s=&threadid=18939&pagenumber=2

Be added to the research repository. I'd also like to consider the posting AGD made at:

http://www.automags.org/forums/showthread.php?s=&threadid=34109

While its not lab data, it IS a report of a well controlled study relative to the discussions here.

Any opinions on the matter?

FatMan

[FatMan]

Barrel ID experiments

First, thanks AGD for posting the Official Data thread!

Next, I've been getting prepared to run the experiments I've proposed. I think I have everything covered except the ball weight issue. I'm having trouble finding a scale that will weigh something that light with reasonable precision. I'm sure they exist around here, but getting someone to let me borrow one is another issue.

How valuable would the proposed data be without that? Obviously we would not be able to correct for variations in weight. Does anyone have actual data that shows that the weight variation is significant? What do you all think?

As for recording the shot data, I'm thinking of using a large sheet of paper with a 1 inch grid and recording the coordinates of each shot. From there I can calculate the centroid of the group, and then the distance from the centroid for each shot. Any thoughts?

50 shots to one sheet of paper would probably result in a mess. I was thinking of doing groups of 10 or 15 or so. I can probably get each sheet of paper mounted pretty close to the same way each time - or I can only calculate the centroids for each subgroup. What do you think? Both approaches will have some error, but which would you think is worse?

Finally, on velocity. Everyone seems to agree I need to adjust the velocity each time I change the barrel. So the question is - what procedure for doing that. As we all know, shot-to-shot velocity varies - so how do I determine that I have returned it to the "same" velocity each time? This might be further compounded by the fact that shot-to-shot variation may be different with different barrel inserts. Ideas would be appreciated.

I'm hoping to get started this weekend. I will probably only get to do one paint this weekend, but that should be a good start. I have pretty much everything, except the scale, ready to go.

One last issue. The only chrono I have is an optical. I know those aren't as accurate as a radar model, but I just don't have access to one of those. Will this compromise the experiment? Thoughts?

FatMan

[bjjb99]

Rather than using paper for your target (it will turn into a soggy mess and will likely be perforated by the paintball impacts), you might want to consider some sort of plastic sheet. If you don't mind plunking down some dollars, you can get a 4x4 foot piece of 1/8" thick transparent polycarbonate for around $50 plus shipping. Paint or otherwise mark lines on the backside of the sheet so that they form 1" squares, hang it against a white-painted plywood back, and you've got a target that you can wash/wipe off after each test.

If the price is a bit steep, you can go with polyethylene sheet. It's a bit less rigid and it's usually not nearly as transparent, but it costs about half as much as polycarbonate for the same size piece. You could also probably get away with a 2x2 foot section instead of a 4x4, depending on the target distance you want to use for testing.

Plastic sheet is sold in various places online. McMaster Carr is always a good place to start looking. Just put "plastic sheet" into their search engine and away you go. Here's their URL:

http://www.mcmaster.com/

I would suggest adjusting the velocity for each insert until the mean of several shots matches the desired value, give or take one standard deviation? Then perform the centroid test and measurement?

BJJB

[bjjb99]

Measuring paintball mass and velocity

Probably your best bet (aside from spending more dollars than you want to) would be a college chemistry department. See if you can borrow one of their triple beam balances, which generally have 0.1 gram sensitivity (that's about 3 percent of an average paintball's mass). If you want more precision than that, you'll probably have to go to an electronic toploading balance. You may have to perform your paintball weighings in a chem lab and then take the paint to your firing range. Just make sure the paint doesn't soak up a lot of humidity during transport, and you should be ok.

Regarding the chronograph... I think for a first run of the experiment an optical chronograph is a reasonable choice. After working out some of the kinks in the testing it may become necessary to go to something more precise, but not yet.

Be sure to note the make and model of your test equipment (what type of scale for weighing paint, the type of chronograph, etc.) so other folks can repeat the test using hardware built to identical specs if they wish to do so.

BJJB

[FatMan]

Thanks.

Hmm, if a triple beam will do the trick, I can probably swing that. i was worried I'd need something more sensitive. Now who do I know in chemistry ...?

I like the polycarb sheet idea. A little pricey. Maybe I can work up something that will do just as good though. Let me think about that. There is likely to be something like that lying around I can scrounge.

Thanks for the ideas,

FatMan