StandbySetting
Fish Aficionado
I did it all at College in our lab, the solutions were ordered from a supplier.
Corrupt Test Kit!
- Thread starter reeeedfish
- Start date
That is good to know then.
In terms of overall accuracy of a home test kit, I still think +- 5 to 10% is going to be the best that can be expected from anything economically feasible for a hobbyist.
Coming in a powder does a few different things: 1) you won't get that sedimentation that will change a liquid reagent's concentration. 2) since they are pre-measured, you also eliminate the source of error of using 'drops' as the volume measure.
However, using a powder does have other issues. 1) Processing of a powder and mixing a powder is far, far more difficult for the manufacturer than mixing a liquid. The granules tend to self-separate. As such, I suspect that there is more variability in their finished product than the liquid tests. That is, the distribution of granules in each blister pack is more varied than the distribution of liquid in the liquid tests. Without some incredibly costly detailed quality checks, it can't not have greater variability due to the nature of solids handling versus liquid handling. And since they aren't charging much differently, they probably don't pay for a huge increase in quality checks. 2) Powders tend to stick together, especially if they've been stored in warm humid conditions (like say an LFS), so care must be taken that some of the powder doesn't spill or stick to the blister pack.
And, no matter if you add liquid or solid to the test tube, you still always get the issues of not enough or too much water in the sample, not mixing the sample correctly, and reading it off a color-match chart. All that added up, I still suspect 5-10% unless a manufacturer increases the costs dramatically.
That isn't to say that one company can't be more accurate or precise than another on the average -- but judging any individual instance it is best to remember that the error margins for any result are going to be significant.
In terms of overall accuracy of a home test kit, I still think +- 5 to 10% is going to be the best that can be expected from anything economically feasible for a hobbyist.
Coming in a powder does a few different things: 1) you won't get that sedimentation that will change a liquid reagent's concentration. 2) since they are pre-measured, you also eliminate the source of error of using 'drops' as the volume measure.
However, using a powder does have other issues. 1) Processing of a powder and mixing a powder is far, far more difficult for the manufacturer than mixing a liquid. The granules tend to self-separate. As such, I suspect that there is more variability in their finished product than the liquid tests. That is, the distribution of granules in each blister pack is more varied than the distribution of liquid in the liquid tests. Without some incredibly costly detailed quality checks, it can't not have greater variability due to the nature of solids handling versus liquid handling. And since they aren't charging much differently, they probably don't pay for a huge increase in quality checks. 2) Powders tend to stick together, especially if they've been stored in warm humid conditions (like say an LFS), so care must be taken that some of the powder doesn't spill or stick to the blister pack.
And, no matter if you add liquid or solid to the test tube, you still always get the issues of not enough or too much water in the sample, not mixing the sample correctly, and reading it off a color-match chart. All that added up, I still suspect 5-10% unless a manufacturer increases the costs dramatically.
That isn't to say that one company can't be more accurate or precise than another on the average -- but judging any individual instance it is best to remember that the error margins for any result are going to be significant.
Tests are judged on two separate attributes, precision and accuracy. The two are related, especially as the words are used colloquially, but still address two distinct testing issues.
1) Accuracy is a measure of how close the statistical test results are to the correct value.
2) Precision is how much variability there are between individual test results.
Let me explain with an example, a dart board, considering throwing a dart to be a 'test result'. Consider that every result is trying to hit the bulls eye (not always the goal when playing an individual game of darts, but consider the goal for this example)
A test ('dart thrower') with poor precision and poor accuracy would be all over the board. Double 2, in the wall behind the board, and single 11.
A test with good precision but poor accuracy would be very tightly clustered together -- say all together in the 20, but distinctly away from the bulls eye.
A test with good accuracy, but poor precision would be darts that all land in the triples ring in the 20,10, and 8 -- evenly spread around the bulls eye, so that their average position is the bulls eye, but they are not clustered together.
Finally, a test with both good accuracy and good precision would be all three darts in the bulls eye.
(see more at the following link: http
/www.butlercc.edu/engineering/en115/en115_accur_vs_prec.cfm )
You can see how these issues could affect our test results. A test with good precision put poor accuracy could report results of 0 every test (remember the results will be close to one another because it has good precision!) when there is actually some in the sample. Or, report something every time when there is nothing -- kind of like KISSfn's report above. Or a test could have good accuracy, but you need to do 10 samples and average them to achieve that accuracy.
So, when judging the quality of a test, both the accuracy and precision need to be addressed. A test that is lacking one or the other can still be useful: a test lacking precision but good accuracy can be used, it just needs to be repeated many times so that the average is well-determined. A test that lacks accuracy but with good precision can be useful if the offset is known -- it is like a rifle with a misaligned scope. You can still hit targets with it, you just have to know where to put the crosshairs when you pull the trigger.
I've never seen a study of both the accuracy and precision of the home test kits. It would be good to know actual numbers for comparison's sake. I have no doubt of what you all are saying in this thread. It is just that without measures of the accuracy and precision, it is hard to be completely objective. The repeats in triplicate SBS did are a good start to measure accuracy, but 3 samples is almost meaningless for measuring precision. I'm not saying somebody should spend their money to do this for the forum, I am just saying that I don't think that it is completely clear that one test is significantly superior to another.
1) Accuracy is a measure of how close the statistical test results are to the correct value.
2) Precision is how much variability there are between individual test results.
Let me explain with an example, a dart board, considering throwing a dart to be a 'test result'. Consider that every result is trying to hit the bulls eye (not always the goal when playing an individual game of darts, but consider the goal for this example)
A test ('dart thrower') with poor precision and poor accuracy would be all over the board. Double 2, in the wall behind the board, and single 11.
A test with good precision but poor accuracy would be very tightly clustered together -- say all together in the 20, but distinctly away from the bulls eye.
A test with good accuracy, but poor precision would be darts that all land in the triples ring in the 20,10, and 8 -- evenly spread around the bulls eye, so that their average position is the bulls eye, but they are not clustered together.
Finally, a test with both good accuracy and good precision would be all three darts in the bulls eye.
(see more at the following link: http
/www.butlercc.edu/engineering/en115/en115_accur_vs_prec.cfm )You can see how these issues could affect our test results. A test with good precision put poor accuracy could report results of 0 every test (remember the results will be close to one another because it has good precision!) when there is actually some in the sample. Or, report something every time when there is nothing -- kind of like KISSfn's report above. Or a test could have good accuracy, but you need to do 10 samples and average them to achieve that accuracy.
So, when judging the quality of a test, both the accuracy and precision need to be addressed. A test that is lacking one or the other can still be useful: a test lacking precision but good accuracy can be used, it just needs to be repeated many times so that the average is well-determined. A test that lacks accuracy but with good precision can be useful if the offset is known -- it is like a rifle with a misaligned scope. You can still hit targets with it, you just have to know where to put the crosshairs when you pull the trigger.
I've never seen a study of both the accuracy and precision of the home test kits. It would be good to know actual numbers for comparison's sake. I have no doubt of what you all are saying in this thread. It is just that without measures of the accuracy and precision, it is hard to be completely objective. The repeats in triplicate SBS did are a good start to measure accuracy, but 3 samples is almost meaningless for measuring precision. I'm not saying somebody should spend their money to do this for the forum, I am just saying that I don't think that it is completely clear that one test is significantly superior to another.
StandbySetting
Fish Aficionado
That is fair enough, I'm heading back to college this week so I can get the spread sheet with results on for you.
This Younger Spouse
Fish Fanatic
Bignose, excellent explanation. I use a rifle target shooting analogy to explain the difference in the terms.
I found this thread using the search feature because I was wondering how accurate/inaccurate the API master kits are. That's what we're using. Well, mine just showed between .25 and .50 ppm ammonia in my 55 gallon. Alarming indeed! Well, we had a second (non API but no brand name on the box or bottle) liquid ammonia test kit that we haven't used yet. Same basic thing, put tank water in test tube, add the drops, check against the color chart.
Well, the new one showed 0 ammonia. Tested it again with the API and it read around .25. Tested it again with the other and got 0 again. The API kit shows nitrites at 0 and nitrates at 10-15ppm. It's only the ammonia test that seems a bit off. None of the fish are showing any signs of high ammonia stress, either. Given the nitrite and nitrate readings, I think it's likely the other brand ammonia test is the accurate one.
And as mentioned, one MUST shake these bottles well before testing. I forgot to shake them the first time I tested this afternoon, and that threw everything off.
I found this thread using the search feature because I was wondering how accurate/inaccurate the API master kits are. That's what we're using. Well, mine just showed between .25 and .50 ppm ammonia in my 55 gallon. Alarming indeed! Well, we had a second (non API but no brand name on the box or bottle) liquid ammonia test kit that we haven't used yet. Same basic thing, put tank water in test tube, add the drops, check against the color chart.
Well, the new one showed 0 ammonia. Tested it again with the API and it read around .25. Tested it again with the other and got 0 again. The API kit shows nitrites at 0 and nitrates at 10-15ppm. It's only the ammonia test that seems a bit off. None of the fish are showing any signs of high ammonia stress, either. Given the nitrite and nitrate readings, I think it's likely the other brand ammonia test is the accurate one.
And as mentioned, one MUST shake these bottles well before testing. I forgot to shake them the first time I tested this afternoon, and that threw everything off.
Similar threads
Most reactions
-
748 -
389
-
296 -
250 -
203 -
195 -
180 -
178 -
172 -
157 -
156 -
110 -
99 -
86 -
80