Precision party: Testing gone wild
You intuitively know that a sunrise means it gets a whole lot less dark and you know that meatloaf is typically a loaf of meat. You know that a stop sign isn’t directing you to floor it (well, unless you are about half the folks that live on my street but that’s another issue…) and you just KNOW that you should never cross Taylor Swift!
But did you know that there are potentially oodles of things you encounter during your daily binder testing duties that are either not as they seem or woefully under-explained? Well, did you? As my training season winds down heading into the holidays, I’ve been perusing the material I commonly teach. This is to be sure I don’t continue saying any of the ignorant things I may have said this year, next year. I dug a little deeper into precision estimates and, WOW, was I wrong! Well, maybe not so much wrong as not as right as I’d prefer… That sounds better, right?
In the context of a short class, I stand by the way I’ve taught the concept of a precision estimate, but I also acknowledge that, as with most things, there’s always more to learn. I’ve taught that, within the laboratories of AI, we hold ourselves at least to expect better than single-operator precision (repeatability) and left it at that. But…
There’s a lot more to a precision estimate than meets the eye!
Repeatability and reproducibility aren’t the only ones in attendance at the precision party but they sure get all the attention. Heck, they are the only ones that were invited to the table and they don’t even like each other! They are the Taylor Swift and Kanye West of precision.
Though they represent conditions of the same thing, there’s a conspicuously huge gap between how they go about doing it.
We know Taylor, all perfect and precise, is there and we know Kanye, willing to do whatever it takes to convince you he’s a genius even if his only real success comes from other people’s work, is there.
But you should be introduced to some of the less extreme precision party guests. Thank goodness they came or it would be a miserable affair (At least we might get another killer Swift song or two if they hadn’t). ASTM E177 calls them the
intermediate precision conditions or any set of conditions that falls between repeatability and reproducibility. They may not get the spotlight and, in fact, do not appear at all in any of the published binder test methods, but knowing they exist may help you better understand the relative value of your results.
All of the precision conditions include four main components called ‘factor levels’-
operator, instrument, day and
site and each factor level is assigned a value of single or multiple to reflect actual testing conditions. Except in the instance of a poorly performing laboratory, repeatability has greater precision than intermediate precision, which has greater precision than reproducibility. So, introducing the guests at the precision party from most to least precise:
Repeatability Single EVERYTHING!
Multi-Operator, Single-Day-Apparatus- Multiple operators, single instrument, day and site
Single-Operator-Apparatus, Multi-Day- Single operator, instrument and site, multiple day
Within-Laboratory- Single/multiple operator/instrument, multiple day, single site
Within-Organization- Multiple operator, instrument, day and site within organization
Within-Industry Sector- Multiple operator, instrument, day and site within industry
Reproducibility- Multiple EVERYTHING!
Each precision condition is established by assessing how many ‘components of variance’ were present when two test results were generated. We start with Taylor, er, repeatability, which represents the set of conditions with the least ‘components of variance.’ We go from there all the way to you-know-who…reproducibility, which represents the precision condition with the most components of variance. The irony here is that, when you compare two results, it’s highly unlikely that either of these extremes are the actual set of conditions under which the data was generated.
So…., what to do? If we aren’t actually replicating the factor levels for the precision conditions represented in the published precision estimates, how do we use them as a tool to evaluate the value of our data? I say play it safe.
At the Asphalt Institute, it is quite possible that a replicate sample could be tested on a different DSR. It may also be tested the next day after it was determined that the replicate was necessary. That sample could easily be tested by Madison rather than Jason depending on scheduling. At this point we no longer satisfy the definition of repeatability conditions:
repeatability conditions, n-
conditions where independent test results are obtained with the same method on identical test items in the same laboratory by the same operator using the same equipment within short intervals of time.
We say SO WHAT!? Boldly hold yourself up to a higher standard in your lab and go with repeatability anyway. If your data falls within the limits of repeatability you, by default, satisfy everything that follows. Keep in mind, the precision estimates were calculated using data from many labs with varying levels of…, well, precision. This can stem from any number of things from calibration protocols to technician proficiency. This is one of those rare instances where you can go rogue in the lab and still walk away a hero.
Obviously it becomes more and more difficult to maintain very high precision as components of variance increase but it is always a valuable exercise to try to live by this one piece of advice: Be a Taylor, not a Kanye.
TECH TIP
Rolling Thin Film AWESOME
Thank you, Eric Casebolt, with Blueknight Energy Partners for this great tip! My personal tip: listen to the awesome technicians that come to your classes. Here is Eric’s tip:
Next time you are overrun by ASTM D7173 and have run out of separation tube holders, grab a Rolling Thin Film Oven (RTFO) bottle.
The tube fits perfectly and, after crimping (if you are a crimper), it hangs vertically in the bottle for the 48-hour conditioning period and freezer time.
This was one of those
“why-didn’t-I-think-of-that” moments. I have to admit that I was skeptical, but darned if it didn’t work just as advertised. This is a handy lab hack.
- Mike Beavin, Asphalt Institute Technical Training Coordinator
TECHNICIAN SPOTLIGHT
Eric Casebolt
Blueknight Energy Partners
Lab Engineer
Eric has been testing asphalt binder for over 14 years. He is responsible for managing lab technicians in other states. He enjoyed meeting other lab technicians at NBTC and sharing knowledge and experience.
"NBTC helps me to make sure I am up-to-date on all the newest test procedures and AASHTO resource requirements," said Eric. "I need to make sure I am fully aware of any changes to the tests, and how they relate to field performance."