How can I find out, how well I can measure something?
The art and importance of measurement system analysis
Abstract
At the recent ENBIS conference in Piraeus, Greece, one of the presentation tracks was around measurement uncertainty and measurement system analysis. This is a unique selling point for statistics, as most of the data we gather come from some measurement. An important part of understanding, what we actually know about a problem is, how well we can measure it. During a problem solving session we saw actually an example, where measurement error seemed very close to machining accuracy.
Table of contents
Why measurement uncertainty and "system" analysis?
In real life we all know that watches might not work exactly at the same speed, even smart watches may have visible differences in the time they are showing. Or you may have two thermometers in close proximity and still they will show different values. But also when you are measuring distances with two different types of measures, say a measurement tape and folding rule may give you different results. The tape and rule example actually introduces us to a measurement system, because neither of these will measure something without a human being operating them, and so how well they handle the measurement tool and process will very much also influence the outcome. E.g. the operator might measure along a straight edge of something, making it easy to receive mostly consistent results. But say if the object had a rounded edge and the length of that edge is required, achieving consistent results with these two measures will be very difficult.
You may now protest that using a modern laser measure will give results that are not varying if you measure twice in the same way. However, it might just be that the error becomes so small you do not care anymore, there still will be a tiny variation, due to the sensor, or maybe some dust particles passing through the laser,… And of course, here the influence of the operator becomes now the major source of uncertainty, because of the low instrument error even the tiniest movement by the operator may lead to measurable deviations.
What is the difference between precision and accuracy?
Accuracy describes, if the value coming out of our measurement tool is near the actual value. E.g. measurement tapes often have at the end a rectangular hook bolted on to them, by usage the hole for that connection is stretched so the measurement might be decreased by a millimeter or so. As this typically happens consistently in each measurement, we speak of a bias and we can simply add the millimeter back to our measurement to restore the error as a “bias correction”. So it essentially is the error in location of your measurement, i.e. in mean or median.
Precision describes how much spread of values we can expect, when measuring the same thing. It also means that if you got the same value from two different things measured, how much they could differ despite giving the same value.
How to learn about uncertainty in your measure?
If you now want to find out about the uncertainty in your measurements, you will need to do a number of measurements on the same thing, if possible. But it is important that a replicate of a measurement means to do all the steps from the beginning. This means in our tape measure example we will need to roll it up at the end of each measurement, unroll it place the end to our object we want to measure, unrole to the correct length and make the reading. This is important because each of these steps of course might introduce some variation to the end result. Important is now that this is also done by the same person, as obviously there are differences between people. If we do several such measurements and note down the values achieved, we get the Repeatability of the measurement. If we now wanted to find out, how the measurement fares in a more general setting, we will have to use different tape measures, and different people all measuring a reference piece repeatedly. Then we can determine the Reproducibility of a measuremen, i.e. the variation to be expected if we get a measurement from somewhere else done in the same way. This is extremely important for industry, as one needs to be able to know, when supplied parts actually deviate from specification to settle disputes. But also when designing a new process or product it is important to understand, how much variation is introduced just from the measurement of the specifications of supplied materials or parts.
Interesting new challenges in the digital world
At the ENBIS conference one talk was particularly interesting: “Uncertainty evaluation for calibrating mobile sensors only using short collocation periods” by Gertjan Kok, VSL. The issue described is that nowadays mobile digital sensors can deliver much more detailed information, particularly for evironmental monitoring. But there is a challenge, how should one callibrate these sensor readings to ensure comparability with fixed installed high quality sensors, as it is not possible to reguarly bring these mobile sensors for any length of time close enough to those fixed installed sensors of higher quality. The suggested solution in the presentation was to use times when those mobile sensors happened to be “close enough” to the fixed measurement stations to attempt a callibration in those short time windows. The presented work was work in progress, but it showed promising results. In a simulation study Gertjan and his colleagues had been able to show that there were sufficient collocation events to do some meaningful analysis. The next steps will be now to see if by incorporating some known influences in the models and making smart use of the time and space correlation to actually come up with a callibration method that will create the chance to monitor and therefore understand issues much better. This could then be used to drive action to give earlier warnings on dangerous concentrations of pollutants in the air.
Summary
Measurement uncertainty is everywhere in daily life and everyone is actually aware of it, but does not necessarily understand the consequences of it. But if you ignore it you may end up in massive disputes with your customers, because a product you delivered does not satisfy them, although you measured it and it seemed to fulfil the specifications. So knowing how well you measure and regularly checking there is no deterioration in your measurement system, and for some systems ensuring regular re-callibration is an economical necessity, not just a nice to have!
The example of the mobile sensor callibration shows that this area is not standing still at all and has intriguing challenges with massive real world impacts.
