Precision of a measuring instrument
The smallest difference in value that a measuring instrument can reliably distinguish. It is typically taken as half the smallest division on the scale or as stated in the instrument's specification. For example, a ruler with mm markings has a precision of ±0.5 mm, while a digital balance with 0.01 g divisions has a precision of ±0.005 g.
Real World
A Vernier caliper resolves to 0.02 mm while a standard ruler resolves to 1 mm — engineers at Rolls-Royce use micrometers (0.001 mm resolution) to check jet engine turbine blades where even tiny deviations cause failure.
Exam Focus
State the resolution as the smallest scale division, and the uncertainty as half that value for analogue instruments.
How well did you know this?