Meaning of Resolution and Accuracy of Digital Display Instruments
When selecting a FLUKE clamp‑on meter, one of its technical specifications reads: Digital display: 6000‑count resolution. What does "6000‑count" mean? Also, what does the accuracy specification "1.6 % ±6 counts {40 A}" stand for?
For digital‑display instruments, the electrical quantity corresponding to one least‑significant digit on the minimum range is defined as resolution capability. For example, a certain digital multimeter is said to have a maximum resolution capability of 10 μV, which reflects the instrument's sensitivity. When resolution capability is expressed as a percentage, i.e. the ratio of the smallest displayable digit to the maximum displayable digit, it is called resolution.
The term "6000‑count" refers to the display‑resolving sensitivity of the meter, namely its ability to identify tiny electrical signals. Simply put, suppose the input signal ranges from 0 to 1 V. The instrument displays the digital value converted by the A/D converter. In other words, the 0‑1 V range is divided into 6000 parts (counts), where 0 V corresponds to 0 and 1 V corresponds to 6000. Therefore, higher resolution yields more accurate results when measuring minor‑signal variations. Take the 317 / 319 clamp‑on meters as an example. They have four digital display tubes. Since the most‑significant digit can show 5 or 6 and the minimum displayable digit is 1, these are 3‑3/4‑digit meters with a maximum displayed reading of 5999 (nominally 6000). Their resolution is 1 / 5999, approximately 0.016 %.
"1.6 % ±6 counts" denotes instrument accuracy. Note that there is a separation between "1.6 %" and "±6 counts", meaning two accuracy‑expression formats are applied. For the 40 A range: one term is 1.6 %, and the other term is ±6 counts.
