Establishing confidence from measurement comparisons

Measurement comparisons can test the compatibility expected from the claimed uncertainties. Traditionally one compares to a reference with a much (4×) smaller uncertainty, although one is not always available. From the highest-accuracy frontiers of measurement science, we report methods developed to treat cases where no undisputed reference is available, and even selecting an ad hoc reference value can be problematic. Unmediated comparisons of pairs of peer measurements can be evaluated and aggregated with rigorous variants of familiar tools: En and χ2. Monte Carlo simulation can rigorously extend these tools into regions where significant departures from the traditional analytic approximations of probabilities are expected. Furthermore, rich data sets can be aggregated to obtain straightforward statements about physically significant 'consensus invariants' (i.e. quantities that should be expected to be consistent within claimed uncertainties), and whether they have been demonstrated to be equivalent by this direct peer-to-peer comparison