An Overview On Assessing Agreement With Continuous Measurements

The limitations of the agreement approach are to determine whether the differences between the devices are, on average, sufficient to be considered clinically acceptable. This is determined by assessing the insparation of their limits of variation in the range of clinically acceptable differences. The probability of coverage (CP) proposed by Lin et al. [6] answers the same question more directly by calculating the probability that the differences between the devices themselves are at the limit of a tolerance interval – what Bland and Altman call the domain of clinically acceptable differences. Higher probabilities clearly indicate closer convergence. In practice, the researcher must decide whether the value of the CP is large enough for the two devices to be interchangeable. On the other hand, the limitations of the agreement and TDI methods have the advantage of being based on the initial unit of measurement and are compared to a clinically acceptable difference [43]. In the reviews of Barnhart et al. [11] and Barnhart [12], the authors point out that it is possible for the LoA to have 95% of the differences in the clinically acceptable difference, but not to reach an agreement (for example, if one of the limits is outside the CAD). This can happen in the event of distorted data or another error in accepting normality.

We agree that this may be a problem in the search for loA interpretation and that it is particularly important to review the assumptions in the implementation of loA. However, we believe that the ability of the methodology (and in particular the Bland-Altman plot) to highlight relative average distortions, patterns in the data and therefore sources of differences of opinion, is valuable; and that the simple calculation of a TDI or CP synthesis index can mask this detail. Therefore, if the TDI or CP is calculated, we recommend that a Bland-Altman style diagram of coupled differences between devices should also be constructed relative to the average, which shows gross average distortion and CAD, and suggest that this provides a solid way to assess match. In particular, outliers or slates in the data can be easily studied in relation to CAD. Choudhary PK. A consistent approach to non-parametric evaluation of matches in comparative method studies. Int J Biostat. 2010;6(1).