This article has Open Peer Review reports available.
Reliability of radiographic measurements for acute distal radius fractures
© The Author(s). 2016
Received: 16 February 2016
Accepted: 13 July 2016
Published: 22 July 2016
The management of distal radial fractures is guided by the interpretation of radiographic findings. The aim of this investigation was to determine the intra- and inter-observer reliability of eight traditionally reported anatomic radiographic parameters in adults with an acute distal radius fracture.
Five observers participated. All were routinely involved in making treatment decisions based on distal radius fracture radiographs. Observers performed independent repeated measurements on 30 radiographs for eight anatomical parameters: dorsal shift (mm), intra-articular gap (mm), intra-articular step (mm), palmar tilt (degrees), radial angle (degrees), radial height (mm), radial shift (mm), ulnar variance (mm). Intraclass correlation coefficients (ICCs) and the magnitude of retest errors were calculated.
Measurement reliability was summarised as high (ICC > 0.80), moderate (0.60–0.80) or low (<0.60). Intra-observer reliability was high for dorsal shift and palmar tilt; moderate for radial angle, radial height, ulnar variance and radial shift; and low for intra-articular gap and step. Inter-observer reliability was high for palmar tilt; moderate for dorsal shift, ulnar variance, radial angle and radial height; and low for radial shift, intra-articular gap and step. Error magnitude (95 % confidence interval) was within 1–2 mm for intra-articular gap and step, 2–4 mm for ulnar variance, 4–6 mm for radial shift, dorsal shift and radial height, and 6–8° for radial angle and palmar tilt.
Based on previous reports of critical values for palmar tilt, ulnar variance and radial angle, error margins appear small enough for measurements to be useful in guiding treatment decisions. Our findings indicate that clinicians cannot reliably measure values ≤1 mm for intra-articular gap and step when interpreting radiographic parameters using the standardised methods investigated in this study. As a guide for treatment selection, palmar tilt, ulnar variance and radial angle measurements may be useful, but intra-articular gap and step appear unreliable.
Comprising 2.5 % of all Emergency Department (ED) presentations, fracture of the distal radius is the most common skeletal fracture type . It occurs in approximately 10 % of Caucasian women over 65 years . After cast removal, immediate functional limitations include loss of strength (particularly grip) and range of movement [3, 4]. A decade following fracture, ongoing pain and reduced function of the wrist and hand can still occur with heavy tasks .
Radiographs of the distal radius are used for diagnosis, to guide treatment choices, assess fracture reduction and monitor healing. There are no standardised, evidence-based methods for interpreting radiographic parameters. Eight anatomic parameters of distal radius fracture have been described. These are dorsal shift (mm), intra-articular gap (mm), intra-articular step (mm), palmar tilt (degrees), radial angle (degrees), radial height (mm), radial shift (mm), and ulnar variance (mm) . Relationships have been described between functional outcome and the anatomical parameters of intra-articular gap and step [7–9], dorsal [10–12] and palmar tilt , radial angle [14, 15], radial height [10, 16, 17], radial shift  and ulnar variance [19, 20]. No such relationships have been described for dorsal shift suggesting it may have no clinical utility, may not be adequately reliable and/or its close correlation with dorsal tilt  renders this measurement less important. This investigation explored the reliability of these eight parameters in preparation for a study of their utility in guiding treatment decisions.
We assessed the error associated with radiographic interpretation of acute distal radius fractures and whether the errors associated with measurements were small enough for measurements to be used confidently in fracture management. We investigated the intra- and inter-observer reliability of eight anatomic parameters in skeletally mature patients with an acute distal radius fracture using digitised radiographs. This investigation extends and updates the work of Kreder et al.  by using a larger sample of radiographs (30 acute fractures) and the computerised images and measurement procedures used in current practice. The majority of choices regarding treatment for distal radius fractures occur in the acute period. This investigation utilized radiographs of acute distal radius fractures in contrast to the healed distal fracture radiographs utilized by Kreder et al. .
Participants: selection of radiographs
Radiographic characteristics for classification as mild or severe deformity
Group A Mild deformity: must meet all criteria
Group B Severe deformity: must have at least one criteria
Intra-articular step: ≤ 2 mm
Intra-articular step: > 2 mm
Intra-articular gap: ≤ 2 mm
Intra-articular gap: > 2 mm
Dorsal tilt: ≤ 10°
Dorsal tilt: > 10°
Volar tilt: ≤ 20°
Volar tilt: > 20°
Participants: selection of assessors
To accommodate the influence of site specific practices, observers were invited from two independent health networks. Health professionals were invited to participate if their role included making treatment decisions based on radiographic images of upper limb fractures. Invitations were sent to ED, orthopaedic and radiology staff, and upper limb specialists. Potentially eligible professional groups included orthopaedic consultants and registrars, radiologists, ED consultants and registrars, and advanced practice musculoskeletal physiotherapists. A total of 10 observers (five from each health network) meeting these criteria were invited to participate. Ensuring at least two observers from each health network, the first five observers to consent were enrolled in the investigation.
The five observers were asked to measure the following parameters for each of the 30 radiographs: dorsal tilt (degrees), intra-articular gap (mm), intra-articular step (mm), palmar tilt (degrees), radial angle (degrees), radial height (mm), radial shift (mm), and ulnar variance (mm) using methods described in the observer training tutorials.
On the first measurement occasion, observers were presented, in random sequence, with 30 fully de-identified radiographs with no unique identifying features. No earlier than two weeks and no later than three weeks later, observers were presented with the same set of radiographs, again in random sequence and without access to measurements taken on the first measurement occasion. To reduce the potential impact of measurement fatigue, observers were instructed to disperse their measurements over a two week period.
Data were analysed using the recommendations by Rankin and Stokes  (correlational indices) and Bland and Altman  (metricated error estimates). The ICC quantifies the relationship between two variables with r = 1 indicating perfect agreement and r = 0 indicating no agreement.
Bland and Altman  analysis was used to quantify agreement between measurements made by the same observer; the difference between two measurements was plotted against the average of the two measurements. The 95 % confidence intervals (CIs) around the mean differences were calculated using the standard errors in estimates of the mean and a t multiplier appropriate for the sample size (2.05) .
The ICC(3,1) was chosen as each radiograph in the current investigation was rated by each of the same k observers who were the only observers of interest. As the observers for this investigation were selected from the general population, ICC(1,1) could have been used, however the more conservative equation was chosen. Bland and Altman  analysis was again applied to quantify inter-observer agreement.
As this investigation involved exploring agreement between more than two fixed observers, a representative average of the reliability between pairs within the five observers was calculated using an overall concordance correlation coefficient (OCCC) based on recommendations and equations by Barnhart et al. . The OCCC provides an overall correlation that takes into account the correlation between individual pairs of observers.
A number of investigations have recommended conservative management for distal radius fractures when intra-articular gap or step is less than 1 mm [7–9]. Intra-articular gap and step measurements were therefore converted to a dichotomy of less than or equal to 1 mm or greater than 1 mm and pairwise inter-observer agreement values (kappa) were calculated. Further pairwise inter-observer agreement values (kappa) were calculated when intra-articular gap and step measurements were converted to a dichotomy of presence (any gap or step recorded) or absence (zero gap or zero step recorded) of intra-articular step or gap. Data from the first set of measurements was used for the conversions to dichotomies.
The professional roles of the five observers who reviewed radiographs were orthopaedic surgeon (upper limb), orthopaedic registrar, ED consultant, ED primary care advanced practice musculoskeletal physiotherapist and radiologist.
Intra- and inter-observer ICCs & OCCCs for each anatomical parameter based on ANOVA output and Equations 1 and 2 are compared to data from Kreder et al. 
Inter-observer (using 1st measurements)
OCCC13 (using 1st measurements)
Kreder et al.  intra-observer
Kreder et al.  inter-observer
Range of measurements, standard error of measurement (SEM) and 95 % confidence intervals (95 % CI) for each anatomical parameter using first set of measurements of 30 radiographs, compared to data from Kreder et al.  based on six radiographs
SEM (inter-observer using 1st measurements)
Upper limit 95 % CI
Minimum, Maximum Kreder et al. 
Dorsal shift (mm)
Intra-articular Gap (mm)
Intra-articular step (mm)
Palmar tilt (degrees)
Radial angle (degrees)
Radial height (mm)
Radial shift (mm)
Ulnar variance (mm)
Kappa values for inter-observer agreement for intra-articular gap (taken from 1st recording) using raw scores and scores dichotomised with recoding
Uncoded data (as recorded)
Coding: 0 = < 1 mm
1 = equal or > 1 mm
Coding: 0 = unable to see gap
1 = able to see gap
Kappa values for inter-observer agreement for intra-articular step (taken from 1st recording) using raw scores and scores dichotomised with recoding
Uncoded data (as recorded)
Coding: 0 = < 1 mm
1 = equal or > 1 mm
Coding: 0 = unable to see step
1 = able to see step
Inter-observer ICCs and SEMs (taken from 1st recording) using data dichotomized for severity of fracture deformity
ICC All 5 observers- 30 radiographs
ICC Grp 1 (mild deformity)- 15 radiographs
ICC Grp 2 (severe deformity)- 15 radiographs
SEM Grp 1 (mild deformity)- 15 radiographs
SEM Grp 2 (severe deformity)- 15 radiographs
Stratifying data based on professional subgroups e.g. isolating the analysis to the orthopaedic surgeon and radiologist, did not result in systematically higher reliability values. The highest correlations were obtained when data for all five observers were included in analysis.
Bland and Altman
Distal radius fractures are typically managed non-operatively with cast immobilisation or surgically with either percutaneous pinning (Kirscher wires), external fixation or internal fixation . However, the evidence behind treatment choices based on deformation and radiographic parameters is limited. Best treatment for the various types of distal radius fracture would ideally include a reliable, standardised, evidence-based method of classifying distal radius fractures and unambiguous decision guidelines for treatment.
The rationale for this investigation was to quantify the intra- and inter-observer reliability of eight traditionally reported anatomic parameters in skeletally mature patients with an acute distal radius fracture using PACS computerised images and display systems (Synapse ). Bland and Altman  graphs (Figs. 2 and 3) indicate no clear relationship between an individual measurement and the magnitude of error in measurement. Consequently, in clinical practice, errors associated with these measurements are better estimated using degrees or millimetres of error (e.g. +/− 4°) than error expressed as a percentage of the range (e.g. 10 % of range).
Despite adopting standardised measurement techniques and observer training, the intra- and inter-observer consistency when applying these measures varied greatly for the eight anatomic parameters. Intra-observer ICC values appeared higher than inter-observer for all anatomic parameters except palmar tilt and may indicate the potential for additional training to remediate inconsistencies between clinicians for measurements.
Kreder et al.  published the results of intra- and inter-observer consistency in assessing these eight anatomic parameters with repeated assessments at 0 and 2–4 weeks of six radiographs of healed fractures conducted by 16 observers. Printed films were assessed on flat view boxes and measured using protractors and rulers. Limitations of the Kreder et al.  investigation were the small sample of radiographs and that radiographs were of healed fractures.
For intra-observer reliability in the current investigation, ICC values were found to be above 0.80 for palmar tilt and dorsal shift (Table 2). Only one parameter, palmar tilt, was associated with an inter-observer ICC value above 0.80. Unlike the current investigation, Kreder et al.  found ulnar variance was the only parameter to have an intra-and inter-observer ICC value above 0.80.
Comparison of error margins in millimeters or degrees (e.g. using the SEM in Table 3) is preferable to comparison of ICC values. This is because the magnitude of ICCs is affected by the range of raw scores included in the computation. This effect is referred to as attenuation of range and has the consequence that the ICC will increase as the variance in raw scores increases despite the same absolute differences (error) in repeated measurements. We were unable to determine the extent to which the higher ICCs obtained in the current investigation were a consequence of a larger range of raw scores as we did not have the variance estimates for Kreder et al.’s  data. However, on examination of the range of raw scores for the data analysed in both studies (Table 3), it is possible that attenuation of range might explain at least some of the observed differences.
It is difficult to be unequivocally confident that the use of computerised images and measurement procedures facilitates additional accuracy. If we were studying a similar spectrum of measurements, some differences in study design may account for observed differences. These include the digital methods we employed, our larger number of radiographs (30 versus 6) and that we studied acute fractures while Kreder et al.  studied healed fractures. This may have afforded us better visibility of the cortical disruption. The use of acute fracture images that mirror authentic practice confirms Kreder et al.’s  findings and extends the validity of claims regarding measurement utility across a representative spectrum of deformity.
Our classification of radiographs into Group 1 (mild deformity) and 2 (severe deformity) was undertaken to enable the spectrum of mild to severe deformity to be represented in the radiographs. While the accuracy of this step cannot be defended based on our analysis of the reliability of radiographic measurements, the range of obtained measurements in this study were comparable to the range of measurements obtained in the study by Kreder et al. , suggesting some success in capturing the spectrum of severity.
Intra-articular gap and step
A number of investigations have recommended conservative management for distal radius fractures when intra-articular gap or step is less than 1 mm; accelerated development of arthritis, increased severity of degenerative changes and poor functional outcome has been linked with intra-articular gap or step greater than 1 mm [7–9]. Intra-articular gap and step measurements in both the current investigation and previous literature (Tables 2, 4 and 5) were associated with low intra- and inter-reliability ICC values. Given the poor reliability for assessing intra-articular gap or step, we question the suitability of using these radiographic interpretations as criteria for guiding treatment choices. It is possible that additional training in measurement technique might improve the accuracy of these measurements and the cost-benefits of computerised tomography for improving reliability warrants exploration.
Dorsal and palmar tilt
It has been argued that functional outcomes are significantly affected when dorsal tilt (negative palmar tilt) exceeds 10° or 12° [10–12] or palmar tilt exceeds 25° . Allowing for error in estimates, dorsal tilt would need to be less than 2.2° to be confident that in 95 % of cases it is actually less than 10°. Error estimates indicate that palmar tilt measurement would need to be less than 17.2° to be confident that in 95 % of cases it is less than 25°.
Positive ulnar variance greater than 3 mm has been reported to negatively impact functional ability [19, 20]. Allowing for error in estimates we would need to see no more than 0.8 mm of positive ulnar variance to be confident that true ulnar variance is no more than 3 mm.
Radial angle generally reduces with displaced distal radius fractures and a radial angle of less than 15° has been used to indicate operative management [14, 15]. We would need to see more than 21.8° radial angle to be confident that in 95 % of cases we have more than 15°.
A reduction in radial height of 3–6 mm has been linked with a decline in functional outcome [16, 17]. Error estimates and a 95 % CI upper limit of 5.2 mm raise questions about the utility of this measure using the standardised methods described.
Dorsal shift and radial shift
There is limited information in the literature linking the measurements of dorsal shift and radial shift with functional outcomes raising questions around the importance of these anatomical parameters for guiding treatment decisions. The current investigation indicates that dorsal shift can be reliably measured [inter-observer ICC value (0.75)] and therefore exploration of its relationship with functional outcomes is warranted.
In summary, when interpreting computerised images of acute distal radius fractures, reliability measures and error margins from this investigation support the use of palmar tilt, radial angle and ulnar variance measurements for guiding treatment choices. However, consideration needs to be given to error margins when using these measurements to guide treatment choices. Reliability measures and error margins indicate that intra-articular gap and step cannot reliably be used to guide treatment choices for acute distal radius fractures when using the methods for interpreting radiographic parameters investigated in this study. This study did not investigate the reliability of the scan itself, and this warrants further investigation.
The next step from this investigation is to use evidence-based methods to develop decision rules for treatment guidelines following acute distal radius fracture. It is known that clinicians do not routinely measure all eight anatomical parameters in clinical practice. Further investigation is required to quantify whether there is consistency and agreement with the anatomical parameters that clinicians deem important for decision making for acute distal radius fractures.
ANOVA, analysis of variance; BMS, between-subjects variance; CIs, confidence intervals; ED, Emergency Department; ICCs, intraclass correlation coefficients; MS, mean squares; OCCC, overall concordance correlation coefficient; PA, posteroanterior; PACS, picture archiving and communication system; SEM, standard error of measurement; WMS, within subjects variance
Dr David Alexander is acknowledged for his substantial contributions to the acquisition of data.
Monash University and Western Health (Melbourne, Australia) are acknowledged for their support of this study and the staff that participated.
Availability of data and materials
The datasets supporting the conclusions of this manuscript are included within the manuscript (and in Additional file 1).
NW conceived of the study and design, coordinated the study, participated in the statistical analysis and coordinated manuscript drafting. SA made substantial contributions to the acquisition of data, was involved with drafting the manuscript and has given final approval of the version to be published. FP made substantial contributions to the acquisition of data, was involved with drafting the manuscript and has given final approval of the version to be published. JR made substantial contributions to the acquisition of data, was involved with drafting the manuscript and has given final approval of the version to be published. PT made substantial contributions to the acquisition of data, was involved with drafting the manuscript and has given final approval of the version to be published. JK made substantial contributions to conception and design of the study, participated in the analysis and interpretation of data, was involved with drafting the manuscript and has given final approval of the version to be published. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Approval was granted by the Melbourne Health, Monash University and Peninsula Health Human Research Ethics Committees involved with this investigation (MH 2010.113, CF10/2330-2010001327, HREC/10/22). Observer consent was obtained for this investigation and a waiver of consent was given for radiograph usage as part of the ethical approval.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Larsen CF, Lauritsen J. Epidemiology of acute wrist trauma. Int J Epidemiol. 1993;22(5):911–6.View ArticlePubMedGoogle Scholar
- Cummings SR, Black DM, Rubin SM. Lifetime risks of hip, Colles’, or vertebral fracture and coronary heart disease among white postmenopausal women. Arch Intern Med. 1989;149:2445–8.View ArticlePubMedGoogle Scholar
- Byl NN, Kohlhase W, Engel G. Functional limitation immediately after cast immobilization and closed reduction of distal radius fractures: Preliminary report. J Hand Ther. 1999;12(3):201–11.View ArticlePubMedGoogle Scholar
- Beaule PE, Dervin GF, Giachino AA, et al. Self-reported disability following distal radius fractures: The influence of hand dominance. J Hand Surg [Am]. 2000;25(3):476–82.View ArticleGoogle Scholar
- Foldhazy Z, Tornkvist H, Elmstedt E, et al. Long-term outcome of nonsurgically treated distal radius fractures. J Hand Surg [Am]. 2007;32(9):1374–84.View ArticleGoogle Scholar
- Kreder HJ, Handel DP, McKee M, et al. X-ray film measurements for healed distal radius fractures. J Hand Surg [Am]. 1996;21(1):31–9.View ArticleGoogle Scholar
- Knirk JL, Jupiter JB. Intra-articular fractures of the distal end of the radius in young adults. J Bone Joint Surg Am. 1986;68(5):647–59.View ArticlePubMedGoogle Scholar
- Trumble TE, Schmitt SR, Vedder NB. Factors affecting the functional outcome of displaced intra-articular distal radius fractures. J Hand Surg [Am]. 1994;19(2):325–40.View ArticleGoogle Scholar
- Dias JJ. Nonoperative treatment of distal radius fractures. In: Slutsky DJ, Osterman AL, editors. Fractures and injuries of the distal radius and carpus: The cutting egde. Philadelphia: Saunders; 2009. p. 11.View ArticleGoogle Scholar
- Solgaard S. Function after distal radius fracture. Acta Orthop Scand. 1988;59(1):39–42.View ArticlePubMedGoogle Scholar
- McQueen M, Caspers J. Colles fracture: does the anatomical results affect the final function? J Bone Joint Surg (Br). 1988;70B:649–51.Google Scholar
- Gliatis JD, Plessas SJ, Davis TR. Outcome of distal radius fracture in young adults. J Hand Surg (Br). 2000;25:535–43.View ArticleGoogle Scholar
- Abramo A, Kopylov P, Tagil M. Evaluation of treatment protocol in distal radius fractures: A prospective study in 581 patients using DASH as outcome. Acta Orthop. 2008;79(3):376–85.View ArticlePubMedGoogle Scholar
- Medoff RJ. Use of standard X-rays and classification systems in guiding treatment. In: Slutsky DJ, Osterman AL, editors. Fractures and injuries of the distal radius and carpus: The cutting edge. Philadelphia: Saunders Elsevier; 2009. p. 24.Google Scholar
- Padegimas EM, Ilyas AM. Distal radius fractures: emergency department evaluation and management. Orthop Clin North Am. 2015;46(2):259–70.View ArticlePubMedGoogle Scholar
- Lidstrom A. Fractures of the distal end of the radius. Acta Orthop Scand. 1959;41:1.View ArticleGoogle Scholar
- Abbaszadegan H, Jonsson U, von Sivers K. Prediction of instability of Codes’ fractures. Acta Orthop Scand. 1989;60(6):646–50.View ArticlePubMedGoogle Scholar
- Mackenney PJ, McQueen MM, Elton R. Prediction of instability in distal radial fractures. J Bone Joint Surg Am. 2006;88(9):1944–51.PubMedGoogle Scholar
- McQueen MM, Hajducka C, Court-Brown CM. Redisplaced unstable fractures of the distal radius: a prospective randomised comparison of four methods of treatment. J Bone Joint Surg (Br). 1996;78(3):404–9.Google Scholar
- Grewal R, MacDermid JC. The risk of adverse outcomes in extra-articular distal radius fractures is increased with malalignment in patients of all ages but mitigated in older patients. J Hand Surg [Am]. 2007;32(7):962–70.View ArticleGoogle Scholar
- van der Linden W, Ericson R. Colles’ fracture. How should its displacement be measured and how should it be immobilized? J Bone Joint Surg Am. 1981;63(8):1285–8.View ArticlePubMedGoogle Scholar
- Gravetter FJ, Wallnau LB. In: Privitera GJ. Essentials of statistics for the behavioral sciences. St. Paul: West Publishing Company. Chapter 10: Testing means; 1991.Google Scholar
- Fujifilm Synapse Picture Archiving and Communication System. USA: Fijifilm Medical Systems.Google Scholar
- Rankin G, Stokes M. Reliability of assessment tools in rehabilitation: an illustration of appropriate statistical analyses. Clin Rehabil. 1998;12:187–99.View ArticlePubMedGoogle Scholar
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;327(8476):307–10.View ArticleGoogle Scholar
- Fleiss JL. Reliability of measurement. In: Fleiss JL, editor. Design and analysis of clinical experiments. New York: John Wiley & Sons; 1986. p. 1–32.Google Scholar
- Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1976;86:420–8.View ArticleGoogle Scholar
- Barnhard HX, Haber M, Song J. Overall concordance correlation coefficient for evaluating agreement among multiple observers. Biometrics. 2002;58(4):1020–7.View ArticleGoogle Scholar
- Handoll HHG, Watts AC. Internal fixation and comparisons of different fixation methods for treating distal radial fractures in adults (Protocol). Cochrane Database Syst Rev. 2008;(1). doi: 10.1002/14651858