Outcome Measures

 

Measurement properties of patient-reported outcome measures in the settin of adult patients with distal radius fractures

 

Introduction

Assessing outcome is an inherent way of determining the comparative effectiveness of interventions. Interest in the patient’s view of their treatment has increased dramatically. Questionnaires known as PROMS which elicit information from patients are becoming the mainstay of clinical studies.

The aim of this review was to critically appraise the evidence concerning the measurement properties of questionnaires used to capture self-reported outcome in the setting of adult patients with DRFs.

 

Review Question

In the setting of adult patients with DRFs, what is the evidence for the measurement properties of questionnaires used to capture patient self-reported symptoms and musculoskeletal disability and/or function?

The review question included four key elements. (Table 10.5.1)    

Table 10.5.1 Key elements of the review question

Measurement instruments of interest	Self-reported patient questionnaires used to capture outcome in studies of adult patients with distal radius fractures.
Construct of interest	Patient self-reported symptoms and musculoskeletal disability and/or function, defined according to the developers of the instrument.
Population of interest	Adult patients with DRFs.
Measurement properties	The measurement properties are divided over three domains: reliability, validity, and responsiveness 5.

 

Methods

The development of an agreed standardised collection of outcomes is known as a Core Outcome Set (COS). This involves a rigorous process of stakeholder consensus, backed by reviews of existing knowledge. A COS for reporting outcome in clinical trials of DRF treatment does not as yet exist.

A core set of domains in the setting of adult distal radius fractures is reported by one group, the Distal Radius Working Group of the International Society for Fracture Repair (ISFR) and the International Osteoporosis Foundation (IOF) group ¹.

Consensus within our committee was that the methodology reported by Goldhanh et al. ² was not as robust as core outcome processes such as supported by initiatives like the Core Outcome Measures in Effectiveness Trials (COMET) or the Outcome Measures in Rheumatology (OMERACT) Initiatives. In addition, recommendations for the use of specific patient–reported measurement tools were made without critical appraisal of their measurement properties in the population of interest ¹.

In order to clarify the later, a systematic review was performed with the aim to critically appraise, compare and summarise the evidence on measurement properties of patient-reported questionnaires used to capture outcome in studies of adult patients with DRFs. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) Initiative provides a framework for this process ^2-4.  Goldhahn J, Beaton D, Ladd A, Macdermid J, Hoang-Kim A. Recommendation for measuring clinical outcome in DRFs: a core set of domains for standardised reporting in clinical practice and research. Arch Orthop Trauma Surg. 2014 Feb;134(2):197-205.. As part of this initiative, the COSMIN group developed a critical appraisal tool (a checklist) containing standards for evaluating the methodological quality of studies on the measurement properties of health measurement instruments (http://www.cosmin.nl/Systematic%20reviews%20of%20measurement%20properties.html). Details of the review protocol, including the quality appraisal process, are registered on PROSPERO and can be accessed at www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016029424.

Combining results of different studies on a measurement property of an instrument is only possible when the studies are sufficiently similar with regards to study population and setting, the (language) version of the questionnaire used and the form of administration. Conclusions should be drawn from studies with sufficient homogeneity.

 

Eligibility Criteria

Study inclusion criteria were as follows: 

(a) Studies concerning questionnaires aiming to measure patient self-reported symptoms and musculoskeletal disability and/or function, according to the developers of the questionnaires.

(b) Study population (or population subgroup) concerning specifically adult patients with DRFs.

(c) Aim of the study is the development of a measurement instrument or the evaluation of one or more of its measurement properties.

(d) Studies concerning the measurement properties over any of three domains: reliability, validity, and responsiveness; measurement properties as defined by the COSMIN group ⁵.

(e) Questionnaires which are self-reported.

(f) Studies published as a full text original article in the English language.

Study exclusion criteria were as follows:

(a) Studies in patient populations with other hand or wrist conditions.

(b) Studies of populations which include a subgroup of patients with DRF, but do not report specific results for the subgroup.

(c) Trials or studies evaluating the effectiveness of interventions where a questionnaire is used as an endpoint (without studying the measurement properties).

(d) Studies in which the instrument of interest is used in the validation process of another instrument.

Evidence

The search strategy returned 4667 citations.  4519 studies were excluded by title/abstract and, after removal of duplicates, 66 full-text articles were retrieved for further review. Twelve studies were included in the final review. The included studies evaluated the measurement properties in the setting of adult patients with DRFs of five PROMs, including the Patient Rated Wrist Evaluation (PRWE) ⁶, Disability of the Arm Shoulder and Hand Outcome Measure (DASH) ⁷, Patient Evaluation Measure (PEM) ⁸, Michigan Hand Questionnaire (MHQ) ⁹ and Short version of MHQ ^10. Details of included studies assessing the above PROM questionnaires in the English language are found in Table 10.5.2 below.  Further included studies assessing the measurement properties of the PROM questionnaires in languages other than English are found in (Table 12.6.1). The results of measurement properties found for English language PROMs in adults with a DRF are listed in (Table 12.6.2). The methodological quality per study and measurement property in accordance to the COSMIN framework ^2-4 is presented in (Table 12.6.3)

Table 10.5.2 Included studies assessing PROMs in the English Language

Study	PROM	Questionnaire Language	Patients	Measurement properties assessed
MacDermid et al. (1998)5	PRWE	English	Adults (n=59)	Reliability Structural Validity Criterion Validity
Waljee et al. (2011)9	Brief MHQ	English	Adults (n=132)	Responsiveness
MacDermid et al. (2000)10	PRWE DASH	English	Adults (n=64)	Responsiveness
Forward at al. (2007)11	PEM	English	Adults (n=200)	Internal Consistency Content Validity
Kotsis et al. (2007)12	MHQ	English	Adults (n=96)	Responsiveness

 

Evidence Statement

There is insufficient evidence to recommend one optimal PROM in the setting of adults patients with DRFs.  A substantial amount of information on the measurement properties of PROMs used in this setting is still lacking or has been assessed in studies of poor methodological quality according to the COSMIN framework. No PROM questionnaire has been fully evaluated in terms of its measurement properties in our population of interest; the majority of measurement properties have yet to be assessed. Most available evidence was for questionnaire languages other than English.

In the English language, there was moderate positive evidence for the reliability and responsiveness of the PRWE and for the responsiveness of the DASH. There was limited positive evidence for the responsiveness of the MHQ and the Brief MHQ. Other measurement properties were either not examined, or the level of evidence was “unknown due to poor methodological quality”. The PEM was examined only for internal consistency and content validity; the level of evidence for the PEM was “unknown due to poor methodological quality” as per the COSMIN appraisal checklist (Table 12.6.4).

Patient-reported questionnaires used for capturing outcome in the setting of adult patients with distal radius fractures should be subjected to further scrutiny. Future studies should aim to supplement existing knowledge by evaluating unknown measurement properties, utilising an appropriate quality framework, such as COSMIN, to guide study design.

 

Recommendation

Best Practice:

There is insufficient evidence to recommend the optimal PROM for capturing outcomes in studies of adult patients with DRFs. However, pending future research, an interim recommendation can be made for the use of either the PRWE or the DASH, based on available evidence for responsiveness in this setting. 

 

Further Research

A UK consensus for a core outcome set for adult patients with DRFs should be developed. This process needs to be inclusive of all relevant stakeholder groups affected by DRFs or involved in their treatment and rehabilitation and use transparent methodology defined a priori via detailed protocol. The process should be informed by systematic reviews of patient-reported and performance outcome measures.

 

Outcome Measures

Supplemental data.

Acknowledgements:

The GDG wish to acknowledge the contribution of the following medical statisticians to the outcomes chapter who provided their expertise in applying the COSMIN critical appraisal checklist to the included studies.

-Trish Hepburn,  Senior Statistician Faculty of Medicine & Health Sciences, University of Nottingham

- Archan Bhattacharya, Research Fellow Statistics/Epidemiology both from the Faculty of Medicine & Health Sciences, University of Nottingham

 

 

Table 12.6.1   Studies of PROMs in languages other than English

Study	PROM	Questionnaire Language	Patients	Measurement properties assessed
Wilcke et al. (2009)13	PRWE	Swedish	Adults (n=99)	Reliability Content Validity Structural Validity Cross-cultural Validity Criterion Validity Responsiveness
Lovgren et al. (2012)14	PRWE DASH	Swedish	Adults (n1=16, n2=16)	Internal Consistency Reliability Criterion Validity
Schonnemann et al. (2011)15	DASH	Danish	Adults (n=60)	Internal Consistency Reliability Content Validity Structural Validity Responsiveness
Schonnemann et al. (2013)16	PRWE	Danish	Adults (n=60)	Reliability Content Validity Structural Validity Cross-cultural Validity Criterion Validity Responsiveness
Hemelaers et al. (2008)17	PRWE	German	Adults (n=44)	Internal Consistency Reliability Structural Validity
Kim et al. (2013)18	PRWE	Korean	Adults (n=63)	Internal Consistency Reliability Structural Validity Cross-cultural Validity Criterion Validity Responsiveness
Mehta et al. (2012)19	PRWE	Hindi	Adults (n=50)	Internal Consistency Reliability Structural Validity Cross-cultural Validity Responsiveness

 

 

Table 12.6.2    Measurement properties for PROMs in the English language in the setting of adults with a distal radius fracture

 

Study	Study size	Measurement property
		Internal consistency	Test-retest reliability	Validity	Responsiveness
PRWE
MacDermid et al. (2000)	N=59				SRM/ Effect size =2.27/1.86 (0-3 months) =0.74/0.50 (3-6 months) =2.95/3.91 (0-6 months)
MacDermid et al. (1998)	N=64		Acute fracture:  ICC=0.90 Treated fracture: ICC=0.97
PEM
Forward et al. (2007)	N=200	Cronbach’s =0.94		vs DASH Spearman’s r=0.73
DASH
MacDermid et al. (2000)	N=59				SRM/Effect Size =2.01/1.86 (0-3 months) =0.68/0.44 (3-6 months) =2.52/2.32 (0-6 months)
MHQ
Kotsis et al. (2007)	N=96				Period 1 (3-6 months) Mean (SD) =9.2 (9) SRM= 0.8 Period 2 (6-12 months) Mean (SD) =2.2 (9.2) SRM = 0.2
Brief 12-item MHQ
Waljee et al. (2011)	N=132				SRM=0.91

 

Table 12.6.3    Methodological quality of each study per PROM and measurement property

Study	Language	Internal Consistency	Reliability	Measurement Error	Content Validity	Structural Validity	Hypotheses Testing	Cross-cultural Validity	Criterion Validity	Responsiveness
PRWE
MacDermid et al. (1998)	English		Good			Poor			Poor
MacDermid et al. (2000)	English									Good
PEM
Forward et al. (2007)	English	Poor			Good
DASH
MacDermid et al. (2000)	English									Good
MHQ
Kotsis at al. (2007)	English									Fair
Brief MHQ
Waljee at al. (2011)	English									Fair

 

 

Table 12.6.4 Levels of evidence for PROM instrument measurement properties in the English language in the setting of adults with a distal radius fracture

 

PROM instrument	Internal Consistency	Reliability	Measurement Error	Content Validity	Structural Validity	Hypotheses Testing	Cross-cultural Validity	Criterion Validity	Responsiveness
PRWE	na	++	na	na	?	na	na	?	++
PEM	?	na	na	?	na	na	na	na	na
MHQ	na	na	na	na	na	na	na	na	+
DASH	na	na	na	na	na	na	na	na	++
Brief MHQ	na	na	na	na	na	na	na	na	+

 

+++ = strong positive evidence, ++ = moderate positive evidence, + = limited positive evidence, +/- = conflicting evidence, ? = unknown due to poor methodological quality, na = no information available

1. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol. 2010 Jul;63(7):737-45.
2. Goldhahn J, Beaton D, Ladd A, Macdermid J, Hoang-Kim A. Recommendation for measuring clinical outcome in distal radius fractures: a core set of domains for standardized reporting in clinical practice and research. Arch Orthop Trauma Surg. 2014 Feb;134(2):197-205.
3. Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010 May;19(4):539-49.
4. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007 Jan;60(1):34-42.
5. MacDermid JC, Turgeon T, Richards RS, Beadle M, Roth JH. Patient rating of wrist pain and disability: a reliable and valid measurement tool. J Orthop Trauma. 1998 Nov-Dec;12(8):577-86.
6. Hudak PL, Amadio PC, Bombardier C. Development of an upper extremity outcome measure: the DASH (disabilities of the arm, shoulder and hand) [corrected]. The Upper Extremity Collaborative Group (UECG). Am J Ind Med. 1996 Jun;29(6):602-8.
7. Macey AC, Burke FD, Abbott K, Barton NJ, Bradbury E, Bradley A, et al. Outcomes of hand surgery. British Society for Surgery of the Hand. J Hand Surg Br. 1995 Dec;20(6):841-55.
8. Chung KC, Pillsbury MS, Walters MR, Hayward RA. Reliability and validity testing of the Michigan Hand Outcomes Questionnaire. J Hand Surg Am. 1998 Jul;23(4):575-87.
9. Waljee JF, Kim HM, Burns PB, Chung KC. Development of a brief, 12-item version of the Michigan Hand Questionnaire. Plast Reconstr Surg. 2011 Jul;128(1):208-20.
10. MacDermid JC, Richards RS, Donner A, Bellamy N, Roth JH. Responsiveness of the short form-36, disability of the arm, shoulder, and hand questionnaire, patient-rated wrist evaluation, and physical impairment measurements in evaluating recovery after a distal radius fracture. J Hand Surg Am. 2000 Mar;25(2):330-40.
11. Forward DP, Sithole JS, Davis TR. The internal consistency and validity of the Patient Evaluation Measure for outcomes assessment in distal radius fractures. J Hand Surg Eur Vol. 2007 Jun;32(3):262-7.
12. Kotsis SV, Lau FH, Chung KC. Responsiveness of the Michigan Hand Outcomes Questionnaire and physical measurements in outcome studies of distal radius fracture treatment. J Hand Surg Am. 2007 Jan;32(1):84-90.
13. Wilcke MT, Abbaszadegan H, Adolphson PY. Evaluation of a Swedish version of the patient-rated wrist evaluation outcome questionnaire: good responsiveness, validity, and reliability, in 99 patients recovering from a fracture of the distal radius. Scand J Plast Reconstr Surg Hand Surg. 2009;43(2):94-101.
14. Lovgren A, Hellstrom K. Reliability and validity of measurement and associations between disability and behavioural factors in patients with Colles' fracture. Physiother Theory Pract. 2012 Apr;28(3):188-97.
15. Schonnemann JO, Larsen K, Hansen TB, Soballe K. Reliability and validity of the Danish version of the disabilities of arm, shoulder, and hand questionnaire in patients with fractured wrists. J Plast Surg Hand Surg. 2011 Feb;45(1):35-9.
16. Schonnemann JO, Hansen TB, Soballe K. Translation and validation of the Danish version of the Patient Rated Wrist Evaluation questionnaire. J Plast Surg Hand Surg. 2013 Dec;47(6):489-92.
17. Hemelaers L, Angst F, Drerup S, Simmen BR, Wood-Dauphinee S. Reliability and validity of the German version of "the Patient-rated Wrist Evaluation (PRWE)" as an outcome measure of wrist pain and disability in patients with acute distal radius fractures. J Hand Ther. 2008 Oct-Dec;21(4):366-76.
18. Kim JK, Kang JS. Evaluation of the Korean version of the patient-rated wrist evaluation. J Hand Ther. 2013 Jul-Sep;26(3):238-43; quiz 44.
19. Mehta SP, Mhatre B, MacDermid JC, Mehta A. Cross-cultural adaptation and psychometric testing of the Hindi version of the patient-rated wrist evaluation. J Hand Ther. 2012 Jan-Mar;25(1):65-77; quiz 8.