Sensitivity of changes in chronic graft-versus-host disease activity to changes in patient-reported quality of life: results from the Chronic Graft-versus-Host Disease Consortium

Abstract

Background The 2005 National Institute of Health Chronic Graft-versus-Host Disease Consensus Conference recommended collection of patient-reported outcomes in clinical trials on chronic graft-versus-host disease. We assessed whether changes in chronic graft-versus-host disease severity, determined using National Institute of Health criteria, clinicians’ assessment or patients’ self-evaluation, correlated with patient-reported quality of life as measured by the Short Form-36 and Functional Assessment of Cancer Therapy – Bone Marrow Transplant (FACT-BMT) instruments.Design and Methods Three-hundred and thirty-six adult patients (median age 52 years; range, 19 – 79) with chronic graft-versus-host disease from six transplant centers contributed baseline and follow-up data (from 936 visits overall).Results While the majority of the patients had stable chronic graft-versus-host disease, improvement or worsening was noted in approximately 40% of follow-up visits. Multivariable analysis demonstrated no association between change in chronic graft-versus-host disease severity evaluated by National Institute of Health criteria and change in quality of life, while clinician-reported changes in severity were associated with changes in some quality of life measures. Patient-reported changes in the severity of chronic graft-versus-host disease were associated with changes in all quality of life measures. Comparison of the Short Form-36 and the FACT-BMT suggested that the data collected in the Functional Assessment of Cancer Therapy – General (FACT-G) core survey are sufficient without the need for the Short Form-36 or the FACT–BMT subscale.Conclusions We conclude that serial National Institute of Health and clinician-reported chronic graft-versus-host disease severity assessments cannot substitute for patient-reported outcomes in clinical trials. Collection of just the FACT-G instead of the Short Form-36 and the full FACT-BMT will decrease respondent burden without compromising quality of life assessment.

Introduction

Chronic graft-versus-host disease (GVHD) is an immune-mediated disorder which is associated with late hematopoietic cell transplantation (HCT)-related morbidity, mortality, prolonged duration of immune suppressive therapy, and impaired quality of life (QOL).1–4 While the anticipated trajectory following HCT for patient-reported QOL is one of recovery and return to pre-HCT functioning, previous studies have demonstrated that those affected by chronic GVHD suffer ongoing impairment in multiple domains of QOL5–14 The global severity of chronic GVHD as determined by National Institute of Health (NIH) criteria is associated with QOL, and patients with moderate to severe chronic GVHD have significant impairment across multiple domains of QOL as shown by comparing their scores with population normative scores.15 Those with moderate to severe chronic GVHD have Short Form-36 (SF-36) physical component scores (PCS) similar to those reported for patients with allied immune-mediated disorders such as systemic sclerosis, systemic lupus erythematosus, and multiple sclerosis, and greater impairment compared to patients with common chronic medical conditions.15

However, these analyses have not considered clinician-assessed or patient-reported severity of chronic GVHD, and have not characterized the relevance of dynamic changes in chronic GVHD activity on patient-reported outcomes. To address these limitations, we performed an analysis of the association between changes in chronic GVHD severity as assessed by NIH global severity criteria, clinicians, and patients with changes in patient-reported QOL in a multi-center cohort of HCT recipients affected by chronic GVHD.

Design and Methods

Patients with chronic graft-versus-host disease

A cohort of HCT recipients with chronic GVHD was prospectively assembled in a multicenter observational study. The protocol was approved by the Institutional Review Board at each of six sites (Fred Hutchinson Cancer Research Center, University of Minnesota, Dana-Farber Cancer Institute, Stanford University, Vanderbilt University, and the Medical College of Wisconsin). All subjects provided written informed consent to participation. Subjects included were allogeneic HCT recipients with chronic GVHD requiring systemic immunosuppressive therapy. Patients with classic chronic GVHD and those with overlap syndrome were eligible.16 Cases were classified as incident (enrollment less than 3 months after the diagnosis of chronic GVHD) or prevalent (enrollment 3 or more months but less than 3 years after the diagnosis of chronic GVHD). Exclusion criteria were primary disease relapse after HCT and inability to comply with study procedures. At enrollment and every 6 months thereafter (incident cases had an extra assessment 3 months after enrollment), clinicians and patients reported standardized information on chronic GVHD organ involvement, severity, and symptoms. The global severity of the chronic GVHD according to the NIH Consensus criteria was calculated from individual organ scores provided by clinicians using the NIH consensus scoring algorithm, resulting in a global grade of none, mild, moderate, or severe.16 Chronic GVHD severity was also independently assessed by both clinicians’ and patients’ report at each study visit using the qualitative categories of “none”, “mild”, “moderate” and “severe” without specific definitions. Medical charts were reviewed to collect objective medical data, laboratory and examination results, information on complications, and medication profiles.

All available longitudinal data on chronic GVHD severity and QOL were included in this analysis, from cohort inception through March, 2010, thus including both baseline enrollment data and data from all available follow-up assessments. Off study visits when a patient relapsed or died were excluded from this analysis since patient-reported information was rarely available.

Data collection forms, the ACCESS database structure, and SAS code for the NIH consensus severity algorithm are available upon request from the authors.

Quality of life instruments

The Functional Assessment of Cancer Therapy – Bone Marrow Transplant (FACT-BMT) and SF-36 questionnaires were completed by patients age 18 years or older. The FACT-BMT v4.0 is a 37-item self-report QOL questionnaire, which includes a 10-item BMT subscale. The instrument measures the effect of cancer therapy on domains including physical, functional, social/family, and emotional well-being, plus BMT-specific concerns. Individual core domain scores are summarized to give a Functional Assessment of Cancer Therapy – General (FACT-G) score based on 27 items. The FACT-G plus the BMT subscale yield a FACT-BMT score based on all 37 items. Additionally, a Functional Assessment of Cancer Therapy – Trial Outcome Index (FACT-TOI) score consists of the sum of physical and functional well-being subscales and the BMT subscale (24 items).17 We analyzed these three overlapping summary scores to see if we could identify a parsimonious battery that would capture the outcomes of interest with the minimum number of items.

The SF-36 v2 is a 36-item self-report questionnaire which assesses the following domains of QOL: physical functioning, role functioning-physical, bodily pain, general health, vitality, social functioning, role functioning-emotional, and mental health. Two summary scales from the SF-36 are the PCS and the mental component score (MCS).18–20 The PCS and MCS are normalized so that a score of 50 represents the average for the 1998 United States general population with a standard deviation of 10. In validation studies, the correlation between the SF-36v1.0 and the SF-12 was 91%.21

Statistical methods

Standard algorithms were used to compute total and subscale scores for the FACT-BMT17 and SF-36 instruments.19,20 Chronic GVHD severity was separately determined according to the three severity grading schemes (NIH consensus criteria, clinician-assessed severity, and patient-reported chronic GVHD severity) for each of the study visits.

Change in chronic GVHD severity between sequential visits was calculated for each grading scheme. The continuous change scale was also categorized into stable (0), improvement (negative values), and worsening (positive values) chronic GVHD severity, so that asymmetry between QOL sensitivity to improvement versus worsening could be detected in regression models. Agreement regarding change in severity for the three grading schemes was calculated using weighted kappa for ordinal measures with Fleiss-Cohen weights.22 Missing QOL data occurred when an office visit was completed, but the patient did not complete the QOL questionnaires at the visit, or did not return them by mail afterwards. A dependence of missing QOL data on health status could result in bias in assessments of associations between change in QOL and change in severity. We, therefore, summarized patterns for missing QOL data with respect to patients’ socio-demographic and chronic GVHD involvement and severity characteristics. For baseline characteristics, statistical comparisons between groups were made with the two-sample t-test of independence for continuous variables, and Fisher’s exact test for categorical variables. Additionally, patterns of missing data by chronic GVHD severity and organ involvement were compared using generalized estimating equations to adjust for multiple observations per patient. If observed covariates were related to missing data, including these variables as predictors in the linear mixed models could assist in implicit imputation of missing QOL data and prevent bias in assessing the relationship between change in QOL and change in chronic GVHD severity.23

Multivariable models were constructed for the primary analysis measuring the association between changes in chronic GVHD severity and changes in patient-reported QOL, using linear mixed models to account for within-patient correlation and data missing at random.23 Separate models were fitted for each of the QOL instruments/domains of interest (SF-36 PCS, SF-36 MCS, FACT-TOI, FACT-G, FACT-BMT). Patients’ socio-demographic, disease, and transplantation covariates considered in the models included the following: age, gender, education, income level, time from HCT to enrollment, disease stage, donor relation, conditioning regimen intensity, incident versus prevalent chronic GVHD, and transplant center. Effects of selected covariates were held constant when exploring relationships between change in patient-reported QOL and stability, worsening, or improvement in chronic GVHD severity.

A final, complementary analysis examined concordance between changes in each severity grading scheme and clinically meaningful change in QOL measures, using weighted kappa for ordinal measures with Fleiss-Cohen weights.22 The threshold for improvement or worsening in QOL was one-half of the population standard deviation (5 points for SF-36, or 7 points for FACT).

Statistical analyses were conducted using SAS/STAT software, version 9.2 (SAS Institute, Inc., Cary, NC, USA) and R version 2.9.2 (R Foundation for Statistical Computing, Vienna, Austria). Recognizing the multiple comparisons involved in comparing several QOL measures and pair-wise comparisons among GVHD severity levels, a type I error was controlled by considering a P value of 0.02 or lower as statistically significant, and by looking for consistency of results across related constructs.

Results

Characteristics of the cohort of patients with chronic graft-versus-host disease

This analysis was conducted on longitudinal data from 336 patients, including data from enrollment visits and 600 follow-up visits through March, 2010. The analysis, therefore, encompasses chronic GVHD severity and QOL data from a total of 936 visits. Baseline socio-demographic, disease, and transplantation characteristics, as well as baseline SF-36 and FACT-BMT scores are summarized in Table 1. The cohort included nearly equal numbers of incident (53%) and prevalent (47%) cases of chronic GVHD. Both male (60%) and female (40%) HCT recipients were well-represented, as were recipients of myeloablative (57%) and non-myeloablative (43%) transplants. Skin, mouth, eye, and lung symptoms were present in at least half of visits; genital involvement was least commonly reported, in only 13% of visits. Average SF-36 subscale scores were generally somewhat below the population norm of 50 points, with the exception of the mental health scale. The lowest average values were found for the role-functioning physical scale (39.5 points; range, 17.7 – 56.9) and the PCS (40.6 points; range 6.2 – 61.1).

A total of 142 patients had 203 visits at which SF-36 and/or FACT-BMT data were missing. Of these, 97 patients had data missing for one visit, 30 for two visits, 14 for three visits, and one for four visits. The most common scenario was lacking data for both the SF-36 and FACT-BMT for a visit, as compared to missing just one instrument, suggesting that patients failed to complete the entire patient-reported battery of instruments. Table 2 shows comparisons of patients with any missing data (n=142) to patients with no missing QOL assessments (n=194). Patients who completed all QOL assessments had a higher educational level than patients with some missing QOL data (P=0.01) but were otherwise similar in gender and case type at enrollment (incident versus prevalent, overlap versus classic chronic GVHD).

Visits for which QOL data were missing (Table 3) tended to occur at a longer time since enrollment (median 6 versus 4 months, P=0.01 for comparison of means) and after a longer interval between visits. The mean time interval between visits for incident cases was 4.2 months, while it was 6.1 months for prevalent cases. After accounting for patient case type, longer time span between visits was moderately associated with a higher chance of missing SF36 and/or FACT-BMT data (P=0.04). For example, the estimated odds of missing data for at least one instrument was 57% higher for visits occurring after an interval of more than 6 months, comparing to visits occurring after an interval from 3 – 6 months (95% CI: 2% higher – 143% higher). Organ involvement and overall chronic GVHD severity as assessed by NIH criteria, clinician assessment or patient assessment were similar (Table 3).

Change in chronic graft-versus-host disease severity on serial assessment

Change in chronic GVHD severity was examined by comparing severity at each visit to severity at the previous visit. In most cases the serial assessments showed that the severity of the patients’ chronic GVHD remained stable (NIH 63%, clinician 63%, or patient 58%). However, follow-up visits demonstrated improvement (NIH 19%, clinician 22%, patient 24%) or worsening (NIH 18%, clinician 15%, patient 18%) in others. The proportion of patients with improvement in chronic GVHD severity by any measure was highest between the first and second visits, perhaps reflecting response to treatment initiated at the time of study entry but remained close to 40% even at subsequent visits (data not shown). Change in the NIH global severity score was largely driven by change in severity in skin (Pearson’s correlation coefficient r = 0.32), eye (r = 0.30), and lung (r = 0.49) involvement. Change in severity in other affected organs was correlated to a lesser extent with change in the NIH global severity score (mouth, r=0.21; gastrointestinal tract, r=0.08; liver, r=0.14; joint, r=0.04; genital, r=0.14).

Although the prevalences of stability, improvement, and worsening were similar according to NIH criteria, clinicians’ assessment and patients’ ratings, the agreements were not strong. As shown in Figure 1, weighted kappa values for agreement between change in pairs of measures were low, ranging from 0.06 to 0.23. For example, Figure 1B shows the agreement between change in severity according to NIH criteria and according to patients’ report. Worsening according to NIH criteria and improvement according to patients’ report (upper left corner of Figure 1B) was roughly as common as worsening according to both criteria (upper right corner). These discrepancies between clinician-assessed and patient-assessed changes in GVHD severity could be associated with physical functioning level as measured by the SF-36 PCS, or with specific organ manifestations (yes/no for skin, mouth, eye, gastrointestinal tract, liver, joint, genital, or lung involvement). Generalized estimating equations were used to model agreement in the three-level classification of severity change; none of these cofactors was related to agreement of clinicians’ and patients’ assessments (P≥0.1).

Table 1.Characteristics and QOL instrument scores for the cohort of patients with chronic GVHD.

Associations between change in chronic graft-versus-host disease severity and change in quality of life

Multivariable analysis considered patients’ socio-demographic, disease, and transplantation covariates (age, gender, education, income level, time from HCT to enrollment, disease stage, donor relation, conditioning regimen intensity, incident versus prevalent chronic GVHD, and transplant center) as predictors of QOL, in addition to three levels of change in chronic GVHD severity ratings (stable, improved, worsened). Separate models were constructed for each of the QOL outcomes (SF-36 PCS, SF-36 MCS, FACT-TOI, FACT-G, and FACT-BMT). For each model, the change in chronic GVHD severity was also analyzed as a continuous predictor (to use the full range of change scores) and as a categorical predictor (to allow for different magnitudes of effects on QOL change for improvement and worsening). In univariable models, statistically significant associations were not found between any socio-demographic, disease, or transplantation covariates and change in PCS, FACT-TOI, or FACT-BMT. For change in FACT-G, a positive association with education level persisted in multiple exploratory models. Since education and months since enrollment were associated with missing data (Tables 2 and 3), multi-variable models adjusted for education level and months since enrollment for all QOL outcomes.

Table 4 shows the multivariable results, with separate models for chronic GVHD change as a continuous predictor of change in QOL (center columns), and as a categorical predictor (right-hand columns). Change in GVHD severity assessed using NIH criteria was not associated with change in any of the QOL outcomes. This conclusion held true for both the studied summary scores (SF-36 PCS, SF-36 MCS, FACT-TOI, FACT-G, and FACT-BMT), as well as detailed investigation of individual domain scores for each (data not shown). Change in clinician-reported severity of GVHD was associated with changes in QOL assessed by FACT-G and FACT-BMT. For the FACT-G, FACT-BMT and SF-36 MCS, as well as the individual domain scores which contribute to these summary scores, clinician-assessed improvement (versus stability) in chronic GVHD severity was more consistently associated with change in QOL than worsening (versus stability). Patient-reported change in severity of GVHD was sensitive to changes in all QOL outcomes, including both the summary scores, and individual domain scores, and for both improvement and worsening in chronic GVHD severity.

Figure 1.Agreement between the NIH-, clinician-, and patient-determined chronic GVHD severity changes. Jittered points were displayed to differentiate tied data in each category. Fleiss-Cohen weighted kappa was used to measure agreement. (A) NIH consensus and clinicians’ assessment (κ=0.23); (B) NIH consensus and patients’ assessment (κ=0.06). (C) Clinicians’ assessment and patients’ assessment (κ=0.14).

Table 2.Patient-level description of characteristics related to missing QOL data.

Association between change in chronic graft-versus-host disease severity and clinically meaningful change in quality of life

To complement the analysis in Table 4, we next examined the relationship between change in chronic GVHD severity according to each severity grading scheme and clinically meaningful change in QOL. We defined clinically meaningful change according to one-half of the population standard deviation (5 points for SF-36, or 7 points for FACT). An overall small proportion of the patients at each of the chronic GVHD cohort visits had clinically meaningful improvement or worsening (approximately 20% each, with the remaining stable) on follow-up across each of the examined QOL domains (SF-36 PCS, SF-36 MCS, FACT-TOI, FACT-G, and FACT-BMT, Table 5). Weighted kappa statistics were calculated to measure the degree of concordance and showed low concordance between clinically meaningful change in QOL and change in the severity of chronic GVHD. We did, however, observe a trend in which agreement was greater between clinically meaningful change in QOL and change in patient-reported chronic GVHD severity, as compared to that observed with change in chronic GVHD severity assessed either using NIH criteria or by clinicians (Table 5).

Discussion

Patient-reported QOL represents a comprehensive assessment of patients’ perceived functional ability and overall well-being, and is cited by both patients and providers as an important goal of treatment. Measurable improvements in QOL may also be recognized by the Food and Drug Administration as evidence of “clinical benefit”, a necessary component for regulatory approval.24 The collection of patient-reported data was, therefore, recommended by the NIH Consensus Conference as a valuable outcome measure in clinical practice and therapeutic trials.

We made several important findings. First, change in NIH-assessed global chronic GVHD severity over a 6-month average timeframe was not associated with change in any of the studied QOL domains. In contrast, changes in clinician-assessed chronic GVHD were associated with changes in the FACT-G and FACT-BMT scores of QOL, primarily when chronic GVHD became less severe. Change in patient-reported chronic GVHD severity, either improvement or worsening, was strongly correlated with change in QOL, demonstrating significant associations on multivariable modeling with all of the studied QOL domains of both the SF-36 and FACT-BMT. While this observation is somewhat intuitive, it confirms that if one wants to know whether patients perceive a change in the severity of their chronic GVHD or whether their QOL has changed, the only way to ascertain this information in clinical trials or in clinical practice is to ask patients directly. One cannot rely on clinician-reported chronic GVHD organ scores or clinician-reported severity of GVHD to extrapolate changes in QOL.

Table 3.Visit-level description of characteristics related to missing QOL data.

We also observed that patients demonstrated clinically relevant improvement or worsening in QOL (change greater than 0.5 times the population standard deviation) compared to that at the prior visit in approximately 40% of follow-up visits. These clinically meaningful changes showed the greatest correlation with change in patient-reported chronic GVHD severity, and little to no correlation with NIH-calculated or clinician-reported changes in severity. These results show that in approximately two of five visits, patients would report a clinically significant change in their QOL over each 6-month period, and that the only way to identify these patients would be to ask them directly.

Table 4.Results of multivariable analyses examining relationships between changes in chronic GVHD severity and changes in patient-reported QOL, holding education level and months since enrollment constant. Linear mixed models with random patient effects were fitted, with separate models for each of the QOL outcomes (SF-36 PCS and SF36-MCS; FACT-TOI, FACT-G, and FACT-BMT). Center columns describe models with chronic GVHD change as a continuous predictor of change in QOL, and right-hand columns model chronic GVHD change as a categorical predictor. Figures in bold P≤0.02.

One of the goals of this analysis was to identify the most parsimonious battery of QOL instruments or items that would capture the outcomes of interest. It appears that both the SF-36 and FACT-BMT perform well in capturing the patients’ perspective, and fairly well in capturing physicians’ assessment. Neither correlates well with changes in NIH global severity scores. The SF-36 has 36 items, the FACT-TOI has 24 items, the FACT-G has 27 items and the FACT-BMT has 37 items: an examination of Table 4 suggests that the battery could be condensed to collection of only the FACT-G without loss of discrimination capabilities. Patients complain about the redundant nature of the surveys and would probably welcome completing only 27 instead of 73 items, reducing the anticipated time commitment from 12 minutes to 4 minutes.

The strengths of this study include the prospective assessment and standardized reporting of data on chronic GVHD activity according to the NIH Consensus criteria, clinicians’ assessment, and patients’ evaluation. One of the limitations is the amount of missing data. Our analysis of the patterns of missing data showed only that more educated participants were more diligent about performing study procedures; this finding is consistent with a large body of published data demonstrating a relationship between patients’ adherence and education level.25 Missing data tended to occur as the time since enrollment increased and after a longer interval between visits. We could not identify any other patient, transplant or chronic GVHD characteristics associated with missing data.

Table 5.Frequency of clinically meaningful change in QOL and weighted kappa statistics between changes in chronic GVHD severity and clinically meaningful changes in QOL.

In summary, these data indicate that NIH and clinician-determined chronic GVHD severity assessments do not reflect the full QOL experience of people with chronic GVHD. Inclusion of patient-reported outcomes in the conduct of clinical trials and routine clinical practice provides a unique perspective that is not currently captured through any clinician-reported assessments. We recommend use of the FACT-G core survey rather than the SF-36 and the FACT-BMT subscale since the core survey would substantially decrease respondent burden without compromising understanding of multi-dimensional QOL and sensitivity to change.

Footnotes

• Presented in abstract form at the ASBMT/CIBMTR meeting, Honolulu, 2/2011
• Authorship and Disclosures The information provided by the authors about contributions from persons listed as authors and in acknowledgments is available with the full text of this paper at www.haematologica.org.
• Financial and other disclosures provided by the authors using the ICMJE (www.icmje.org) Uniform Format for Disclosure of Competing Interests are also available at www.haematologica.org.

• Received April 19, 2011.
• Revision received June 8, 2011.
• Accepted June 10, 2011.