Psychometric Validation of the QLQ-NMIBC24 in Low Grade, Intermediate Risk Non-Muscle Invasive Bladder Cancer

Introduction

Non–muscle-invasive bladder cancer (NMIBC) accounts for approximately 75% of newly diagnosed bladder cancer cases.^1,2 NMIBC is classified by histological grade as either low grade (LG) or high grade (HG),³ and by risk of recurrence/progression as low, intermediate, or high risk.² LG intermediate-risk (LG-IR) NMIBC includes LG tumors that do not meet the criteria for low risk, such as multiple or large (>3 cm) LG Ta tumors, or recurrent LG Ta tumors without carcinoma in situ (CIS).^4,5

LG-IR-NMIBC is typically managed with transurethral resection of the bladder tumor (TURBT)^6–8 which can be followed by a single dose of adjuvant intravesical chemotherapy, or office based biopsies and fulgurations.^9–11 Recently, UGN-102, an intravesical mitomycin-containing reverse thermal gel, has been approved as an additional chemoablative treatment option for recurrent LG-IR-NMIBC.¹²

Patient-reported outcomes (PROs) provide direct measures of patient based disease impact perspectives, including symptom burden, physical and social functioning, and overall health-related quality of life (HRQoL).^13,14 The European Organisation for Research and Treatment of Cancer (EORTC) quality of life questionnaire core 30 (QLQ-C30) is a widely used questionnaire to assess HRQoL in cancer patients across several tumor types.¹⁵ To address NMIBC-specific concerns, the EORTC developed the QLQ-NMIBC24, a complementary module tailored for NMIBC patients to be administered alongside the QLQ-C30.¹⁶

Psychometric validation of the QLQ-NMIBC24 has been conducted in UK patients with high- and intermediate-risk NMIBC,¹⁷ in the Netherlands in patients with high-risk NMIBC,¹⁸ and in Denmark and South Korea in patients with NMIBC receiving TURBT,^19,20 demonstrating its reliability and validity across populations. Psychometric properties of PRO instruments may differ in LG-IR-NMIBC compared with higher-risk populations due to differences in symptom burden, treatment and changes in health status, which could influence PROs item relevance, score distribution and responsiveness.^21,22 Furthermore, establishing minimal clinically important difference (MCID) thresholds for LG-IR-NMIBC is clinically relevant, as they provide guidance for interpreting whether observed changes in QLQ-NMIBC24 scores reflect meaningful improvements or worsening from the patient’s perspective.

However, the psychometric properties of the QLQ-NMIBC24 have not yet been specifically evaluated in patients with LG-IR-NMIBC. This study evaluates psychometric properties (validity, reliability, and responsiveness) of the EORTC QLQ-NMIBC24 in patients with LG-IR-NMIBC and proposes distribution-based MCID thresholds for this patient population.

Materials and Methods

Quality of Life Questionnaires

The EORTC QLQ-C30 is a widely used questionnaire to assess HRQoL in cancer patients across several tumor types.¹⁵ QLQ-C30 consists of 30 items covering functional domains, symptom scales, and global health status, with established scoring procedures and validated properties.^15,23,24

The QLQ-NMIBC24 was developed to address NMIBC-specific concerns and is administered alongside the QLQ-C30.¹⁶ The QLQ-NMIBC24 includes 24 items spanning 11 domains, such as urinary symptoms, malaise, intravesical treatment-related issues, sexual function, and future worries, with separate scoring for functional and symptom scales.¹⁶

In both QLQ-NMIBC24 and QLQ-C30, items are scored using a 1 (not at all) to 4 (very much) Likert scale, except the global health status/QoL in QLQ-C30, which has a 1–7 scale. Scales and single-item measures’ scores are linearly transformed to range from 0 to 100. In the QLQ-C30, higher functional scales scores indicate higher level of functioning (“better”) and in symptom scales higher scores indicate more symptoms (“worse”).²⁵ In the NMIBC24, higher scores indicate lower level of functioning (“worse”) in functional scales and more symptoms in symptom scales (“worse”).

Data

This study used data from two phase 3 clinical studies conducted in patients with LG-IR-NMIBC: ATLAS²⁶ and ENVISION.²⁷

ATLAS was a randomized, controlled, open-label phase 3 study. Participants with LG-IR-NMIBC were recruited across 72 sites in 10 countries and randomized to either (i) UGN-102 (mitomycin) for intravesical solution, administered once weekly for 6 weeks, with or without TURBT (UGN-102 was the primary treatment, and TURBT was performed in cases of residual disease at the 3 month assessment) or (ii) TURBT alone. After randomization, participants had follow-up clinic visits every three months. Assessment of patient reported outcome (PRO) measures was included as a secondary study objective. ATLAS received Institutional Review Board approval (IRB No. BL006) and was registered with ClinicalTrials.gov (NCT04688931).²⁸

ENVISION was a single arm phase 3 study. Participants with LG-IR-NMIBC were recruited across 56 locations in 10 countries. Participants received UGN-102 intravesical instillations once weekly for six weeks, followed by follow-up clinic visits every three months from the start of the treatment. Assessment of PRO measures was included as an exploratory study objective. ENVISION received Institutional Review Board approval (IRB Tracking Number: 20216408) and was registered with ClinicalTrials.gov (NCT05243550).²⁹ ENVISION included both the QLQ-C30¹⁵ and the QLQ-NMIBC24¹⁶ questionnaires, whereas ATLAS included only the QLQ-NMIBC24.¹⁶ Questionnaires were administered at baseline and at treatment and follow-up visits.

In ENVISION, missing values were handled following the EORTC scoring manual:²⁵ (i) if at least 50% of the items in a scale were answered, the scale scores were calculated according to the standard equations given on the manual (any items with missing values will be ignored); (ii) if less than 50% of the items were answered, the score was set to missing; (iii) in single-item measures the score was set to missing. In ATLAS, scale scores were only calculated if all items had non-missing answers. The difference in missing data handling reflects variations in the respective study protocols and statistical analysis plans, which were developed independently for each trial.

This study used all available data, regardless of the participants’ adherence to treatment, collected at baseline, 6 weeks (UGN-102 arms only) and three months. Although psychometric analyses in this study focused on these time points, PROs data were also collected at later follow-up visits as part of the trial protocols.^26,27 All data analyses were performed using STATA version 18.³⁰

Psychometric Evaluation Methods

Convergent Validity

Convergent validity was evaluated using polychoric correlations at two levels. First, item-item correlations were examined to assess the association between individual items within the same scale. Second, item-scale correlations were calculated using item-omitted correlations, which correlate each item with the total score of its own scale after removing that item. Correlations ≥ 0.40 were considered indicative of convergent validity, in line with previous studies.^17,18,20

Internal Consistency

Internal consistency was assessed using Cronbach’s alpha, which measures the average strength of association among all possible pairs of items within a domain.³¹ A threshold of 0.7 was used, with higher values indicating internal consistency, in line with previous studies.^17–20

Known-Groups Validity

Two groups based on the baseline score on the QLQ-C30’s physical function scale (PFS) were defined: PFS < 90 and PFS ≥ 90.¹⁷ Participants in the PFS ≥ 90 group were expected to be “healthier”, with higher scores on functional scales and lower scores on symptoms scales in the QLQ-C30, and lower scores in the NMIBC24 domains. Additionally, analysis by sex (male vs female)¹⁷ was performed to assess the absence of differences across general domains and to explore potential differences in domains more likely to be sex-related.

Differences in mean scores were examined using t-tests assuming unequal variances. Effect sizes comparing the difference in means for the two groups defined by PFS were calculated; values of 0.5 and 0.8 were considered indicative of moderate and large effects, respectively.³²

Test-Retest Reliability

Test-Retest Reliability was evaluated using QLQ-NMIBC24 scores at baseline (“test”) and week 6/month 3 (“re-test”) from participants who reported no change in the QLQ-C30 global QoL domain between time points (test and re-test). Individual intraclass correlation coefficients (ICCs) were calculated using subjects as targets and visits as raters. ICC values were interpreted based on previously published thresholds: < 0.5 indicated poor, 0.5–0.75 moderate, 0.75–0.9 good and > 0.9 excellent reliability.³³

Responsiveness to Change

To investigate whether QLQ-NMIBC24 scores reflect clinical change, analysis of covariance (ANCOVA) models were applied, using the change from baseline in QLQ-NMIBC24 domains as the dependent variable, and a clinical indicator of change (0 = no; 1 = yes) and the baseline domain score as explanatory variables. Specifically, the clinical indicator of change was Complete Response (CR; ie, having no detectable disease and having received no further treatment) at month 3.

Meaningful Change

Meaningful change from the patient’s perspective, referred to as the minimal clinically important difference (MCID) or minimal important change (MIC), can be estimated using distribution-based and anchor-based approaches.³⁴

The distribution-based approach uses estimates derived from the observed distribution of scores, including half of the standard deviation (SD) and the standard error of measurement (SEM), calculated as:

where r is the ICC obtained from test-retest reliability.³⁴

The anchor-based approach uses an external indicator, either clinical or patient-reported, to identify patients who experience minimal change (eg, “a little better” or “a little worse”) and estimates the MCID from their mean PRO change.^34,35 Anchors should assess a construct similar to that of the PRO of interest, and their correlation should be sufficient (≥0.37).^35–37 For the QLQ-NMIBC24, the QLQ-C30 global QoL domain is a suitable candidate anchor. The suitability of anchor-based MCID estimation was evaluated by examining correlations between changes in QLQ-NMIBC24 domains and the anchor QLQ-C30, and anchor-based MCID estimates were only derived when this criterion was met.

Results

Patient Characteristics at Baseline

A total of 270 patients in ATLAS and 240 in ENVISION were enrolled to receive treatment, following screening of 396 and 318 patients, respectively. Baseline demographic characteristics for patients treated in the ATLAS and enrolled in the ENVISION studies are summarized in Table 1.

In ATLAS (n = 270), the mean age was 66.5 years (SD 10.5), and most participants were male (70.4%). Almost all patients were White (98.9%), with two (0.7%) identifying as Asian and one (0.4%) with race not reported (Table 1).

In ENVISION (n = 240), the mean age was 68.8 years (SD 11.6), and the majority were male (61.3%). Most participants were White (97.5%), with two Asian (0.8%), three Black or African American (1.3%), and one with race not reported (0.4%, Table 1).

At baseline, all patients in both studies completed at least one item of the patient-reported outcome questionnaires, and most had complete datasets (96.3% in ATLAS and 93.8% in ENVISION). At week 6, the proportion of fully completed questionnaires declined in ATLAS (46.3%), reflecting the absence of a Week 6 visit for patients in the TURBT-alone arm, but remained high in ENVISION (95.4%). At month 3, completion rates were 88.9% and 89.6% in ATLAS and ENVISION, respectively (Table 1).

Table 1 Baseline Demographic Characteristics of Patients in the ATLAS and ENVISION Studies

Item Correlations and Consistency of QLQ-NMIBC24 Domains

Analysis of within-scale correlations for the QLQ-NMIBC24 questionnaire demonstrated good convergent validity across all multi-item domains in the ATLAS study cohort. Item-item correlations ranged from 0.40 to 0.96 across domains and timepoints, indicating moderate to strong associations between items within each domain (Table 2). Item-omitted correlations ranged from a minimum of 0.44 for Urinary Symptoms at baseline to a maximum of 0.96 for Sexual Function at Month 3, supporting good convergent validity (Table 2).

Table 2 Convergent Validity and Internal Consistency of QLQ-NMIBC24 Domains

The Malaise domain exhibited lower internal consistency, with Cronbach’s alpha values of 0.34 at baseline, 0.55 at Week 6, and 0.67 at Month 3, all below the commonly accepted threshold of 0.70, indicating limited reliability of this subscale in this population. Cronbach’s alpha coefficients for the other multi-items domains ranged from 0.78 to 0.90 at baseline, demonstrating good to strong internal consistency (Table 2). Similar values were observed at Week 6 and Month 3, indicating that most domains maintained stable and acceptable internal consistency over time. The only exception was the Bloating and Flatulence domain, which showed a borderline Cronbach’s alpha value of 0.69 at Month 3 (Table 2).

Differentiation Between Patient Subgroups

To assess known-groups validity, we compared QLQ-C30 and QLQ-NMIBC24 scores between patient subgroups defined by physical function scores (PFS). Patients with lower physical function scores (PFS < 90) from the ENVISION dataset reported significantly worse level of functioning and more symptoms across most QLQ-C30 scales and QLQ-NMIBC24 domains, compared to patients with higher physical function scores (PFS ≥ 90), supporting known-groups validity (p < 0.05, Table 3). As expected, the largest differences were observed for Physical Functioning (p < 0.0001, effect size = −1.63), Fatigue (p < 0.0001, effect size = 0.97), and Global Quality of Life (p < 0.0001, effect size = −0.79, Table 3).

Most scales and items were similar between males and females, except that females reported significantly more problems than males in the QLQ-NMIBC24 Future Worries (p = 0.006, effect size = 0.39), Sexual Function (p < 0.0001, effect size = 1.01) and Sexual Enjoyment (p = 0.0085, effect size = 0.73) domains, and in the QLQ-C30 Emotional Function scale (p = 0.0223, effect size = - 0.31; Table 3).

Table 3 Mean QLQ-C30 and QLQ-NMIBC24 Scores by Physical Function Category and Sex

Stability of Patient-Reported Scores Over Time

Test-retest reliability, assessed on ENVISION data via intraclass correlation coefficient (ICC) between baseline and Week 6, and between baseline and Month 3, indicated good reliability (0.75–0.9)³³ for the Urinary Symptoms and Sexual Function at both Week 6 (ICC = 0.79 for both domains) and Month 3 (ICC = 0.80 and 0.82, respectively; Table 4). The Male Sexual Problem domain showed good reliability at Week 6 (ICC = 0.78) but moderate reliability at Month 3 (ICC = 0.63).

Table 4 Test-Retest Reliability of QLQ-NMIBC24 Domains by Intraclass Correlation Coefficients (ICC) Between Band Follow-Up Visits (Week 6 and Month 3)

The Future Worries and Risk of Contaminating Partner domains demonstrated moderate reliability (0.5–0.75)³³ between baseline and both Week 6 (ICC = 0.70 and 0.61, respectively) and Month 3 (ICC = 0.58 and 0.70, respectively; Table 4).

The Malaise and Sexual Intimacy domains showed moderate reliability at Week 6 (ICC = 0.66 and 0.64, respectively) but low (ICC = 0.47) and poor reliability (ICC = −0.13) at Month 3, respectively (Table 4). In contrast, the Intravesical Treatment Issues, Bloating and Flatulence, and Sexual Enjoyment domains showed low reliability at Week 6 (ICC = 0.46, 0.45 and 0.44, respectively), but moderate reliability at Month 3 (ICC = 0.55, 0.61 and 0.71 respectively; Table 4).

Responsiveness to Change

Changes from baseline in QLQ-NMIBC24 domain scores were assessed using ANCOVA models adjusted for baseline values on the ENVISION data, comparing patients with and without complete response (CR) after 3 months of treatment with UGN-102. Adjusted coefficients and 95% confidence intervals at Week 6 and Month 3 are presented in Figure 1 and Supplementary Table 1.

Figure 1 Differences in QLQ-NMIBC24 domain scores by clinical response at month 3, at Week 6 and Month 3. Estimated differences in change from baseline between patients with and without complete response (CR) at month 3 across QLQ-NMIBC24 domains. Results from Week 6 are shown on the left panel, results from Month 3 on the right panel. Differences were estimated using ANCOVA models adjusting for baseline domain scores. Points represent the estimated coefficients, and horizontal error bars indicate 95% confidence intervals. Confidence intervals excluding zero indicate statistically significant differences in changes from baseline associated with CR.

The QLQ-NMIBC24 Urinary Symptoms (coefficient = 5.04, 95% CI: 0.61 to 9.47) and Sexual Intimacy (coefficient = −7.58, 95% CI: −15.06 to −0.10) domains demonstrated sensitivity to symptom changes at Week 6 (Figure 1, Supplementary Table 1), while the Malaise domain (coefficient = −3.81, 95% CI: −6.96 to −0.66) showed sensitivity to symptom changes at Month 3 (Figure 1, Supplementary Table 1).

Meaningful Change

The possible Minimal Clinically Important Difference (MCID) ranges for each QLQ-NMIBC24 domain at Week 6 and Month 3 are shown in Figure 2. Improvements are depicted on the negative side and worsening on the positive side. Overall, the figure highlights the variability of MCID ranges across domains and over time, showing that some domains, such as Malaise and Urinary Symptoms, show narrower consistent ranges after 6 weeks and 3 months, while others, such as Sexual Intimacy, show variable ranges across time (Figure 2). The detailed results of distribution-based metrics, including one-half SD and SEM, performed on ENVISION data are shown in Table 5. The one-half SD at baseline ranged from 4.37 for Malaise to 16.29 for Male Sexual Problems, indicating variability in baseline scores across domains. The SEM was generally comparable to or slightly higher than one-half SD, ranging from 5.13 (Malaise, Baseline–Week 6) to 19.84 (Male Sexual Problems, Baseline–Month 3).

Figure 2 Possible Minimal Clinically Important Difference (MCID) ranges for QLQ-NMIBC24 domains at Week 6 and Month 3. Bars on the negative side represent improvements, and bars on the positive side represent worsening, with shading indicating whether changes are within or beyond the MCID (Table 5). The grey area indicates the change is not clinically relevant/no change. Domains with unavailable MCID estimates (Sexual Function at week 6) are indicated by missing bars.

The correlations between the change from baseline (CFB) in QLQ-NMIBC24 domains and the CFB in the QLQ-C30 QoL domain did not reach the minimum correlation threshold, neither for CFB to Week 6 (correlations between −0.37 and −0.06) nor to Month 3 (correlations between −0.26 and 0.01). Therefore, no potential MCID could be derived from anchor-based approaches.

Table 5 Estimated Difference Distribution-Based Approach, QLQ-NMIBC24 Domains

Discussion

This study provides the first comprehensive psychometric evaluation of the EORTC QLQ-NMIBC24 in patients with LG-IR-NMIBC within global clinical trial settings, demonstrating psychometric properties largely consistent with previous reports.^17–20 The module showed good convergent validity across all the multi-item domains, confirming that items designed to measure the same construct were appropriately correlated, in line with prior findings from the UK,¹⁷ Netherlands,¹⁸ Denmark,¹⁹ and Korea.²⁰

Internal consistency was generally good, with Cronbach’s alpha values exceeding the commonly accepted threshold of 0.7^17–20 for most domains. The only exception was the Malaise domain, which demonstrated lower internal consistency. These findings are in line with previous reports, where reduced internal consistency in the Malaise domain at baseline and during follow-up visits has been observed across different cultural and clinical settings.^17,18,20 Consistent observations across studies suggests that the two items comprising the Malaise domain (“feeling unwell” and “feeling tired”) may not form a robust unidimensional scale in this population. Accordingly, the utility of the Malaise domain as a standalone scale warrants further psychometric evaluation in future research.

Known-groups comparisons showed that patients reporting lower physical functioning in the QLQ-C30 also had worse functional scores and more symptoms across QLQ-NMIBC24 domains. These results reinforce the sensitivity of the instrument to clinical differences, in line with previous validation studies.^17,20 While no sex-related differences were observed across general domains, female patients reported greater impairment in the Sexual Function and Sexual Enjoyment domains. Sex-specific variation in sexual QLQ-NMIBC24 domains has been previously observed by Blazeby et al¹⁷ and Park et al,²⁰ with the trend in Blazeby et al consistent with our findings, whereas the trend in Park et al was opposite to that observed here.

Test-retest reliability, assessed via ICCs, was generally good, particularly for the Urinary Symptoms and Sexual Function domains. However, negative or low ICCs were observed in domains with very small sample sizes, such as Sexual Intimacy at Month 3, which may be due to limited data.³⁸ This finding highlights the importance of adequately powered samples, especially in domains where missing data can be common.

Responsiveness to change was observed in specific domains of the QLQ-NMIBC24 following ANCOVA analysis. In particular, the Urinary Symptoms, Sexual Intimacy, and Malaise domains demonstrated sensitivity to change, while other domains did not exhibit significant changes. This pattern of variable domain-level responsiveness has been reported in previous validation studies of the QLQ-NMIBC24, where some domains showed significant change over time while others remained stable.^17,20 These results suggest that the QLQ-NMIBC24 can detect meaningful changes in selected domains, although responsiveness may vary across different aspects of patients’ health-related quality of life.

Anchor-based MCID estimation requires a minimum correlation (≥0.37) between the anchor and the PRO to ensure the anchor is sufficiently related to the outcome. In this study, correlations between the QLQ-C30 global QoL domain and QLQ-NMIBC24 domains did not meet this threshold. The reasons for these low correlations are unclear. One possibility is that, although the QLQ-C30 is a widely used cancer-specific HRQoL instrument, it may not fully capture disease-specific concerns in LG-IR-NMIBC. Consequently, distribution-based methods were used to estimate MCID. These estimates demonstrated variability across QLQ-NMIBC24 domains, with lower threshold values ranging from 4.37 for Malaise to 16.29 for Male Sexual Problems. While these thresholds are context-specific, a change equal to or exceeding the MCID can be considered meaningful in that domain, with the direction of the change (positive or negative) indicating improvement or worsening. The analysis of meaningful change suggests that not all domains of the QLQ-NMIBC24 are equally sensitive to detecting change. Multi-item domains tended to capture smaller and more nuanced shifts in health status, reflecting the finer granularity of their scoring. By contrast, single-item domains required larger changes before these could be interpreted as meaningful. Taken together, these findings could aid in the clinical interpretation of HRQoL in patients with LG-IR-NMIBC and enable clinicians to evaluate the impact of different treatment interventions on patients with LG-IR-NMIBC.

Recently, a new NMIBC-specific patient-reported outcome measure, the NMIBC Symptom Index (NMIBC-SI), was published.³⁹ Developed to assess patient-reported symptom burden and treatment experience,³⁹ its relationship to the QLQ-NMIBC24 is not yet established. Future research could explore whether the NMIBC-SI may have a role as a complementary instrument to existing measures, including its potential use as a source of anchor questions in NMIBC clinical trials.

Although this study provides valuable insights into the psychometric properties of the QLQ-NMIBC24 in the LG-IR-NMIBC population, it has some limitations. In the ATLAS study, participants completed only the QLQ-NMIBC24 and not the QLQ-C30, which limited the availability of patient-reported data. Score changes reflecting meaningful differences were estimated using distribution-based approaches, as the correlations between the QLQ-C30 QoL domain and the QLQ-NMIBC24 domains were insufficient to support anchor-based methods, which are typically given greater weight when available.

Patient Global Impression of Change (PGIC) data, typically used to calculate anchor-based MCID estimates,^40,41 were not collected.

This study used phase 3 clinical trial data to establish responder thresholds, which may limit the generalizability of the findings.³⁷ In addition, the study population was relatively homogeneous, consisting predominantly of white and male participants, which may further restrict applicability to the broader LG-IR-NMIBC population. Thresholds derived from this context could be influenced by trial-specific efficacy outcomes, and future research in the broader LG-IR-NMIBC population will be necessary to validate and refine these estimates.

Sample size limitations, particularly in certain subgroups and domains, may have contributed to variability in reliability estimates and to the occurrence of low ICCs. Missing responses were most frequent for sex-related items, consistent with prior validation studies from the UK,¹⁷ the Netherlands,¹⁸ and Korea.²⁰ As a consequence, missing responses may have affected the results for sex-related domains. It should be noted that some questions were only administered to participants who were sexually active during the previous four weeks. Despite this, overall completion rates were high, supporting the feasibility and acceptability of the QLQ-NMIBC24 in this trial context. However, findings may not be generalizable to other patient groups (ie, other than LG-IR) within NMIBC, as MCIDs are specific to the data used to estimate them.³⁴

Finally, no formal multiplicity adjustment was applied.

Conclusion

This study demonstrates the reliability and validity of the EORTC QLQ-NMIBC24 in patients with LG-IR-NMIBC, supporting its use as a valuable tool for assessing patient-reported outcomes in this population, both in clinical research and potentially in routine practice. The distribution-based MCID estimates generated in this analysis may further assist clinicians in interpreting HRQoL scores and evaluating the impact of treatment interventions. Moreover, these MCID estimates provide foundational thresholds for contextualizing clinical trial outcomes within health technology assessments, value frameworks, and economic models. Some limitations, including the generalizability of findings, should be considered when interpreting these results. Also, the Malaise domain showed suboptimal performance, suggesting that scoring modifications or careful interpretation is advised when using this domain in LG-IR-NMIBC populations. Future studies could build on these findings by applying anchor-based methods, although identifying suitable anchors for disease-specific instruments in lower-risk populations may be challenging. Furthermore, evaluating responsiveness in broader NMIBC populations, including multi-center, cross-cultural, or international confirmatory factor analyses would improve generalizability.