Development and Internal Validation of a Prediction Model for Major Cardiovascular and Respiratory Events in Chronic Obstructive Pulmonary Disease: Nationwide Primary Care Electronic Medical Records Cohort Study

Introduction

Chronic obstructive pulmonary disease (COPD) is a prevalent public health problem, affecting an estimated 10.3% of people aged 30–79 years globally in 2019 which translated to 392 million patients with COPD.¹ Across Europe in 2020 alone, COPD incurred estimated direct costs of between €1963 and €10,701 per patient-year.² Notably, COPD is not a single-system disease and is associated with multiple comorbidities, with cardiovascular disease (CVD) being one of the most prevalent. Compared to those without COPD, patients with COPD have a higher prevalence of CVDs, including ischaemic heart disease, arrhythmia, heart failure (HF) and stroke,^3–5 and higher risk of incident cardiovascular diseases and cardiovascular mortality.⁶ There is also an increased risk of adverse cardiovascular events shortly after COPD exacerbations.^7–9 Such elevation in cardiovascular risks in patients with COPD may be due to shared risk factors, pulmonary and systemic inflammation, deconditioning and frailty, arterial stiffening, hyperinflation of the lungs, and haemodynamic alterations.¹⁰ Nonetheless, it is unclear whether other aspects of COPD in the stable setting influence cardiovascular risk. The presence of CVD is associated with worse prognosis in patients with COPD.^11–13 The clinical presentation of some CVDs, such as HF, may overlap with COPD, meaning that the diagnosis of such conditions in patients with COPD can be difficult.^14,15 Notwithstanding this, both CVDs and COPD exacerbations contribute to the substantial health and financial burden on patients with COPD.^16,17

Whilst a number of primary prevention cardiovascular risk prediction models exist, including QR4,¹⁸ QRISK3,¹⁹ the SCORE2 family,^20–23 PREVENT,²⁴ and the pooled cohort equation,²⁵ none have been validated in patients with COPD. The QRISK3 equation underestimates incident myocardial infarction (MI) and stroke in COPD,²⁶ and all – with the sole exception of QR4 – do not include COPD as a predictor. Indeed, no primary prevention cardiovascular risk model has considered COPD-related factors such as COPD exacerbation or hospitalisation history, use of oral steroids, and COPD severity. Current cardiovascular risk prediction models are not optimised in patients with COPD, something that has been identified as a key gap in evidence by the Global Working Group on Cardiopulmonary Risk in COPD (now the International Cardiovascular and Respiratory Alliance).²⁷

Real-world primary care data provide an excellent opportunity for deriving and validating a COPD-specific cardiovascular risk prediction model. This proof-of-concept study thus seeks to demonstrate the viability of using only electronic medical records data to predict cardiorespiratory risk and identify high-risk patients. Specifically, we aimed to use the Optimum Patient Care Research Database (OPCRD), a large primary care electronic medical records database representative of the United Kingdom (UK),²⁸ to develop and validate a prediction model for MACRE in patients with COPD.

Methods

This study was performed in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD),²⁹ the Strengthening The Reporting of OBservational studies in Epidemiology (STROBE),³⁰ the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement,³¹ the CODE-EHR best-practice framework for using structured electronic healthcare records in clinical research,³² and the Declaration of Helsinki.

Source of Data

Electronic health records were extracted from OPCRD, which comprises data extracted through the Optimum Patient Care (OPC) Clinical Service Evaluation.²⁸ At the time of writing, OPCRD contains anonymized, research-quality data for over 23 million patients from over 1100 general practices across the UK, with mean follow-up duration of 11.7 years. The OPCRD encodes diagnostic, prescription and procedural data using SNOMED-International codes, SNOMED-UK codes, Read codes v2 and v3, and ICD-10 codes.²⁸ The OPCRD is approved by the Health Research Authority for clinical research use (Research Ethics Committee reference: 15/EM/0150), is governed by the Anonymised Data Ethics & Protocol Transparency Committee, and has been used extensively for research.^33–36 There was no linkage to other databases.

Study Design and Key Timepoints

For this population-based, retrospective, observational study, the index date was 1 January 2011. The baseline period, during which covariates were assessed, was defined as the entire period available for each patient prior to the index date. All patients were followed up from the index date to the date of the outcome, end of data availability, or 6 July 2023, whichever was earliest. The time horizons of prediction were five and 10 years. Non-informative censoring was presumed.

Inclusion and Exclusion Criteria

To be included patients were required to fulfil all of the following criteria: 1) recorded with any diagnostic code for COPD on or before the index date; 2) aged ≥40 years old on the index date; 3) with at least two years of continuous practice data prior to index date; and 4) ever recorded to be a smoker or ex-smoker. Patients who were recorded with any diagnostic code for MI on or before the index date were excluded. This is because it was not possible to accurately distinguish new episodes of MI from follow-up visits in these patients.

Outcomes

The primary outcome was MACRE, a five-component composite of the following clinical events after index date: 1) HF hospitalization; 2) MI; 3) coronary revascularization (percutaneous coronary intervention and/or coronary artery bypass grafting surgery); 4) severe COPD exacerbation (defined as a COPD-related hospitalization); and 5) all-cause mortality. Only the first occurrence of any component of MACRE was analyzed.

The components of MACRE were chosen with consideration of the conventional definition of major adverse cardiovascular events (MACE) in the existing literature. MI and HF were chosen because occurrence of COPD exacerbation is associated with subsequent MI and HF.^7–9 Severe COPD exacerbation was added to reflect respiratory risks. Although many studies used cardiovascular mortality as a clinical outcome measure, ascertaining cause of death is challenging in patients with COPD, as it is clinically difficult to definitively arbitrate the cardiovascular and respiratory deaths. Given this ascertainment issue with cause-specific mortality, and that 80% of deaths in patients with COPD result from cardiorespiratory causes,^37,38 all-cause mortality was chosen as a component of MACRE instead of cause-specific counterparts.

Candidate Predictor Variables

Candidate predictor variables were identified from the literature, including the predictors in the Cambridge Multimorbidity Score and QRISK3 (Supplementary Table 1).^19,39,40 Briefly, the candidate predictor variables included demographics, anthropometric measurements, smoking status, family history of CVD, comorbid diagnoses, lung function test parameters, COPD-specific parameters (time since COPD diagnosis, severity measures, exacerbation history, and prescription of inhalers and oral corticosteroids), prescription of other medications, lipid measurements, blood pressure, and vaccination history.

Statistical Analysis

Continuous variables were expressed as means with standard deviations (sd) or medians with inter-quartile ranges (iqr), depending on the distribution of the variables on visual inspection of histograms. Categorical variables were expressed as counts with percentages. The number of patients with missing values for each variable were reported. For all analyses, two-sided p-values <0.05 were considered statistically significant. All statistical analyses were performed using Stata version 18.0 (StataCorp LLC, College Station, Texas, United States of America).

The hold-out method was used for the derivation and validation of the prediction model: 80% of practices were randomly allocated to the derivation set using the Mersenne Twister pseudorandom number generator for building the model, which was then validated in the remaining 20% of practices (ie the validation set). The risk prediction model was constructed for the primary outcome using multivariable Cox regression. The least absolute shrinkage and selection operator (LASSO) was used to select predictor variables from the candidate predictor list, with the penalty parameters chosen by minimizing the Bayesian Information Criterion. The performance of the resultant risk prediction model was then evaluated within both datasets separately, with the Brier score at five and 10 years used as a measure of the overall predictive accuracy. Discrimination was evaluated using Harrell’s c-statistic, and the area under the receiver operating characteristics curve at five and 10 years. Calibration was evaluated by overlaying a plot of the model-predicted cumulative freedom from the primary outcome over time on the Kaplan–Meier survival curve for the observed cumulative freedom from the primary outcome over time, with the patients divided into quintiles by the 10-year predicted risk. Calibration slope and intercept with 95% confidence intervals were also calculated.

Baseline cardiopulmonary risk was further stratified by the presence of CVDs and/or cardiovascular risk factors and mMRC score, according to the heatmap recommended by the Global Working Group on Cardiopulmonary Risk in COPD, in which the top right-hand corner denotes patients with the highest cardiorespiratory risk, and the bottom left-hand corner denotes those with the lowest risk.⁴¹ Definitions of cardiovascular risk factors, established CVDs, and recent cardiovascular events are detailed in Supplementary Table 2. The predicted risks of MACRE were calculated for each risk category.

Multiple imputation was not performed. Missing data were imputed where possible according to the rules set in Supplementary Table 3. Where imputation was not possible, a separate category of “missing” was used. This approach was used as it does not require knowledge of the dependency of missing data on other patient characteristics, meaning that further-refined models in the future will be more straightforward to apply in a clinical setting.

Results

Of 23,063,807 patients in OPCRD, 122,077 informed the analytical cohort after applying inclusion and exclusion criteria (Figure 1), of whom 98,959 (81.1%) were included in the derivation set and 23,118 (18.9%) in the validation set. Key baseline characteristics of the whole cohort, specifically variables included in the eventual multivariable model, are summarized in Table 1. A detailed summary of all baseline characteristics can be found in Supplementary Table 4. The mean age of the whole cohort was 69.3 (SD 11.2) years and 47.9% were women. Cardiometabolic risk factors were prevalent, and most patients had moderate or severe COPD by GOLD grading.

Table 1 Key Baseline Characteristics of Included Patients. Only Variables Included in the Eventual Multivariable Model are Listed

Figure 1 Patient flow diagram. COPD, chronic obstructive pulmonary disease.

Over median follow-up of 9.9 [iqr 3.8–12.4] years in the derivation set and 10.5 [iqr 4.2–12.4] years in the validation set, MACRE cumulatively occurred in 49,461 (50.0%) patients in the derivation set and 11,602 (50.2%) patients in the validation set – such high cumulative incidences of MACRE were congruent with the high prevalence of cardiometabolic risk factors and moderate or severe COPD as aforementioned, as well as the long follow-up duration. The most frequent component of MACRE was severe COPD exacerbation, accounting for 56.3% of first MACRE after the index date in the derivation set and 54.3% in the validation set (Supplementary Table 5), followed by all-cause mortality (34.4% in the derivation set, 35.9% in the validation set), MI (4.7% in the derivation set, 5.0% in the validation set), HF hospitalization (2.4% in the derivation set, 2.6% in the validation set), and revascularization (2.1% in the derivation set, 2.2% in the validation set). Notably, 15.2% of all first MACRE (14.8% in the derivation set and 16.7% in the validation set) occurred in the first year of follow-up, with similar proportions of constituent events (Supplementary Table 6). The median time to MACRE was 7.2 years in both derivation and validation sets.

Predictor Selection and Model Validation

In total, 61 predictor variables were selected (Table 2 and Supplementary Table 7). The predictors with the greatest magnitudes of effects were age (HR [in multivariable model] per 10 years: 1.539 [95% CI: 1.521, 1.557]), underweight (HR: 1.479 [95% CI: 1.421, 1.540]), dementia (HR: 1.580 [95% CI: 1.469, 1.699]), chronic kidney disease stage 4 (HR: 1.510 [95% CI: 1.378, 1.654]) or 5 (HR: 1.559 [95% CI: 1.393, 1.744]), mMRC score 5 (HR: 1.670 [95% CI: 1.493, 1.868]), and GOLD classification severe (HR: 1.489 [95% CI: 1.441, 1.539]) or very severe (HR: 1.951 [95% CI: 1.855, 2.053]).

Table 2 Summary Statistics of the Variables Selected by the Least Absolute Shrinkage and Selection Operator (LASSO) in the Multivariable Model in the Derivation Set

In the derivation set, the model’s Harrell’s c-statistic was 0.719 [95% CI: 0.716, 0.721] and had an AUROC of 0.779 [95% CI: 0.776, 0.783] at five years and 0.813 [95% CI: 0.809, 0.817] at 10 years (Supplementary Figure 1). Overlaid predicted and observed (Kaplan-Meier) curves of cumulative freedom from MACRE demonstrated good calibration (Supplementary Figure 2), as did the calibration slope, intercept and plots for both five and 10 years (Supplementary Table 8). Overall, the model achieved a Brier score of 0.183 at five years and 0.162 at 10 years.

In the validation set, the model’s Harrell’s c-statistic was 0.717 [95% CI: 0.712, 0.722] and had an AUROC of 0.781 [95% CI: 0.774, 0.789] at five years and 0.817 [95% CI: 0.810, 0.825] at 10 years (Figure 2). Overlaid predicted and observed (Kaplan-Meier) curves of cumulative freedom from MACRE demonstrated good calibration (Figure 3), as did the calibration slopes, intercepts and plots for both five and 10 years (Supplementary Table 8). The 95% confidence intervals of all of the above quantitative measures of discrimination and calibration for the validation set overlapped with those of the derivation set. Overall, the model achieved a Brier score of 0.184 at five years and 0.156 at 10 years.

Figure 2 Receiver-operating characteristics (ROC) curves for major adverse cardiorespiratory events (MACRE) at five (A) and 10 (B) years in the validation set.

Figure 3 Overlaid predicted and observed (Kaplan–Meier) curves of cumulative freedom from major adverse cardiorespiratory events for the validation set.

Risk Classification Heatmap

Per the heatmap recommended by the Global Working Group on Cardiopulmonary Risk in COPD, patients with COPD in higher cardiopulmonary risk categories had higher five- and ten- year MACRE risks as predicted by the model (Figure 4).

Figure 4 Risk classification heatmap showing the predicted risks of major adverse cardiorespiratory events (MACRE) in the overall cohort at (A) five years and (B) 10 years, stratified by the cardiovascular risk profile and the modified Medical Research Council (mMRC) score. Numbers represent the predicted risks in percentages.

Discussion

In this proof-of-concept historical cohort study of a large British electronic medical record database, we constructed a risk prediction model for MACRE in patients with COPD with good discrimination and calibration. As one of the first studies predicting cardiorespiratory risk in patients with COPD, this study demonstrated that such prediction using electronic medical records data is feasible, which can serve as a basis for further efforts to construct more refined and clinically usable cardiorespiratory risk models for patients with COPD.

Our model achieved discriminative performances that were largely comparable with contemporary guideline-recommended cardiovascular risk scores for the general population, such as SCORE2 (internal validation c-statistic 0.739) and the pooled cohort equation (cross-validation c-statistics 0.713–0.818 in different subsets).^20,25 Our model also achieved comparable performance in both the derivation and validation sets. However, as a proof-of-concept study, our model was intended to demonstrate the viability of using electronic medical records data for cardiorespiratory risk prediction, and it was not yet optimized for clinical use. As a result, it included a much larger number (ie 61) of predictors compared to the aforementioned risk scores, with SCORE2 including six predictors and the pooled cohort equation including nine.^20,25 Although the most important predictors were clinically sensible, some predictors exhibited associations with MACRE which were counterintuitive and contradicted medical consensus or conclusive prior literature, such as dyslipidaemia being associated with a lower risk of MACRE, and having a pneumococcal vaccination being associated with a higher risk of MACRE. These may have resulted from bias by indication, eg sicker patients were more likely to have received pneumococcal vaccination, as well as being a result of the reliance on data-driven variable selection methods in a large derivation cohort. Some of these variables may also be surrogate markers of higher quality of care, such as coding for dyslipidemia potentially reflecting more thorough health monitoring, and pneumococcal vaccination possibly indicating greater attentiveness to preventive care. Overall, our model was not set up to be causal in nature, and given the risk of other biases such as over adjustment bias, the estimates should be interpreted with caution outside causal contexts. Clinically, the large number of predictors meant that clinical use of the resultant model will be difficult, and the counterintuitive associations meant the predictors cannot be interpreted as treatment targets. Further studies using more deliberate and knowledge-driven predictor selection processes are needed, with greater a priori emphasis on the clinical utility of the model and interpretability of associations. This study, having demonstrated the viability of using routinely available electronic medical records data from primary care practices to predict MACRE and identify high-risk patients with COPD, should serve as a basis and stepping stone for these future endeavours.

Contrasting existing cardiovascular risk tools which focus solely on incident cardiovascular diseases, this study focused on MACRE, a novel outcome, which provided a potential means of meaningfully capturing cardiac and respiratory events in a single composite outcome. Additionally, this circumvented the issue of misdiagnosis and thus miscoding between COPD and cardiac events, especially HF. Our finding that higher-risk categories among those stipulated in the heatmap recommended by the Global Working Group on Cardiopulmonary Risk had higher incidences of MACRE also provided a qualitative validation of said heatmap. Nevertheless, with MACRE being a novel outcome that is yet to be considered by international guidelines to guide management, these estimates of MACRE incidences may not yet be clinically actionable, although they may be used cautiously as a potential reference for understanding the cardiorespiratory risk of different patients with COPD. Moving forward, further studies on MACRE are warranted. These include studies that examine the mortality- and morbidity-equivalence of cardiovascular and respiratory events in patients with COPD, which is a gap in evidence identified by the Global Working Group on Cardiopulmonary Risk and a means of examining the validity of MACRE as a composite outcome.²⁷ In the longer run, together with further variable reduction and, subsequently, external validation of the model, these efforts may lead to the establishment of clinically actionable thresholds that guide cardiorespiratory interventions in patients with COPD to prevent MACRE.

This study represented an effort to focus on the cardiorespiratory interface, building on a prior study which observed an association between COPD diagnosis and cardiovascular mortality,⁶ as well as associations between cardiovascular conditions and COPD outcomes.^7–9 Mechanistically, these associations are multifactorial in nature. Aside from overlapping risk factors such as smoking, other biological factors likely contribute significantly to this complex relationship as well, including pulmonary and systemic inflammation, deconditioning and frailty, arterial stiffening, hyperinflation of the lungs, haemodynamic alterations, subclinical atherosclerosis, and left ventricular hypertrophy and concentric remodelling.^10,42–46 Furthermore, many medications used for treating COPD (such as systemic steroids) or cardiovascular diseases have adverse effects on each other, further compounding the complexity of the overlap between these broad clinical entities.¹⁰ Notwithstanding the above, studies of the cardiorespiratory overlap in patients with COPD remain relatively scarce and crude. Further studies with greater nuance and consideration of COPD-specific factors, such as severity and healthcare resources utilization measures, remain warranted.

Limitations

This study had a number of limitations. First, as a proof-of-concept study, external validation was not performed, which remains required for developing a clinically useful model. Second, the dataset represented information collected for clinical and routine use rather than specifically for research purposes. The validity and completeness of individual patient records could not be adjudicated. Nonetheless, the source of data (OPCRD) has been used extensively in other studies of respiratory and cardiovascular conditions or outcomes.^33,35,36,47 Third, non-linear associations, inter-predictor interactions, and time-varying associations were not considered but may be useful in improving the predictive accuracy of the model. Fourth, patients with prior occurrence of MI were excluded. This ensured accurate ascertainment of MACRE, but also limited the generalizability and clinical relevance of our model, especially since these patients were at an elevated risk of cardiorespiratory events, meaning that the model may underestimate risks in these patients. Fifth, cardiac events constituted only a small proportion of MACRE, which may be partly due to the aforementioned exclusion of patients with prior MI. Further studies and validation of MACRE or similar composite cardiorespiratory outcomes remain required. Sixth, the study period included the COVID-19 pandemic period, during which the skewed rates of cardiorespiratory events may have affected our results and limited the model’s generalizability.^48,49 Nonetheless, the model demonstrated good predictive performance at both five years – which is unaffected by COVID-19 – and 10 years, suggesting that the model is robust and not substantially impacted by the COVID-19 pandemic. Lastly, it is possible that some important predictive variables were unobserved, including behavioural data which were not widely available from the source of data used in this study.

Conclusions

In a large British cohort of patients with COPD, cardiorespiratory events (MACRE) were common. As one of the first studies predicting cardiorespiratory risk in patients with COPD, this study demonstrated that prediction of MACRE using primary care electronic medical records data was feasible and showed a good predictive value. Further studies with more rigorous predictor selection processes, greater emphasis on clinical utility and interpretability, and external validation are required, which may allow establishment of clinically actionable thresholds that guide cardiorespiratory interventions in patients with COPD to prevent MACRE.