Long COVID risk and pre-COVID vaccination in an EHR-based cohort study from the RECOVER program

Our four analyses yielded consistent results. We see protective associations of vaccination with long COVID diagnosis in both logistic and time-to-event models, and in both clinic-based and model-based outcomes. While these findings are similar to those of other large observational studies^16,17,18,19, previous sources have only looked for evidence of COVID-associated symptoms as evidence of long COVID. A major finding of our analysis is that the protective association remains consistent in results requiring a clinical diagnosis, and among those who contracted COVID-19 in a later period that includes Omicron infections.

The use of a clinical diagnosis resulted in a significantly lower long COVID prevalence in our study (less than 2% in both cohorts) than studies based on long COVID symptoms, which have reported prevalences between 8 and 38%, depending on which and how many symptoms were required^17,18,19. However, both of our cohorts are large, and the use of a CP allowed us to expand our sample from six to eleven sites and 47,404 to 198,514 COVID-positive patients, providing a sufficient sample of strictly defined long COVID diagnoses. Due to the underdiagnosis of long COVID in a clinical setting, our conclusions are limited to associations with diagnosis and not with long COVID onset more generally.

Interestingly, the protective association of vaccination with long COVID diagnosis is weaker or reversed in the unadjusted coefficients and cross-tabulations (Table 6 and Fig. 2). Several features that are associated with a higher likelihood of long COVID (coefficients in Supplementary Tables 3–6) are also associated with a higher likelihood of vaccination (coefficients in Supplementary Tables 12, 13). The most significant is age: Supplementary Table 14 shows how older adults are both more likely to be vaccinated and more likely to contract long COVID in comparison to younger adults. Failing to account for the substantial differences between individuals who were and were not vaccinated prior to COVID-19 could lead one to inaccurately conclude that vaccination is harmful.

The sensitivity analysis presents other instructive complexities. Reducing the CP score threshold lowers the amount of evidence required to denote someone as having long COVID; it also moderates the protective association of vaccination with long COVID (key results in Fig. 2, full range of thresholds in Supplementary Fig. 3). We expect that including healthy adults in the long COVID, population would dilute the observed association, but individuals with a CP score between 0.6 and 0.9 are not entirely healthy—they have some evidence of long COVID. In fact, our sample’s long COVID incidence rate at lower thresholds is closer to long COVID incidence rates reported elsewhere (although the true incidence rate of long COVID is unknown). This suggests a hypothesis that vaccination may be more effective at preventing clinically diagnosed long COVID than undiagnosed long COVID. More research is needed to determine the differences between high confidence and clinically diagnosed long COVID cases compared to low confidence and undiagnosed cases. If they are more severe, then our results could suggest that vaccination is associated with reduced severity of long COVID symptoms.

Healthcare utilization is one of the most important features of the CP model¹⁰. If fully vaccinated patients are more likely to utilize the healthcare system, the CP model’s marginal predictions may be assigning more fully vaccinated individuals to long COVID because they are more likely to interact with the healthcare system, depressing the observed benefit of vaccination. A known challenge of analyzing EHR data is that they tend to provide more information on individuals who regularly utilize healthcare systems²⁵, though we attempt to control for this by requiring multiple recorded encounters outside of COVID-19 for inclusion in the study.

Our use of long COVID diagnosis and a computable phenotype as outcomes differentiate this study from others^17,18,20, which measure the association between vaccination and a curated list of long COVID symptoms. Each approach has its strengths. Our clinical outcomes reduce measurement error due to false positives (e.g., long COVID symptoms caused by something other than long COVID). However, other studies show that long COVID symptoms differ in their relationship with vaccination. Our outcomes obscure such variation. We conclude that it is beneficial to study this relationship from both perspectives.

Vaccination reduces the risk of developing COVID-19 for a period of time after vaccination^14,15, offering one mechanism for preventing long COVID. However, there is evidence that widely circulated vaccines are less effective against now-dominant Omicron than earlier SARS-CoV-2 variants^26,27,28, increasing interest in whether or not vaccination reduces the risk of long COVID in breakthrough infections. That is the aim of this study, in which all eligible patients had a COVID-19 diagnosis. As a result, we are excluding any effect due to vaccination’s primary prevention of COVID-19 in the first place that is present in the general population.

Several studies conclude that the protective effect of vaccination on acute COVID-19 infection severity wanes over time^27,29, but we are unaware of any studies making the same claim for long COVID. As can be seen in Supplementary Tables 8–11, the subanalysis incorporating time between vaccination and acute COVID-19 does not offer any evidence that the association between vaccination and long COVID diagnosis changes over time. The reference level for Weeks Since Last Vaccination is those who received their last vaccine dose at least 25 weeks prior to their COVID-19 infection. Negative coefficients for the modeled indicators suggest stronger protective associations. Three models present statistical significance (alpha = 0.05) for at least one indicator, indicating a significant difference between that level and those vaccinated 25+ weeks prior to COVID-19, but results are not consistent between models. Contrary to intuition and previously reported results with acute COVID-19, those vaccinated at least 25 weeks prior to COVID-19 are among the least likely to be diagnosed with long COVID across the four models. We do not present this as evidence that the benefits of vaccination with respect to long COVID do not wane. Caution should be used when interpreting conditional coefficients and investigating the time between vaccination and COVID-19 was not a primary focus in this study³⁰.

IPTW is often used to estimate causal effects from observational data and is employed here to provide more robust associations. However, we do not interpret these results as causal effects. This is for two reasons: (1) we are unwilling to assume that there are no unmeasured confounders in our treatment model and (2) our causal model includes several latent variables, which obstruct the estimation of treatment effects through covariate adjustment. We explore each reason in the Supplementary Discussion and provide a directed acyclic graph of confounders in Supplementary Fig. 4.

Our study is limited by its reliance on EHRs and other factors. Those who choose to not seek healthcare, or are unable to do so, are not represented in EHRs. This could be particularly problematic among long COVID patients, who may lack the energy or resources required to receive a clinical diagnosis, or whose providers may not be familiar enough with long COVID symptoms to make a diagnosis. If vaccinated long COVID patients are less likely to be clinically diagnosed than unvaccinated long COVID patients, then our estimate of the association between vaccination and long COVID diagnosis will overstate the association between vaccination and long COVID onset. Furthermore, we had previously identified a heterogeneous set of features that were differentially present in clinical observations versus patient-reported symptoms⁹. This agreed with the WHO suggestion that a definition of long COVID must necessarily include both clinician and patient-reported features – which are not commonly available in the EHR.

We are forced to assume that those without a recorded condition or symptom do not exhibit it, including our exposure (vaccination), our outcome (long COVID diagnosis), and potentially unrecorded reinfections of COVID-19. We take two steps to mitigate the risk of unrecorded records relevant to the outcome: (1) we require that all participants in the study had established care at the partner facility prior to COVID-19 infection, as evidenced by two healthcare visits in the year prior, and (2) we require that all participants in the study were seen at the partner facility at least 120 days after COVID-19 infection. Our utilization-related inclusion criteria result in a cohort that is disproportionately female, which may be due in part to females being higher healthcare utilizers than males on average^31,32,33. We account for confounding due to sex by including sex as a covariate in treatment weighting and in all primary models. The utilization criteria result in a significantly smaller cohort and biases the sample towards high utilizers and those with hospitalizations. However, it remains sufficiently large for analysis and has a lower risk that long COVID will go undiagnosed, as patients were active users of the partner facility both before and four months after COVID-19 infection. In the clinic-based cohort, there is an additional requirement that the facilities have a track record of diagnosing long COVID (though the variation between doctors remains).

A sensitivity analysis that censors individuals after their last healthcare visit (rather than at the end of the study period) yields an association similar to our four primary results (Supplementary Table 7). Censoring is not possible in logistic regression models, but allows the proportional hazards model to relax the assumption that individuals that established care at a facility continue to use that facility after their last recorded visit. For this analysis, we did not require a recorded visit after the acute COVID-19 infection, but individuals remained ineligible for long COVID designation until 45 days after infection.

Our cohort is further refined by our requirement that the partner facility have reasonably high recorded vaccine ratios, as defined in our Methods. Most facilities fail to achieve recorded vaccine ratios greater than 66%, as they are not the primary provider of vaccinations in their community, do not link to their state’s vaccine registry, or do not consistently record the vaccinations they provide in the EHR. We do not use a facility’s vaccination rate as an individual characteristic in our models, but rather as a facility-wide inclusion criterion. By limiting to partner facilities with a high vaccination rate, as with our utilization criteria, we refine our cohort to be smaller but more data-rich.

We strictly define our study cohort to minimize the underreporting of vaccination and long COVID, though we acknowledge that it is not entirely resolved. Our sensitivity analysis using only sites with the highest recorded vaccine ratios (≥89%) offers some evidence that incomplete vaccine records result in a conservative estimate. The cohorts are small (the model-based cohort has 10,122 patients; the clinic-based cohort has 5545), resulting in wide confidence intervals that include the primary estimates for every model (Supplementary Table 7). However, the mean estimated associations are stronger than our primary results in three of the four models and remain significant in all four models with 95% confidence. We conclude that our primary estimates are likely conservative, but our primary result—that pre-COVID-19 vaccination is associated with a reduced risk of long COVID diagnosis—is not threatened.

The confidence intervals around the CP model-based risk estimates are likely too narrow, as there remains residual misclassification of long COVID outcomes in that cohort not factored into the confidence interval boundaries. We did not distinguish between vaccine types, though previous studies and initial tabulations failed to detect significant differences in their associations with long COVID^17,18,19. The ICD-10 code for long COVID, U09.9, was not implemented until October 2021, and it has not been fully adopted. The previously recommended ICD-10 code, B94.8, is more general and is used to diagnose long-term complications from any viral infection. We accepted B94.8 as a long COVID diagnosis because the use of the code in our data by mid-2021 was 40 times higher than its baseline use in 2018 and 2019. A sensitivity analysis using only U09.9 returned nearly identical results.

In conclusion, vaccination was consistently associated with lower odds of both a long COVID clinical diagnosis as well as a high-confidence computationally derived diagnosis, regardless of viral epoch and taking into account age, sex, and demographics. This multi-method strategy provides additional evidence on the controversial and yet understudied and challenging topic of whether vaccination reduces the risk of long COVID.