I’ve previously written about methods from causal inference (G-formula and G-estimation) that can be used to exploit data observed after patients experience intercurrent events (ICEs) to improve the precision (and hence statistical power) of estimates of treatment effects in clinical trials. Thanks to useful comments from reviewers, in our revised paper (open access version) on the topic, we explore the robustness of these methods, and those that do not make use of such data, to misspecification in model assumptions.
We explore robustness under in very simple setting, in which the ICE occurs (or not) at a single time after baseline, following which the outcome is measured. Our derivations were made assuming the following DAG, in which A denotes randomised treatment group, L0 denotes baseline covariates, L1 post-baseline confounders, R a binary indicator of occurrence of the ICE, and Y the final outcome.
The hypothetical estimand of interest is the mean difference in Y between randomised arms under the hypothetical scenario where the ICE R is set to zero. We explored a missing data imputation type estimator that does not make use of outcomes Y in those patients who in the trial experience the ICE, and G-formula / G-estimator methods that do. The imputation estimator requires a model for the ‘no-ICE’ outcomes, conditional on A, L0 and L1. The G-formula / G-estimator requires a model for this, and a model for the outcomes after the ICE occurred (or a model for both with suitable modelling of the effect of R).
I won’t go into the full details here, but we derived some analytical results and performed simulation studies to examine how the different methods performed when in truth some of the modelling assumptions were incorrect. In the first, we considered the impact of the post-baseline variable L1 having a non-linear (quadratic) effect, but which is not modelled as such by the estimators. Interestingly, despite the model misspecification, the G-formula and G-estimators which use post-ICE data (pre-post linear and Loh-linear in the plot below) remain approximately unbiased for the hypothetical estimand if the proportion of patients experiencing the ICE is similar in the two arms. In contrast, the imputation data estimator that only uses pre-ICE data and wrongly assumes a linear effect of L1 (pre-linear) is biased, with the bias increasing as a greater proportion of patients experience the ICE.
Next we considered the setting where in truth there is an interaction between L1 and R in the model generating Y. Here the G-formula / G-estimator that does not allow for this interaction shows bias, while the imputation type estimators that only pre-ICE data, and thus do not require the outcome model to specify how R affects the outcome, remained unbiased (shown below).
Lastly we ran simulations in which there was both an L1*R interaction in the data generating model for Y and a quadratic effect of L1 on Y (when R=0). Here all estimators showed bias, except the imputation type estimator that used pre-ICE data and correctly captured the quadratic effect of L1 on Y in those patients that did not experience an ICE.
As we wrote in our conclusions, the G-formula / G-estimator methods that use post-ICE data can increase statistical power for the hypothetical estimand. However, while the pre-ICE data imputation estimator only requires a model for outcomes under no-ICE, the G-formula / G-estimator methods require the analyst to additionally model the outcomes after ICE, and therefore to model the effect of the ICE on outcomes. If in truth there exist important interactions between randomised treatment (A) and or post-baseline variables (L1) that modify the effect of the ICE on subsequent outcomes, G-formula / G-estimators that assume the absence of such interactions can be biased. When post-baseline covariates L1 have non-linear effects on outcomes, these could be important to correctly model, particularly for the imputation type estimator that only uses pre-ICE data.