The hazards of period specific and weighted hazard ratios

In recent years increasing attention has been focused on violations of the proportional hazards assumption made in Cox’s famous proportional hazards model. This has occurred in particular due to trials of immuno-oncology treatments, where the mechanism of action may imply that any benefit of treatment would only occur some time after initiation of treatment. As a result, a large number of papers have been published recently looking at alternative methods of analysis (to Cox’s model), which I have written about before.

Recently a paper was published in the journal Statistics in Biopharmaceutical Research proposing methods of analysis for time to event data in randomised trials when it is anticipated that the proportional hazards assumption may be violated. The paper proposed an approach for hypothesis testing, the ‘max combo’ method, which is based on a series of weighted log rank tests. Weighted log rank tests are a generalisation of the standard log rank test for censored time to event data which weight the different follow-up periods differentially, ideally with highest weight in the period where one expects the largest treatment effect. Since one may not correctly anticipate when this will occur, the max combo test uses the weighted log rank test which is most statistically significant (among a small collection of weighted log rank tests), with an appropriate penalty incorporated for the fact that the particular weighted log rank test is being selected because it is the most significant. As well as hypothesis testing, effect estimation is obviously of crucial importance. In the presence of non-proportional hazards, the aforementioned paper advocated use of period specific hazard ratios (estimating separate hazard ratios in distinct portions of follow-up), and also a weighted hazard ratio that corresponds to the ‘winning’ weighted log rank test in order to characterise how the treatment effect changes over time.

Read more

Robustness of ANCOVA in randomised trials with unequal randomisation

In my previous post I wrote about a new paper in Biometrics which shows that when ANCOVA is used to analyse a randomised trial with adjustment for baseline covariates, as well as the treatment effect estimator being consistent, the usual model based standard error (SE) is also valid, irrespective of whether the regression model is correctly specified. As I wrote, these results were proved assuming that the trial used simple randomisation to the two groups, with equal probability of randomisation to the two.

In a pre-print available on arXiv, I extend this paper’s results to consider the case where the randomisation probabilities are not equal. Although 1:1 randomisation is by far the most common approach used I think, unequal randomisation is not that uncommon. In this situation, the point estimator for the treatment effect is still consistent – this isn’t affected by the unequal randomisation.

The analyses in the paper show that the model based SE is no longer generally consistent if the outcome model is misspecified when the randomisation is not 1:1. It is valid if the true regression coefficients of the outcome on the covariates are the same in the two treatment groups and the variance of the errors in the two groups are equal. But otherwise in general it is not. So for example even if the true regression coefficients are equal in the two groups, if the error variances are not, the model based SE is not valid. Alternatively, if the true regression coefficients differ in the two groups (i.e. interactions between treatment and some baseline covariates), again the model based SE would not in general be expected to be valid.

The impact of such invalidity in the SEs is that the type I error will not generally be controlled at the desired level, and confidence intervals will not have the correct coverage, even for large sample sizes. The results show that depending on the configuration the SEs can be biased upwards or downwards (see the pre-print for details).

These results mean that in trials with simple randomisation and where the randomisation is not 1:1, if one is concerned about the ANCOVA model being misspecified, the model based SE shouldn’t be used. Instead, robust sandwich SEs, which are widely available in statistical packages, are recommended. These provided asymptotically valid variance estimation under essentially arbitrary model misspecification.

December 2019 – this work has now been published in Biometrics.

ANCOVA in RCTs – model based standard errors are valid even under misspecification

A nice paper by Wang and colleagues has just been published in Biometrics which examines the robustness of ANCOVA (i.e. linear regression) for analysing continuous outcomes in randomised trials. By this they mean a linear regression which adjusts for randomised treatment group and baseline covariates. Yang and Tsiatis in 2001 showed that this estimator is consistent for the average treatment effect even if the model is misspecified, and as such can be recommended for general use in the analysis of such trials. They offered a variance estimator for the resulting treatment effect estimate that is valid even if the model is misspecified, and compared this in simulations to the usual model based variance estimator from ANCOVA (which they refer to as the OLS variance). Yang and Tsiatis reported that this OLS variance estimator performed well even when the linear model was misspecified, but suggested that the OLS variance estimator had some bias as the sample size was increased.

In their new paper Wang and colleagues prove that so long as the randomisation ratio is 1:1, the standard model based variance estimator for the adjusted treatment effect from ANCOVA is valid even under model misspecification. As such, this further strengthens support for using a linear regression model with adjustment for baseline covariates in randomised trials.

An important caveat to the result of Wang and colleagues is that they assumed that the treatment group is assigned completely at random, and in particular independently of the baseline covariates. This rules out certain randomisation schemes, such as stratified randomisation, where randomisation depends on the subject’s baseline covariates. Indeed we know that in this setting, if we don’t properly model the effects of the variables used to stratify the randomisation, our treatment effect variance estimates in general are not correct.