The ICH E9 R1 addendum on estimands in clinical trials has made big waves in the clinical trial world in the last few years. It aims to provide a framework to think about and define more precisely what exactly the treatment effect(s) of interest is in a clinical trial, in light of what the addendum calls ‘intercurrent events’ (ICEs):
Events occurring after treatment initiation that affect either the interpretation or the existence of the
measurements associated with the clinical question of interest. It is necessary to address intercurrent
events when describing the clinical question of interest in order to precisely define the treatment effect
that is to be estimated.
A couple of weeks ago a really nice paper was published by Harrison and Brummel in the American Statistican which explored the five different ‘strategies’ described in the E9 addendum for handling ICEs in a simple example using potential outcomes. For each strategy they gave an example of an estimand defined using the strategy and a simple estimator for estimating the estimand from the data. In this post, I want to focus on the while on treatment strategy, as I think it’s one area where there is some debate as to what exactly the E9 addendum meant. I of course do not claim to have the definitive answer, but the following is my view.
While on treatment according to E9
The E9 addendum describes a while on treatment strategy as follows:
For this strategy, response to treatment prior to the occurrence of the intercurrent event is of interest.
Terminology for this strategy will depend on the intercurrent event of interest; e.g. “while alive”, when
considering death as an intercurrent event.
If a variable is measured repeatedly, its values up to the time of the intercurrent event may be
considered relevant for the clinical question, rather than the value at the same fixed timepoint for all
subjects. The same applies to the occurrence of a binary outcome of interest up to the time of the
intercurrent event. For example, subjects with a terminal illness may discontinue a purely
symptomatic treatment because they die, yet the success of the treatment can be measured based on
the effect on symptoms before death. Alternatively, subjects might discontinue treatment and, in some circumstances, it will be of interest to assess the risk of an adverse drug reaction while the
patient is exposed to treatment.
Thus the idea is that the outcome is defined as some function of the observed data on patients up until the ICE occurs, if it occurs. For example, in the case of the ICE being death, the outcome could be the last outcome before death, or, if death does not occur within a specified time window (e.g. 12 months), it takes the value at the specified time point.
Later in Section A5 on impacts on trial analysis, the addendum says:
Estimands constructed based on a while on treatment strategy can be estimated provided outcomes
are collected up to the time of the intercurrent event.
Although this is saying that while on treatment estimands can be estimated provided outcomes are collected up to the time of the ICE, in the same spirit, I think the intention is that the estimand should be defined based on an outcome that is well-defined for all patients. That is, it must be in the context of the trial’s design be possible to define what the outcome will be for all patients. For example, in the case of the ICE being death, and the outcome being a quality of life measure, we must define what quality of life measure to use for patients should they die before the first post-baseline planned quality of life measure is obtained. Although one could criticise it on various fronts, if the chances of death before this first measure would be low, one pragmatic solution would be to define the outcome using a baseline measure of the same quality of life instrument.
Ensuring that the outcome is well-defined for all patients is crucial, in light of the addendum’s description of an estimand as being:
It summarises at a population level what the outcomes would be in the same patients under different treatment conditions being compared.
For an estimand to satisfy this requires us to be able to specify a well-defined value of the outcome for all patients in the trial.
While on treatment in Harrison and Brummel’s paper
This brings me back to the Harrison and Brummel paper. They illustrate the five different strategies using an example in which there is a binary randomised treatment variable A, an outcome Y and an ICE denoted M (e.g. treatment discontination). They assume a causal structure where M precedes Y. Thus if a patient discontinues treatment, it could (and generally would) affect the subsequent outcome Y.
For while on treatment for this example they write:
In this example where Y is measured at a single time-point, we illustrate an ‘on treatment’ strategy defined as the outcome Y for those that remain on treatment, i.e. in those individuals for which 𝑀=0.
and write the estimand in potential outcome notation as
This estimand contrasts the mean outcome under treatment level 1 in patients who do not experience the ICE when given treatment 1 compared to the mean outcome under treatment level 0 in patients who do not experience the ICE when given treatment 0. As they note, this estimand is non-causal, because it conditions on the post-baseline variable M. In the trial, the patients who do not discontinued when assigned treatment level 1 are not in general the same as the patients who would not discontinued when assigned treatment level 0.
But I do not believe this estimand is an ‘estimand’ as defined or proposed by the E9 addendum. Indeed the addendum is clear about this:
The subset of subjects who experience an intercurrent event on the test treatment will often be a different subset from those who experience the same intercurrent event on control. Treatment effects defined by comparing outcomes in these subsets confound the effects of the different treatments with the differences in outcomes possibly due to the differing characteristics of the subjects.
Following my earlier comments, I think for a while on treatment estimand to be defined in line with the addendum, one must define what the outcome would be for every patient in trial, covering the case where the ICE occurs before the first planned outcome measurement is to take place.
Harrison and Brummel go on to describe how an estimator of the estimand is a version of what might be called a per-protocol estimator, because it compares outcomes between arms only among those who have stayed on randomised treatment. They then explain why it is in general a biased estimator of a hypothetical estimand which is the effect in all patients had they (somehow all) been made to stay on randomised treatment.
Later in Section 3 they extend their example to one where the outcome is equal to some initial/early measure of outcome if the patient experiences the ICE and if they do not, is some final measure of outcome. This is inline with my understanding of a while on treatment strategy and estimand.
I have said nothing above about whether while on treatment estimands are sensible and useful. This is obviously a very important question, and although there are clearly some potential complications in interpretation when outcomes for some patients are in the end measured at one time point (relative to baseline) and at another time point for others, I think there probably are situations where while on treatment estimands could be useful.