Alternatives to MMRM for Preclinical Alzheimer’s Trials

class: middle, center

# Alternatives to MMRM for Preclinical Alzheimer’s Trials
### Simulated Impact of COVID-19 Hiatus on A4

Michael Donohue, PhD

Alzheimer's Therapeutic Research Institute

University of Southern California

### Joint Statistical Meetings

August 5, 2020

.pull-left[

]

.pull-right[

]

???

Thank you to Sue-Jane and Tristan for organizing the session and for the invitation.

This investigation started as a luxury, with the question: "can an alternative analysis approach provide more power than MMRM?"

But since COVID-19, MMRM is becoming untennable, and finding an alternative has become more of a necessity.

---

# Disclosures

- M. Donohue has served on scientific advisory boards for Biogen, Eli Lilly, and Neurotrack; and has consulted for Roche. His spouse is a full-time employee of Janssen.

- This talk will discuss alternative analysis approaches for the A4 study, but no decisions have been made regarding any changes to the analysis plan. The opinions and interpretations expressed are those of M. Donohue.

???

Here are my disclosures, and a note that no decision has yet been made regarding any changes to the analysis approach for the ongoing A4 trial.

---

# Alternatives to MMRM for Preclinical AD Trials

+ The Mixed Model for Repeated Measures [MMRM; Mallinckrodt, Clark, and David (2001) @] is the most commonly used approach for assessing treatment effects in Alzheimer’s clinical trials.

+ An alternative nonlinear Disease Progression Model (DPM) which assumes the ratio of group means is fixed over time was proposed for DIAN-TU [e.g. Wang, Berry, Xiong, Hassenstab, Quintana, McDade, Delmar, Vestrucci, Sethuraman, Bateman, and others (2018)].

+ We assess alternative linear, nonlinear, and generalized additive models for Preclinical Alzheimer’s clinical trials like the A4 Study (Sperling, Rentz, Johnson, Karlawish, Donohue, Salmon, and Aisen, 2014) with 
  
  - real ADNI data and

- simulated A4 data.

+ We also simulate the impact of variable lengths of participant hiatus in treatment and follow-up in A4 due to the COVID-19 pandemic.

???

- The Mixed Model for Repeated Measures (or MMRM), has been the standard analysis approach for continuous outcomes, collected longitudinally in AD clinical trials. This approach treats time as categorical and allows flexible estimation of the group means at each timepoint.

- The present work was originally motivated by work from the DIAN-TU study team. Peter [Wong’s] 2018 paper describes the Disease Progression Model (or DPM) and advantages of DPM over MMRM for DIAN-TU (as [kenGEE] discussed). This motivated us to start exploring DPM for A4.

- We have extended this work in the setting of variable lengths of participant hiatus in A4
due to the COVID-19 pandemic.

---

# MMRM fit to ADNI cognitively normal amyloid pos. vs neg.

.pull-leftWider[

]

.pull-rightNarrower[

MMRM:

+ Group 1 mean at `$t_i$`: `$\beta_i$`

+ Group 2 mean at `$t_i$`: `$\gamma_i$`

+ `$i=1,\ldots,m$` ( `$m$` visits)

]

???

Here is a demonstration of the familiar categorical time MMRM, fit to a cognitive composite outcome from cognitively normal Amyloid positive and negative participants of the Alzheimer’s Disease Neuroimaging Initiative (ADNI).

Because these are not randomized groups, but rather high-risk vs low-risk groups from a natural history study, this variant of MMRM allows different means at baseline.

The mean over time is estimated by 12 separate parameters in this case, one parameter per visit per group.

---

# MMRM assumptions and fitting in R

$$
Y_i = X_i\beta + \epsilon_i,
$$

where:

+ `$Y_i$` is vector of observations (one for each visit) for individual `$i=1,\ldots,n$`
+ `$X_i$` is matrix of fixed effect covariates for subject `$i$`
+ `$\beta$` is vector of fixed effects to be estimated
+ `$\epsilon_i \sim N(0,\Sigma)$` vector of correlated residuals for subject `$i$`

Fit in `R`, using `nlme::gls`:

```r
gls(pacc.ch ~ pacc.bl.c + age.c + edu.c + apoe4 + suvr.c + # bl covs
  MONTH + active:MONTH,         # mean over time (MONTH categorical)
  correlation = corSymm(form = ~ visNo | id), # general correlation
  weights = varIdent(form = ~ 1 | wk))        # heterogeneous variance
```

???

MMRM is a linear model with residuals typically assumed to follow a general correlation structure within subject, and heterogeneous variance with respect to time.

MMRM can be fit with the `gls()` function from the `nlme` R package as shown here at the bottom.

---

# Disease Progression Model (DPM) fit to ADNI

.pull-leftWider[

]

.pull-rightNarrower[

DPM:

+ Group 1 mean at `$t_i$`: `$\beta_i$`

+ Group 2 mean at `$t_i$`: `$\beta_i \times \theta$`

+ `$i=1,\ldots,m$` ( `$m$` visits)

+ Assumes:
$$
\frac{\textrm{Group 1 mean at }t}{\textrm{Group 2 mean at }t}=\theta
$$
where `$\theta$`, _disease rate ratio_, is constant over time
]

???

The solid lines here depict the DPM fit to the same ADNI data.

Dashed lines now depict the same MMRM as the previous slide.

This variant of DPM treats one group exactly the same as MMRM, with a parameter for each visit.

The other group is assumed to differ by a constant rate, Theta.

Theta, the disease rate ratio, is assumed to be constant over time.

DPM usually assumes monotone decline, but because of learning effects in preclinical AD, this assumption is probably unreasonable, and has been relaxed.

However, note that if the placebo trend is allowed to go above and below zero, the constant
disease rate ratio will represent a benefit on one side of zero and harm on the other;
complicating the model interpretation.

DPM also struggles here to capture some features which are apparent from the MMRM: namely the general decline in the amyloid positive group and the separation between groups.

---

# DPM assumptions and fitting in R

$$
Y_i = X_i\beta + (Z_i\gamma)(1+\textrm{active}_i\theta) + W_ib_i + \epsilon_i,
$$

where:

+ `$Y_i$` is vector of observations (change from baseline at each visit) for individual `$i=1,\ldots,n$`
+ `$X_i, Z_i, W_i$` are covariates for subject `$i$`
+ `$\beta$` is fixed effect parameter vector for baseline covariates
+ `$\gamma$` is fixed effect parameter for placebo group temporal mean trend
+ `$\theta$` is *multiplicative* fixed effect parameter for relative effect of treatment
+ `$b_i \sim N(0,\Sigma)$` is vector of random effects

???

DPM is a nonlinear model since the parameter theta is multipled by the mean parameters
for placebo decline (represented by "gamma" here)

This complicates estimation, but ...

---

# DPM assumptions and fitting in R

Fit in `R`, using `nlme::nlme`:

```r
nlme(pacc ~ my_nonlinear_function(...), ...)

my_nonlinear_function <- function(active,
  theta,
  beta_m0, beta_m12, beta_m24, beta_m36, beta_m48,
  beta_age, beta_edu,
  M, age, edu)
{
  (I(M==0) * beta_m0 + I(M==12) * beta_m12 + I(M==24) * beta_m24 + 
   I(M==36) * beta_m36 + I(M==48) * beta_m48) * (1 + theta*active) +
  age*beta_age + edu*beta_edu
}
```

???

... It can be fit by defining your own nonlinear function, and passing it to nlme().

Unlike gls, nlme appears to require explicit random effects (so at least a subject-specific random interept).

Of course, DPM can also be fit with MCMC techniques, with more flexibility.

---

# Quadratic time model fit to ADNI amyloid positive vs negative

.pull-leftWider[

]

.pull-rightNarrower[

Quadratic time model:

+ Group 1 mean at `$t$`: `$\beta_0 + t\beta_1 + t^2\beta_2$`

+ Group 2 mean at `$t$`: `$\gamma_0 + t\gamma_1 + t^2\gamma_2$`

+ `$i=1,\ldots,m$` ( `$m$` visits)

]

???

Here solid lines depict a quadratic time model fit to ADNI.

The quadratic model provides a reasonable smooth of the trends estimated by MMRM, assuming that they each follow quadratic trends.

---

# Quadratic time model assumptions and fitting in R

$$
Y_i = X_i\beta + \epsilon_i,
$$

where:

Fit in `R`, using `nlme::gls`:

```r
gls(pacc ~  age.c + edu.c + apoe4 + suvr.c +  # bl covs
    (months + I(months^2)) +                  # mean over time
    (months + I(months^2)):active,            # (months continuous)
  correlation = corSymm(form = ~ visNo | id), # general correlation
  weights = varIdent(form = ~ 1 | wk))        # heterogeneous variance
```

???

This is obviously still a linear model, and can be fit with gls

---

# Hybrid: Categorical + Quadratic time model fit to ADNI

.pull-leftWider[

]

.pull-rightNarrower[

Hybrid model:

+ Group 1 mean at `$t_i$`: `$\beta_i$`

+ Group 2 mean at `$t_i$`: `$\beta_i + \gamma_0 + t_i\gamma_1 + t_i^2\gamma_2$`

+ `$i=1,\ldots,m$` ( `$m$` visits)

]

???

This is another linear model which is a hybrid of categorical time (for one group) and quadratic time (for the group difference over time).

This captures more of the visit-to-visit zig-zag of MMRM in the first few visits, but smooths out the group difference over time under a quadratic assumption.

This is similar to the DPM, in that the placebo group mean is given *no* structure, but *some* structure is imposed on the group difference over time.

Here the group difference is a more traditional additive effect, rather than multiplicative as with DPM.

<!--

# Hybrid model assumptions and fitting in R

$$
Y_i = X_i\beta + \epsilon_i,
$$

where:

+ `$Y_i$` is vector of observations (one for each visit) for individual `$i=1,\ldots,n$`
+ `$X_i$` is matrix of fixed effect covariates for subject `$i$`

Fit in `R`, using `nlme::gls`:

```r
gls(pacc ~  age.c + edu.c + apoe4 + suvr.c +   # bl covs
    MONTH +                          # categorical time placebo trend
    months:active,                   # treatment difference over time
  correlation = corSymm(form = ~ visNo | id), # general correlation
  weights = varIdent(form = ~ 1 | visNo))     # heterogeneous variance
```
-->

---

# Natural Cubic Spline fit to ADNI

.pull-leftWider[

]

.pull-rightNarrower[

Natural Cubic Spline (Schoenberg, 1968):

+ Group 1 mean at `$t$`: `$f_1(t)$`

+ Group 2 mean at `$t$`: `$f_2(t)$`

+ `$f_1$` and `$f_2$` are two smooth functions

+ cubic basis functions

+ ``natural'' implies second derivatives are zero at boundaries

]

???

This model uses natural cubic splines to model the mean in each group. In this case we assume:

- cubic spline basis functions,  
- just one interior knot, and
- second derivatives are fix at 0 on the boundaries.

This constraint helps to avoids spurious flips of the curves near the last observations.

---

# Natural Cubic Spline model assumptions and fitting in R

$$
Y_i = X_i\beta + \epsilon_i,
$$

where:

+ `$Y_i$` is vector of observations (one for each visit) for individual `$i=1,\ldots,n$`
+ `$X_i$` is matrix of fixed effect covariates for subject `$i$` 
  + (including terms from the basis expansion)

Fit in `R`, using `nlme::gls`:

```r
gls(pacc ~  age.c + edu.c + apoe4 + suvr.c +   # bl covs
    ns(months, df=2) +        # natural spline for placebo (1 knot)
    ns(months, df=2):active + # natural spline for active (1 knot)
  correlation = corSymm(form = ~ visNo | id), # general correlation
  weights = varIdent(form = ~ 1 | visNo))     # heterogeneous variance
```

???

And once we have the basis expansion, this is also just a linear model which can 
be fit with gls.

One downside is that we have to pre-specify the number and location of spline knots.

Inference could be based on a likelihood ratio for the main group effect, or we could
derive contrasts at any given timepoint.

---

# Generalized Additive Mixed Model (GAMM) fit to ADNI

.pull-leftWider[

]

.pull-rightNarrower[

GAMM:

+ Group 1 mean at `$t$`: `$f_1(t)$`

+ Group 2 mean at `$t$`: `$f_2(t)$`

+ `$f_1$` and `$f_2$` are two smooth functions

+ Thin plate regression spline
  
  + Smoothness penalty tuned by Generalized Cross Validation (Wood, 2017)
  
]

???

The previous model could be considered a semiparametric Generalized Additive Mixed
Effect Model, or GAMM.

In this talk we will reserve GAMM to indicate a model with so-called Thin Plate Regression Splines for each group mean; and a smoothness penalty which is tuned by Generalized Cross Validation

---

# GAMM assumptions and fitting in R (Wood, 2017)

$$
Y_i = X_i\beta + f(Z_i) + \epsilon_i,
$$

where:

+ `$Y_i$` is vector of observations (one for each visit) for individual `$i=1,\ldots,n$`
+ `$X_i, Z_i$` are fixed effect covariates for subject `$i$`
+ `$f$` is a smooth semiparametric function to be estimated

Fit in `R`, using `mgcv::gamm`:

```r
gamm(pacc ~ age.c + edu.c + apoe4 + suvr.c +  # bl covs
  s(months) + s(months, by=active, pc=0),
  correlation = corSymm(form = ~ visNo | id), # general correlation
  weights = varIdent(form = ~ 1 | visNo))     # heterogeneous variance
```

`s(months, by=active, pc=0)` provides constraint such that `f(0)=0`.

???

This model can be fit with the `gamm` function in Simon Woods' mgcv package.

Unlike the prior approach, this approach does not require prespecification of
degrees of freedom, knot location, etc.

The generalized cross validation approach auto-tunes the smoothness penalty and
effective degrees of freedom.

---

# AIC for each model fit to ADNI

- All models fit with random intercept and slope

- Similar order with other correlation structures (when supported)

???

In this ADNI dataset, AIC seems to prefer MMRM, followed by the Natural Cubic Spline, Quadratic, Hybrid, GAMM, and DPM.

---

class: inverse, middle, center

## Anti-Amyloid Treatment in Asymptomatic Alzheimer's (A4) Trial:

## Simulated Impact of COVID-19 Hiatus

???

Switching gears now from the model demonstrations on actual ADNI data to a simulation study of the A4 trial

---

## Anti-Amyloid Treatment in Asymptomatic Alzheimer's (A4) Trial

- A4 is an ongoing randomized trial in a Preclinical AD population (approx 1150 randomized).

- Double-blind period is 4.5 years; monthly infusions; biannual cognitive testing.

- In March 2020, infusions and cognitive testing were paused for most A4 participants due to COVID-19.

- Participants have been returning to their sites and will be able to receive all of the originally planned monthly doses after their "hiatus".

???

Starting in March of this year, many A4 follow-up visits for monthly infusions and biannual cognitive testing were put on hiatus until conditions at each site would allow them to resume.

The idea of this hiatus is to allow each subject to have the opportunity to return to recieve the same number of infusions as originally planned.

As a result, the total follow-up for participants impacted by a hiatus will be longer than the originally planned 4.5-year follow-up.

This raises the question: how should we handle these post-hiatus delayed observations at the end of the double-blind period?

---

# A4 COVID-19 Hiatus Simulations

We simulated A4 trial data under 0, 3, 6, 9, and 12 month hiatus scenarios. The simulated scenarios assume:

- All visits scheduled to occur after 2020-03-16 are shifted by 0, 3, 6, 9, or 12 months.
- Placebo group simulated based on models fit to data from the Harvard Aging Brain Study (HABS).
  - Fixed effects included age, APOE, amyloid PET SUVR, PACC version
- Treatment group trajectories are simulated with a piecewise linear or quadratic functions (next slide)
- Attrition:
  - 30% total attrition for no hiatus
  - 32% total attrition for 3 month hiatus
  - 34% total attrition for 6 month hiatus
  - 36% total attrition for 9 month hiatus
  - 38% total attrition for 12 month hiatus

???

To address this question, we have been simulating A4 data imposing various hiatus durations.

We assume every visit projected to occur after March 16, 2020 is postponed by the same number of days. This is simplistic, and hopefully pessimistic for the longer scenarios.

The placebo trends, including the effect of alternating cognitive test versions, are simulated based on models fit to Harvard Aging Brain Study.

The assumed attrition rate is dependent on the hiatus duration as shown here.

---

# Assumed treatment effect over time

- Under no hiatus, assumes average week 240 benefit of about 0.73 PACC points (34% of HABS amyloid group difference).
- Residual standard deviation at week 240 about 4.0. (PACC as sum of 4 `$Z$`-scores)

<!--

# Handling time
**Calendar time (exam dates) approach**: using exam dates to calculate calendar time from baseline. Categorizing calendar time after the hiatus results in new time categories post 240 weeks.

**Dose schedule (visit codes) approach**: using the number of doses administered (i.e. visit codes) to categorize visits according to dose schedule, essentially *ignoring the hiatus*. This entails *carrying visits back* with respect to calendar time. No new time categories beyond week 240 are necessary.

-->

???

These are plots of the relative treatment benefit over time assumed in the simulations.

Relative treatment benefit is on the vertical axis.

Time from baseline is on the horizontal axis.

We assumed two different types of benefits: linear and quadratic.

The linear effect is immediate and constant. The quadratic effect starts more modestly and grows over time.

The A4 study increased the dose of Solanezumab from 400 mg to 1600 mg midway through.

The first vertical dashed line indicates when this simulated participant switched from low to high dose.

The second and third vertical dashed lines indicate when this pariticipant went on hiatus.

On the left, under no hiatus, we see both trends result in the same benefit at week 240.

In the middle, we assume no benefit during this exaggerated 360-day hiatus, and then a return to the high-dose benefit after hiatus.

On the right, we assume a residual benefit (about 50% as strong) during the hiatus, 
and a greater net benefit as a result.

---

# Visit codes vs exam dates

???

Here are MMRM trends from a simulated trial with a (hopefully) pessimistic 360-day hiatus.

On the left, we ignore the hiatus, and categorize time according to the original planned visit codes.

This framework results in carrying observations back in time in the analysis.

One the right, we use actual exam dates, respecting the hiatus, and therefore must re-categorize time accordingly.

We end up with more time categories, and fewer observations per category.

P-values are for constrasts at week 168 (pre-COVID), week 240 (as originally planned), and week 288 (the new final time category).

---

# PACC test versions can get mixed together using exam dates

For example, with a 180 day hiatus, we might see these version counts:

- With a categorical time analysis model, it would be awkward/difficult/impossible to control for version effects.
- With a continuous time analysis model, assuming a reasonable parametric mean structure over time, we can simply add a fixed effect for version.

???

One drawback of categorizing time under the exam date approach is that PACC test versions will get mixed together within the same visit category.

Here, based on simulated data, we see counts of alternating test versions (A vs B).

From week 24 to week 144, all participants were administered the same test version per visit. 
But after the hiatus, PACC versions begin to get mixed together at the same visit.

This would likely increase the unexplained variance in a categorical time based MMRM.

But under a continuous time model imposing some structure on the mean trend over time, like GAMM, a test version effect which should be EStimable

---

# MMRM vs GAMM fit to simulated A4 data

???

On the right is what the GAMM fit looks like from a typical simulated dataset.

The simulated PACC version effect is apparent from the MMRM on the left. The GAMM controls for PACC version, and hides its effect in the plot on the right.

On the left, again, are p-values for contrasts at Weeks 168, 240, and the last time category, 288.

On the right, the p-value is from a Chi-square test for the regression
spline controlling the treatment group difference over time.

---

# MMRM vs GAMM (representing version effect)

???

We can also plot the GAMM trends demonstrating the PACC version effect, if desired,
as shown here on the right.

Note that the version effect is assumed to be constant over time in this model.

---

# Power based on 1,000 simulated trials

GAMM is more powerful than MMRM and preserves power better with prolonged hiatus

???

These are simulated power results from 1,000 trials.

Power is on the vertical axis. Hiatus duration is on the x-axis.

Again, we hope that a hiatus longer than 90 days for *all* participants is pessimistic.

The top row is for the stronger, immediate linear treatment effect.

The bottom row is for the delayed onset quadratic treatment effect.

The first two columns are from the MMRM using exam dates, and group contrasts at either week 240, or the last visit.

The third column shows the power for MMRM ignoring the hiatus and using visit codes.

The far-right column is using the GAMM-based Chi-square test.

GAMM improves power by about 10% even under no hiatus, and sustains power better under longer hiatus scenarios.

Using the visit codes (third columns) seems to preserve power better, likely because this approach does not mix PACC versions at the same visit or deplete the number of observations per category.

However, carrying visits back in time might introduce bias, similar to the bias induced by Last Observation Carried Forward approaches.

---

# Summary

- AIC chooses MMRM as the most predictive model in ADNI data, followed by 
Natural Cubic Spline, Quadratic, Hybrid, GAMM, and DPM approaches.

- The COVID-19 hiatus introduces some novel modeling issues for A4, and in particular, challenges for the MMRM framework.

- Regression splines are an attractive continuous time approach, with 
  - Flexible assumptions about group trends over time
  - Adjustment for test version effects

- Under no hiatus, the GAMM approach with an ANOVA test for the main effect of treatment provides more power (95-97%) compared to MMRM timepoint-specific contrasts (82-85%)

- With a 90 day hiatus, the GAMM approach also provides more power (90-97%) compared to MMRM (59-82%). Difference grows with longer hiatuses.

???

To sum up, we found that model selection based on AIC applied to natural history ADNI data, indicates a preference for MMRM, with Natural Cubic Splines, Quadratic, Hybrid, GAMM, and DPM following in order of preference.

The COVID-19 hiatus introduces some novel modeling issues, and particular challenges for the MMRM framework.

Regression splines are an attractive approach, with flexible assumptions about the trends over time.

Simulations of A4 indicate GAMM might improve power relative to MMRM by about 10%, under no hiatus, and better sustain power under exaggerated hiatus durations.

---

# Summary (continued)

- Type I error was well controlled for the main effect tests derived from GAMM in these scenarios, but more recent simulations have revealed some possible Type I error inflation.

- Natural Cubic Splines:
  - Might provide more reliable Type I error control
  - But they require pre-specifying knots

- Primary test?
  - Main effect or timepoint contrast?
  - Last timepoint? How do we ensure sufficient data?
  - Originally planned timepoint (week 240)?
  - Area between curves after week 240?

- Investigation continues. No decision to change A4 analysis plan has been made.

???

Though Type I error was well controlled for the GAMM Chi-square test of the main effect
in this simulation, more recent work has revealed inflated error rates in other scenarios.

As a result, we will be exploring the Natural Cubic Spline approach, which we hope
will provide more robust Type I error control.

But we will have to pre-specify knot locations.

There is not an obvious best choice for a timepoint contrast, which makes the main
effect tests attractive.

We are also considering the area betwen the curves after week 240.

So the outlines of our approach are starting to take shape, but we obviously have
more work to do; and we gladly welcome any suggestions or recommendations.

---

# Thank you!

- Collaborators:

- Gopalan Sethuraman, Oliver Langford, Rema Raman, Wenyi Lin, Philip Insel, Wesley K Thompson, Kathryn Papp, Reisa Sperling, Paul Aisen
  - Scott Andersen, Saptarshi Chatterjee, JonDavid Sparks, Michael Case, Thomas Jensen, Hong Liu-Seifert

- ADNI, HABS, and A4 study teams, investigators, and participants.

???

Finally, thank you to my collaborators at USC and Lilly; and all of the study teams
and study participants invovled.

---

# References

Hastie, T. J. et al. (1990). _Generalized additive models_. Vol. 43.
CRC press.

Mallinckrodt, C. H. et al. (2001). "Accounting for dropout bias using
mixed-effects models". In: _Journal of biopharmaceutical statistics_
11.1-2, pp. 9-21.

Schoenberg, I. J. (1968). "Spline functions and the problem of
graduation". In: _Proceedings of the National Academy of Sciences_ 52,
pp. 947-950.

Sperling, R. A. et al. (2014). "The A4 study: stopping AD before
symptoms begin?" In: _Science translational medicine_ 6.228, pp.
228fs13-228fs13.

Wahba, G. (1990). _Spline models for observational data_. SIAM.

Wang, G. et al. (2018). "A novel cognitive disease progression model
for clinical trials in autosomal-dominant Alzheimer's disease". In:
_Statistics in medicine_ 37.21, pp. 3047-3055.

Wood, S. N. (2017). _Generalized additive models: an introduction with
R_. CRC press.