#+TITLE: Overview of the package LMMstar
#+Author: Brice Ozenne
#+BEGIN_SRC R :exports none :results silent :session *R* :cache no
options(width = 100, digits = 5)
if(system("whoami",intern=TRUE)=="bozenne"){  
  setwd("~/Documents/GitHub/LMMstar/inst/doc-overview/")
}else if(system("whoami",intern=TRUE)=="unicph\\hpl802"){  
  setwd("c:/Users/hpl802/Documents/Github/LMMstar/inst/doc-overview/")
}
library(ggpubr, quietly = TRUE, verbose = FALSE, warn.conflicts = FALSE)
#+END_SRC

This vignette describes the main functionalities of the *LMMstar*
package. This package implements specific types of linear mixed
models, mainly useful when having repeated observations over a
discrete variable: \(\VY=(Y_1,\ldots,Y_T)\) where \(T\) can be
for example be time (chronological ordering of the repetitions) or
brain region (arbitrary ordering of the repetitions). Denoting by
\(\VX\) the associated covariates and \(\boldsymbol{\varepsilon} =
(\varepsilon_1,\ldots,\varepsilon_T)\), the model can be written:
#+BEGIN_EXPORT latex
\[ \VY = \VX \beta + \boldsymbol{\varepsilon} \text{ where } \varepsilon \sim \Gaus(0,\Omega) \]
#+END_EXPORT
where \(\beta\) are the mean parameters and the residual
variance-covariance matrix, \(\Omega\), depends on a set of
variance-covariance parameters (say \(\gamma\)) distinct of
\(\beta\). Key assumptions are:
- we observe \(n\) independent replicates of
  \(\mathcal{O}=(\VY,\VX)\), i.e. at the cluster level,
  observations \(\left(\mathcal{O}_1,\ldots,\mathcal{O}_n\right)\) are
  independent. The replicates should also be identically distributed
  up to a categorical variable (called strata variable in the following).
- the residual variance is independent of the mean value.
Additional assumptions are necessary in presence of missing values,
typically correct specification of the conditional mean to have
consistent estimates of the mean parameters. This case will sometimes
be examplified by considering that only last outcome may be missing:
the conditional mean \(\Esp[Y_T|Y_1,Y_2,\ldots,Y_{T-1}]\) is then
abreviated as \(\Esp[Y_T|Y_{T-1}]\). Note that we do not require the
residuals to be normally distributed to have valid estimates or
statistical inference in large samples.

\bigskip

To get start, one should load the *LMMstar* package in the \Rlogo session:
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
library(LMMstar)
#+END_SRC

This package is under active development. Newer package versions may
include additional functionalities and fix previous bugs. The version
of the package that is being used for this overview is:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
utils::packageVersion("LMMstar")
#+END_SRC

#+RESULTS:
: [1] ‘1.1.0’

It is recommanded to also the following packages, as some of the
methods implemented in the package are relative to a generic method
implmented in other packages:
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
library(ggplot2) ## autoplot method
library(nlme) ## ranef method
library(lava) ## iid, information, manifest methods
#+END_SRC

\clearpage

The user interface of the *LMMstar* package is made of the following
functions:
- functions to describe or visualize the dataset: 
    + =scatterplot= to visualize the marginal and bivariate distribution of continuous variables.
    + =summarize= to compute summary statistics, possibly stratified on a categorical variable.
    + =summarizeNA= to identify missing data patterns.
    + =partialCor= to compute partial correlation between two variables.
- the function =mt.test= to perform multiple Student's t-Tests and
  adjust the results for multiple testing.
- the function =lmm= is the main function of the package which fits
  linear mixed models. The user can interact with /lmm/ objects using:
    + =anova= to perform Wald tests, i.e. test linear combinations of
      coefficients (\(\widehat{\beta}_1+\widehat{\beta}_2=0\) or
      \(\widehat{\beta}_1=\widehat{\beta}_2=0\)). The output obtained
      with different =lmm= can be combined using =rbind=.
    + =coef= to extract the estimated model parameters (\(\widehat{\beta}\) and possibly \(\widehat{\gamma}\)).
    + =confint= to extract the estimates with their confidence intervals.
    + =effects= to evaluate marginal effects, e.g. \(\Esp\left[\Esp[Y|X_1=1]-\Esp[Y|X_1=0]\right]\) when \(\VX=(X_1,X_2)\).
    + =estimate= to test non-linear combinations of coefficients (Wald test via a first order delta method, e.g. \(\widehat{\beta}_1/\widehat{\beta}_2=1\)).
    + =fitted= to output the fitted mean (\(X\widehat{\beta}\)) or the
      conditional mean for observations with missing outcome
      (e.g. \(X\widehat{\beta} +
      \widehat{\Esp}[\varepsilon_T|\varepsilon_{-T}]\)).
    + =iid= to extract the influence function of the estimated
      parameters (\(\varphi\)), which satisfies \newline
      \(\sqrt{n}(\widehat{\beta}-\beta) = \frac{1}{\sqrt{n}}
      \sum_{i=1}^n \varphi\left(\mathcal{O}_i\right) +o_p(1)\)
    + =levels= to extract the reference level for the mean structure.
      (i.e. what =(Intercept)= refers to in presence of categorical.
      covariates).
    + =logLik= to output the log-likelihood of the estimated model.
    + =model.tables= to extract the estimates, standard errors, p-value, and confidence intervals.
    + =plot= to obtain a diagnostic plots, partial residual plots, or a graphical display of the fitted values.
    + =predict= to compute the mean conditional on covariates and
      possible outcome values.
    + =profile= to display the likelihood or profile likelihood of the model.
    + =resample= to use non-parametric bootstrap or permutation test for statistical inference.
    + =residuals= to extract the observed residuals of the fitted
      model, possibly normalized
      (\(\widehat{\Omega}^{-\frac{1}{2}}\widehat{\varepsilon}\)).
    + =sigma= to extract the modeled residual variance covariance matrix (\(\widehat{\Omega}\)).
    + =summary= to obtain a summary of the input, model fit, and estimated values.
    + =vcov= to extract the variance-covariance matrix of the mean
      parameters (\(\widehat{\Sigma}_{\widehat{\beta}}\)).
- the =mlmm= function to fit group-specific linear mixed models and
  gather the estimated coefficients.
- the =sampleRem= function to simulate longitudinal data.
- the =LMMstar.options= function enables the user to display the
  default values used in the *LMMstar* package. The function
  can also change the default values to better match the user needs.


\clearpage

* Illustrative dataset

To illustrate the functionalities of the package, we will use the
gastricbypass dataset. The long format can be imported using:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
data(gastricbypassL, package = "LMMstar")
head(gastricbypassL)
#+END_SRC

#+RESULTS:
:   id visit time weight glucagonAUC
: 1  1     1  -13  127.2      20.690
: 2  2     1  -13  165.2      49.922
: 3  3     1  -13  109.7      42.434
: 4  4     1  -13  146.2      27.517
: 5  5     1  -13  113.1      29.151
: 6  6     1  -13  158.8      42.700

See =?gastricbypassL= for a presentation of the dataset. It is
convenient to encode the time variable in two formats:
- numeric, e.g. here with the time in week since surgery (=time=
  variable taking values -13,-1,1,13 - negative times refering to
  before an intervention and positive times after the intervention).
- factor, e.g. here with the visit index (=visit= variable taking
  value 1,2,3,4)

To illustrate certain functionalities we will use an (artificial)
group variable:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassL$group <- as.factor(as.numeric(gastricbypassL$id)%%2)
#+END_SRC

#+RESULTS:

and dichotomize time as before and after the intervention:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassL$baseline <- gastricbypassL$time<0
#+END_SRC

#+RESULTS:

The corresponding wide format is
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
data(gastricbypassW, package = "LMMstar")
head(gastricbypassW)
#+END_SRC

#+RESULTS:
:   id weight1 weight2 weight3 weight4 glucagonAUC1 glucagonAUC2 glucagonAUC3 glucagonAUC4
: 1  1   127.2   120.7   115.5   108.1       20.690       20.535       92.600       43.434
: 2  2   165.2   153.4   149.2   132.0       49.922       58.513       49.633       35.747
: 3  3   109.7   101.6    97.7    87.1       42.434       25.770       91.240       83.137
: 4  4   146.2   142.4   136.7   123.0       27.517       27.552       59.360       21.371
: 5  5   113.1   105.6    99.9    87.7       29.151           NA       86.859       57.970
: 6  6   158.8   143.6   134.6   108.7       42.700       31.616       53.408       37.636

for which we can also add the group variable:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassW$group <- as.numeric(gastricbypassW$id)%%2
#+END_SRC

#+RESULTS:

In some cases we will, for comparison, perform complete case analyses
with the following dataset:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassL.NNA <- gastricbypassL[!is.na(gastricbypassL$glucagonAUC),]
#+END_SRC

#+RESULTS:

\clearpage

* Visualization & descriptive statistics
** Graphical display

A scatterplot of the data can obtained by specifying which columns to
display when using the wide format:
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
scatterplot(gastricbypassW, ## left panel
            columns = c("weight1","weight2","weight3","weight4")) 
#+END_SRC

#+RESULTS:

\noindent When using the long format, a formula should describe the
structure of the data: =outcome ~ order|cluster=
- the left hand side indicates the values to be displayed (here weight)
- the right hand side indicates the ordering of the repetitions (here over time) and
  how the repetitions are grouped within clusters (here within subject).

When calling =scatterplot=, the argument =group= leads to different
color per group and the argument =type.diag= enables to use histograms
(or density plots) instead of boxplots:
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
scatterplot(weight~time|id, data = gastricbypassL, ## right panel
            type.diag = "hist", group = "group")
#+END_SRC

#+RESULTS:


\bigskip

#+LaTeX: \begin{minipage}{0.48\linewidth} 
#+ATTR_LaTeX: :width \textwidth :options trim={0 0 0 0} :placement [!h]
[[./figures/scatterplot.pdf]]
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.48\linewidth} 
#+ATTR_LaTeX: :width \textwidth :options trim={0 0 0 0} :placement [!h]
[[./figures/scatterplot-group.pdf]]
#+LaTeX: \end{minipage}


#+RESULTS:

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
pdf("./figures/scatterplot.pdf", width = 6.5, height = 6)
scatterplot(weight~time|id, data = gastricbypassL, size.cor = 6)
dev.off()

pdf("./figures/scatterplot-group.pdf", width = 6.5, height = 6)
scatterplot(weight~time|id, data = gastricbypassL, size.cor = 6, type.diag = "hist", group = "group")
dev.off()
#+END_SRC


#+RESULTS:
: null device 
:           1
: null device 
:           1

\bigskip

By default the resulting object will be of class =list=. A =ggplot2=
object can be obtained by setting the argument =facet= to
="grid2"=. This requires to have installed the package ggh4x and will
produce a slightly different graphical display.

\bigskip

There is (currently) not dedicated function to obtain spaghetti
plots. Instead one can use the ggplot2 package with the long format, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gg.spa <- ggplot(gastricbypassL, aes(x=time,y=weight,group=id,color=id))
gg.spa <- gg.spa + geom_point() + geom_line()
gg.spa
#+END_SRC

#+RESULTS:

\clearpage

** Missing data patterns

The =summarizeNA= function identifies the possible combinations of
observed/missing data:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
mp <- summarizeNA(gastricbypassL)
mp
#+END_SRC

#+RESULTS:
:  frequency missing.pattern n.missing id visit time weight glucagonAUC group baseline
:         78         0000000         0  0     0    0      0           0     0        0
:          2         0000100         1  0     0    0      0           1     0        0

A graphical representation can be obtained using =plot=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
plot(mp)
#+END_SRC

#+RESULTS:

See =help(plot.summarizeNA)= for options to customize the graphical
display.

#+RESULTS:

#+ATTR_LaTeX: :width 1\textwidth :options trim={0 0 0 0} :placement [!h]
[[./figures/summarizeNA.pdf]]



#+BEGIN_SRC R :exports none :results output :session *R* :cache no
ggsave(autoplot(mp)$plot, filename = "./figures/summarizeNA.pdf", width = 12)
#+END_SRC
#+RESULTS:

\clearpage

** Summary statistics

Mean, standard deviation, and other summary statistic can be computed
with respect to a categorical variable (typically time) using the
=summarize= function: \newline (\Warning this function has the same
name as a function from the dplyr package. If you have loaded dplyr,
you should use =LMMstar:::summarize=)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sss <- summarize(weight+glucagonAUC ~ time, data = gastricbypassL, na.rm = TRUE)
print(sss, digits = 3)
#+END_SRC

#+RESULTS:
:       outcome time observed missing  mean   sd    min    q1 median    q3   max
: 1      weight  -13       20       0 129.0 20.3 100.90 115.3  123.1 139.8 173.0
: 2               -1       20       0 121.2 18.9  95.70 107.8  114.5 134.5 162.2
: 3                1       20       0 115.7 18.3  89.90 102.2  110.6 128.4 155.0
: 4               13       20       0 102.4 17.1  78.80  90.4   98.5 108.2 148.0
: 5 glucagonAUC  -13       20       0  32.3 15.5  10.28  21.3   27.9  42.5  69.1
: 6               -1       19       1  29.7 13.7   9.87  21.2   25.8  33.6  67.7
: 7                1       19       1  76.9 27.9  35.85  56.5   73.8  91.9 135.9
: 8               13       20       0  52.0 21.0  21.37  37.2   51.2  57.9 109.2

\noindent Specifying a cluster (=id=) and ordering variable (=time=)
enable to output correlation matrices: \newline (\Warning there should be
no duplicated value of the ordering variable within cluster)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sss2 <- summarize(weight ~ time|id, data = gastricbypassL, na.rm = TRUE)
print(sss2, digits = 3)
#+END_SRC

#+RESULTS:
#+begin_example
  time observed missing mean   sd   min    q1 median  q3 max
1  -13       20       0  129 20.3 100.9 115.3  123.1 140 173
2   -1       20       0  121 18.9  95.7 107.8  114.5 135 162
3    1       20       0  116 18.3  89.9 102.2  110.6 128 155
4   13       20       0  102 17.1  78.8  90.4   98.5 108 148

 Pearson's correlation: 
      -13    -1     1    13
-13 1.000 0.990 0.986 0.946
-1  0.990 1.000 0.997 0.959
1   0.986 0.997 1.000 0.966
13  0.946 0.959 0.966 1.000
#+end_example

Graphical displays of the summary statistics can be obtained via the
=plot= method, where the argument =type= specifies the summary
statistic to be displayed:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
plot(sss2, type = "mean") ## left panel
plot(sss2, type = "sd") ## middle panel
plot(sss2, type = "cor") ## right panel
#+END_SRC

#+RESULTS:

See =help(plot.summarize)= for options to customize the graphical
display.

#+ATTR_LaTeX: :width 1\textwidth :options trim={0 0 0 0} :placement [!h]
[[./figures/summarize.pdf]]

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
pdf("./figures/summarize.pdf", width = 12)
ggarrange(autoplot(sss2,type="mean")$plot + ggtitle("mean"),
          autoplot(sss2,type="sd")$plot + ggtitle("sd"),
          autoplot(sss2,type="cor")$plot + ggtitle("cor") + theme(legend.position ="bottom"),
          nrow = 1)
dev.off()
#+END_SRC

#+RESULTS:
: X11cairo 
:        2

\clearpage

** Correlation and partial correlations

The =partialCor= function can be used to evaluate group-specific
correlations, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
partialCor(weight + glucagonAUC ~ 1, by = "group", data = gastricbypassL)
#+END_SRC

#+RESULTS:
:                            estimate    se   df  lower    upper p.value
: 0: rho(weight,glucagonAUC)   -0.328 0.143 21.8 -0.587 -0.00886  0.0447
: 1: rho(weight,glucagonAUC)   -0.354 0.141 22.5 -0.607 -0.03631  0.0313

This willl lead to the same estimate as the =cor.test= function
(Pearson correlation):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassL.0 <- gastricbypassL[gastricbypassL$group==0,]
rho <- cor.test(gastricbypassL.0$weight, gastricbypassL.0$glucagonAUC)
c(rho$estimate, p.value = rho$p.value)
#+END_SRC

#+RESULTS:
:       cor   p.value 
: -0.328481  0.038505

However the p-value may differ, especially in small samples, as
=partialCor= uses a different (and probably more crude) small sample
approximation for the estimator's distribution. Nevertheless
=partialCor= enables to compare correlation coefficients across
groups, by specifying the argument =effects=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
partialCor(weight + glucagonAUC ~ 1, by = "group", effects = "Dunnett",
           data = gastricbypassL)
#+END_SRC

#+RESULTS:
:                                                       estimate se df lower upper p.value
: 1:rho(weight,glucagonAUC) - 0:rho(weight,glucagonAUC)  -0.0255 NA NA    NA    NA   0.899


Partial correlations can be also computed by specifying covariate to
adjust for on the right-hand side:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
partialCor(weight4 + glucagonAUC4 ~ weight1,
           data = gastricbypassW)
#+END_SRC

#+RESULTS:
:                           estimate    se   df  lower upper p.value
: rho(weight4,glucagonAUC4)    0.112 0.233 9.12 -0.397 0.568   0.645

When the set of covariates is outcome-dependent, a list of formulas
can be used instead:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
partialCor(list(weight1 ~ glucagonAUC1, weight4 ~ glucagonAUC4),
           data = gastricbypassW)
#+END_SRC

#+RESULTS:
:                      estimate     se   df lower upper  p.value
: rho(weight1,weight4)    0.946 0.0252 26.4 0.861 0.979 5.51e-08

These partial correlations are defined as the residual correlation
between the outcomes, i.e. the correlation once the covariate effects
have been substracted from the outcome, and a linear mixed model is
used to estimated them.

\clearpage

* Multiple Student's t-tests

When working with multiple outcomes and having no missing data, mean
comparisons between exposure groups can be carried out using Student's
t-tests at each timepoint, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
restt <- t.test(weight1 ~ group, data = gastricbypassW)
c(estimate = unname(diff(restt$estimate)), p.value = restt$p.value)
#+END_SRC

#+RESULTS:
:  estimate   p.value 
: -10.60000   0.25282

And so on for the three other timepoints. Morever results would
typically need to be adjusted for multiple comparisons, e.g. when
looking for any mean difference. This can be faciliated by
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
## single step max-test adjustment (see help(confint.Wald_lmm) for details)
mt.test(weight1+weight2+weight3+weight4~group, data = gastricbypassW)
#+END_SRC

#+RESULTS:
:        by parameter estimate     se     df   lower   upper p.value
: 1 weight1     group   -10.60 8.9717 17.965 -30.968  9.7680 0.31894
: 2 weight2     group    -9.50 8.3951 17.985 -28.559  9.5590 0.34164
: 3 weight3     group    -8.92 8.1295 17.959 -27.376  9.5358 0.35891
: 4 weight4     group    -4.59 7.7607 17.682 -22.209 13.0286 0.66331

The method used to adjust confidence intervals and p-values for
multiple comparisons can be specified via the =method= argument, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
## no adjustment
mt.test(weight1+weight2+weight3+weight4~group, data = gastricbypassW, method = "none")
#+END_SRC

#+RESULTS:
:        by parameter estimate     se     df   lower   upper p.value
: 1 weight1     group   -10.60 8.9717 17.965 -29.452  8.2516 0.25281
: 2 weight2     group    -9.50 8.3951 17.985 -27.139  8.1386 0.27266
: 3 weight3     group    -8.92 8.1295 17.959 -26.002  8.1622 0.28703
: 4 weight4     group    -4.59 7.7607 17.682 -20.916 11.7356 0.56171

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
## bonferroni adjustment
mt.test(weight1+weight2+weight3+weight4~group, data = gastricbypassW, method = "bonferroni")
#+END_SRC

#+RESULTS:
:        by parameter estimate     se     df   lower  upper p.value
: 1 weight1     group   -10.60 8.9717 17.965 -35.498 14.298       1
: 2 weight2     group    -9.50 8.3951 17.985 -32.795 13.795       1
: 3 weight3     group    -8.92 8.1295 17.959 -31.481 13.641       1
: 4 weight4     group    -4.59 7.7607 17.682 -26.165 16.985       1


\clearpage

* Linear mixed model (LMM)
** Classical covariance patterns

Several build-in covariance patterns can be used when specifying the
linear model. The most basic ones are the *identity* structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eId.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id, 
               structure = "ID", data = gastricbypassL)
eId.lmm
cat(" modeled residual variance-covariance: \n");sigma(eId.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
		Linear regression 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       1 variance (sigma) 
 log-restr.likelihood: -316.461119970244 
 convergence         : TRUE (0 iterations)
 modeled residual variance-covariance: 
       -13     -1      1     13
-13 381.35   0.00   0.00   0.00
-1    0.00 381.35   0.00   0.00
1     0.00   0.00 381.35   0.00
13    0.00   0.00   0.00 381.35
#+end_example

and the *independence* structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eInd.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id, 
                structure = "IND", data = gastricbypassL)
eInd.lmm
cat(" modeled residual variance-covariance: \n");sigma(eInd.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
		Linear regression with heterogeneous residual variance 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       4 variance (sigma k.-1 k.1 k.13) 
 log-restr.likelihood: -310.428096419287 
 convergence         : TRUE (0 iterations)
 modeled residual variance-covariance: 
       -13     -1      1     13
-13 209.44   0.00   0.00   0.00
-1    0.00 174.81   0.00   0.00
1     0.00   0.00 768.23   0.00
13    0.00   0.00   0.00 382.95
#+end_example

\clearpage

The most common linear mixed model uses a *compound symmetry* structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eCS.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
               structure = "CS", data = gastricbypassL)
eCS.lmm
cat(" modeled residual variance-covariance: \n");sigma(eCS.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
		Linear Mixed Model with a compound symmetry covariance matrix 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       1 variance (sigma) 
                       1 correlation (rho(id)) 
 log-restr.likelihood: -314.394203759159 
 convergence         : TRUE (6 iterations)
 modeled residual variance-covariance: 
        -13      -1       1      13
-13 380.580  82.741  82.741  82.741
-1   82.741 380.580  82.741  82.741
1    82.741  82.741 380.580  82.741
13   82.741  82.741  82.741 380.580
#+end_example

\noindent A more flexible model can be obtained with a *toeplitz* covariance matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eTOE.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
                structure = "TOEPLITZ", data = gastricbypassL)
eTOE.lmm
cat(" modeled residual correlation: \n");cov2cor(sigma(eTOE.lmm))
#+END_SRC

#+RESULTS:
#+begin_example
		Linear Mixed Model with a block Toeplitz covariance matrix 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       4 variance (sigma k.-1 k.1 k.13) 
                       4 correlation (rho(12) rho(14) rho(26) rho(2)) 
 log-restr.likelihood: -297.525485582536 
 convergence         : TRUE (15 iterations)
 modeled residual correlation: 
          -13       -1        1        13
-13  1.000000 0.700020 0.093615 -0.082963
-1   0.700020 1.000000 0.016795  0.093615
1    0.093615 0.016795 1.000000  0.700020
13  -0.082963 0.093615 0.700020  1.000000
#+end_example

\clearpage

\noindent And an even more flexible model can be obtained with an
*unstructured* covariance matrix:

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
               structure = "UN", data = gastricbypassL)
eUN.lmm
cat(" modeled residual variance-covariance: \n");sigma(eUN.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
		Linear Mixed Model with an unstructured covariance matrix 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       4 variance (sigma k.-1 k.1 k.13) 
                       6 correlation (rho(-13,-1) rho(-13,1) rho(-13,13) rho(-1,1) rho(-1,13) rho(1,13)) 
 log-restr.likelihood: -295.314056198772 
 convergence         : TRUE (8 iterations)
 modeled residual variance-covariance: 
        -13       -1        1      13
-13 209.442 150.2502 106.4000 -24.202
-1  150.250 168.1138   1.3064 -23.884
1   106.400   1.3064 748.0769 288.184
13  -24.202 -23.8844 288.1839 382.952
#+end_example

\noindent Stratification of the covariance structure on a categorical
variable is also possible:
- e.g. to get a *stratified compound symmetry*
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eSCS.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
                structure = CS(group~1), data = gastricbypassL)
eSCS.lmm
#+END_SRC

#+RESULTS:
: 		Linear Mixed Model with a stratified compound symmetry covariance matrix 
: 
:  outcome/cluster/time: glucagonAUC/id/time 
:  data                : 78 observations from 20 clusters 
:  parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
:                        2 variance (sigma:0 sigma:1) 
:                        2 correlation (rho(id):0 rho(id):1) 
:  log-restr.likelihood: -314.123797063042 
:  convergence         : TRUE (7 iterations)

\clearpage

- e.g. *stratified unstructured* covariance matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eSUN.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
                structure = UN(group~1), data = gastricbypassL)
eSUN.lmm
#+END_SRC
#+RESULTS:
: 		Linear Mixed Model with a stratified unstructured covariance matrix 
: 
:  outcome/cluster/time: glucagonAUC/id/time 
:  data                : 78 observations from 20 clusters 
:  parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
:                        8 variance (sigma:0 sigma:1 k.-1:0 k.1:0 k.13:0 k.-1:1 k.1:1 k.13:1) 
:                        12 correlation (rho(-13,-1):0 rho(-13,1):0 rho(-13,13):0 rho(-1,1):0 rho(-1,13):0 rho(1,13):0 rho(-13,-1):1 rho(-13,1):1 rho(-13,13):1 rho(-1,1):1 rho(-1,13):1 rho(1,13):1) 
:  log-restr.likelihood: -286.536815485471 
:  convergence         : TRUE (10 iterations)

with modeled residual variance-covariance:

\bigskip

#+LaTeX: \begin{minipage}{0.47\linewidth} 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigma(eSCS.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
$`0`
        -13      -1       1      13
-13 334.289  50.782  50.782  50.782
-1   50.782 334.289  50.782  50.782
1    50.782  50.782 334.289  50.782
13   50.782  50.782  50.782 334.289

$`1`
       -13     -1      1     13
-13 428.46 115.09 115.09 115.09
-1  115.09 428.46 115.09 115.09
1   115.09 115.09 428.46 115.09
13  115.09 115.09 115.09 428.46
#+end_example
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.47\linewidth} 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigma(eSUN.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
$`0`
       -13      -1       1      13
-13 309.85 251.512 102.189 -42.250
-1  251.51 274.752 -79.811 -90.718
1   102.19 -79.811 579.110 163.767
13  -42.25 -90.718 163.767 173.439

$`1`
         -13     -1       1       13
-13 109.0309 48.667 104.908  -6.1549
-1   48.6665 59.395  93.976  43.2144
1   104.9077 93.976 967.583 450.8899
13   -6.1549 43.214 450.890 592.4655
#+end_example
#+LaTeX: \end{minipage}

\clearpage

\noindent Finally the some covariance patterns like the compound
symmetry structure may depend on covariates:
- e.g. to obtain a *block compound symmetry* structure[fn::similar to
  nested random effects]:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eBCS.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
                structure = CS(~baseline, type = "homogeneous"), data = gastricbypassL)
eBCS.lmm
cat(" modeled residual variance-covariance: \n");sigma(eBCS.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
		Linear Mixed Model with a block compound symmetry covariance matrix 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       1 variance (sigma) 
                       2 correlation (rho(id/baseline) rho(id)) 
 log-restr.likelihood: -308.994835006264 
 convergence         : TRUE (6 iterations)
 modeled residual variance-covariance: 
        -13      -1       1      13
-13 380.957 226.403  15.465  15.465
-1  226.403 380.957  15.465  15.465
1    15.465  15.465 380.957 226.403
13   15.465  15.465 226.403 380.957
#+end_example

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
library(lme4)
e.lmer <- lmer(glucagonAUC ~ visit*group + (1|id/baseline), data = gastricbypassL)
logLik(e.lmer)
#+END_SRC

#+RESULTS:
: 'log Lik.' -308.99 (df=11)

- e.g. to obtain a *block unstructured* covariance matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eBUN.lmm <- lmm(glucagonAUC ~ visit*group, repetition = ~time|id,
                structure = CS(~baseline, type = "heterogeneous"), data = gastricbypassL)
eBUN.lmm
cat(" modeled residual variance-covariance: \n");sigma(eBUN.lmm)
#+END_SRC

#+RESULTS:
#+begin_example
		Linear Mixed Model with a block unstructured covariance matrix 

 outcome/cluster/time: glucagonAUC/id/time 
 data                : 78 observations from 20 clusters 
 parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
                       2 variance (sigma k.TRUE) 
                       3 correlation (rho(FALSE) rho(FALSE,TRUE) rho(TRUE)) 
 log-restr.likelihood: -300.047474124556 
 convergence         : TRUE (7 iterations)
 modeled residual variance-covariance: 
        -13      -1       1      13
-13 189.420 150.356  15.353  15.353
-1  150.356 189.420  15.353  15.353
1    15.353  15.353 570.908 300.071
13   15.353  15.353 300.071 570.908
#+end_example

\clearpage

** User-specific covariance patterns

It is possible input user-specific covariance patterns under the
following model for the residuals: \[\Omega =
\trans{\boldsymbol{\sigma}} R \boldsymbol{\sigma}\]
- \(\boldsymbol{\sigma}=f(\boldsymbol{\theta}_{\sigma},Z_{\sigma})\)
  is a vector of residual standard errors depending on a vector of
  parameters \(\boldsymbol{\theta}_{\sigma}\) and possible covariates
  via the design matrix \(Z_{\sigma}\). 
- \(R=g(\boldsymbol{\theta}_{R},Z_R)\) is a matrix of residual
  correlations depending on a vector of parameters
  \(\boldsymbol{\theta}_{R}\) and possible covariates via the design
  matrix \(Z_R\).

\bigskip

To be more concrete, consider the following correlation matrix
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
rho.2block <- function(p,n.time,X){
  rho <- matrix(1, nrow = n.time, ncol = n.time)
  rho[1,2] <- rho[2,1] <- rho[4,5] <- rho[5,4] <- p["rho1"]
  rho[1,3] <- rho[3,1] <- rho[4,6] <- rho[6,4] <- p["rho2"]
  rho[2,3] <- rho[3,2] <- rho[5,6] <- rho[6,5] <- p["rho3"]
  rho[4:6,1:3] <- rho[1:3,4:6] <- p["rho4"]
  return(rho)
}
Rho <- rho.2block(p = c(rho1=0.25,rho2=0.5,rho3=0.4,rho4=0.1),
                  n.time = 6)
Rho
#+END_SRC

#+RESULTS:
:      [,1] [,2] [,3] [,4] [,5] [,6]
: [1,] 1.00 0.25  0.5 0.10 0.10  0.1
: [2,] 0.25 1.00  0.4 0.10 0.10  0.1
: [3,] 0.50 0.40  1.0 0.10 0.10  0.1
: [4,] 0.10 0.10  0.1 1.00 0.25  0.5
: [5,] 0.10 0.10  0.1 0.25 1.00  0.4
: [6,] 0.10 0.10  0.1 0.50 0.40  1.0

and the corresponding dataset:
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
set.seed(11)
Y <- mvtnorm::rmvnorm(1000, mean = rep(0,6), sigma = Rho)
dfW <- cbind(id = 1:NROW(Y), as.data.frame(Y))
dfL <- reshape2::melt(dfW, id.vars = "id", variable.name = "time")
dfL[dfL$id %in% 1:2,]
#+END_SRC

#+RESULTS:
#+begin_example
     id time      value
1     1   V1 -0.9842079
2     2   V1  1.2402726
1001  1   V2 -0.3681245
1002  2   V2  0.6494215
2001  1   V3 -1.6174652
2002  2   V3  0.3272105
3001  1   V4 -1.4994103
3002  2   V4 -1.0626973
4001  1   V5  0.7493107
4002  2   V5 -0.9013244
5001  1   V6 -1.0719657
5002  2   V6 -0.6696714
#+end_example

#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
dfL[dfL$id==1,]
#+END_SRC

#+RESULTS:
:      id time      value
: 1     1   V1 -0.9842079
: 1001  1   V2 -0.3681245
: 2001  1   V3 -1.6174652
: 3001  1   V4 -1.4994103
: 4001  1   V5  0.7493107
: 5001  1   V6 -1.0719657

#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
dfL[dfL$id==2,]
#+END_SRC

#+RESULTS:
:      id time      value
: 2     2   V1  1.2402726
: 1002  2   V2  0.6494215
: 2002  2   V3  0.3272105
: 3002  2   V4 -1.0626973
: 4002  2   V5 -0.9013244
: 5002  2   V6 -0.6696714

#+LaTeX: \end{minipage}

\clearpage

To estimate the corresponding mixed model we first define a new
covariance structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
myStruct <- CUSTOM(~time,
                   FCT.sigma = function(p,n.time,X){rep(p,n.time)}, ## function f
                   init.sigma = c("sigma"=1),
                   FCT.rho = rho.2block, ## function g
                   init.rho = c("rho1"=0.25,"rho2"=0.25,"rho3"=0.25,"rho4"=0.25))
#+END_SRC

#+RESULTS:

and then call =lmm= with this structure structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmmCUSTOM <- lmm(value~time, repetition=~time|id,
                   structure = myStruct, data=dfL,
                   df = FALSE) ## df = FALSE to save computation time
logLik(e.lmmCUSTOM)
#+END_SRC

#+RESULTS:
: [1] -7962.243

The optimization procedure may be slow but should eventually reaches
an optimum. We can then output the estimated correlation matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
cov2cor(sigma(e.lmmCUSTOM))
#+END_SRC

#+RESULTS:
:            V1         V2         V3         V4         V5         V6
: V1 1.00000000 0.24898095 0.50058994 0.09053785 0.09053785 0.09053785
: V2 0.24898095 1.00000000 0.36110943 0.09053785 0.09053785 0.09053785
: V3 0.50058994 0.36110943 1.00000000 0.09053785 0.09053785 0.09053785
: V4 0.09053785 0.09053785 0.09053785 1.00000000 0.24898095 0.50058994
: V5 0.09053785 0.09053785 0.09053785 0.24898095 1.00000000 0.36110943
: V6 0.09053785 0.09053785 0.09053785 0.50058994 0.36110943 1.00000000

*Comparison to build-in structure*: consider the following model using
a build-in compound symmetry structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
system.time(
  e.lmmDEFAULT.CS <- lmm(value~time, repetition = ~time|id,
                         structure = "CS", data = dfL,
                         df = FALSE)
)
#+END_SRC

#+RESULTS:
:    user  system elapsed 
:   0.097   0.000   0.097

Using instead =CUSTOM= to specifying this structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
myCS <- CUSTOM(~1,
               FCT.sigma = function(p,n.time,X){rep(p,n.time)},
               init.sigma = c("sigma"=1), 
               FCT.rho = function(p,n.time,X){p+diag(1-p,n.time,n.time)},
               init.rho = c("rho"=0.5))
#+END_SRC

#+RESULTS:

is considerably slower than using the pre-specified structure:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
system.time(
  e.lmmCUSTOM.CS <- lmm(value~time, repetition = ~time|id,
                        structure = myCS, data = dfL,
                        df = FALSE)
)
#+END_SRC

#+RESULTS:
:    user  system elapsed 
:   0.952   0.019   0.972


but will lead to the same estimates:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
logLik(e.lmmDEFAULT.CS)
logLik(e.lmmCUSTOM.CS)
#+END_SRC

#+RESULTS:
: [1] -8186.859
: [1] -8186.859

There are two reasons for the slower execution time: slower evaluation
of the derivatives (since they are obtained by numerical
differentiation) and worse starting point, as reflected by the larger
number of interations needed to reach convergence:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmmDEFAULT.CS$opt$n.iter
e.lmmCUSTOM.CS$opt$n.iter
#+END_SRC

#+RESULTS:
: [1] 1
: [1] 4

Faster execution time can be obtained by specifying the first and
second derivative regarding each parameter:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
myCS.wD <- CUSTOM(~1,
                  FCT.sigma = function(p,n.time,X){rep(p,n.time)},
                  dFCT.sigma = function(p,n.time,X){list(sigma = rep(1,n.time))},
                  d2FCT.sigma = function(p,n.time,X){list(sigma = rep(0,n.time))},
                  init.sigma = c("sigma"=1),
                  FCT.rho = function(p,n.time,X){p+diag(1-p,n.time,n.time)},
                  dFCT.rho = function(p,n.time,X){list(rho = 1-diag(1,n.time,n.time))},
                  d2FCT.rho = function(p,n.time,X){list(rho = matrix(0,n.time,n.time))},
                  init.rho = c("rho"=0.5))
#+END_SRC

#+RESULTS:

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
system.time(
  e.lmmCUSTOMwD.CS <- lmm(value~time,
                          repetition = ~time|id,
                          structure = myCS.wD, 
                          data = dfL, df = FALSE
                          )
)
#+END_SRC

#+RESULTS:
:    user  system elapsed 
:   0.699   0.004   0.703


\clearpage

** Estimation procedure

*Initialiation*: by default the mean parameters are initialized using
 Ordinary Least Squares (OLS) and the variance and correlation
 parameters are initialized by minimizing the difference between the
 observed and residuals variance-covariance matrix. These values can
 be visualized by specifying the argument =control=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eCS.lmm.bis <- update(eCS.lmm, control = list(trace = 2))
#+END_SRC

#+RESULTS:
#+begin_example
Initialization:
  (Intercept)        visit2        visit3        visit4        group1 visit2:group1 visit3:group1 
     38.72897      -4.73433      31.43303       4.52138     -12.82462       3.75946      27.00150 
visit4:group1         sigma       rho(id) 
     30.22391      19.52828       0.22819 

Loop:
,******
  (Intercept)        visit2        visit3        visit4        group1 visit2:group1 visit3:group1 
     38.72897      -4.73433      31.43303       4.52138     -12.82462       3.80337      27.48103 
visit4:group1         sigma       rho(id) 
     30.22391      19.50846       0.21741 
Convergence after 6 iterations: max score=1.2413e-05 | max change in coefficient=4.5167e-06
#+end_example

It is possible to input user-defined value:
- for all parameters (vector)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
init.all <- coef(eCS.lmm, effects = "all")
eCS.lmm.bis <- update(eCS.lmm, control = list(init = init.all, trace = 1))
#+END_SRC

#+RESULTS:
: Convergence after 0 iteration: max score=1.2413e-05

- the mean parameters only (vector)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
init.mean <- coef(eCS.lmm, effects = "mean")
eCS.lmm.bis <- update(eCS.lmm, control = list(init = init.mean, trace = 2))
#+END_SRC

#+RESULTS:
#+begin_example
Initialization:
  (Intercept)        visit2        visit3        visit4        group1 visit2:group1 visit3:group1 
     38.72897      -4.73433      31.43303       4.52138     -12.82462       3.80337      27.48103 
visit4:group1         sigma       rho(id) 
     30.22391      19.52904       0.22849 

Loop:
,******
  (Intercept)        visit2        visit3        visit4        group1 visit2:group1 visit3:group1 
     38.72897      -4.73433      31.43303       4.52138     -12.82462       3.80337      27.48103 
visit4:group1         sigma       rho(id) 
     30.22391      19.50846       0.21741 
Convergence after 6 iterations: max score=1.4893e-05 | max change in coefficient=5.3866e-06
#+end_example

- a full data variance-covariance matrix (matrix). 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
init.vcov <- sigma(eCS.lmm)
eCS.lmm.bis <- update(eCS.lmm, control = list(init = init.vcov, trace = 1))
#+END_SRC

#+RESULTS:
: Convergence after 0 iteration: max score=1.2413e-05

*Optimizer*: by default the optimizer is a Newton Raphson algorithm
with backtracking. At each iteration:
- it computes the first two moments (score, information) according to
  the current parameters values.
- it updates the variance-covariance parameters according to the
  gradient multiplied by the inverse of the information.
- it updates the mean parameters by generalized least squares (using
  the updated variance-covariance parameters).
- it checks whether the log-likelihoood at the updated estimates is
  well defined and higher than at the previous estimates. If this is
  not the case, the step is re-run with half the update of the
  variance-covariance parameters (backtracking).

One can modify the maximum number of iterations (=n.iter=), maximum
number of backtracking steps (=n.backtracking=), the maximum score
(absolute) value over all parameters (=tol.score=) and (absolute)
maximum difference in parameter value between to iterations
(=tol.param=) used to declare convergence. It is also possible to use
another optimizer (=optimizer=). All these elements should be passed
to the argument =control= of =lmm= using a list.

 \clearpage

** Model output

The =summary= method can be used to display the main information
relative to the model fit:
#+BEGIN_SRC R :exports code :results none :session *R* :cache no
summary(eUN.lmm)
#+END_SRC

#+BEGIN_SRC R :exports results :results output :session *R* :cache no
summary(eUN.lmm, hide.mean = TRUE)
#+END_SRC
#+RESULTS:
#+begin_example
		Linear Mixed Model 
 
Dataset: gastricbypassL 

  - 20 clusters 
  - 78 observations were analyzed, 2 were excluded because of missing values 
  - between 3 and 4 observations per cluster 

Summary of the outcome and covariates: 

    $ glucagonAUC: num  20.7 49.9 42.4 27.5 29.2 ...
    $ visit      : Factor w/ 4 levels "1","2","3","4": 1 1 1 1 1 1 1 1 1 1 ...
    $ group      : Factor w/ 2 levels "0","1": 2 1 2 1 2 1 2 1 2 1 ...
    reference level: visit=1;group=0 

Estimation procedure 

  - Restricted Maximum Likelihood (REML) 
  - log-likelihood :-295.31
  - parameters: mean = 8, variance = 4, correlation = 6
  - convergence: TRUE (8 iterations) 
    largest |score| = 4.6771e-05 for rho(-1,1)
            |change|= 1.68033723859651e-05 for visit3:group1
 
Residual variance-covariance: unstructured 

  - correlation structure: ~0 + time 
            -13       -1       1      13
    -13  1.0000  0.80072 0.26880 -0.0855
    -1   0.8007  1.00000 0.00368 -0.0941
    1    0.2688  0.00368 1.00000  0.5384
    13  -0.0855 -0.09413 0.53842  1.0000

  - variance structure: ~time 
              standard.deviation ratio
    sigma.-13               14.5 1.000
    sigma.-1                13.0 0.896
    sigma.1                 27.4 1.890
    sigma.13                19.6 1.352
#+end_example

\clearpage

#+BEGIN_SRC R :exports results :results output :session *R* :cache no
oo <- capture.output(summary(eUN.lmm, hide.fit = TRUE, hide.data = TRUE, hide.cor = TRUE, hide.var = TRUE, hide.sd = TRUE))
cat(sapply(oo[-(1:2)],paste0,"\n"))
#+END_SRC

#+RESULTS:
#+begin_example
Fixed effects: glucagonAUC ~ visit * group 
 
                  estimate     se   df   lower  upper p.value    
    (Intercept)     38.729  4.576   18  29.114 48.344 < 1e-04 ***
    visit2          -4.734  2.776 17.5 -10.577  1.109 0.10574    
    visit3          31.433   8.63 17.6  13.272 49.594 0.00192  **
    visit4           4.521  8.005   18 -12.297  21.34 0.57917    
    group1         -12.825  6.472   18 -26.422  0.773 0.06302   .
    visit2:group1    3.987  3.996 17.9   -4.41 12.383 0.33169    
    visit3:group1   27.571  12.42 17.8   1.461 53.682 0.03963   *
    visit4:group1   30.224 11.321   18   6.439 54.008 0.01562   *
    -------------------------------------------------------- 
   Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.
   Columns lower and upper contain 95% pointwise confidence intervals for each coefficient.
   Model-based standard errors are derived from the observed information (column se). 
   Degrees of freedom were computed using a Satterthwaite approximation (column df).
#+end_example

_Note:_ the calculation of the degrees of freedom, especially when
using the observed information can be quite slow. Setting the
arguments =df= to =FALSE= and =type.information= to ="expected"= when
calling =lmm= should lead to a more reasonnable computation time.

** Extract estimated coefficients
The value of the estimated coefficients can be output using =coef=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
coef(eUN.lmm)
#+END_SRC

#+RESULTS:
:   (Intercept)        visit2        visit3        visit4        group1 visit2:group1 visit3:group1 
:       38.7290       -4.7343       31.4330        4.5214      -12.8246        3.9866       27.5714 
: visit4:group1 
:       30.2239

Variance coefficients can be output by specifying the =effects= argument:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
coef(eUN.lmm, effects = "variance")
#+END_SRC

#+RESULTS:
:    sigma     k.-1      k.1     k.13 
: 14.47212  0.89592  1.88991  1.35220

The first coefficient is the residual standard deviation at the
reference timepoint (here -13 week) and the remaining coefficient the
residual standard deviation at later timepoints relative to the
reference timepoint. It is possible to apply specific transformation
on the variance coefficients, for instance to obtain the residual
variance at each timepoint:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
coef(eUN.lmm, effects = "variance", transform.k = "sd")
#+END_SRC

#+RESULTS:
: sigma.-13  sigma.-1   sigma.1  sigma.13 
:    14.472    12.966    27.351    19.569

** Extract estimated coefficient and associated uncertainty

The uncertainty about the mean coefficients can be obtained using the
=model.tables= method [fn:: it is equivalent to =confint= method
except that by default it also outputs =se= and =p.value=]:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
model.tables(eUN.lmm)
#+END_SRC

#+RESULTS:
:               estimate      se     df    lower    upper    p.value
: (Intercept)    38.7290  4.5765 18.003  29.1143 48.34369 1.0891e-07
: visit2         -4.7343  2.7759 17.543 -10.5772  1.10851 1.0574e-01
: visit3         31.4330  8.6297 17.585  13.2719 49.59411 1.9229e-03
: visit4          4.5214  8.0050 17.995 -12.2968 21.33958 5.7917e-01
: group1        -12.8246  6.4721 18.003 -26.4219  0.77265 6.3015e-02
: visit2:group1   3.9866  3.9957 17.937  -4.4102 12.38329 3.3169e-01
: visit3:group1  27.5714 12.4199 17.831   1.4605 53.68232 3.9634e-02
: visit4:group1  30.2239 11.3208 17.995   6.4394 54.00840 1.5624e-02

Values for the all correlation parameters can be displayed
too, by specifying @@latex:\texttt{effect=c("variance","correlation")}@@:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
model.tables(eUN.lmm, effect = c("variance","correlation"))
#+END_SRC

#+RESULTS:
#+begin_example
              estimate       se      df    lower    upper    p.value
sigma       14.4721183 2.412020 15.3158 10.15148 20.63170         NA
k.-1         0.8959206 0.127032 20.2671  0.66670  1.20396 0.44721963
k.1          1.8899095 0.431098 25.9157  1.18244  3.02067 0.00974152
k.13         1.3521979 0.317550 29.8074  0.83694  2.18468 0.20874407
rho(-13,-1)  0.8007214 0.085177 13.4142  0.52949  0.92343 0.00042923
rho(-13,1)   0.2688043 0.219200  7.9286 -0.26374  0.67576 0.27735748
rho(-13,13) -0.0854578 0.233981  8.5882 -0.55306  0.42309 0.72505145
rho(-1,1)    0.0036838 0.237237  8.1487 -0.49424  0.49979 0.98798445
rho(-1,13)  -0.0941328 0.233649  8.9191 -0.55697  0.41331 0.69821381
rho(1,13)    0.5384239 0.176221 10.2233  0.05058  0.81883 0.03522642
#+end_example

Because these parameters are constrained (e.g. strictly positive),
they uncertainty is by default computed after transformation
(e.g. =log=) and then backtransformed. The column argument can be used
to extract more or less information, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
model.tables(eUN.lmm, columns = c("estimate","p.value"))
#+END_SRC

#+RESULTS:
:               estimate    p.value
: (Intercept)    38.7290 1.0891e-07
: visit2         -4.7343 1.0574e-01
: visit3         31.4330 1.9229e-03
: visit4          4.5214 5.7917e-01
: group1        -12.8246 6.3015e-02
: visit2:group1   3.9866 3.3169e-01
: visit3:group1  27.5714 3.9634e-02
: visit4:group1  30.2239 1.5624e-02

All parameters can be displayed by specifying
@@latex:\texttt{effect="all"}@@.  The functions =add= (resp. =remove=)
can be used to add (resp. remove) one or several columns from the
default display, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
model.tables(eUN.lmm, columns = add("statistic"))
#+END_SRC

#+RESULTS:
:               estimate      se statistic     df    lower    upper    p.value
: (Intercept)    38.7290  4.5765   8.46260 18.003  29.1143 48.34369 1.0891e-07
: visit2         -4.7343  2.7759  -1.70552 17.543 -10.5772  1.10851 1.0574e-01
: visit3         31.4330  8.6297   3.64242 17.585  13.2719 49.59411 1.9229e-03
: visit4          4.5214  8.0050   0.56482 17.995 -12.2968 21.33958 5.7917e-01
: group1        -12.8246  6.4721  -1.98151 18.003 -26.4219  0.77265 6.3015e-02
: visit2:group1   3.9866  3.9957   0.99772 17.937  -4.4102 12.38329 3.3169e-01
: visit3:group1  27.5714 12.4199   2.21995 17.831   1.4605 53.68232 3.9634e-02
: visit4:group1  30.2239 11.3208   2.66977 17.995   6.4394 54.00840 1.5624e-02

** Extract estimated residual variance-covariance structure

The method =sigma= can be used to output the modeled residual
covariance structure and then converted to a correlation matrix using
=cov2cor=:

\medskip

#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
Sigma <- sigma(eUN.lmm)
Sigma
#+END_SRC

#+BEGIN_SRC R :exports results :results output :session *R* :cache no
round(Sigma,3)
#+END_SRC

#+RESULTS:
:         -13      -1       1      13
: -13 209.442 150.250 106.400 -24.202
: -1  150.250 168.114   1.306 -23.884
: 1   106.400   1.306 748.077 288.184
: 13  -24.202 -23.884 288.184 382.952
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.05\linewidth}
#+LaTeX:\hphantom{x}
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
cov2cor(Sigma)
#+END_SRC

#+BEGIN_SRC R :exports results :results output :session *R* :cache no
round(cov2cor(Sigma), 3)
#+END_SRC

#+RESULTS:
:        -13     -1     1     13
: -13  1.000  0.801 0.269 -0.085
: -1   0.801  1.000 0.004 -0.094
: 1    0.269  0.004 1.000  0.538
: 13  -0.085 -0.094 0.538  1.000
#+LaTeX: \end{minipage}

The method can also be used to extract the residual covariance
relative to a "known" individual:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigma(eUN.lmm, cluster = 5)
#+END_SRC

#+RESULTS:
:         -13      1      13
: -13 209.442 106.40 -24.202
: 1   106.400 748.08 288.184
: 13  -24.202 288.18 382.952

or for a new individual:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
newdata <- data.frame(id = "X", time = c("-13","-1","1","13"))
sigma(eUN.lmm, cluster = newdata)
#+END_SRC

#+RESULTS:
:         -13       -1        1      13
: -13 209.442 150.2502 106.4000 -24.202
: -1  150.250 168.1138   1.3064 -23.884
: 1   106.400   1.3064 748.0769 288.184
: 13  -24.202 -23.8844 288.1839 382.952

\clearpage

** Marginal effects

The =effects= method can be used to evaluate marginal means with
respect to a categorical variable:
- \(\Esp[Y_t \mid \text{group}]\)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eUN.lmm, variable = "group")
#+END_SRC

#+RESULTS:
#+begin_example
		Average counterfactual outcome
		 w.r.t 'group' values 

                  estimate    se   df  lower   upper
   group=0(t=-13)   38.729 4.576   18 29.114  48.344
   group=0(t=-1)    33.995   4.1 17.9 25.377  42.612
   group=0(t=1)     70.162 8.649 17.7 51.968  88.356
   group=0(t=13)     43.25 6.188   18 30.249  56.251
   group=1(t=-13)   25.904 4.576   18  16.29  35.519
   group=1(t=-1)    25.157 4.167 18.7 16.425  33.889
   group=1(t=1)     84.909 8.951 18.2 66.115 103.702
   group=1(t=13)     60.65 6.188   18 47.649  73.651
#+end_example

- \(\Esp[Y_t-Y_0 \mid \text{group}]\)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eUN.lmm, type = "change", variable = "group")
#+END_SRC

#+RESULTS:
#+begin_example
		Average counterfactual change in outcome
		 w.r.t 'group' values 

                  estimate    se   df   lower  upper
   group=0(dt=-1)   -4.734 2.776 17.5 -10.577  1.109
   group=0(dt=1)    31.433  8.63 17.6  13.272 49.594
   group=0(dt=13)    4.521 8.005   18 -12.297  21.34
   group=1(dt=-1)   -0.748 2.874 18.3  -6.779  5.283
   group=1(dt=1)    59.004 8.932   18  40.242 77.767
   group=1(dt=13)   34.745 8.005   18  17.927 51.563
#+end_example

- \(\Esp[\int_0^T Y_t dt \mid \text{group}]\)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eUN.lmm, type = "auc", variable = "group")
#+END_SRC

#+RESULTS:
: 		Average counterfactual area under the outcome curve
: 		 w.r.t 'group' values 
: 
:                 estimate      se   df    lower    upper
:    group=0(auc) 1220.972 104.098 17.8 1002.072 1439.873
:    group=1(auc) 1289.782 105.512 18.5 1068.508 1511.056

It can also be used to contrast these marginal means:
- \(\Esp[Y_t \mid \text{group}=1]-\Esp[Y_t \mid \text{group}=0]\)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eUN.lmm, type = "difference", variable = "group")
#+END_SRC

#+RESULTS:
: 		Difference in average counterfactual outcome
: 		 w.r.t 'group' values 
: 
:                     estimate     se   df   lower  upper p.value  
:    group=1-0(t=-13)  -12.825  6.472   18 -26.422  0.773  0.0630 .
:    group=1-0(t=-1)    -8.838  5.846 18.3 -21.106   3.43  0.1477  
:    group=1-0(t=1)     14.747 12.447 17.9 -11.409 40.903  0.2516  
:    group=1-0(t=13)    17.399  8.752   18  -0.987 35.785  0.0622 .

- \(\Esp[Y_t-Y_0 \mid \text{group}=1]-\Esp[Y_t-Y_0 \mid \text{group}=0]\)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eUN.lmm, type = c("change","difference"), variable = "group")
#+END_SRC

#+RESULTS:
: 		Difference in average counterfactual change in outcome
: 		 w.r.t 'group' values 
: 
:                     estimate     se   df lower  upper p.value  
:    group=1-0(dt=-1)    3.987  3.996 17.9 -4.41 12.383  0.3317  
:    group=1-0(dt=1)    27.571  12.42 17.8 1.461 53.682  0.0396 *
:    group=1-0(dt=13)   30.224 11.321   18 6.439 54.008  0.0156 *

- \(\Esp[\int_0^T Y_t dt \mid \text{group}=1]-\Esp[\int_0^T Y_t dt \mid \text{group}=0]\)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eUN.lmm, type = c("auc","difference"), variable = "group")
#+END_SRC

#+RESULTS:
: 		Difference in average counterfactual area under the outcome curve
: 		 w.r.t 'group' values 
: 
:                   estimate     se   df   lower   upper p.value  
:    group=1-0(auc)   68.809 148.22 18.1 -242.44 380.059   0.648

It is possible to control the set of covariates used to condition on
via the =conditional= argument. This can be useful when considering an
interaction with a biomarker to obtain biomarker-specific effects.

\clearpage

** Random effects

Mixed model having a compound symmetry structure with positive
correlation parameters may be equivalent to random intercept models,
possibly with nested random effects. Indeed in some case the residual
variance-covariance matrix can then be decomposed as:
#+BEGIN_EXPORT latex
\[ \Omega = Z \Psi \trans{Z} + \Delta \]
#+END_EXPORT
- \(Z\) is the design matrix associated to the random effect (e.g. patient id)
- \(\Psi\) is the variance-covariance of the random effects
- \(\Delta\) the residual variance covariance conditional to the random effects.
One can the use =lme4= syntax to fit random intercept models with
=lmm=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRI.lmm <- lmm(glucagonAUC ~ visit*group + (1|id), data = gastricbypassL)
eRI.lmm
#+END_SRC

#+RESULTS:
: 		Linear Mixed Model with a random intercept 
: 
:  outcome/cluster/time: glucagonAUC/id/XXtimeXX 
:  data                : 78 observations from 20 clusters 
:  parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
:                        1 variance (sigma) 
:                        1 correlation (rho(id)) 
:  log-restr.likelihood: -314.394203759159 
:  convergence         : TRUE (6 iterations)

It is also possible to specify cross or nested random effects, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eNRI.lmm <- lmm(glucagonAUC ~ visit*group + (1|id/baseline), data = gastricbypassL)
eNRI.lmm
#+END_SRC

#+RESULTS:
: 		Linear Mixed Model with nested random intercepts 
: 
:  outcome/cluster/time: glucagonAUC/id/XXtimeXX 
:  data                : 78 observations from 20 clusters 
:  parameter           : 8 mean ((Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1) 
:                        1 variance (sigma) 
:                        2 correlation (rho(id/baseline) rho(id)) 
:  log-restr.likelihood: -308.994835006264 
:  convergence         : TRUE (6 iterations)

We obtain the same log-likelihood as, respectively, =eCS.lmm= and
=eBCS.lmm=. Indeed, as previously mentioned, with positive residual
correlation the random effect structure is equivalent to a compound
symmetry structure. \newline \Warning random slopes are not currently
supported in LMMstar. \newline \Warning the proposed implementation can
be very inefficient compared to =lme4=.

\bigskip

The joint distribution between the outcome \(\VY\)
and the random effects \(\Veta\) can be expressed as:
#+BEGIN_EXPORT latex
\[
\begin{bmatrix} \VY \\ \Veta \end{bmatrix} \sim \Gaus\left(\begin{bmatrix} \boldsymbol{\mu} \\ \mathbf{0} \end{bmatrix}, \begin{bmatrix} \Omega & Z \Psi \\ \Psi \trans{Z} & \Psi \end{bmatrix}\right)
\]
#+END_EXPORT
Denote by \(\varepsilon_i=\VY_i-\boldsymbol{\mu}_i\) the vector of
marginal residuals relative to individual \(i\), \(\Omega_i\) its
variance-covariance matrix, and \(\psi_j=(\Psi)_{jj}\) the variance of the
\(j\)-th random effect. We can re-express the expected value of the
\(j\)-th random effect for individual \(i\) as:
#+BEGIN_EXPORT latex
\[ \eta_{ij} = \psi_{j} Z_{ij} \Omega_i^{-1}\varepsilon_i \]
#+END_EXPORT
This is what the =ranef= method returns:

\bigskip

#+LaTeX: \begin{minipage}{0.48\linewidth} 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
head(ranef(eRI.lmm, format = "wide"))
#+END_SRC

#+RESULTS:
:   id estimate
: 1  1 -2.51154
: 2  2  1.01043
: 3  3  6.08384
: 4  4 -6.62350
: 5  5  0.39519
: 6  6 -2.73384
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.48\linewidth} 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
head(ranef(eNRI.lmm, format = "wide"))
#+END_SRC

#+RESULTS:
:   id  estimate estimate.FALSE estimate.TRUE
: 1  1 -0.494271       -3.50959      -3.23209
: 2  2  0.186051      -10.39431      12.93198
: 3  3  1.088409        9.36327       5.48225
: 4  4 -1.219596      -11.06703      -5.56784
: 5  5  0.081686       -0.71254       1.82672
: 6  6 -0.503386       -7.81700       0.95098
#+LaTeX: \end{minipage}


#+BEGIN_SRC R :exports none :results output :session *R* :cache no
library(lme4)
e.lmer <- lmer(glucagonAUC ~ visit*group + (1|id), data = gastricbypassL)
range(ranef(e.lmer)[[1]][,1]-ranef(eRI.lmm))
e.lmer2 <- lmer(glucagonAUC ~ visit*group + (1|id/baseline), data = gastricbypassL)
range(matrix(c(ranef(e.lmer2)[[2]][,1],ranef(e.lmer2)[[1]][,1]), ncol = 3, byrow = FALSE)-ranef(eNRI.lmm, format = "wide")[,-1])
#+END_SRC

#+RESULTS:
: [1] -5.6960e-06  5.8867e-06
: [1] -0.00027523  0.00028697

It is also possible to extract the variance decomposition by setting
the argument =effects= to ="variance"=:

\medskip

#+LaTeX: \begin{minipage}{0.47\linewidth} 
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
ranef(eRI.lmm, effects = "variance",
      format = "wide")
#+END_SRC

#+BEGIN_SRC R :exports results :results output :session *R* :cache no
ranef(eRI.lmm, effects = "variance", format = "wide")
cat(" \n")
cat(" \n")
#+END_SRC

#+RESULTS:
:       type absolute relative
: 1    total  380.580  1.00000
: 2       id   82.741  0.21741
: 3 residual  297.839  0.78259
: 
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.47\linewidth} 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
ranef(eNRI.lmm, effects = "variance",
      format = "wide")
#+END_SRC

#+RESULTS:
:       type absolute relative
: 1    total  380.957 1.000000
: 2       id   15.465 0.040595
: 3 baseline  210.938 0.553705
: 4 residual  154.554 0.405700
#+LaTeX: \end{minipage}


Confidence intervals can also be obtained setting the argument =se= to
=TRUE= and =format= equal to ="long"=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
head(ranef(eRI.lmm, se = TRUE))
#+END_SRC

#+RESULTS:
:   id estimate     se      df    lower   upper
: 1  1 -2.51154 2.3019 11.1302  -7.5708  2.5477
: 2  2  1.01043 2.1163 15.7355  -3.4821  5.5030
: 3  3  6.08384 2.9771  6.2085  -1.1421 13.3098
: 4  4 -6.62350 3.1114  5.8319 -14.2902  1.0432
: 5  5  0.39519 1.9661 23.8446  -3.6640  4.4543
: 6  6 -2.73384 2.2940 10.0189  -7.8438  2.3761

\clearpage

** Sum of squares

\Warning The definition of the sum of squares is not straightforward with mixed
models. Intuitively summing residuals across several outcomes will be
hard to interpret unless all outcomes have the same variance. This is
why LMMstar does not provide them. Nevertheless for specific
covariance structure, namely independence and compound symmetry (with
positive correlation) structure, sum of squares can be deduced from
the =lmm= object - see appendix [[#SM:sumSquares]] for the theoretical
derivations. Importantly, with these structures the residuals can be
reparametrised as random effects plus independent residuals,
i.e. \(\Omega = Z \Psi \trans{Z} + \delta I\) where \(I\) is the
identity matrix and \(\delta\) the variance of these independent
residuals.

\bigskip

Appendix [[#SM:sumSquares]] illustrate how to extract the sum of squares
for univariate linear regression (i.e. independence structure) and
here we illustrate the case of a compound symmetry structure.  A key
step is to extract from the =lmm= object the conditional residual variance
\(\delta\):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigma2 <- coef(eCS.lmm, effect = "variance")^2
tau <- coef(eCS.lmm, effect = "correlation")*sigma2
delta <- unname(sigma2 - tau)
#+END_SRC

#+RESULTS:

This step will typically depend on the covariance structure. The
residual sum of squares (SSE) equals the residual degrees of freedom
times the conditional variance:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
df.res <- df.residual(eCS.lmm)
SSE <- df.res * delta
c(df.res = df.res, SSE = SSE)
#+END_SRC

#+RESULTS:
: df.res    SSE 
:     70  20849

For the regression sum of squares (SSR), we first extract the mean
parameters and their variance-covariance based on the expected
information:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eBeta.lmm <- coef(eCS.lmm)
eVcov.lmm <- vcov(eCS.lmm, type.information = "expected")
#+END_SRC

#+RESULTS:

Parameters are grouped with respect to the original variable:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
attr(model.matrix(eCS.lmm),"assign")
#+END_SRC

#+RESULTS:
: [1] 0 1 1 1 2 3 3 3

\bigskip

So we respect this grouping when computing the normalized SSR: 
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
SSRstar.time <- eBeta.lmm[2:4] %*% solve(eVcov.lmm[2:4,2:4]) %*% eBeta.lmm[2:4] 
SSRstar.group <- eBeta.lmm[5] %*% solve(eVcov.lmm[5,5]) %*% eBeta.lmm[5] 
#+END_SRC
#+RESULTS:

The SSR is obtained by multiplying the normalized SSR by the
conditional variance:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
SSR.time <- as.double(SSRstar.time * delta)
SSR.group <- as.double(SSRstar.group * delta)
c(time = SSR.time, group = SSR.group)
#+END_SRC
#+RESULTS:
:    time   group 
: 7872.19  643.57

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
library(lme4) ## note type I anova so only look at the last line
df.tempo <- gastricbypassL
df.tempo$visit.group <- df.tempo$visit
df.tempo[df.tempo$group==0,"visit.group"] <- levels(df.tempo$visit)[1]
anova(lmer(glucagonAUC ~ visit.group + visit + group + (1|id), data = df.tempo))
anova(lmer(glucagonAUC ~ visit.group + group + visit + (1|id), data = df.tempo))
#+END_SRC

#+RESULTS:
#+begin_example
Analysis of Variance Table
            npar Sum Sq Mean Sq F value
visit.group    3  20840    6947   23.32
visit          3   9586    3195   10.73
group          1    644     644    2.16
Analysis of Variance Table
            npar Sum Sq Mean Sq F value
visit.group    3  20840    6947   23.32
group          1   2357    2357    7.91
visit          3   7872    2624    8.81
#+end_example

** Proportion of explained variance and partial correlation

For a univariate linear model with homoschedastic residual variance,
the proportion of explained variance, also called partial \(R^2\) or
partial \(\eta^2\), is defined as the ratio between sum of squares
(e.g. cite:lakens2013calculating, equation 12):
#+BEGIN_EXPORT latex
\[ R^2=\frac{SSR}{SSR + SSE} \]
#+END_EXPORT

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
c(SSR.time/ (SSR.time + SSE),
  SSR.group/ (SSR.group + SSE))
#+END_SRC

#+RESULTS:
: [1] 0.274092 0.029944

Computing the SSR for each individual coefficients, taking its squared
root, and multiplying by the sign of the corresponding coefficient
leads to the partial correlation. This procedure extends to covariance
structures that can be reparametrised as random effects plus
independent residuals (see previous subsection) such as the compound
symmetry with non-negative correlation.
- \Warning :: for other covariance structures, especially when the
  variance may be repetition-dependent, the definition of explained
  variance/partial correlation is not straightforward.
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eCS.R2 <- partialCor(eCS.lmm, R2 = TRUE)
summary(eCS.R2)
#+END_SRC

#+RESULTS:
#+begin_example

		Partial correlation 

                 estimate    se   df  lower upper p.value
   visit2          -0.073 0.119 52.4 -0.311 0.165 0.54028
   visit3           0.438 0.089 51.4   0.26 0.616 < 1e-04
   visit4            0.07 0.119 52.4 -0.168 0.308 0.55876
   group1          -0.173 0.114 60.7 -0.402 0.056 0.13527
   visit2:group1    0.041 0.119 52.8 -0.198  0.28 0.73256
   visit3:group1    0.284 0.106   52  0.071 0.497 0.01007
   visit4:group1    0.314 0.103   52  0.107 0.521 0.00365
   ------------------------------------------------------ 
  Columns lower and upper contain 95% pointwise confidence intervals for each coefficient.
  Degrees of freedom were computed using a Satterthwaite approximation (column df). 

		Coefficient of determination (R2)

               estimate    se   df  lower upper p.value
   visit          0.274  0.08 50.5  0.114 0.434  0.0012
   group           0.03  0.04 60.7 -0.049 0.109  0.4520
   visit:group    0.147 0.073 51.7 <0.001 0.295  0.0500
   global         0.598 0.053 40.4  0.492 0.705  <1e-04
   ---------------------------------------------------- 
  Columns lower and upper contain 95% pointwise confidence intervals for each coefficient.
  Degrees of freedom were computed using a Satterthwaite approximation (column df).
#+end_example

Here the line "global" refer to the R2 for all covariates, computed
based on the SSR relative to all mean parameters but the intercept.
 - \Warning :: =partialCor= will compute values for all types of mixed
   models. But their interpretation as partial correlation and
   proportion of explained variance outside the compound symmetry with
   non-negative correlation is questionnable.

\bigskip

_Note:_ Other software packages like =effectsize::eta_squared= uses
another formula to estimate the partial R2:
#+BEGIN_EXPORT latex
\[ R^2=\frac{F df_{num}}{F df_{num} + df_{denom}} \]
#+END_EXPORT

where \(F\) denote the F-statistic, \(df_{num}\)
(resp. \(df_{denom}\)) the degrees of freedom of the numerator
(resp. denominator) of this statistic. However since the calculation
of degrees of freedom in LMM is approximate, I would expect this
approach to be less reliable than the one of =partialCor= based on the
SSR and SSE.

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
aCS.aov <- anova(eCS.lmm)$multivariate
setNames(with(aCS.aov, statistic*df.num/(statistic*df.num+df.denom)), aCS.aov$test)
#+END_SRC

#+RESULTS:
:       visit       group visit:group 
:    0.335374    0.033811    0.186290


#+BEGIN_SRC R :exports none :results output :session *R* :cache no
effectsize::eta_squared(lmer(glucagonAUC ~ visit*group + (1|id), data = gastricbypassL))
cat("\n")
#+END_SRC

#+RESULTS:
#+begin_example
# Effect Size for ANOVA (Type III)

Parameter   | Eta2 (partial) |       95% CI
-------------------------------------------
visit       |           0.64 | [0.50, 1.00]
group       |           0.01 | [0.00, 1.00]
visit:group |           0.19 | [0.03, 1.00]

- One-sided CIs: upper bound fixed at
# Effect Size for ANOVA (Type III)

Parameter               | Eta2 (partial) |       95% CI
-------------------------------------------------------
visit                   |           0.06 | [0.00, 1.00]
as.numeric(group)       |           0.01 | [0.00, 1.00]
visit:as.numeric(group) |           0.19 | [0.03, 1.00]

- One-sided CIs: upper bound fixed at
#+end_example

\bigskip


\clearpage

** Model diagnostic

The method =residuals= returns the residuals in the wide format:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.diagW <- residuals(eUN.lmm, type = "normalized", format = "wide")
colnames(eUN.diagW) <- gsub("normalized.","",colnames(eUN.diagW))
head(eUN.diagW)
#+END_SRC

#+RESULTS:
:   id    r.-13     r.-1       r.1     r.13
: 1  1 -0.36029 -0.11344  0.377177 -1.45539
: 2  2  0.77339  2.12301 -0.232908 -0.10708
: 3  3  1.14219 -1.44778 -0.654876  2.01259
: 4  4 -0.77473  0.20612 -0.127117 -1.39519
: 5  5  0.22435       NA  0.011432 -0.15398
: 6  6  0.27439 -0.67308 -1.031131  0.42724

or in the long format:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.diagL <- residuals(eUN.lmm, type = "normalized", format = "long", keep.data = TRUE)
head(eUN.diagL)
#+END_SRC

#+RESULTS:
:   id visit time weight glucagonAUC group baseline fitted r.normalized
: 1  1     1  -13  127.2      20.690     1     TRUE 25.904     -0.36029
: 2  2     1  -13  165.2      49.922     0     TRUE 38.729      0.77339
: 3  3     1  -13  109.7      42.434     1     TRUE 25.904      1.14219
: 4  4     1  -13  146.2      27.517     0     TRUE 38.729     -0.77473
: 5  5     1  -13  113.1      29.151     1     TRUE 25.904      0.22435
: 6  6     1  -13  158.8      42.700     0     TRUE 38.729      0.27439

Various type of residuals can be extract but the normalized one are
recommanded when doing model checking. Diagnostic plots can then be
generated by the user, or directly from the =lmm= object via the
method =plot= (which internally calls the =residuals= method):
- misspecification of the mean structure
#+BEGIN_SRC R :file ./figures/diag-scatterplot.pdf :results graphics file :session *R* :cache no
plot(eUN.lmm, type = "scatterplot")
#+END_SRC

#+RESULTS:
[[file:./figures/diag-scatterplot.pdf]]

#+ATTR_LaTeX: :width 0.4\textwidth :placement [!h]
[[./figures/diag-scatterplot.pdf]]

\clearpage

- misspecification of the variance structure
#+BEGIN_SRC R :file ./figures/diag-scatterplot2.pdf :results graphics file :session *R* :cache no
plot(eUN.lmm, type = "scatterplot2")
#+END_SRC

#+RESULTS:
[[file:./figures/diag-scatterplot2.pdf]]

#+ATTR_LaTeX: :width 0.4\textwidth :placement [!h]
[[./figures/diag-scatterplot2.pdf]]

- misspecification of the correlation structure
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
plot(eUN.lmm, type = "correlation", type.residual = "response")
plot(eUN.lmm, type = "correlation", type.residual = "normalized")
#+END_SRC

#+RESULTS:

#+BEGIN_SRC R :exports none :results output raw drawer :session *R* :cache no
library(ggpubr)
gg <- ggarrange(autoplot(eUN.lmm, type = "correlation", type.residual = "response")$plot,
                autoplot(eUN.lmm, type = "correlation", type.residual = "normalized")$plot,
                common.legend = TRUE)
ggsave(gg, filename = "./figures/diag-correlation.pdf", width = 12)
#+END_SRC


#+RESULTS:
:results:
[1m[22mSaving 12 x 7 in image
:end:

#+ATTR_LaTeX: :width 0.6\textwidth :placement [!h]
[[./figures/diag-correlation.pdf]]

- residual distribution vs. normal distribution [fn::see cite:oldford2016self for guidance
  about how to read quantile-quantile plots.]:

#+BEGIN_SRC R :file ./figures/diag-qqplot.pdf :results graphics file :session *R* :cache no :width 10 :height 4
plot(eUN.lmm, type = "qqplot", engine = "qqtest",
     facet = ~time, labeller = "label_both", facet_nrow=1)
## Note: the qqtest package to be installed to use the argument engine.plot = "qqtest" 
#+END_SRC

#+RESULTS:
[[file:./figures/diag-qqplot.pdf]]

#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/diag-qqplot.pdf]]

\Warning Deviation from the normal distribution does not necessarily
question the validity of the statistical inference. Moreover, for
variance and correlation parameters, normally distributed data is not
enougth to ensure valid statistical inference. Instead one could
assess whether the log-likelihood is locally quadratic as this ensures
normally distributed estimates in finite samples
citep:geyer2013asymptotics. Since the likelihood function is a
multi-dimensional function this is not an easy task but one can look
at specific 'slices' using the =profile= method:

#+header: :width 8 :height 5
#+BEGIN_SRC R :file ./figures/diag-profileUN.pdf :results graphics file :session *R* :cache no
eUN.lmm_profile <- profile(eUN.lmm, effects = c("sigma","rho(-13,-1)"))
plot(eUN.lmm_profile)
#+END_SRC

#+RESULTS:
[[file:./figures/diag-profileUN.pdf]]


#+ATTR_LaTeX: :width 0.75\textwidth :placement [!h]
[[./figures/diag-profileUN.pdf]]

\clearpage

** Visualize model fit

The fitted values can be displayed via the =plot= method:
#+BEGIN_SRC R :file ./figures/fit-autoplot.pdf :results graphics file :session *R* :cache no
## left panel
plot(eUN.lmm, type = "fit", color = "group", size.text = 20)
#+END_SRC

#+RESULTS:
[[file:./figures/fit-autoplot.pdf]]

\Warning the shaded area represent 95% confidence intervals (CIs),
  i.e. is not adjusted for multiplicity over time. More explicit (but
  sometimes less readable) representation of the CIs can be obtained
  by setting the argument =ci.alpha= to =NA=:

#+BEGIN_SRC R :file ./figures/fit-autoplot2.pdf :results graphics file :session *R* :cache no
## middle panel
plot(eUN.lmm, type = "fit", color = "group", ci.alpha = NA, size.text = 20)
#+END_SRC

#+RESULTS:
[[file:./figures/fit-autoplot2.pdf]]

\noindent It is also possible to display the observed values along with the
fitted values by setting the argument =obs.alpha= to a strictly
positive value below or equal to 1. This argument controls the
transparency of the color used to display the observed values:
#+BEGIN_SRC R :file ./figures/fitAll-autoplot.pdf :results graphics file :session *R* :cache no
## right panel
plot(eUN.lmm, type = "fit", obs.alpha = 0.25, ci = FALSE, size.text = 20)
#+END_SRC

#+RESULTS:
[[file:./figures/fitAll-autoplot.pdf]]

#+latex: \begin{minipage}{0.3\linewidth}
#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/fit-autoplot.pdf]]
#+latex: \end{minipage}
#+latex: \begin{minipage}{0.3\linewidth}
#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/fit-autoplot2.pdf]]
#+latex: \end{minipage}
#+latex: \begin{minipage}{0.3\linewidth}
#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/fitAll-autoplot.pdf]]
#+latex: \end{minipage}


When considering continuous covariates, e.g.:
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
## add baseline weight
gastricbypassLB <- merge(gastricbypassL, gastricbypassW[c("id","weight1")], by = "id")

eUN.lmmB <- lmm(glucagonAUC ~ weight1 + visit*group, repetition = ~time|id,
                structure = "UN", data = gastricbypassLB)
#+END_SRC

#+RESULTS:


\noindent The default graphical display can be confusing as it shows
one curve per distinct set of covariate values, i.e. one line per
subject:
#+BEGIN_SRC R :file ./figures/fit-baseline-autoplot.pdf :results graphics file :session *R* :cache no
## left panel
plot(eUN.lmmB, type = "fit", color = "group", ci = FALSE, size.text = 20)
#+END_SRC

#+RESULTS:
[[file:./figures/fit-baseline-autoplot.pdf]]

It is possible to restrict the display specific to a covariate value
via the argument =at=:
#+BEGIN_SRC R :file ./figures/fit10-autoplot.pdf :results graphics file :session *R* :cache no
## middel panel
plot(eUN.lmmB, type = "fit", color = "group", ci = FALSE, size.text = 20,
     at = data.frame(weight1 = 150), obs.alpha = 0.2)
#+END_SRC

#+RESULTS:
[[file:./figures/fit10-autoplot.pdf]]

\clearpage

The =plot= method calls the =autoplot= methods which returns a list
containing:
- a ggplot2 object (element =plot=)
- the dataset used to generate the ggplot2 object (element =data=)
This should ease further customization of the graphical display, e.g.:
#+BEGIN_SRC R :file ./figures/fit-autoplot-indiv.pdf :results graphics file :session *R* :cache no :height 5 :width 5
## right panel
gg.traj <- autoplot(eUN.lmmB, type = "fit", color = "group", size.text = 20, facet =~id)
gg.traj$plot + theme(legend.position = "bottom")
#+END_SRC

#+RESULTS:
[[file:./figures/fit-autoplot-indiv.pdf]]

#+latex: \begin{minipage}{0.3\linewidth}
#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/fit-baseline-autoplot.pdf]]
#+latex: \end{minipage}
#+latex: \begin{minipage}{0.3\linewidth}
#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/fitAll-autoplot.pdf]]
#+latex: \end{minipage}
#+latex: \begin{minipage}{0.3\linewidth}
#+ATTR_LaTeX: :width \textwidth :placement [!h]
[[./figures/fit-autoplot-indiv.pdf]]
#+latex: \end{minipage}

\clearpage

** Partial residuals

In a linear model where we split the covariates and mean parameters into two sets:
#+BEGIN_EXPORT latex
\begin{align*}
Y_i = X_{1,i} \beta_1 + X_{2,i} \beta_2 + \varepsilon_i
\end{align*}
#+END_EXPORT

\noindent the partial residuals w.r.t. to the covariate(s) \(X_2\) are defined
by \(\varepsilon^{X_2}_{i} = Y_i - X_{1,i} \beta_1\). \newline They can be
computed via the =residuals= method:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
df.pres <- residuals(eUN.lmmB, type = "partial", variable = "weight1", keep.data = TRUE)
head(df.pres)
#+END_SRC

#+RESULTS:
:   id visit time weight glucagonAUC group baseline weight1  fitted r.partial
: 1  1     1  -13  127.2      20.690     0     TRUE   127.2 -20.684  -25.3242
: 2  1     1    1  115.5      92.600     0    FALSE   127.2 -20.684  -12.2923
: 3  1     1   -1  120.7      20.535     0     TRUE   127.2 -20.684  -24.7703
: 4  1     1   13  108.1      43.434     0    FALSE   127.2 -20.684  -37.3259
: 5 10     1   13   90.9      57.942     0    FALSE   118.0 -19.188   -7.1423
: 6 10     1    1   99.3     103.728     0    FALSE   118.0 -19.188   11.7323

In the output, the \(X_1\) covariates (=time= and =group=) have been
set to the reference level (=-13= and =0=) for all
observations. Confusion with the ordering variable from the
=repetition= argument of =lmm= was avoided by using a different 'time'
variable in the mean (=time=) and repetition argument (=visit=) when
calling =lmm=.  These residuals can be directly displayed via the
=plot= method:
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
## left panel
plot(eUN.lmmB, type = "partial", variable = "weight1")
## right panel
plot(eUN.lmmB, type = "partial", variable = c("(Intercept)","weight1"))
#+END_SRC

#+RESULTS:

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
gg1 <- autoplot(eUN.lmmB, type = "partial", variable = "weight1")$plot
gg2 <- autoplot(eUN.lmmB, type = "partial", variable = c("(Intercept)","weight1"))$plot
ggexport(ggarrange(gg1,gg2), filename = "./figures/fit-pres.pdf", width = 12, height = 5)
#+END_SRC

#+RESULTS:
: file saved to ./figures/fit-pres.pdf

#+ATTR_LaTeX: :width 0.75\textwidth :placement [!h]
[[./figures/fit-pres.pdf]]

The =plot= methods can handle one continuous and one categorical
covariate (in addition to the intercept) to display interaction
plots. In that case each observation/fitted line is colored according
to the categorical covariate.

\clearpage

** Statistical inference (single model, linear)

The =anova= method can be used to test one or several linear
combinations of the model coefficients using Wald tests. By default,
it will simultaneously test all parameters associated to a variable:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
anova(eUN.lmm)
#+END_SRC

#+RESULTS:
: 		Multivariate Wald test 
: 
:                      F-statistic       df p.value   
:    mean: visit             5.803 (3,16.9) 0.00647 **
:        : group             3.926 (1,18.0) 0.06302  .
:        : visit:group       2.762 (3,17.3) 0.07332  .

Note that here the p-values are not adjust for multiple comparisons
over variables. It is possible to specify a null hypothesis to be
test: e.g. is there a change in average weight just after taking the
treatment in the reference group:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
anova(eUN.lmm, effects = c("visit3-visit2=0"))
#+END_SRC

#+RESULTS:
: 		Multivariate Wald test 
: 
:           F-statistic       df p.value   
:    all: 1      14.318 (1,17.8) 0.00138 **

One can also simulateneously tests several null hypotheses:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.anova <- anova(eUN.lmm, effects = c("visit3-visit2=0","visit4-visit2=0"))
summary(e.anova)
#+END_SRC

#+RESULTS:
#+begin_example
		Multivariate Wald test 

          F-statistic       df p.value   
   all: 1       8.512 (2,17.2)  0.0027 **
   -------------------------------------- 
  Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.
  Degrees of freedom were computed using a Satterthwaite approximation (column df). 

		Univariate Wald test 

                   estimate    se   df  lower  upper p.value   
   visit3 - visit2   36.167 9.558 17.8 13.381 58.953 0.00263 **
   visit4 - visit2    9.256 7.738   18 -9.192 27.704 0.38153   
   ------------------------------------------------------------ 
  Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.
  Columns lower/upper/p.value adjusted for multiple comparisons -- max-test.
  (1e+05 samples have been used)
  Model-based standard errors are derived from the observed information (column se). 
  Degrees of freedom were computed using a Satterthwaite approximation (column df).
#+end_example

\clearpage

or return all pairwise comparisons for a given factor using the =mcp=
function of the multcomp package:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(multcomp)
summary(anova(eUN.lmm, effects = mcp(visit = "Tukey")))
#+END_SRC

#+RESULTS:
#+begin_example
Singular contrast matrix: contrasts "3 - 2" "4 - 2" "4 - 3" have been removed. 

		Multivariate Wald test 

              F-statistic       df p.value   
   all: visit       5.803 (3,16.9) 0.00647 **
   ------------------------------------------ 
  Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.
  Degrees of freedom were computed using a Satterthwaite approximation (column df). 

		Univariate Wald test 

         estimate    se   df   lower  upper p.value   
   2 - 1   -4.734 2.776 17.5 -12.451  2.982 0.32482   
   3 - 1   31.433  8.63 17.6   7.444 55.422 0.00860 **
   4 - 1    4.521 8.005   18 -17.731 26.774 0.93260   
   3 - 2   36.167 9.558 17.8   9.597 62.737 0.00660 **
   4 - 2    9.256 7.738   18 -12.256 30.767 0.60663   
   4 - 3  -26.912 7.448 16.4 -47.615 -6.209 0.00916 **
   --------------------------------------------------- 
  Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.
  Columns lower/upper/p.value adjusted for multiple comparisons -- max-test.
  (1e+05 samples have been used)
  Model-based standard errors are derived from the observed information (column se). 
  Degrees of freedom were computed using a Satterthwaite approximation (column df). 

Warning message:
In mcp2matrix(model, linfct = linfct) :
  covariate interactions found -- default contrast might be inappropriate
#+end_example

Here the =summary= method prints not only the global test but also the
result associated to each hypothesis. The warning is triggered by the
presence of an interaction between =visit= and =group=: the time
effect is only tested here for the reference group. One should look
also at the time effect in the other group before concluding about the
possible absence of a time effect.

\bigskip

*Special characters*: special characters, such as parentheses or
mathematical operators, can cause problems when using this
formula-like interface to specify linear contrasts on parameters. This
typically arises when testing (transformed) variance or correlation parameters,
parentheses:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
try(
  anova(eUN.lmm,
        effects = c("log(k).-1=0","log(k).1=0","log(k).13=0"))
)
#+END_SRC

#+RESULTS:
: Error in .anova_Wald(object, effects = effects, robust = robust, multivariate = multivariate,  : 
:   Possible mispecification of the argument 'effects' as running mulcomp::glht lead to the following error: 
: Error in parse(text = ex[i]) : <text>:1:7: unexpected symbol
: 1: log(k).
:           ^

It is then advised to build a contrast matrix, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
name.coef <- rownames(confint(eUN.lmm, effects = "all"))
name.varcoef <- grep("^k",name.coef, value = TRUE)
C <- matrix(0, nrow = 3, ncol = length(name.coef), dimnames = list(name.varcoef, name.coef))
diag(C[name.varcoef,name.varcoef]) <- 1
C[,1:9]
#+END_SRC

#+RESULTS:
:      (Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1 sigma
: k.-1           0      0      0      0      0             0             0             0     0
: k.1            0      0      0      0      0             0             0             0     0
: k.13           0      0      0      0      0             0             0             0     0

And then call the =anova= method specifying the null hypothesis via the
contrast matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
anova(eUN.lmm, effects = C)
#+END_SRC

#+RESULTS: 
: 		Multivariate Wald test 
: 
:           F-statistic       df p.value  
:    all: 1       3.388 (3,25.7)  0.0332 *

\clearpage

** Statistical inference (multiple models, linear)

It is possible to adjust for multiple testing across several linear
contrasts that may originate from differente =lmm= using the approach
of cite:pipper2012versatile:
- fit the mixed models using =lmm=. The LMM must be fitted on the same
  dataset (or on subsets on a common larger dataset) with the same =repetition= argument.
- use the =anova= method to indicate which hypotheses are being tested
- combine the tests using =rbind=.

Here is an (artificial) example:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
Manova <- rbind(anova(eInd.lmm, effects = "visit3:group1 = 0", robust = FALSE),
                anova(eCS.lmm, effects = "visit3:group1 = 0", robust = FALSE),
                anova(eUN.lmm, effects = "visit3:group1 = 0", robust = FALSE),
                name = c("Ind","CS","UN"))
summary(Manova) 
#+END_SRC

#+RESULTS:
#+begin_example
		Multivariate Wald test 

          Chi2-statistic      df p.value    
   all: 1          116.9 (3,Inf)  <1e-04 ***
   ----------------------------------------- 
  Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.

		Univariate Wald test 

                      estimate     se   df  lower  upper p.value  
   Ind: visit3:group1   27.001 14.285 25.3 -1.482 55.485  0.0631 .
   CS: visit3:group1    27.481  11.09 52.8  5.369 49.593  0.0137 *
   UN: visit3:group1    27.571  12.42 17.8  2.808 52.335  0.0268 *
   --------------------------------------------------------------- 
  Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1.
  Columns lower/upper/p.value adjusted for multiple comparisons -- max-test.
  (error when computing the adjusted columns lower/upper/p.value by numerical integration: 0.00073)
  Model-based standard errors are derived from the observed information (column se).
#+end_example

\clearpage

** Statistical inference (single model, non-linear)

The =estimate= function can be used to test one or several non-linear
combinations of model coefficients, using a first order delta method
to quantify uncertainty. The combination has to be specified via a
function (argument =f=). To illustrate its use consider an ANCOVA
analysis:
#+BEGIN_EXPORT latex
\[ Y_{i1} = \textcolor{\darkred}{\alpha} + \textcolor{\darkblue}{\beta} Y_{i,0} + \textcolor{\darkgreen}{\gamma} X_{i} + e_{i} \]
#+END_EXPORT

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.ANCOVA <- lm(weight4 ~ weight1 + group, data = gastricbypassW)
summary(e.ANCOVA)$coef
#+END_SRC

#+RESULTS:
:             Estimate Std. Error  t value   Pr(>|t|)
: (Intercept) -5.92851   8.780064 -0.67522 5.0861e-01
: weight1      0.82363   0.064116 12.84598 3.5247e-10
: group        4.14046   2.533355  1.63438 1.2056e-01

We can replicate this analysis by first fitting a mixed model:
#+BEGIN_EXPORT latex
\[ Y_{ij} = \alpha_j + \gamma_j X_{i} + \varepsilon_{i,j} \text{ where } \varepsilon_i \sim \Gaus \left( \begin{bmatrix} 0 \\ 0 \end{bmatrix}, \begin{bmatrix} \sigma^2_1 & \rho \sigma_1 \sigma_2 \\ \rho \sigma_1 \sigma_2 & \sigma^2_2 \end{bmatrix} \right) \]
#+END_EXPORT
#+BEGIN_SRC R :exports code :results output :session *R* :cache no
gastricbypassL14 <- gastricbypassL[gastricbypassL$visit %in% c(1,4),]
gastricbypassL14$visit <- droplevels(gastricbypassL14$visit)
e.lmmANCOVA <- lmm(weight ~ visit + visit:group, repetition = ~visit|id,
                   data = gastricbypassL14)
#+END_SRC

#+RESULTS:

and then perform a first order delta-method:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
lava::estimate(e.lmmANCOVA, f = function(p){
  c(Y1 = as.double(p["rho(1,4)"]*p["k.4"]),
    X1 = as.double(p["visit4:group1"]-p["rho(1,4)"]*p["k.4"]*p["visit1:group1"]))
})
#+END_SRC

#+RESULTS:
:    estimate       se      df    lower  upper    p.value
: Y1  0.82363 0.062309  9.8746  0.68456 0.9627 1.3327e-07
: X1  4.14046 2.461978 15.1613 -1.10227 9.3832 1.1309e-01

Indeed:
#+BEGIN_EXPORT latex
\begin{align*}
\Esp[Y_{i2}|Y_{i1},X_{i}] &= \alpha_2 + \gamma_2 X_{i} + \rho \frac{\sigma_2}{\sigma_1}\left(Y_{i1} - \alpha_1 - \gamma_1 X_{i}\right) \\
                         &= \textcolor{\darkred}{\alpha_2 - \rho \frac{\sigma_2}{\sigma_1} \alpha_1}
                         + \textcolor{\darkblue}{\rho \frac{\sigma_2}{\sigma_1}Y_{i1}}
                         + \textcolor{\darkgreen}{\left(\gamma_2 - \rho \frac{\sigma_2}{\sigma_1} \gamma_1\right)  X_{i} }
\end{align*}
#+END_EXPORT

We obtain identical estimate but different standard-errors/degrees of
freedom compared to the univariate linear model approach. The later is
to be prefer as it does not rely on approximation. The former is
nevertheless useful as it can handle missing data in the outcome
variable.

\clearpage

** Baseline adjustment

In clinical trial the group and intervention variable often do not
coincide, e.g., in presence of baseline measurement. In our running
example, the first two measurement are pre-treatment (i.e. treatment
should be ="none"=) while the last two measurements are post-treatment
(i.e. treatment should be =1= or =2=). The =baselineAdjustment=
function can be helpful to define a time varying treatment variable:
- where baseline takes a specific value
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
gastricbypassL$treat <- baselineAdjustment(gastricbypassL, variable = "group",
                                repetition = ~visit|id, constrain = c("1","2"),
                                new.level = "none")
table(treat = gastricbypassL$treat,
      visit = gastricbypassL$visit,
      group = gastricbypassL$group)
#+END_SRC

#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
table(treat = gastricbypassL$treat, visit = gastricbypassL$visit, group = gastricbypassL$group)[,,1,drop=FALSE]
#+END_SRC

#+RESULTS:
: , , group = 0
: 
:       visit
: treat   1  2  3  4
:   none 10 10  0  0
:   0     0  0 10 10
:   1     0  0  0  0
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.05\linewidth} 
#+LaTeX: \hphantom{x}
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
table(treat = gastricbypassL$treat, visit = gastricbypassL$visit, group = gastricbypassL$group)[,,2,drop=FALSE]
#+END_SRC

#+RESULTS:
: , , group = 1
: 
:       visit
: treat   1  2  3  4
:   none 10 10  0  0
:   0     0  0  0  0
:   1     0  0 10 10

#+LaTeX: \end{minipage}


- where baseline corresponds to the reference group
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
gastricbypassL$treat2 <- baselineAdjustment(gastricbypassL, variable = "group",
                                 repetition = ~visit|id, constrain = c("1","2"))
table(treat = gastricbypassL$treat2,
      visit = gastricbypassL$visit,
      group = gastricbypassL$group)
#+END_SRC

#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
table(treat = gastricbypassL$treat2, visit = gastricbypassL$visit, group = gastricbypassL$group)[,,1,drop=FALSE]
#+END_SRC

#+RESULTS:
: , , group = 0
: 
:      visit
: treat  1  2  3  4
:     0 10 10 10 10
:     1  0  0  0  0
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.05\linewidth} 
#+LaTeX: \hphantom{x}
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
table(treat = gastricbypassL$treat2, visit = gastricbypassL$visit, group = gastricbypassL$group)[,,2,drop=FALSE]
#+END_SRC

#+RESULTS:
: , , group = 1
: 
:      visit
: treat  1  2  3  4
:     0 10 10  0  0
:     1  0  0 10 10

#+LaTeX: \end{minipage}

- including interactions with group
#+BEGIN_SRC R :exports code :results silent :session *R* :cache no
gastricbypassL$visitXtreat <- baselineAdjustment(gastricbypassL, variable = "group",
                                      repetition = ~visit|id, constrain = c("1","2"),
                                      collapse.time = ".")

table(treat = gastricbypassL$visitXtreat,
      visit = gastricbypassL$visit,
      group = gastricbypassL$group)
#+END_SRC

#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
table(treat = gastricbypassL$visitXtreat, visit = gastricbypassL$visit, group = gastricbypassL$group)[,,1,drop=FALSE]
#+END_SRC

#+RESULTS:
#+begin_example
, , group = 0

     visit
treat  1  2  3  4
  1   10  0  0  0
  2    0 10  0  0
  3.0  0  0 10  0
  4.0  0  0  0 10
  3.1  0  0  0  0
  4.1  0  0  0  0
#+end_example
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.05\linewidth} 
#+LaTeX: \hphantom{x}
#+LaTeX: \end{minipage}
#+LaTeX: \begin{minipage}{0.45\linewidth} 
#+BEGIN_SRC R :exports results :results output :session *R* :cache no
table(treat = gastricbypassL$visitXtreat, visit = gastricbypassL$visit, group = gastricbypassL$group)[,,2,drop=FALSE]
#+END_SRC

#+RESULTS:
#+begin_example
, , group = 1

     visit
treat  1  2  3  4
  1   10  0  0  0
  2    0 10  0  0
  3.0  0  0  0  0
  4.0  0  0  0  0
  3.1  0  0 10  0
  4.1  0  0  0 10
#+end_example

#+LaTeX: \end{minipage}

We would then typically like to model group differences only after
baseline (i.e. only at 1 week and 3 months after). This can be
performed using the time varying treatment variable, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eC.lmm <- lmm(glucagonAUC ~ visitXtreat, data = gastricbypassL,
              repetition = ~visit|id, structure = "UN")
coef(eC.lmm) ## change from baseline
#+END_SRC

#+RESULTS:
:    (Intercept)   visitXtreat2 visitXtreat3.0 visitXtreat4.0 visitXtreat3.1 visitXtreat4.1 
:        32.3167        -2.7478        34.3703        11.6559        56.0581        27.6108

or
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eC2.lmm <- lmm(glucagonAUC ~ 0 + visitXtreat, data = gastricbypassL,
              repetition = ~visit|id, structure = "UN")
coef(eC2.lmm) ## absolute value
#+END_SRC

#+RESULTS:
:   visitXtreat1   visitXtreat2 visitXtreat3.0 visitXtreat4.0 visitXtreat3.1 visitXtreat4.1 
:         32.317         29.569         66.687         43.973         88.375         59.927

The parametrization however does not (directly) output treatment
effects. Instead one may be tempted to use a formula like
=treatment*time=. However this will lead to a non-indentifiable
model. Indeed we are only able to estimate a total of 6 means when
constraining the expected baseline value between the two groups to be
the same. Therefore can at most identify 6 effects. However the design
matrix for the interaction model:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
colnames(model.matrix(glucagonAUC ~ treat*visit, data = gastricbypassL))
#+END_SRC

#+RESULTS:
:  [1] "(Intercept)"   "treat0"        "treat1"        "visit2"        "visit3"        "visit4"       
:  [7] "treat0:visit2" "treat1:visit2" "treat0:visit3" "treat1:visit3" "treat0:visit4" "treat1:visit4"

contains 12 parameters (i.e. 6 too many). Fortunately, the =lmm= will
 drop non-identifiable effects from the model and fit the resulting
 simplified model:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eC3.lmm <- lmm(glucagonAUC ~ treat2*visit, data = gastricbypassL,
               repetition = ~visit|id, structure = "UN")
#+END_SRC

#+RESULTS:
: Constant values in the design matrix for the mean structure.
: Coefficients "treat21" "treat21:visit2" relative to interactions "treat2:visit" have been removed.

with the following coefficients:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
model.tables(eC3.lmm)
#+END_SRC

#+RESULTS:
:                estimate      se     df   lower   upper    p.value
: (Intercept)     32.3167  3.4764 19.003 25.0407 39.5927 1.6802e-08
: visit2          -2.7478  1.9950 19.007 -6.9232  1.4276 1.8441e-01
: visit3          34.3703  8.6111 15.161 16.0331 52.7076 1.1573e-03
: visit4          11.6559  7.5601 17.328 -4.2715 27.5833 1.4119e-01
: treat21:visit3  21.6878 12.2967 13.748 -4.7316 48.1072 9.9982e-02
: treat21:visit4  15.9549  9.6174 12.374 -4.9296 36.8395 1.2224e-01

One can vizualize the baseline adjustment via the =plot= function:
#+BEGIN_SRC R :file ./figures/gg-baseAdj.pdf :results graphics file :session *R* :cache no
plot(eC3.lmm, color = "group", ci = FALSE, size.text = 20, obs.alpha = 0.1)
#+END_SRC

#+RESULTS:
[[file:./figures/gg-baseAdj.pdf]]

#+ATTR_LaTeX: :width 0.4\textwidth :placement [!h]
[[./figures/gg-baseAdj.pdf]]

and retrieve the treatment at each timepoint using the =effects= method:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eC3.lmm, variable = "treat2", type = "difference")
#+END_SRC

#+RESULTS:
: 		Difference in average counterfactual outcome
: 		 w.r.t 'treat2' values 
: 
:                    estimate     se   df  lower  upper p.value  
:    treat2=1-0(t=1)        0      0  Inf      0      0      NA  
:    treat2=1-0(t=2)        0      0  Inf      0      0      NA  
:    treat2=1-0(t=3)   21.688 12.297 13.7 -4.732 48.107   0.100 .
:    treat2=1-0(t=4)   15.955  9.617 12.4  -4.93  36.84   0.122

\clearpage

** Predictions

Two types of predictions can be performed with the =predict= method:
- *static predictions* that are only conditional on the covariates:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
news <- gastricbypassL[gastricbypassL$id==2,]
news$glucagon <- 0
predict(eUN.lmm, newdata = news, se = TRUE)
#+END_SRC

#+RESULTS:
:   estimate     se     df  lower  upper
: 1   38.729 4.5765 18.003 29.114 48.344
: 2   33.995 4.1002 17.897 25.377 42.612
: 3   70.162 8.6491 17.695 51.968 88.356
: 4   43.250 6.1883 18.005 30.249 56.251

which can be computing by creating a design matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
X.12 <- model.matrix(formula(eUN.lmm), news)
X.12
#+END_SRC

#+RESULTS:
#+begin_example
   (Intercept) visit2 visit3 visit4 group1 visit2:group1 visit3:group1 visit4:group1
2            1      0      0      0      0             0             0             0
22           1      1      0      0      0             0             0             0
42           1      0      1      0      0             0             0             0
62           1      0      0      1      0             0             0             0
attr(,"assign")
[1] 0 1 1 1 2 3 3 3
attr(,"contrasts")
attr(,"contrasts")$visit
[1] "contr.treatment"

attr(,"contrasts")$group
[1] "contr.treatment"
#+end_example

and then multiplying it with the regression coefficients:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
X.12 %*% coef(eUN.lmm)
#+END_SRC

#+RESULTS:
:      [,1]
: 2  38.729
: 22 33.995
: 42 70.162
: 62 43.250

\clearpage

- *dynamic predictions* that are conditional on the covariates and the
  outcome measured at other timepoints. Consider two subjects for who
  we would like to predict the weight 1 week before the intervention
  based on the weight 3 months before the intervention:
  
#+ATTR_LATEX: :options otherkeywords={}, deletekeywords={}
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
newd <- rbind(
  data.frame(id = 1, time = -13, visit = "1", group = 0, glucagonAUC = coef(eUN.lmm)["(Intercept)"]),
  data.frame(id = 1, time = 1, visit = "3", group = 0, glucagonAUC = NA),
  data.frame(id = 2, time = -13, visit = "1", group = 0, glucagonAUC = 50),
  data.frame(id = 2, time = 1, visit = "3", group = 0, glucagonAUC = NA)
)
predict(eUN.lmm, newdata = newd, type = "dynamic", keep.data = TRUE)
#+END_SRC

#+RESULTS:
:   id time visit group glucagonAUC estimate     se     df  lower  upper
: 1  1  -13     1     0      38.729       NA     NA     NA     NA     NA
: 2  1    1     3     0          NA   70.162 8.3308 17.592 52.630 87.694
: 3  2  -13     1     0      50.000       NA     NA     NA     NA     NA
: 4  2    1     3     0          NA   75.888 9.6360 12.711 55.022 96.753
  
The first subjects starts with the average glucagon while the second
  starts with a much higher glucagon. The predicted glucagon after the
  operation for the first subject is then the average glucagon while
  it is predicted to be higher for the second subject due to the
  positive correlation over time. The predicted value is computed
  using the formula of the conditional mean for a Gaussian vector:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
mu1 <- coef(eUN.lmm)["(Intercept)"]
mu3 <- mu1 + coef(eUN.lmm)["visit3"]
Omega_11 <- sigma(eUN.lmm)[1,1]
Omega_31 <- sigma(eUN.lmm)[3,1]
as.double(mu3 + Omega_31 * (50 - mu1) / Omega_11)
#+END_SRC

#+RESULTS:
: [1] 75.888


\clearpage

* Equivalence with other statistical methods
** Welch two sample t-test

A two sample t-test:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
t.test(weight4 ~ group, data = gastricbypassW)
#+END_SRC

#+RESULTS:
#+begin_example

	Welch Two Sample t-test

data:  weight4 by group
t = 0.591, df = 17.7, p-value = 0.56
alternative hypothesis: true difference in means between group 0 and group 1 is not equal to 0
95 percent confidence interval:
 -11.736  20.916
sample estimates:
mean in group 0 mean in group 1 
         104.66          100.07
#+end_example

is equivalent to an independent covariance pattern with a different
variable for each group:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.ttest4 <- lmm(weight4 ~ group, structure = IND(~group), 
               data = gastricbypassW, trace = FALSE)
model.tables(e.ttest4)
#+END_SRC

#+RESULTS:
:             estimate     se      df   lower   upper    p.value
: (Intercept)   104.66 5.1045  9.0018  93.113 116.207 7.2710e-09
: group          -4.59 7.7607 17.6824 -20.916  11.736 5.6171e-01

\clearpage

** Paired t-test

With complete data, a paired t-test:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
t.test(gastricbypassW$weight4, gastricbypassW$weight1, paired = TRUE)
#+END_SRC

#+RESULTS:
#+begin_example

	Paired t-test

data:  gastricbypassW$weight4 and gastricbypassW$weight1
t = -17.2, df = 19, p-value = 5e-13
alternative hypothesis: true mean difference is not equal to 0
95 percent confidence interval:
 -29.848 -23.362
sample estimates:
mean difference 
        -26.605
#+end_example

is equivalent to a LMM with an unstructured covariate pattern:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmm2tt <- lmm(weight ~ visit, repetition = ~visit|id, structure = "UN",
                data = gastricbypassL)
model.tables(e.lmm2tt)["visit4",,drop=FALSE]
#+END_SRC

#+RESULTS:
:        estimate     se     df   lower   upper    p.value
: visit4  -26.605 1.5494 18.964 -29.848 -23.362 5.1692e-13

\clearpage

** Welch two sample t-test on the change

With complete data, a two sample t-test comparing the change from baseline:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassW.0 <- gastricbypassW[gastricbypassW$group==0,]
gastricbypassW.1 <- gastricbypassW[gastricbypassW$group==1,]
t.test(gastricbypassW.0$weight4-gastricbypassW.0$weight1,
       gastricbypassW.1$weight4-gastricbypassW.1$weight1)
#+END_SRC

#+RESULTS:
#+begin_example
	Welch Two Sample t-test

data:  gastricbypassW.0$weight4 - gastricbypassW.0$weight1 and gastricbypassW.1$weight4 - gastricbypassW.1$weight1
t = -2.11, df = 13, p-value = 0.055
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -12.16771   0.14771
sample estimates:
mean of x mean of y 
   -29.61    -23.60
#+end_example

is equivalent to a LMM with a stratified unstructured covariate pattern:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmm2tt2 <- lmm(weight ~ visit*group, repetition = ~visit|id, structure = UN(~group),
                 data = gastricbypassL)
model.tables(e.lmm2tt2)["visit4:group1",,drop=FALSE]
#+END_SRC

#+RESULTS:
:               estimate     se     df    lower  upper p.value
: visit4:group1     6.01 2.8511 13.009 -0.14908 12.169   0.055

\clearpage

** Multiple Student's t-test


Multiple t-tests:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.ttest1 <- lmm(weight1 ~ group, structure = IND(~group), 
                data = gastricbypassW, trace = FALSE)
e.ttest2 <- lmm(weight2 ~ group, structure = IND(~group), 
                data = gastricbypassW, trace = FALSE)
e.ttest3 <- lmm(weight3 ~ group, structure = IND(~group), 
                data = gastricbypassW, trace = FALSE)
#+END_SRC

#+RESULTS:

can be adjusted for multiple comparison by first using the =anova=
function to specify the parameter of interest and combining the
results using =rbind=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.mttest <- rbind(anova(e.ttest1, effects = "group=0"),
                  anova(e.ttest2, effects = "group=0"),
                  anova(e.ttest3, effects = "group=0"),
                  anova(e.ttest4, effects = "group=0"))
model.tables(e.mttest, method = "bonferroni")
#+END_SRC

#+RESULTS:
:                estimate     se     df   lower  upper p.value
: weight1: group   -10.60 8.9717 17.965 -35.498 14.298       1
: weight2: group    -9.50 8.3951 17.985 -32.795 13.795       1
: weight3: group    -8.92 8.1295 17.959 -31.481 13.641       1
: weight4: group    -4.59 7.7607 17.682 -26.165 16.985       1

\Warning efficient adjustment for multiple comparisons (like
="single-step"=) will not be valid as the correlation structure has
not be specified. To do so it is more conveniently to work with a the
long format:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.mttest2 <- mlmm(weight ~ group, structure = IND(~group),
                  data = gastricbypassL, trace = FALSE,
                  effects = "group1=0", by = "time", repetition = ~time|id)
model.tables(e.mttest2, method = "single-step2")
#+END_SRC

#+RESULTS:
:    by parameter estimate     se     df   lower   upper p.value
: 1 -13    group1   -10.60 8.9717 17.965 -30.989  9.7893 0.31768
: 2  -1    group1    -9.50 8.3951 17.985 -28.579  9.5789 0.34123
: 3   1    group1    -8.92 8.1295 17.959 -27.395  9.5551 0.35785
: 4  13    group1    -4.59 7.7607 17.682 -22.227 13.0470 0.66473

or call the dedicated function =mt.test=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
mt.test(weight1+weight2+weight3+weight4~group, data = gastricbypassW)
#+END_SRC

#+RESULTS:
:        by parameter estimate     se     df   lower   upper p.value
: 1 weight1     group   -10.60 8.9717 17.965 -30.976  9.7758 0.31870
: 2 weight2     group    -9.50 8.3951 17.985 -28.566  9.5663 0.34115
: 3 weight3     group    -8.92 8.1295 17.959 -27.383  9.5429 0.35844
: 4 weight4     group    -4.59 7.7607 17.682 -22.215 13.0354 0.66272

\clearpage

** Linear regression on the change 

A widely spread approach to analyze longitudinal data is to reduce the
number of repetitions to 1 by working on the change and then apply
'usual' statistical methods. For instance one could compare the pre-
and post- operation values using:

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassW$changeG41 <- gastricbypassW$glucagonAUC4-gastricbypassW$glucagonAUC1
e.change41 <- lm(changeG41 ~ weight1, data = gastricbypassW)
summary(e.change41)$coef
#+END_SRC

#+RESULTS:
:             Estimate Std. Error t value Pr(>|t|)
: (Intercept) 88.41370   41.01024  2.1559 0.044871
: weight1     -0.53331    0.31432 -1.6967 0.106975

This turns out to be equivalent to the following mixed model:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassL41 <- gastricbypassL[gastricbypassL$visit %in% c(1,4),]
gastricbypassL41$visit <- droplevels(gastricbypassL41$visit)
gastricbypassL41$weight1 <- gastricbypassW$weight1[gastricbypassL41$id]

e.lmm41 <- lmm(glucagonAUC ~ visit + visit*weight1,
               repetition =~ visit|id, structure = "UN",
               data = gastricbypassL41)
model.tables(e.lmm41)
#+END_SRC

#+RESULTS:
:                  estimate       se     df     lower     upper p.value
: (Intercept)    31.7805917 23.58747 18.003 -17.77425  81.33543 0.19458
: visit4         88.4137014 41.01024 18.001   2.25477 174.57264 0.04487
: weight1         0.0041566  0.18078 18.003  -0.37565   0.38396 0.98191
: visit4:weight1 -0.5333052  0.31432 18.001  -1.19366   0.12705 0.10697

This equivalence only holds as there is no missing data.
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
index.missing41 <- which(is.na(gastricbypassW$changeG41))
index.missing41
#+END_SRC

#+RESULTS:
: integer(0)

\clearpage

** Correlation between changes 

In some studies, one is interested in studying the relation between
two evolutions. Say weight and glucagon before and after the
operation:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassW$changeG41 <- gastricbypassW$glucagonAUC4-gastricbypassW$glucagonAUC1
gastricbypassW$changeW41 <- gastricbypassW$weight4-gastricbypassW$weight1
#+END_SRC

#+RESULTS:

\bigskip

One can evaluate their correlation:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
cor.test(gastricbypassW$changeW41, gastricbypassW$changeG41)
#+END_SRC

#+RESULTS:
#+begin_example

	Pearson's product-moment correlation

data:  gastricbypassW$changeW41 and gastricbypassW$changeG41
t = 1.89, df = 18, p-value = 0.075
alternative hypothesis: true correlation is not equal to 0
95 percent confidence interval:
 -0.043829  0.719624
sample estimates:
    cor 
0.40658
#+end_example

or regress one against the other:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e2.change41 <- lm(changeG41 ~ changeW41, data = gastricbypassW)
summary(e2.change41)$coef
#+END_SRC

#+RESULTS:
:             Estimate Std. Error t value Pr(>|t|)
: (Intercept)  65.0794   24.83368  2.6206 0.017331
: changeW41     1.7082    0.90473  1.8881 0.075246

This problem can be recast using all measurement as outcomes:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
keep.col <- c("id","weight1","weight4","glucagonAUC1","glucagonAUC4")
gastricbypassL4 <- reshape(gastricbypassW[,keep.col], direction = "long",
                           idvar = "id", varying = 2:5, timevar = "type", v.names = "value")
gastricbypassL4$type <- factor(gastricbypassL4$type, labels = keep.col[-1])
gastricbypassL4 <- gastricbypassL4[order(gastricbypassL4$id),]
head(gastricbypassL4)
#+END_SRC

#+RESULTS:
:     id         type   value
: 1.1  1      weight1 127.200
: 1.2  1      weight4 108.100
: 1.3  1 glucagonAUC1  20.690
: 1.4  1 glucagonAUC4  43.434
: 2.1  2      weight1 165.200
: 2.2  2      weight4 132.000

fitting an unstructured mixed model:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmm4 <- lmm(value ~ type,
              repetition = ~type|id, structure = "UN",
              data = gastricbypassL4)
#+END_SRC

#+RESULTS:

extract the residual covariance matrix:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigma.lmm4 <- sigma(e.lmm4)
sigma.lmm4
#+END_SRC

#+RESULTS:
:                weight1 weight4 glucagonAUC1 glucagonAUC4
: weight1       410.8475  326.84       1.7077     -217.399
: weight4       326.8357  290.84     -24.6003     -161.696
: glucagonAUC1    1.7077  -24.60     241.7007      -81.649
: glucagonAUC4 -217.3994 -161.70     -81.6493      442.464

Deduce the residual covariance matrix for the change:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
Mcon <- cbind(c(-1,1,0,0),c(0,0,-1,1))
sigmeChange.lmm4 <- t(Mcon) %*% sigma.lmm4 %*% Mcon
dimnames(sigmeChange.lmm4) <- list(c("d.weight","d.glucagonAUC"),
                                   c("d.weight","d.glucagonAUC"))
sigmeChange.lmm4
#+END_SRC

#+RESULTS:
:               d.weight d.glucagonAUC
: d.weight        48.011        82.011
: d.glucagonAUC   82.011       847.464

and the corrrelation or covariance:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
cov2cor(sigmeChange.lmm4)[1,2]
sigmeChange.lmm4[1,2]/sigmeChange.lmm4[1,1]
#+END_SRC

#+RESULTS:
: [1] 0.40658
: [1] 1.7082

The uncertainty can be quantified using a delta method:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
estimate(e.lmm4, function(p){
  Sigma.change <- t(Mcon) %*% sigma(e.lmm4, p = p) %*% Mcon
  c(cor = cov2cor(Sigma.change)[1,2],
    beta = Sigma.change[1,2]/Sigma.change[1,1])
})
#+END_SRC

#+RESULTS:
:      estimate      se     df    lower  upper p.value
: cor   0.40658 0.19150 2.5925 -0.26078 1.0739 0.13791
: beta  1.70818 0.88073 2.6876 -1.28836 4.7047 0.15837

The standard errors and degrees of freedom do not match the univariate
analysis, suggesting poor small sample properties of this
technic.

\clearpage

* Missing values and imputation

We reconsider the example of the previous section, but now in presence
of missing values. The =summarize= function can be used to describe
the amount of missing data at each repetition:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sss <- summarize(glucagonAUC ~ time, data = gastricbypassL, na.rm = TRUE)
cbind(sss[,1:4], pc = paste0(100 * sss$missing / (sss$missing + sss$observed), "%"))
#+END_SRC

#+RESULTS:
:       outcome time observed missing pc
: 1 glucagonAUC  -13       20       0 0%
: 2 glucagonAUC   -1       19       1 5%
: 3 glucagonAUC    1       19       1 5%
: 4 glucagonAUC   13       20       0 0%

For more detail about the missing data patters, see the =summarizeNA=
function:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
summarizeNA(data = gastricbypassL, repetition = ~ time|id)
#+END_SRC

#+RESULTS:
#+begin_example
    variable frequency missing.pattern n.missing id -13 -1 1 13
       visit        20           00000         0  0   0  0 0  0
      weight        20           00000         0  0   0  0 0  0
 glucagonAUC        18           00000         0  0   0  0 0  0
                     1           00010         1  0   0  0 1  0
                     1           00100         1  0   0  1 0  0
       group        20           00000         0  0   0  0 0  0
    baseline        20           00000         0  0   0  0 0  0
       treat        20           00000         0  0   0  0 0  0
      treat2        20           00000         0  0   0  0 0  0
  timeXtreat        20           00000         0  0   0  0 0  0
 visitXtreat        20           00000         0  0   0  0 0  0
      group2        20           00000         0  0   0  0 0  0
#+end_example

To begin with we will only consider 1 week before and 1 week after
surgery:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
## long format
gastricbypassL32 <- gastricbypassL[gastricbypassL$visit %in% c(3,2),]
gastricbypassL32$visit <- droplevels(gastricbypassL32$visit)
gastricbypassL32$weight1 <- gastricbypassW$weight1[gastricbypassL32$id]
## wide format
gastricbypassW$changeG32 <- gastricbypassW$glucagonAUC3-gastricbypassW$glucagonAUC2
#+END_SRC

#+RESULTS:

\clearpage

** Full information approach

LMM uses a full information approach:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmm32 <- lmm(glucagonAUC ~ visit + visit*weight1,
               repetition =~ visit|id, structure = "UN",
               data = gastricbypassL32)
model.tables(e.lmm32)
#+END_SRC

#+RESULTS:
:                 estimate       se     df     lower     upper    p.value
: (Intercept)      9.24975 20.66248 17.011 -34.34202  52.84153 0.66004717
: visit3         170.75489 39.66297 17.808  87.36171 254.14806 0.00043568
: weight1          0.15750  0.15734 17.011  -0.17444   0.48945 0.33083625
: visit3:weight1  -0.95238  0.30205 17.648  -1.58787  -0.31688 0.00560372

whereas a linear model would perform a complete case approach:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.change32 <- lm(changeG32 ~ weight1, data = gastricbypassW)
summary(e.change32)$coef
#+END_SRC

#+RESULTS:
:              Estimate Std. Error t value   Pr(>|t|)
: (Intercept) 173.46620   41.75201  4.1547 0.00074599
: weight1      -0.96982    0.31589 -3.0701 0.00732363

In the former the likelihood is evaluated using all observations, even
those from individuals with some (but not all) missing outcome values:
baseline is used even if follow-up is missing. In the later the
likelihood is only evaluated on individuals with no missing outcome
values: if follow-up is missing then baseline is not used. Indeed:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
coef(lm(changeG32 ~ weight1, data = gastricbypassW[-c(5,15),]))
#+END_SRC

#+RESULTS:
: (Intercept)     weight1 
:   173.46620    -0.96982

The estimates of the LMM can be retrived using a linear model where we
have imputed the conditional expectation of the missing values given
the observed value and the estimated model parameters: (see section
[[#imputation]] for a graphical representation)
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassWA <- fitted(e.lmm32, type = "outcome", format = "wide")
gastricbypassWA$change32 <- gastricbypassWA$glucagonAUC_3 - gastricbypassWA$glucagonAUC_2
gastricbypassWA$weight1 <- gastricbypassW$weight1[match(gastricbypassW$id,gastricbypassWA$id)]
coef(lm(change32 ~ weight1, data = gastricbypassWA))
#+END_SRC

#+RESULTS:
: (Intercept)     weight1 
:   170.75489    -0.95238


\Warning Standard errors, confidence intervals, and p-values from this
linear model should not be trusted as they do not account for the
uncertainty in the imputed values.

** Complete case approach

The =lmmCC= can be used to obtain the LMM that is equivalent to a
linear regression. In the case of the comparing the change between
groups, the =repetition= argument should indicate how the change has
been computed:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmmCC <- lmmCC(e.change32, repetition = changeG32 ~ glucagonAUC3-glucagonAUC2|id)
model.tables(e.lmmCC)
#+END_SRC

#+RESULTS:
: Remove 2 clusters (4 observations) 
:  - 2 observations with missing data (2 clusters) 
:  - 0 missing repetitions (0 clusters)
:                estimate       se df      lower     upper    p.value
: (Intercept)  -165.90910 55.22956 16 -282.99061 -48.82760 0.00840925
: time          173.46620 41.75201 16   84.95611 261.97630 0.00074594
: weight1         1.13813  0.41786 16    0.25231   2.02395 0.01502328
: time:weight1   -0.96982  0.31589 16   -1.63948  -0.30017 0.00732343

As output, the data from two clusters (i.e. 4 observations) has been
excluded before fitting the LMM (instead of just the 2 observations
with missing values for the full information approach). The
interaction term of the LMM matches the regression coefficient of the
linear model:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
summary(e.change32)$coef
#+END_SRC

#+RESULTS:
:              Estimate Std. Error t value   Pr(>|t|)
: (Intercept) 173.46620   41.75201  4.1547 0.00074599
: weight1      -0.96982    0.31589 -3.0701 0.00732363

In the case of regressing two changes:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassW$changeW32 <- gastricbypassW$weight3 - gastricbypassW$weight2

e2g.change32 <- lm(changeG32 ~ changeW32 + group, data = gastricbypassW)
summary(e2g.change32)$coef
#+END_SRC

#+RESULTS:
:             Estimate Std. Error  t value Pr(>|t|)
: (Intercept)   16.998    30.9485  0.54924  0.59093
: changeW32     -3.288     5.0127 -0.65594  0.52180
: group         26.361    15.7415  1.67463  0.11472

the =repetition= argument should indicate how each change has
been computed:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e2.lmmCC <-  lmmCC(e2g.change32, repetition = list(changeG32 ~ glucagonAUC3-glucagonAUC2|id,
                                                   changeW32 ~ weight3-weight2|id))
model.tables(e2.lmmCC)
#+END_SRC

#+RESULTS:
: Remove 2 clusters (8 observations) 
:  - 2 observations with missing data (2 clusters) 
:  - 0 missing repetitions (0 clusters)
:      estimate      se     df    lower   upper p.value
: cor  -0.16699 0.24255 1.7257  -1.3868  1.0529 0.57192
: beta -3.28804 4.83126 2.3131 -21.5987 15.0226 0.55791

We retrieve the same estimate for the effect of change in weights but
the uncertainty (standard error, confidence intervals, p.value) do not
match. They should be asymptotically correct but may not have very
good small smaple properties.

** Imputation 
:PROPERTIES:
:CUSTOM_ID: imputation
:END:

When fitting a linear mixed model on a dataset with missing values:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.lmmNA <- lmm(glucagonAUC ~ time, repetition = ~time|id, data = gastricbypassL)
nobs(eUN.lmmNA)
#+END_SRC

#+RESULTS:
:             obs         cluster     missing.obs missing.cluster 
:              78              20               2               0

It is possible to extract the most likely value for these missing
observations using the =fitted= function with argument =impute=TRUE=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eData <- fitted(eUN.lmmNA, type = "outcome", keep.data = TRUE)
eData <- eData[order(eData$id,eData$time),]
eData[eData$id %in% eData[eData$impute,"id"],c("id","visit","time","glucagonAUC","impute")]
#+END_SRC

#+RESULTS:
:    id visit time glucagonAUC impute
: 15 15     1  -13      22.244  FALSE
: 35 15     2   -1      32.544  FALSE
: 55 15     3    1      69.719   TRUE
: 75 15     4   13      85.222  FALSE
: 5   5     1  -13      29.151  FALSE
: 25  5     2   -1      26.270   TRUE
: 45  5     3    1      86.859  FALSE
: 65  5     4   13      57.970  FALSE

Missing outcome values in the dataset have been replaced by its most
likely value (which is the same as the dynamic prediction, describedy
previously). A column =impute= has also been added to differentiate
between the the modeled and observed value. Visually:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
ggplot(eData, aes(x=time,y=glucagonAUC, group=id)) + geom_line() + geom_point(aes(color=impute))
#+END_SRC

#+RESULTS:

#+BEGIN_SRC R :exports none :results output raw drawer :session *R* :cache no
gg <- ggplot(eData, aes(x=time,y=glucagonAUC, group=id)) + geom_line() + geom_point(aes(color=impute), size=2)
ggsave(gg+ theme(text = element_text(size=20)), filename = "./figures/imputation.pdf", width = 12)
#+END_SRC
#+RESULTS:
:results:
[1m[22mSaving 12 x 7 in image
:end:

#+ATTR_LaTeX: :width 1\textwidth :options trim={0 0 0 0} :placement [!h]
[[./figures/imputation.pdf]]

It is possible to sample from the estimated distribution of the
missing value instead of using the most likely value, e.g. accounting
for residual variance and uncertainty related to parameter estimation:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
index.na <- which(is.na(gastricbypassL$glucagonAUC))
set.seed(1)
fitted(eUN.lmmNA, type = "impute", se = c(TRUE,TRUE))[index.na]
set.seed(2)
fitted(eUN.lmmNA, type = "impute", se = c(TRUE,TRUE))[index.na]
set.seed(3)
fitted(eUN.lmmNA, type = "impute", se = c(TRUE,TRUE))[index.na]
#+END_SRC

#+RESULTS:
: [1] 21.932 75.390
: [1] 20.060 75.428
: [1] 19.610 60.684

\clearpage

** Multiple imputation

The =mlmm= function can used to perform stratify analyses, typically
useful when performing multiple imputations. Consider the wide format
of the dataset where a few values are missing:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
data(gastricbypassW, package = "LMMstar")
colSums(is.na(gastricbypassW))
#+END_SRC

#+RESULTS:
:           id      weight1      weight2      weight3      weight4 glucagonAUC1 glucagonAUC2 
:            0            0            0            0            0            0            1 
: glucagonAUC3 glucagonAUC4 
:            1            0

We use =mice= to generate a number of imputed datasets (here 5):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(mice)
set.seed(10)
gastricbypassW.mice <- mice(gastricbypassW, m = 5, printFlag = FALSE)
gastricbypassW.NNA <- complete(gastricbypassW.mice, action = "long")
table(gastricbypassW.NNA$.imp)
#+END_SRC

#+RESULTS:
: Warning message:
: Number of logged events: 108
: 
:  1  2  3  4  5 
: 20 20 20 20 20

We can then use =mlmm= to perform a separate linear regression per dataset:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.mlmm <- mlmm(glucagonAUC3~glucagonAUC2+weight2, data=gastricbypassW.NNA,
               by = ".imp", effects = "weight2=0", trace = FALSE)
model.tables(e.mlmm)
#+END_SRC

#+RESULTS:
:   by parameter estimate      se     df   lower    upper   p.value
: 1  1   weight2 -0.79725 0.31970 17.003 -1.4717 -0.12276 0.0232403
: 2  2   weight2 -0.83352 0.29798 17.003 -1.4622 -0.20486 0.0123742
: 3  3   weight2 -0.90658 0.28187 17.003 -1.5013 -0.31189 0.0050657
: 4  4   weight2 -0.90648 0.28183 17.003 -1.5011 -0.31189 0.0050638
: 5  5   weight2 -0.82477 0.30247 17.003 -1.4629 -0.18663 0.0143469

and pool the results using Rubin's rule:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
model.tables(e.mlmm, method = "pool.rubin")
#+END_SRC

#+RESULTS:
:         estimate     se     df   lower    upper  p.value
: <1, 5> -0.85372 0.30212 14.741 -1.4987 -0.20878 0.012949

\clearpage

This matches (almost exactly, only the degrees of freedom are a little
different) the results obtained with the mice package:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.mice <- with(data=gastricbypassW.mice,exp=lm(glucagonAUC3~glucagonAUC2+weight2))
summary(pool(e.mice))
#+END_SRC

#+RESULTS:
:           term   estimate std.error statistic     df    p.value
: 1  (Intercept) 178.988359  36.52589  4.900314 14.703 0.00020353
: 2 glucagonAUC2   0.027599   0.41848  0.065951 15.132 0.94828055
: 3      weight2  -0.853721   0.30212 -2.825775 14.737 0.01295073

One can use the =plot= function to obtain a forest plot of the
individual estimates along with the pooled estimate:
#+BEGIN_SRC R :exports both :results code :session *R* :cache no
plot(e.mlmm, method = c("pool.rubin","none"))
#+END_SRC

#+RESULTS:
#+begin_src R
#+end_src

#+BEGIN_SRC R :exports none :results output raw drawer :session *R* :cache no
ggsave(autoplot(e.mlmm, method = c("pool.rubin","none"))$plot + theme(text = element_text(size=20)), filename = "./figures/forestplot.pdf", width = 12)
#+END_SRC

#+RESULTS:
:results:
[1m[22mSaving 12 x 7 in image
:end:

#+ATTR_LaTeX: :width 1\textwidth :options trim={0 0 0 0} :placement [!h]
[[./figures/forestplot.pdf]]

\clearpage

* Data generation
Simulate some data in the wide format:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
set.seed(10) ## ensure reproductibility
n.obs <- 100
n.times <- 4
mu <- rep(0,4)
gamma <- matrix(0, nrow = n.times, ncol = 10) ## add interaction
gamma[,6] <- c(0,1,1.5,1.5)
dW <- sampleRem(n.obs, n.times = n.times, mu = mu, gamma = gamma, format = "wide")
head(round(dW,3))
#+END_SRC

#+RESULTS:
:   id X1 X2 X3 X4 X5     X6     X7     X8    X9    X10     Y1     Y2     Y3     Y4
: 1  1  1  0  1  1  0 -0.367  1.534 -1.894 1.729  0.959  1.791  2.429  3.958  2.991
: 2  2  1  0  1  2  0 -0.410  2.065  1.766 0.761 -0.563  2.500  4.272  3.002  2.019
: 3  3  0  0  2  1  0 -1.720 -0.178  2.357 1.966  1.215 -3.208 -5.908 -4.277 -5.154
: 4  4  0  0  0  1  0  0.923 -2.089  0.233 1.307 -0.906 -2.062  0.397  1.757 -1.380
: 5  5  0  0  2  1  0  0.987  5.880  0.385 0.028  0.820  7.963  7.870  7.388  8.609
: 6  6  0  0  1  1  2 -1.075  0.479  2.202 0.900 -0.739  0.109 -1.602 -1.496 -1.841

Simulate some data in the long format:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
set.seed(10) ## ensure reproductibility
dL <- sampleRem(n.obs, n.times = n.times, mu = mu, gamma = gamma, format = "long")
head(dL)
#+END_SRC

#+RESULTS:
:   id visit      Y X1 X2 X3 X4 X5       X6     X7      X8      X9      X10
: 1  1     1 1.7914  1  0  1  1  0 -0.36653 1.5338 -1.8944 1.72887  0.95925
: 2  1     2 2.4286  1  0  1  1  0 -0.36653 1.5338 -1.8944 1.72887  0.95925
: 3  1     3 3.9583  1  0  1  1  0 -0.36653 1.5338 -1.8944 1.72887  0.95925
: 4  1     4 2.9912  1  0  1  1  0 -0.36653 1.5338 -1.8944 1.72887  0.95925
: 5  2     1 2.5002  1  0  1  2  0 -0.40975 2.0654  1.7658 0.76133 -0.56302
: 6  2     2 4.2724  1  0  1  2  0 -0.40975 2.0654  1.7658 0.76133 -0.56302

\clearpage

* Modifying default options
The =LMMstar.options= method enable to get and set the default options
used by the package. For instance, the default option for the information matrix is:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
LMMstar.options("type.information")
#+END_SRC

#+RESULTS:
: $type.information
: [1] "observed"

To change the default option to "expected" (faster to compute but less accurate p-values and confidence intervals in small samples) use:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
LMMstar.options(type.information = "expected")
#+END_SRC

#+RESULTS:

To restore the original default options do:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
LMMstar.options(reinitialise = TRUE)
#+END_SRC

#+RESULTS:

\clearpage

* R session
Details of the R session used to generate this document:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sessionInfo()
#+END_SRC

#+RESULTS:
#+begin_example
R version 4.3.3 (2024-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.10.0 
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Europe/Copenhagen
tzcode source: system (glibc)

attached base packages:
[1] grid      parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] mice_3.16.0         emmeans_1.10.0      rlang_1.1.3         numDeriv_2016.8-1.1
 [5] doParallel_1.0.17   iterators_1.0.14    foreach_1.5.2       copula_1.1-3       
 [9] multcomp_1.4-25     TH.data_1.1-2       MASS_7.3-60.0.1     survival_3.5-8     
[13] mvtnorm_1.2-4       lme4_1.1-35.2       Matrix_1.6-5        lava_1.8.0         
[17] nlme_3.1-163        LMMstar_1.1.0       ggpubr_0.6.0        ggplot2_3.5.1      

loaded via a namespace (and not attached):
 [1] pbapply_1.7-2       gridExtra_2.3       pspline_1.0-19      remotes_2.5.0      
 [5] sandwich_3.1-0      magrittr_2.0.3      butils.base_1.3     compiler_4.3.3     
 [9] mgcv_1.9-1          systemfonts_1.0.6   vctrs_0.6.5         gsl_2.1-8          
[13] stringr_1.5.1       profvis_0.3.8       shape_1.4.6.1       pkgconfig_2.0.3    
[17] fastmap_1.1.1       backports_1.4.1     ellipsis_0.3.2      labeling_0.4.3     
[21] utf8_1.2.4          promises_1.2.1      qqtest_1.2.0        sessioninfo_1.2.2  
[25] nloptr_2.0.3        ragg_1.3.0          purrr_1.0.2         jomo_2.7-6         
[29] glmnet_4.1-8        cachem_1.0.8        later_1.3.2         pan_1.9            
[33] broom_1.0.5         R6_2.5.1            stringi_1.8.3       rpart_4.1.23       
[37] parallelly_1.37.1   car_3.1-2           boot_1.3-30         pkgload_1.3.4      
[41] estimability_1.5    Rcpp_1.0.12         future.apply_1.11.2 zoo_1.8-12         
[45] usethis_2.2.3       nnet_7.3-19         httpuv_1.6.15       splines_4.3.3      
[49] tidyselect_1.2.1    abind_1.4-5         codetools_0.2-19    miniUI_0.1.1.1     
[53] listenv_0.9.1       pkgbuild_1.4.4      lattice_0.22-5      tibble_3.2.1       
[57] shiny_1.8.1.1       withr_3.0.0         coda_0.19-4.1       future_1.33.2      
[61] urlchecker_1.0.1    pillar_1.9.0        carData_3.0-5       stats4_4.3.3       
[65] pcaPP_2.0-4         generics_0.1.3      munsell_0.5.1       scales_1.3.0       
[69] minqa_1.2.6         globals_0.16.3      xtable_1.8-4        glue_1.7.0         
[73] ADGofTest_0.3       tools_4.3.3         data.table_1.15.4   ggsignif_0.6.4     
[77] fs_1.6.3            cowplot_1.1.3       tidyr_1.3.1         devtools_2.4.5     
[81] colorspace_2.1-0    cli_3.6.2           textshaping_0.3.7   fansi_1.0.6        
[85] dplyr_1.1.4         gtable_0.3.5        rstatix_0.7.2       stabledist_0.7-1   
[89] digest_0.6.35       pbkrtest_0.5.2      htmlwidgets_1.6.4   farver_2.1.1       
[93] memoise_2.0.1       htmltools_0.5.8.1   lifecycle_1.0.4     mitml_0.4-5        
[97] mime_0.12
#+end_example

\clearpage

* References
:PROPERTIES:
:UNNUMBERED: t
:END:

#+BEGIN_EXPORT latex
\begingroup
\renewcommand{\section}[2]{}
#+END_EXPORT

bibliographystyle:apalike
[[bibliography:bibliography.bib]]

#+BEGIN_EXPORT latex
\endgroup
#+END_EXPORT

\clearpage

#+BEGIN_EXPORT LaTeX
\appendix
\titleformat{\section}
{\normalfont\Large\bfseries}{Appendix~\thesection}{1em}{}

\renewcommand{\thefigure}{\Alph{figure}}
\renewcommand{\thetable}{\Alph{table}}
\renewcommand{\theequation}{\Alph{equation}}

\setcounter{figure}{0}    
\setcounter{table}{0}    
\setcounter{equation}{0}    
#+END_EXPORT

* Likelihood in a linear mixed model
:PROPERTIES:
:CUSTOM_ID: SM:likelihood
:END:

Denote by \(\VY\) a vector of \(m\) outcomes, \(\VX\) a vector of
\(p\) covariates, \(\mu(\Vparam,\VX)\) the modeled mean, and
\(\Omega(\Vparam,\VX)\) the modeled residual variance-covariance. We
consider \(n\) replicates (i.e. \(\VY_1,\ldots,\VY_n)\) and
\(VX_1,\ldots,\VX_n\)) along with a vector of weights
\(\omega=(w_1,\ldots,w_n)\), which are by default all equal to 1.

** Log-likelihood

The restricted log-likelihood in a linear mixed model can then be
written:
#+BEGIN_EXPORT LaTeX
\begin{align}
\Likelihood(\Vparam|\VY,\VX) =& \textcolor{\darkred}{ \frac{p}{2} \log(2\pi)-\frac{1}{2} \log\left(\left|\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \trans{\VX}_i\right|\right)} \notag \\
& + \sum_{i=1}^{n} w_i \left(\textcolor{\darkblue}{-\frac{m}{2} \log(2\pi) - \frac{1}{2} \log\left|\Omega_i(\Vparam)\right| - \frac{1}{2} (\VY_i-\mu(\Vparam,\VX_i)) \Omega_i(\Vparam)^{-1} \trans{(\VY_i-\mu(\Vparam,\VX_i))}} \right)  \label{eq:log-likelihood}
\end{align}
 #+END_EXPORT
 
 This is what the =logLik= method is computing for the REML
 criteria. The red term is specific to the REML criteria and prevents
 from computing individual contributions to the likelihood[fn::The REML is the
 likelihood of the observations divided by the prior on the estimated
 mean parameters \(\VparamHat_{\mu} \sim \Gaus(\mu,\left(\VX
 \Omega^{-1}(\Vparam) \trans{\VX}\right)^{-1})\). This corresponds to
 \(\frac{1}{\sqrt{2\pi}^p \left|\left(\sum_{i=1}^n \VX_i
 \Omega_i^{-1}(\Vparam) \trans{\VX}_i\right)^{-1}\right|}
 \exp\left(-(\VparamHat_{\mu}-\mu)\left(2\sum_{i=1}^n \VX_i
 \Omega_i^{-1}(\Vparam)
 \trans{\VX}_i\right)^{-1})\trans{(\VparamHat_{\mu}-\mu)}\right)\)
 Since \(\mu\) will be estimated to be \(\Vparam_{\mu}\), the
 exponential term equals 1 and thus does not contribute to the
 log-likelihood. One divided by the other term gives \(\sqrt{2\pi}^p
 \left(\left|\sum_{i=1}^n \VX_i \Omega_i^{-1}(\Vparam)
 \trans{\VX}_i\right|\right)^{-1}\). The log of this term equals the red
 term]. The blue term is what =logLik= outputs for the ML criteria
 when setting the argument =indiv= to =TRUE=.

\bigskip

** Score

 Using that \(\partial \log(\det(X))=tr(X^{-1}\partial(X))\), the
score is obtained by derivating once the log-likelihood, i.e., for
\(\theta \in \Vparam\):
#+BEGIN_EXPORT LaTeX
\begin{align*}
   \Score(\theta) =& \dpartial[\Likelihood(\Vparam|\VY,\VX)][\theta]
= \textcolor{\darkred}{ \frac{1}{2} tr \left( \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \trans{\VX}_i\right)^{-1} \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \trans{\VX}_i\right)  \right) } \\
&+ \sum_{i=1}^n w_i \left( \textcolor{\darkblue}{ -\frac{1}{2} tr\left(\Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta]\right) + \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \trans{(\VY_i-\mu(\Vparam,\VX_i))} } \right. \\
 & \qquad \qquad \left. \textcolor{\darkblue}{ + \frac{1}{2} (\VY_i-\mu(\Vparam,\VX_i)) \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \trans{(\VY_i-\mu(\Vparam,\VX_i))} } \right).
\end{align*}
#+END_EXPORT

 This is what the =score= method is computing for the REML
 criteria. The red term is specific to the REML criteria and prevents
 from computing the score relative to each cluster. The blue term is
 what =score= outputs for the ML criteria when setting the argument
 =indiv= to =TRUE=.

\bigskip

\clearpage

** Hessian

Derivating a second time the log-likelihood gives the hessian, \(\Hessian(\Vparam)\), with element[fn::if one is relative to the mean and the other to the variance then they are respectively \(\theta\) and \(\theta'\)]:
#+BEGIN_EXPORT LaTeX
\begin{align*}
& \Hessian(\theta,\theta^{\prime}) = \ddpartial[\Likelihood(\Vparam|\VY,\VX)][\theta][\theta^{\prime}] = \dpartial[\Score(\theta)][\theta^{\prime}] \\
=& \textcolor{\darkred}{\frac{1}{2} tr \left( \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \trans{\VX}_i\right)^{-1} \left\{ \sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \left(\ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] - 2 \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime}]\right)\Omega_i(\Vparam)^{-1} \trans{\VX}_i \right.  \right.}  \\
& \textcolor{\darkred}{ \left. \left. + \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \trans{\VX}_i\right) \left(\sum_{i=1}^n w_i \VX_i\Omega_i^{-1}(\Vparam) \trans{\VX}_i \right)^{-1} \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \trans{\VX}_i\right) \right\} \right) } \\
& +\sum_{i=1}^n w_i \left( \textcolor{\darkblue}{ \frac{1}{2} tr\left(\Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta] - \Omega_i(\Vparam)^{-1} \ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] \right) } \right.\\
& \qquad \textcolor{\darkblue}{ -  \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \trans{\Vvarepsilon_i(\Vparam)} - \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \trans{\dpartial[\mu(\Vparam,\VX_i)][\theta^{\prime}]} } \\
& \qquad \left. \textcolor{\darkblue}{ + \frac{1}{2} \Vvarepsilon_i(\Vparam) \Omega_i(\Vparam)^{-1} \left(\ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] - \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta] - \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \right) \Omega_i(\Vparam)^{-1} \trans{\Vvarepsilon_i(\Vparam)} } \right).
\end{align*}
#+END_EXPORT
where \(\Vvarepsilon_i(\Vparam) = \VY_i-\mu(\Vparam,\VX_i)\).

\bigskip

The =information= method will (by default) return the (observed)
information which is the opposite of the hessian. So multiplying the
previous formula by -1 gives what =information= output for the REML
criteria. The red term is specific to the REML criteria and prevents
from computing the information relative to each cluster. The blue term
is what =information= outputs for the ML criteria (up to a factor -1)
when setting the argument =indiv= to =TRUE=.

\bigskip

A possible simplification is to use the expected hessian at the maximum likelihood. Indeed for
any deterministic matrix \(A\):
- \(\Esp[A \trans{(\VY_i-\mu(\Vparam,\VX_i))}|\VX_i] = 0\)
- \(\Esp[(\VY_i-\mu(\Vparam,\VX_i)) A \trans{(\VY_i-\mu(\Vparam,\VX_i))}||\VX_i] = tr(A \Var(\VY_i-\mu(\Vparam,\VX_i)))\)
when \(\Esp[\VY_i-\mu(\Vparam,\VX_i)]=0\). This leads to:
#+BEGIN_EXPORT LaTeX
\begin{align}
 & \Esp[\Hessian(\theta,\theta^{\prime})|\VX] \notag\\ 
 &= \textcolor{\darkred}{ \frac{1}{2} tr \left( \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \trans{\VX}_i\right)^{-1}  \left\{ \sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \left( \ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] - 2 \dpartial[\Omega_i(\Vparam)][\theta]  \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime}]\right) \Omega_i(\Vparam)^{-1} \trans{\VX}_i \right.  \right.} \notag \\
 & \textcolor{\darkred}{ \left. \left. +  \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \trans{\VX}_i\right) \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \trans{\VX}_i \right)^{-1} \left(\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \trans{\VX}_i\right) \right\} \right) } \notag\\
 & + \sum_{i=1}^n w_i \left( \textcolor{\darkblue}{
- \frac{1}{2} tr\left(\Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta]\right)
 - \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \trans{\dpartial[\mu(\Vparam,\VX_i)][\theta^{\prime}]}
 } \right) \label{eq:expectedInfo}
\end{align}
#+END_EXPORT

This is what =information= output when the argument =type.information=
is set to ="expected"= (up to a factor -1).

\clearpage

** Degrees of freedom

Degrees of freedom are computed using a Satterthwaite approximation,
i.e. for an estimate coefficient \(\widehat{\beta}\in\widehat{\Vparam}\) with standard
error \(\sigma_{\widehat{\beta}}\), the degree of freedom is:
#+begin_export latex
\begin{align*}
df\left(\sigma_{\widehat{\beta}}\right) = \frac{2 \sigma^4_{\widehat{\beta}}}{\Var[\widehat{\sigma}_{\widehat{\beta}}]}
\end{align*}
#+end_export
Using a first order Taylor expansion we can approximate the variance term as:
#+begin_export latex
\begin{align*}
\Var[\widehat{\sigma}_{\widehat{\beta}}] & \approx \dpartial[\widehat{\sigma}_{\widehat{\beta}}][\Vparam] \Sigma_{\Vparam}  \trans{\dpartial[\widehat{\sigma}_{\widehat{\beta}}][\Vparam]} \\
& \approx c_{\beta} \left(\widehat{\Information}_{\widehat{\Vparam}}\right)^{-1} \dpartial[\widehat{\Information}_{\widehat{\Vparam}}][\Vparam] \left(\widehat{\Information}_{\widehat{\Vparam}}\right)^{-1} \trans{c_{\beta}} \Sigma_{\Vparam} \trans{c_{\beta}} \left(\widehat{\Information}_{\widehat{\Vparam}}\right)^{-1} \trans{\dpartial[\widehat{\Information}_{\widehat{\Vparam}}][\Vparam]} \left(\widehat{\Information}_{\widehat{\Vparam}}\right)^{-1} c_{\beta}
\end{align*}
#+end_export

where \(\Sigma_{\Vparam}\) is the variance-covariance matrix of all
  model coefficients, \(\Information_{\Vparam}\) the information
  matrix for all model coefficients, \(c_{\beta}\) a matrix used to
  select the element relative to \(\beta\) in the first derivative of
  the information matrix, and \(\dpartial[.][\Vparam]\) denotes the
  vector of derivatives with respect to all model coefficients.

\bigskip

The derivative of the information matrix (i.e. negative hessian) can
then be computed using numerical derivatives or using analytical
formula. To obtain the later we first notice that:
#+BEGIN_EXPORT LaTeX
\begin{align}
&\Hessian(\theta,\theta^{\prime}) = \Esp[\Hessian(\theta,\theta^{\prime})|\VX] \notag \\
& + \sum_{i=1}^n  w_i \left( \textcolor{\darkblue}{ tr\left(\Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta] - \Omega_i(\Vparam)^{-1} \ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] \right) } \right. \notag \\
& \qquad \textcolor{\darkblue}{ -  \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \trans{\Vvarepsilon_i(\Vparam)}} \notag \\
& \qquad \left. \textcolor{\darkblue}{ + \frac{1}{2} \Vvarepsilon_i(\Vparam) \Omega_i(\Vparam)^{-1} \left(\ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] - \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta] - \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \right) \Omega_i(\Vparam)^{-1} \trans{\Vvarepsilon_i(\Vparam)} } \right) \label{eq:diffInfo}
\end{align}
#+END_EXPORT
where
#+BEGIN_EXPORT latex
\begin{align*}
\Esp[\Hessian(\theta,\theta^{\prime})|\VX] &=& \textcolor{\darkred}{
\frac{1}{2} tr \left(A(\Vparam)^{-1} \left(\sum_{i=1}^n w_i b_i(\Vparam) B_i(\Vparam) \trans{b}_i(\Vparam) + C(\Vparam)A(\Vparam)^{-1} \trans{C}(\Vparam) \right)\right)
}  + \sum_{i=1}^n w_i \textcolor{\darkblue}{E_i(\Vparam)} \\
\textcolor{\darkblue}{E_i(\Vparam)} &=& \textcolor{\darkblue}{\frac{1}{2} tr\left(\Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta]\right)
                                        - \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \trans{\dpartial[\mu(\Vparam,\VX_i)][\theta^{\prime}]}} \\
\textcolor{\darkred}{A(\Theta)} &=& \textcolor{\darkred}{\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \trans{\VX}_i }\\
\textcolor{\darkred}{B(\Theta)} &=& \textcolor{\darkred}{\ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime}] - 2 \dpartial[\Omega_i(\Vparam)][\theta]  \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] }\\
\textcolor{\darkred}{b_i(\Theta)} &=& \textcolor{\darkred}{\VX_i \Omega_i^{-1} }\\
\textcolor{\darkred}{C(\Theta)} &=& \textcolor{\darkred}{\sum_{i=1}^n w_i \VX_i \Omega_i^{-1}(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta] \Omega_i(\Vparam)^{-1} \trans{\VX}_i }
\end{align*}
#+END_EXPORT
So we will first derive the derivative of
\(\Esp[\Hessian(\theta,\theta^{\prime})|\VX]\) and then the one of the
blue term in autoref:eq:diffInfo.  To simplify the derivation of the
formula we will only derive them at the maximum likelihood, i.e. when
\(\Esp\left[\dpartial[\Hessian(\theta,\theta^{\prime}|\VX)][\theta^{\prime\prime}]\right]=\frac{\partial
\Esp[\Hessian(\theta,\theta^{\prime}|\VX)]}{\partial
\theta^{\prime\prime}}\) where the expectation is taken over
\(\VX\). We first notice that the derivative with respect to the mean
parameters is 0. So we just need to compute the derivative with
respect to a variance parameter \(\theta^{\prime\prime}\):
#+BEGIN_EXPORT latex
\begin{align*}
 & \frac{\partial \textcolor{\darkred}{ A(\Vparam)^{-1} \left(\sum_{i=1}^n w_i b_i(\Vparam) B_i(\Vparam) \trans{b}_i(\Vparam) + C(\Vparam)A(\Vparam)^{-1} \trans{C}(\Vparam) \right)}}{\partial \theta^{\prime\prime}} \\
 =& \textcolor{\darkred}{A(\Vparam)^{-1} \dpartial[A(\Vparam)][\theta^{\prime\prime}] A(\Vparam)^{-1} \left(\sum_{i=1}^n w_i b_i(\Vparam) B_i(\Vparam) \trans{b}_i(\Vparam) + C(\Vparam)A(\Vparam)^{-1} \trans{C}(\Vparam) \right)} \\
 & +\textcolor{\darkred}{A(\Vparam)^{-1} \left(\sum_{i=1}^n w_i \left(
 \dpartial[b_i(\Vparam)][\theta^{\prime\prime}]  B_i(\Vparam) \trans{b}_i(\Vparam)
 + b_i(\Vparam) \dpartial[B_i(\Vparam)][\theta^{\prime\prime}]   \trans{b}_i(\Vparam)
 + b_i(\Vparam) B_i(\Vparam) \dpartial[\trans{b}_i(\Vparam)][\theta^{\prime\prime}] \right. \right. } \\
& \qquad \qquad \qquad \qquad \quad + \textcolor{\darkred}{\left. \left.
 \dpartial[C(\Vparam)][\theta^{\prime\prime}]  A^{-1}(\Vparam) \trans{C}(\Vparam)
 + C(\Vparam) A^{-1}\dpartial[A(\Vparam)][\theta^{\prime\prime}]A^{-1}   \trans{C}(\Vparam)
 + C(\Vparam) A^{-1}(\Vparam) \dpartial[\trans{C}(\Vparam)][\theta^{\prime\prime}]
\right) \right)}
\end{align*}
#+END_EXPORT

and

#+BEGIN_EXPORT LaTeX
\begin{align*}
 \dpartial[\textcolor{\darkblue}{E(\Vparam)}][\theta^{\prime\prime}]=&
 \sum_{i=1}^n w_i \left( \textcolor{\darkblue}{
- \frac{1}{2} tr\left(
-2\Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta] \right. } \right. \\
& \qquad \qquad \textcolor{\darkblue}{\left. + \Omega_i(\Vparam)^{-1} \ddpartial[\Omega_i(\Vparam)][\theta^{\prime}][\theta^{\prime\prime}] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta]
+ \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime}] \Omega_i(\Vparam)^{-1} \ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime\prime}]
\right)} \\
& \qquad \qquad  \textcolor{\darkblue}{\left. + \dpartial[\mu(\Vparam,\VX_i)][\theta] \Omega_i(\Vparam)^{-1} \dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}] \Omega_i(\Vparam)^{-1}   \trans{\dpartial[\mu(\Vparam,\VX_i)][\theta^{\prime}]}
 \right)}
\end{align*}
#+END_EXPORT

where:
#+BEGIN_EXPORT latex
\begin{align*}
\textcolor{\darkred}{\dpartial[A(\Vparam)][\theta^{\prime\prime}]} &= \textcolor{\darkred}{\sum_{i=1}^n w_i \VX_i \Omega^{-1}_i(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}]\Omega^{-1}_i(\Vparam) \trans{\VX}_i} \\
\textcolor{\darkred}{\dpartial[b_i(\Vparam)][\theta^{\prime\prime}]} &= \textcolor{\darkred}{\VX_i \Omega^{-1}_i(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}]\Omega^{-1}_i(\Vparam)} \\
\textcolor{\darkred}{\dpartial[B_i(\Vparam)][\theta^{\prime\prime}]} &= \textcolor{\darkred}{
  \frac{\partial^3 \Omega_i(\Vparam)}{\theta\theta^{\prime}\theta^{\prime\prime}} } \\
  & \textcolor{\darkred}{ - 2 \left(
  \ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime\prime}]\Omega^{-1}_i(\Vparam)\dpartial[\Omega_i(\Vparam)][\theta^{\prime}]
+ \dpartial[\Omega_i(\Vparam)][\theta]\Omega^{-1}_i(\Vparam)\dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}]\Omega^{-1}_i(\Vparam)\dpartial[\Omega_i(\Vparam)][\theta^{\prime}]
+ \dpartial[\Omega_i(\Vparam)][\theta]\Omega^{-1}_i(\Vparam)\ddpartial[\Omega_i(\Vparam)][\theta^{\prime}][\theta^{\prime\prime}]
\right)
  } \\
\textcolor{\darkred}{\dpartial[C(\Vparam)][\theta^{\prime\prime}]} &= \textcolor{\darkred}{\sum_{i=1}^n w_i \VX_i \Omega^{-1}_i(\Vparam) \left(
\dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}] \Omega^{-1}_i(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta]
+ \ddpartial[\Omega_i(\Vparam)][\theta][\theta^{\prime\prime}]
+ \dpartial[\Omega_i(\Vparam)][\theta] \Omega^{-1}_i(\Vparam) \dpartial[\Omega_i(\Vparam)][\theta^{\prime\prime}]
\right) \Omega^{-1}_i(\Vparam) \trans{\VX}_i} 
\end{align*}
#+END_EXPORT



\clearpage
  
* Likelihood ratio test with the REML criterion
:PROPERTIES:
:CUSTOM_ID: SM:LRT-REML
:END:

The blue term of autoref:eq:log-likelihood in the log-likelihood is
invariant to re-parameterisation while the red term is not. This means
that a re-parametrisation of \(X\) into \(\tilde{X} = B X\) with \(B\)
invertible would not change the likelihood when using ML but would
decrease the log-likelihood by \(\log(|B|)\) when using REML. \newline
Let's take an example:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
## data(dfL, package = "LMMstar")
dfTest <- gastricbypassL[!is.na(gastricbypassL$glucagonAUC),]
dfTest$gluc <- dfTest$glucagonAUC
dfTest$gluc2 <- dfTest$glucagonAUC*2
#+END_SRC

#+RESULTS:

\noindent where we multiply one column of the design matrix by 2. As mentionned
previously this does not affect the log-likelihood when using ML:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eML.UN <- lmm(weight ~ time+gluc, data = dfTest, repetition = ~time|id, method = "ML")
eML.UN2 <- lmm(weight ~ time+gluc, data = dfTest, repetition = ~time|id, method = "ML")
c(logLik(eML.UN), logLik(eML.UN2), logLik(eML.UN) - logLik(eML.UN2))
#+END_SRC

#+RESULTS:
: [1] -230.62 -230.62    0.00

but it does when using REML:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eREML.UN <- lmm(weight ~ time + gluc, data = dfTest, repetition = ~time|id, method = "REML")
eREML.UN2 <- lmm(weight ~ time + gluc2, data = dfTest, repetition = ~time|id, method = "REML")
c(logLik(eREML.UN), logLik(eREML.UN2), logLik(eREML.UN) - logLik(eREML.UN2), log(2))
#+END_SRC

#+RESULTS:
: [1] -235.23462 -235.92777    0.69315    0.69315


Therefore, when comparing models with different mean effects there is
a risk that the difference (or part of it) in log-likelihood is due to
a new parametrisation and no only to a difference in model fit. This
would typically be the case when adding an interaction where we can
have a smaller restricted log-likehood when considering a more complex
model:

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
set.seed(5) 
dfTest$ff <- rbinom(NROW(dfTest), size = 1, prob = 0.5)
logLik(lmm(weight ~ time+gluc, data = dfTest, repetition = ~time|id, method = "REML"))
logLik(lmm(weight ~ time+gluc*ff, data = dfTest, repetition = ~time|id, method = "REML"))
#+END_SRC

#+RESULTS:
: [1] -235.23
: [1] -238.93

This is quite counter-intuitive as more complex model should lead to
better fit and would never happen when using ML:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
logLik(lmm(weight ~ time + gluc, data = dfTest, repetition = ~time|id, method = "ML"))
logLik(lmm(weight ~ time + gluc*ff, data = dfTest, repetition = ~time|id, method = "ML"))
#+END_SRC

#+RESULTS:
: [1] -230.62
: [1] -230.44

This is why, unless one knows what he/she is doing, it is not
recommanded to use likelihood ratio test to assess relevance of mean
parameters in mixed models estimated with REML.

\clearpage

* Sum of squares in a linear mixed model
:PROPERTIES:
:CUSTOM_ID: SM:sumSquares
:END:

All mixed models implemented in LMMstar can be written as:
#+BEGIN_EXPORT latex
\[ Y_{it} = X_{it}\beta + \varepsilon_{it} \text{ where } \varepsilon_{i} \sim \Gaus\left(0,\Omega\right)\]
#+END_EXPORT
where \(Y\) denote the outcome repeteadly measured within each cluster
\(i\) where \(t\) indexes the repetitions. \(X\) denotes the
covariates, \(\beta\) the mean parameters, \(\varepsilon\) the
residuals, and \(\Omega\) the residual variance-covariance matrix.
\(\Omega\) must be positive definite so there must exist a square
postive definite matrix \(\Omega^{1/2}\) such that
\(\Omega^{1/2}\Omega^{1/2} = \Omega\). Therefore the previous model is
equivalent to:
#+BEGIN_EXPORT latex
\[ Y^*_{it} = X^*_{it}\beta + \varepsilon^*_{it} \text{ where } \varepsilon_{i} \sim \Gaus\left(0,I_T\right)\]
#+END_EXPORT
where \(Y^*_{i} = \Omega^{-1/2} Y_{i}\), \(X^*_{i} = \Omega^{-1/2}
X_{i}\), \(\varepsilon^*_{i} = \Omega^{-1/2} \varepsilon_{i}\), and
\(I_x\) is the identity matrix with \(x\) rows and columns. One can
then introduce the projectors \(H= X \left(\trans{X}\Omega^{-1}
X\right)^{-1}\trans{X} \Omega^{-1}\) and \(H^*= X^*
\left(\trans{X^*}X^*\right)^{-1}\trans{X^*}\) onto the space spanned
by \(X\) and \(X^*\) respectively. We can now define the "normalized"
residual sum of squares as the squared sum of the normalized
residuals:
#+BEGIN_EXPORT latex
\begin{align*}
SSE^* = \trans{\varepsilon^*} \varepsilon^* &= \trans{Y^*} (I_{nT}-H^*) Y^* \\
&= \trans{Y} \Omega^{-1} Y - \trans{Y} \Omega^{-1} X \left(\trans{X}\Omega^{-1} X\right)^{-1} \trans{X} \Omega^{-1} Y \\
&= \trans{Y} (I_{nT}-\trans{H}) \Omega^{-1} (I_{nT}-H) Y 
\end{align*}
#+END_EXPORT
The previous to last line uses that: \((I_{nT}-\trans{H}) \Omega^{-1}
(I_{nT}-H)= \Omega^{-1} - \trans{H} \Omega^{-1} - \Omega^{-1}H +
\trans{H} \Omega^{-1} H = \Omega^{-1} - \trans{H}\Omega^{-1}\) as
\(\trans{H} \Omega^{-1} H = \Omega^{-1}HH=\Omega^{-1}H\) since \(H\)
is a projector. Note that compared to the "traditional" SSE defined
for linear regression and random effect models (e.g. see
cite:christensen2002plane section 2.7), \(SSE=\delta SSE^{*}\) where
\(\delta\) is the residual variance conditional on any random effects,
i.e. \(SSE^{*}\) are the residual degrees of freedom. This is because
the same definition for the sum of squares is used except that
\(\varepsilon_{i} \sim \Gaus\left(0,\delta\Omega\right)\).

\bigskip

We can also define the "normalized" regression sum of squares:
#+BEGIN_EXPORT latex
\begin{align*}
SSR^* = \trans{(X^*\beta)}X^*\beta &= \trans{\left(H^* Y^*\right)} H^* Y^* = \trans{Y^*} H^* Y^* \\
&= \trans{Y} \trans{H} \Omega^{-1} Y^* = \trans{Y} \trans{H} \trans{H} \Omega^{-1} Y^* = \trans{Y} \trans{H} \Omega^{-1} H Y^* \\
&= \widehat{\beta} \trans{X} \Omega^{-1} X \widehat{\beta}
\end{align*}
#+END_EXPORT
where \(\widehat{\beta}= \left(\trans{X}\Omega^{-1}
X\right)^{-1}\trans{X} \Omega^{-1} Y\). Note that when using the
expected information \(SSR^* = \widehat{\beta}
\Sigma^{-1}_{\widehat{\beta}} \widehat{\beta} \), i.e. it is the
F-statistics times the number of parameters. Again the "traditional"
SSR defined for linear regression and random effect models is
proportional to this normalized SSR: \(SSR=\delta SSR^{*}\).

\bigskip

The proportion of explained variance of \(p\) parameters can thus be
re-expressed as:
#+BEGIN_EXPORT latex
\begin{align*}
R^2 &= \frac{SSR}{SSR+SSE} = \frac{SSR^*}{SSR^*+SSE^*}= \frac{Fp}{Fp+df}
\end{align*}
#+END_EXPORT

where \(df\) denotes the residual degrees of freedom, typically
\(n-p\) in a univariate linear model fitted with \(n\)
observations. \newline \Warning In practice \(df\) is estimated using the
Satterthwaite approximation of the degrees of freedom of the
regression coefficient. This is only equivalent to the "SSR/SSE"
formula in univariate linear regression.

\bigskip
\bigskip

*Illustration for a univariate linear model:*

\bigskip

Data without missing values:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
df.aov <- gastricbypassL[!is.na(gastricbypassL$glucagon),]
#+END_SRC

#+RESULTS:

Traditional anova decomposition:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lm <- lm(weight ~ visit + glucagonAUC, data = df.aov)
car::Anova(e.lm, type = "II")
#+END_SRC

#+RESULTS:
: Anova Table (Type II tests)
: 
: Response: weight
:             Sum Sq Df F value Pr(>F)   
: visit         5837  3    5.94 0.0011 **
: glucagonAUC   2133  1    6.51 0.0128 * 
: Residuals    23925 73                  
: ---
: Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
## Which can be reproduced "manually":
tXX <- crossprod(model.matrix(e.lm))
beta <- coef(e.lm)

SSE <- sigma(e.lm)^2*df.residual(e.lm)
SSR.glucagon <- t(beta[5]) %*% solve(solve(tXX)[5,5]) %*% beta[5]
SSR.time <- t(beta[2:4]) %*% solve(solve(tXX)[2:4,2:4]) %*% beta[2:4]
c(SSE = SSE, SSR.glucagon = SSR.glucagon, SSR.time = SSR.time,
  F.time = (SSR.time/3)/(SSE/73),
  F.glucagon = (SSR.glucagon/1)/(SSE/73))
#+END_SRC

#+RESULTS:
:          SSE SSR.glucagon     SSR.time       F.time   F.glucagon 
:    23925.249     2132.629     5837.410        5.937        6.507


Fit =lmm=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmm <- lmm(weight ~ visit + glucagonAUC, data = df.aov)
#+END_SRC

#+RESULTS:

Residual sum of squares (SSE):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
SSEstar <- crossprod(residuals(e.lmm, type = "normalized"))
c(SSEstar = SSEstar, SSE = SSEstar * sigma(e.lmm))
#+END_SRC

#+RESULTS:
: SSEstar     SSE 
:      73   23925

The normalized SSE can also be obtained using the =df.residual= method:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
df.residual(e.lmm)
#+END_SRC

#+RESULTS:
: [1] 73

Regression sum of squares (SSR):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eBeta.lmm <- coef(e.lmm)
eVcov.lmm <- vcov(e.lmm, type.information = "expected")

SSRstar.glucagon <- eBeta.lmm[5] %*% solve(eVcov.lmm[5,5]) %*% eBeta.lmm[5] 
SSRstar.time <- eBeta.lmm[2:4] %*% solve(eVcov.lmm[2:4,2:4]) %*% eBeta.lmm[2:4] 
c(SSR.glucagon = SSRstar.glucagon * sigma(e.lmm),
  SSR.time = SSRstar.time * sigma(e.lmm),
  F.glucagon = SSRstar.glucagon,
  F.time = SSRstar.time/3)
#+END_SRC

#+RESULTS:
: SSR.glucagon     SSR.time   F.glucagon       F.time 
:     2132.629     5837.410        6.507        5.937

So the proportion of explained variance is:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
R2.glucagon <- SSRstar.glucagon/(SSRstar.glucagon+SSEstar)
R2.glucagon
#+END_SRC

#+RESULTS:
:          [,1]
: [1,] 0.081842

and the corresponding partial correlation is:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sign(coef(e.lmm)["glucagonAUC"])*sqrt(R2.glucagon)
#+END_SRC

#+RESULTS:
:          [,1]
: [1,] -0.28608

which matches the output of =partialCor=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
summary(partialCor(e.lmm, R2 = TRUE))
#+END_SRC

#+RESULTS:
#+begin_example

		Partial correlation 

               estimate    se df  lower  upper p.value
   visit2        -0.151 0.113 73 -0.377  0.074 0.18450
   visit3        -0.013 0.117 73 -0.246   0.22 0.91230
   visit4        -0.381 0.092 73 -0.565 -0.197 < 1e-04
   glucagonAUC   -0.286 0.103 73 -0.491 -0.081 0.00695
   --------------------------------------------------- 
  Columns lower and upper contain 95% pointwise confidence intervals for each coefficient.
  Degrees of freedom were computed using a Satterthwaite approximation (column df). 

		Coefficient of determination (R2)

               estimate    se df  lower upper  p.value
   visit          0.196 0.075 73  0.047 0.345 0.010548
   glucagonAUC    0.082 0.059 73 -0.036 0.199 0.169016
   global          0.29 0.075 73   0.14  0.44 0.000257
   --------------------------------------------------- 
  Columns lower and upper contain 95% pointwise confidence intervals for each coefficient.
  Degrees of freedom were computed using a Satterthwaite approximation (column df).
#+end_example

\clearpage

* Equivalence with other R packages

** nlme package

The model class obtained with the =lmm= function overlaps the model
class of the =lme= and =gls= functions from the nlme package.
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(nlme)
#+END_SRC

#+RESULTS:

For instance, the compound symmetry is equivalent to =corCompSymm=
correlation structure, or to a random intercept model (when the within
subject correlation is positive):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRI.lmm <- lmm(weight ~ visit*group, structure = "RE",
               data = gastricbypassL, repetition = ~visit|id)
eCS.gls <- gls(weight ~ visit*group, correlation = corCompSymm(form=~visit|id),
               data = gastricbypassL, na.action = na.omit)
eCS.lme <- lme(weight ~ visit*group, random = ~1|id,
               data = gastricbypassL, na.action = na.omit)
logLik(eRI.lmm)
logLik(eCS.lme)
logLik(eCS.gls)
#+END_SRC

#+RESULTS:
: [1] -236.21
: 'log Lik.' -236.21 (df=10)
: 'log Lik.' -236.21 (df=10)

The estimated random effect also match:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
range(ranef(eRI.lmm)-ranef(eCS.lme))
#+END_SRC

#+RESULTS:
: [1] -1.7303e-08  2.6979e-08

Unstructured residual covariance matrix can also be obtained with
=gls=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.gls <- gls(glucagonAUC ~ visit*group,
               correlation = corSymm(form=~as.numeric(visit)|id),
               weights = varIdent(form=~1|visit),
               data = gastricbypassL, na.action = na.omit)
logLik(eUN.gls)
logLik(eUN.lmm)
#+END_SRC

#+RESULTS:
: 'log Lik.' -295.31 (df=18)
: [1] -295.31

\clearpage

** lme4 package

The model class obtained with the =lmm= function overlaps the model
class of the =lmer= function from the lme4 package.
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(lme4)
library(lmerTest)
#+END_SRC

#+RESULTS:

For instance, the compound symmetry is equivalent to a random
intercept model (when the within subject correlation is positive):
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRI.lmer <- lmer(weight ~ visit*group + (1|id),
                 data = gastricbypassL)
logLik(eRI.lmer)
logLik(eRI.lmm)
#+END_SRC

#+RESULTS:
: 'log Lik.' -236.21 (df=10)
: [1] -236.21

The estimated random effects match:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
range(ranef(eRI.lmm)-ranef(eRI.lmer)$id)
#+END_SRC

#+RESULTS:
: [1] -1.5513e-08  2.4171e-08

Nested random effects correspond to block unstructured:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eNRI.lmm <- lmm(weight ~ visit*group, structure = RE(~(1|id/baseline)),
               data = gastricbypassL, repetition = ~visit|id)
eNRI.lmer <- lmer(weight ~ visit*group + (1|id/baseline),
                  data = gastricbypassL)
logLik(eNRI.lmer)
logLik(eNRI.lmm)
#+END_SRC

#+RESULTS:
: 'log Lik.' -234.97 (df=11)
: [1] -234.97

And the estimated random effects still match:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRanefNRI.lmm <- ranef(eNRI.lmm, format = "wide")
eRanefNRI.lmer <- ranef(eNRI.lmer)
## id
range(eRanefNRI.lmm$estimate-eRanefNRI.lmer$id)
## baseline
range(c(eRanefNRI.lmm$estimate.FALSE,eRanefNRI.lmm$estimate.TRUE)-ranef(eNRI.lmer)$`baseline:id`)
#+END_SRC

#+RESULTS:
: [1] -5.8317e-06  9.0913e-06
: [1] -8.5850e-05  7.8971e-05

\clearpage

An unstructure residual covariance matrix can also be obtained using
random slopes:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.lmer <- lmer(glucagonAUC ~ visit*group + (0 + visit|id),
                 data = gastricbypassL,
                 control = lmerControl(check.nobs.vs.nRE = "ignore"))
logLik(eUN.lmer)
logLik(eUN.lmm)
#+END_SRC

#+RESULTS:
: Warning message:
: In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv,  :
:   Model failed to converge with max|grad| = 0.00203036 (tol = 0.002, component 1)
: 'log Lik.' -295.31 (df=19)
: [1] -295.31

The uncertainty is quantified in a slightly different way, e.g.:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
anova(eUN.lmm)
#+END_SRC

#+RESULTS:
: 		Multivariate Wald test 
: 
:                      F-statistic       df p.value   
:    mean: visit             5.803 (3,16.9) 0.00647 **
:        : group             3.926 (1,18.0) 0.06302  .
:        : visit:group       2.762 (3,17.3) 0.07332  .

is very similar but not identical to:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
## only the last line is comparable
anova(eUN.lmer)
#+END_SRC

#+RESULTS:
: Type III Analysis of Variance Table with Satterthwaite's method
:             Sum Sq Mean Sq NumDF DenDF F value  Pr(>F)    
: visit         1339     446     3  17.4   18.29 1.3e-05 ***
: group            5       5     1  18.1    0.22   0.647    
: visit:group    203      68     3  17.4    2.77   0.073 .  
: ---
: Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

It is also possible to fit cross-random effects such as:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
data("Penicillin")
eCRI.lmer <- lmer(diameter ~ 1 + (1|plate) + (1|sample), Penicillin)
logLik(eCRI.lmer)
#+END_SRC

#+RESULTS:
: 'log Lik.' -165.43 (df=4)


#+RESULTS:

using =lmm=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
Penicillin$index <- paste(Penicillin$sample,Penicillin$plate,sep=".")
Penicillin$id <- 1

eCRI.lmm <- lmm(diameter ~ 1 + (1|plate) + (1|sample), data = Penicillin)
logLik(eCRI.lmm)
#+END_SRC

#+RESULTS:
: [1] -165.43

Despite being significantly slower, the loglikelihood and random
effect still match:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
range(ranef(eCRI.lmm)$estimate-rbind(ranef(eCRI.lmer)$plate,ranef(eCRI.lmer)$sample))
#+END_SRC

#+RESULTS:
: [1] -4.3812e-07  6.0172e-07

** mmrm package

The package =mmrm= is an alternative implementation of mixed models
specified via covariance structures:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(mmrm)
e.mmrm <- mmrm(
  formula = FEV1 ~ RACE + SEX + ARMCD * AVISIT + us(AVISIT | USUBJID),
  data = fev_data
)
#+END_SRC

#+RESULTS:

It leads nearly identical results compared to =lmm=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
e.lmm <- lmm(
  formula = FEV1 ~ RACE + SEX + ARMCD * AVISIT,
  repetition = ~ AVISIT | USUBJID, structure = "UN",
  data = fev_data, type.information = "expected"
)
#+END_SRC
#+RESULTS:
: Warning message:
: In .lmmNormalizeData(as.data.frame(data)[unique(stats::na.omit(var.all))],  :
:     3 clusters have been removed.

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
logLik(e.mmrm) - logLik(e.lmm)
range(coef(e.mmrm) - coef(e.lmm))
range(vcov(e.mmrm) - vcov(e.lmm))
#+END_SRC

#+RESULTS:
: [1] -2.5413e-06
: [1] -0.00018301  0.00016268
: [1] -0.00039710  0.00020479

The main differences are:
- =mmrm= uses the expected information matrix to quantify uncertainty
  instead of the observed information matrix.
- =mmrm= implements the Kenward and Roger method for computing the degrees of
  freedom and not only the Satterthwaite approximation
- =mmrm= implements different covariance patterns
- =mmrm= is faster and probably more memorry efficient
- =mmrm= has currently fewer post-processing methods (e.g. adjustment
  multiple comparisons when testing several model parameters). This
  being said, the latest version of the package (0.3.7) included
  several additional extractor of model feature so this may be
  improved in the future.

#+BEGIN_SRC R :exports none :results output :session *R* :cache no
set.seed(10)
dataS4 <- sampleRem(100, n.times = 4, format = "long")
dataS4$id.f <- as.factor(dataS4$id)
e.lmm1 <- lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = dataS4)
e.lmm2 <- mmrm(Y ~ visit*X1 + X6 + us(visit|id.f), data = dataS4)
logLik(e.lmm1)
logLik(e.lmm2)
microbenchmark::microbenchmark(
                  lmm = lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = dataS4),
                  lmm0 = lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = dataS4, df = FALSE),
                  mmrm = mmrm(Y ~ visit*X1 + X6 + us(visit|id.f), data = dataS4),
                  times = 50
                )
#+END_SRC

#+RESULTS:
: [1] -760.3082
: [1] -760.3082
: Unit: milliseconds
:  expr      min       lq      mean   median       uq      max neval cld
:   lmm 143.3612 163.3131 186.89159 196.0709 200.6004 220.6009    50 a  
:  lmm0  59.5744  67.4667  78.11758  79.9020  85.0430 122.4184    50  b 
:  mmrm  27.9402  34.9598  36.14161  37.1766  38.4489  45.1623    50   c


#+BEGIN_SRC R :exports none :results output :session *R* :cache no
dataL3 <- sampleRem(5000, n.times = 3, format = "long")
dataL3$id.f <- as.factor(dataL3$id)
e.lmm1 <- lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = dataL3)
e.lmm2 <- mmrm(Y ~ visit*X1 + X6 + us(visit|id.f), data = dataL3)
logLik(e.lmm1)
logLik(e.lmm2)
microbenchmark::microbenchmark(
                  lmm = lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = dataL3),
                  lmm0 = lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = dataL3, df = FALSE),
                  mmrm = mmrm(Y ~ visit*X1 + X6 + us(visit|id.f), data = dataL3),
                  times = 20
                )
#+END_SRC

#+RESULTS:
: [1] -29929.47
: [1] -29929.47
: Unit: milliseconds
:  expr      min       lq     mean   median       uq       max neval cld
:   lmm 852.9435 878.4181 909.1854 892.9624 906.9167 1164.7141    20 a  
:  lmm0 563.8959 587.0884 637.6633 603.6918 633.1166  857.8971    20  b 
:  mmrm 669.4953 678.5426 704.9954 687.4764 703.5992  966.9066    20   c


#+BEGIN_SRC R :exports none :results output :session *R* :cache no
data <- sampleRem(5000, n.times = 4, format = "long")
data$id.f <- as.factor(data$id)
e.lmm1 <- lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = data)
e.lmm2 <- mmrm(Y ~ visit*X1 + X6 + us(visit|id.f), data = data)
logLik(e.lmm1)
logLik(e.lmm2)
microbenchmark::microbenchmark(
                  lmm = lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = data),
                  lmm0 = lmm(Y ~ visit*X1 + X6, repetition = ~visit|id, data = data, df = FALSE),
                  mmrm = mmrm(Y ~ visit*X1 + X6 + us(visit|id.f), data = data),
                  times = 20
                )
#+END_SRC

#+RESULTS:
: [1] -37769.25
: [1] -37769.25
: Unit: milliseconds
:  expr      min        lq      mean    median        uq      max neval cld
:   lmm 1144.132 1438.2282 1493.0397 1472.8144 1549.4191 1886.949    20  a 
:  lmm0  739.710  885.3336  907.0729  895.3615  914.7741 1176.493    20   b
:  mmrm 1260.906 1522.0530 1554.2281 1550.6463 1574.8884 1932.091    20  a

** emmeans package

To illustrate a key difference between the emmeans package and the
=effects.lmm= function we consider an informative and unbalanced group
variable:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassLB$group2 <- gastricbypassLB$weight1>150
#+END_SRC

Since =lmm=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eCS.lmm_2 <- lmm(glucagonAUC ~ visit*group2, repetition =~visit|id, structure = "CS", data = gastricbypassLB)
logLik(eCS.lmm_2)
#+END_SRC

#+RESULTS:
: [1] -315.2

we will use the equivalent with the random effect specification:

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRI.lmer_2 <- lmer(glucagonAUC ~ visit*group2 + (1|id), data = gastricbypassLB)
logLik(eRI.lmer_2)
#+END_SRC

#+RESULTS:
: 'log Lik.' -315.2 (df=10)

While the two models are equivalent, the average outcome output by
=effects=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
effects(eCS.lmm_2, variable = NULL)
#+END_SRC

#+RESULTS:
: 		Average counterfactual outcome
: 
:          estimate    se   df  lower  upper
:    (t=1)   32.317 4.426 64.3 23.476 41.158
:    (t=2)   29.653 4.535 65.2 20.598 38.709
:    (t=3)   77.308 4.535 65.1  68.25 86.366
:    (t=4)    51.95 4.426 64.3 43.109 60.791

substantially differ from the one of emmeans:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(emmeans)
emmeans(eRI.lmer_2, specs=~visit)
#+END_SRC

#+RESULTS:
#+begin_example
NOTE: Results may be misleading due to involvement in interactions
 visit emmean   SE   df lower.CL upper.CL
 1       33.6 5.53 64.2     22.6     44.7
 2       32.0 5.57 64.4     20.9     43.2
 3       70.0 5.57 64.4     58.9     81.1
 4       47.2 5.53 64.2     36.1     58.2

Results are averaged over the levels of: group2 
Degrees-of-freedom method: kenward-roger 
Confidence level used: 0.95
#+end_example

This is because when averaging over the level of a covariate, emmeans
considers /balanced groups/. In the example, the groups are not
balanced:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
table(gastricbypassLB$group2)/NROW(gastricbypassLB)
#+END_SRC

#+RESULTS:
: 
: FALSE  TRUE 
:   0.8   0.2

Based on the group and timepoint specific means:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eCS.elmm_2 <- model.tables(effects(eCS.lmm_2, variable = "group2"))
eCS.elmm_2
#+END_SRC

#+RESULTS:
:   group2 visit estimate     se     df  lower  upper    p.value
: 1  FALSE     1   31.430 4.9484 64.349 21.545 41.314 2.4688e-08
: 2  FALSE     2   28.067 5.0996 65.383 17.884 38.251 6.6737e-07
: 3  FALSE     3   82.173 5.1008 65.211 71.986 92.359 0.0000e+00
: 4  FALSE     4   55.126 4.9484 64.349 45.241 65.010 0.0000e+00
: 5   TRUE     1   35.864 9.8967 64.349 16.095 55.633 5.7374e-04
: 6   TRUE     2   35.997 9.8967 64.349 16.228 55.766 5.4953e-04
: 7   TRUE     3   57.848 9.8967 64.349 38.079 77.617 1.8339e-07
: 8   TRUE     4   39.246 9.8967 64.349 19.477 59.015 1.8651e-04

We illustrate the difference:
- emmeans:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
0.5*eCS.elmm_2[eCS.elmm_2$group2==FALSE,"estimate"]+0.5*eCS.elmm_2[eCS.elmm_2$group2==TRUE,"estimate"]
#+END_SRC

#+RESULTS:
: [1] 33.647 32.032 70.010 47.186

- effects:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
0.8*eCS.elmm_2[eCS.elmm_2$group2==FALSE,"estimate"]+0.2*eCS.elmm_2[eCS.elmm_2$group2==TRUE,"estimate"]
#+END_SRC

#+RESULTS:
: [1] 32.317 29.653 77.308 51.950

The "emmeans" approach gives equal "weight" to the expected value of
both group:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
mu.group1 <-  as.double(coef(e.group)["(Intercept)"])
mu.group2 <-  as.double(coef(e.group)["(Intercept)"] + coef(e.group)["group2TRUE"])
p.group1 <- 14/20          ; p.group2 <- 6/20
c(emmeans = (mu.group1+mu.group2)/2, predict = mu.group1 * p.group1 + mu.group2 * p.group2)
#+END_SRC

#+RESULTS:
:  emmeans  predict 
: 4.450435 4.514352

\clearpage

** effectsize package (\(R^2\) or \(\eta^2\))

Partial \(\eta^2\) can be computed based on =lmer= using the effectsize package:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(effectsize)
eta_squared(eCS.lmer)
cat("\n")
#+END_SRC

#+RESULTS:
: # Effect Size for ANOVA (Type III)
: 
: Parameter   | Eta2 (partial) |       95% CI
: -------------------------------------------
: visit       |           0.64 | [0.50, 1.00]
: group       |           0.01 | [0.00, 1.00]
: visit:group |           0.19 | [0.03, 1.00]
: 
: - One-sided CIs: upper bound fixed at

and are approximately equal to what one can compute "manually":
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eCS.Wald <- anova(eCS.lmm)$multivariate
eCS.Wald$df.num*eCS.Wald$statistic/(eCS.Wald$df.num*eCS.Wald$statistic+eCS.Wald$df.denom)
#+END_SRC

#+RESULTS:
: [1] 0.335374 0.033811 0.186290

The will not be true for heteroschedastic models:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eUN.Wald <- anova(eUN.lmm)$multivariate
eUN.Wald$df.num*eUN.Wald$statistic/(eUN.Wald$df.num*eUN.Wald$statistic+eUN.Wald$df.denom)
#+END_SRC

#+RESULTS:
: [1] 0.50787 0.17905 0.32380

compared to:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eta_squared(eUN.lmer)
cat("\n")
#+END_SRC

#+RESULTS:
: # Effect Size for ANOVA (Type III)
: 
: Parameter   | Eta2 (partial) |       95% CI
: -------------------------------------------
: visit       |           0.76 | [0.54, 1.00]
: group       |           0.01 | [0.00, 1.00]
: visit:group |           0.32 | [0.00, 1.00]
: 
: - One-sided CIs: upper bound fixed at

But in that case both may be misleading as the proportion of explained
variance is not clearly defined.

** MuMIn package (\(R^2\))

#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(MuMIn)
r.squaredGLMM(eCS.lmer)
cat("\n")
#+END_SRC

#+RESULTS:
:          R2m     R2c
: [1,] 0.51728 0.62222

To reproduce these R2, we extract from the random intercept model:
- the residual variance
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigmaW <- sigma(eCS.lmm)[1,1]-sigma(eCS.lmm)[1,2]
#+END_SRC

#+RESULTS:

- the variance of the random effect
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigmaB <- sigma(eCS.lmm)[1,2]
#+END_SRC

#+RESULTS:

- the variance of the fitted values:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
sigma2_XB <- var(fitted(eCS.lmm))
#+END_SRC

#+RESULTS:

and evalutae the ratios:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
c(R2m = sigma2_XB/(sigmaW + sigmaB + sigma2_XB),
  R2c = (sigma2_XB + sigmaB)/(sigmaW + sigmaB + sigma2_XB))
#+END_SRC

#+RESULTS:
:     R2m     R2c 
: 0.52549 0.62865

** EMAtools for Cohen's D :noexport:

Consider again the random intercept model:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRI.lmm <- lmm(weight ~ time + glucagon + (1|id), data = dfL)
eRI.lmer <- lmer(weight ~ time + glucagon + (1|id), data = dfL)
eRI.lme <- lme(weight ~ time + glucagon, random =~ 1|id, data = dfL)
#+END_SRC

#+RESULTS:
To estimate standardized effect sizes one can use the function
=lme.dscore= of the EMAtools packages that calculate Cohen's D:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
library(EMAtools)
## lme.dscore(eRI.lmer, type = "lme4") ## error
lme.dscore(weight ~ time + glucagon + (1|id), data = dfL, type = "lme4")
## lme.dscore(eRI.lme, type = "nlme") ## very simular but warning
#+END_SRC

#+RESULTS:
:                   t       df          d
: timeB1w   -7.230297 53.97873 -1.9682252
: timeA1w  -10.159167 54.20609 -2.7597138
: timeA3m  -24.888739 54.01461 -6.7729406
: glucagon   1.327685 54.45612  0.3598336

Internally the Cohen's D is evaluated as twice the test statistic
divided by the number of degrees of freedom:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eRI.lmertable <- summary(eRI.lmer)$coefficient[-1,]
eRI.lmmtable <- model.tables(eRI.lmm, columns = add("statistic"))[-1,]

rbind(lmer = 2*eRI.lmertable[,"t value"]/sqrt(eRI.lmertable[,"df"]),
      lmm = 2*eRI.lmmtable$statistic/sqrt(eRI.lmmtable$df))
## small difference due to expected vs. observed information
#+END_SRC

#+RESULTS:
:        timeB1w   timeA1w   timeA3m  glucagon
: lmer -1.968225 -2.759714 -6.772941 0.3598336
: lmm  -1.968416 -2.765843 -6.776385 0.3614009

I am a bit surprised by the formula as to me an analogue of the
Cohen's d with a random intercept model would simply be the estimated
coefficient divided by the residual standard deviation:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
coef(eRI.lmm)["glucagon"]/sqrt(sigma(eRI.lmm)[1,1])
#+END_SRC

#+RESULTS:
:   glucagon 
: 0.04359717



#+BEGIN_SRC R :exports none :results output :session *R* :cache no
library(mvtnorm)
n <- 1e4
mu <- c(0,0,0)
sigma <- diag(0.5,3) + 0.5
dfW.sim <- rbind(data.frame(id = 1:n, group = "C", rmvnorm(n, mean = mu, sigma = sigma)),
                 data.frame(id = n+(1:n), group = "T", rmvnorm(n, mean = mu+1, sigma = sigma))
                 )
dfL.sim <- reshape2::melt(dfW.sim, id.vars = c("id","group"))
lme.dscore(value ~ group + (1|id), data = dfL.sim, type = "lme4")
eSim.lmer <- lmer(value ~ group + (1|id), data = dfL.sim)

eSim.lmm <- lmm(value ~ group + (1|id), data = dfL.sim)
coef(eSim.lmm)["groupT"]/sqrt(sigma(eSim.lmm)[1,1])

1/c(sqrt(1),sqrt(0.5))
#+END_SRC

#+RESULTS:
:               t    df        d
: groupT 86.16901 19998 1.218675
:   groupT 
: 1.408396
:    groupT 
: 0.9952911
: [1] 1.000000 1.414214


** stats package (partial residuals)

The function =residuals.lm= can be used to extract partial residuals
from =lm= objects. For instance:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
gastricbypassW$group <- as.factor(as.numeric(gastricbypassW$id)%%2)
eIID.lm <- lm(weight4 ~ group + weight1, data = gastricbypassW)
pRes.lm <- residuals(eIID.lm, type = "partial")
head(pRes.lm)
#+END_SRC

#+RESULTS:
:       group  weight1
: 1   7.19282   3.6648
: 2  -0.20504  31.7052
: 3   0.60631 -17.3352
: 4   6.44389  22.7052
: 5  -1.59403 -16.7352
: 6 -18.23382   8.4052

Those generally differ (by a constant) from the one provided by
=residuals.lmm=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
eIID.lmm <- lmm(weight4 ~ group + weight1, data = gastricbypassW)
(residuals(eIID.lmm, type = "partial", variable = "group") - pRes.lm[,"group"])
(residuals(eIID.lmm, type = "partial", variable = "weight1") - pRes.lm[,"weight1"])
#+END_SRC

#+RESULTS:
:      1      2      3      4      5      6      7      8      9     10     11     12     13     14 
: 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702 
:     15     16     17     18     19     20 
: 2.0702 2.0702 2.0702 2.0702 2.0702 2.0702
:      1      2      3      4      5      6      7      8      9     10     11     12     13     14 
: 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 106.22 
:     15     16     17     18     19     20 
: 106.22 106.22 106.22 106.22 106.22 106.22

Indeed, =residuals.lm= centers the design matrix of the variable
relative to which the partial residuals are computed:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
coef(eIID.lm)["group1"] * mean(gastricbypassW$group=="1")
coef(eIID.lm)["weight1"] * mean(gastricbypassW$weight1)
#+END_SRC

#+RESULTS:
: group1 
: 2.0702
: weight1 
:  106.22

For continuous variable with a linear effect, these residuals can be
obtained by setting the =type= argument to ="partial-center"=:
#+BEGIN_SRC R :exports both :results output :session *R* :cache no
(residuals(eIID.lmm, type = "partial-center", variable = "weight1") - pRes.lm[,"weight1"])
#+END_SRC

#+RESULTS:
:           1           2           3           4           5           6           7           8 
:  1.7675e-13  6.7502e-14 -6.3949e-14  5.6843e-14 -3.9080e-14  8.1712e-14 -3.7303e-14  5.9508e-14 
:           9          10          11          12          13          14          15          16 
: -4.2633e-14  4.4409e-14 -2.9310e-14  5.5123e-14 -4.6185e-14  4.4409e-14 -4.2633e-14  4.6185e-14 
:          17          18          19          20 
: -3.9968e-14  5.3291e-14 -1.4211e-14  3.5527e-14

\Warning When evaluating the partial residuals relative to categorical
variables, interactions, or non-linear terms, the output obtained with
=partial-center= will not match the one of =residuals.lm=. Indeed
=partial-center= will, when numeric, center the original variable
whereas =residuals.lm= will center the column relative to the
coefficient in the design matrix.



* CONFIG                                                           :noexport:
#+LANGUAGE:  en
#+LaTeX_CLASS: org-article
#+LaTeX_CLASS_OPTIONS: [12pt]
#+OPTIONS:   title:t author:t toc:nil todo:nil
#+OPTIONS:   H:3 num:t 
#+OPTIONS:   TeX:t LaTeX:t
** Display of the document
# ## space between lines
#+LATEX_HEADER: \RequirePackage{setspace} % to modify the space between lines - incompatible with footnote in beamer
#+LaTeX_HEADER:\renewcommand{\baselinestretch}{1.1}
# ## margins
#+LaTeX_HEADER: \geometry{a4paper, left=10mm, right=10mm, top=10mm}
# ## personalize the prefix in the name of the sections
#+LaTeX_HEADER: \usepackage{titlesec}
# ## fix bug in titlesec version
# ##  https://tex.stackexchange.com/questions/299969/titlesec-loss-of-section-numbering-with-the-new-update-2016-03-15
#+LaTeX_HEADER: \usepackage{etoolbox}
#+LaTeX_HEADER: 
#+LaTeX_HEADER: \makeatletter
#+LaTeX_HEADER: \patchcmd{\ttlh@hang}{\parindent\z@}{\parindent\z@\leavevmode}{}{}
#+LaTeX_HEADER: \patchcmd{\ttlh@hang}{\noindent}{}{}{}
#+LaTeX_HEADER: \makeatother
** Color
# ## define new colors
#+LATEX_HEADER: \RequirePackage{colortbl} % arrayrulecolor to mix colors
#+LaTeX_HEADER: \definecolor{myorange}{rgb}{1,0.2,0}
#+LaTeX_HEADER: \definecolor{mypurple}{rgb}{0.7,0,8}
#+LaTeX_HEADER: \definecolor{mycyan}{rgb}{0,0.6,0.6}
#+LaTeX_HEADER: \newcommand{\lightblue}{blue!50!white}
#+LaTeX_HEADER: \newcommand{\darkblue}{blue!80!black}
#+LaTeX_HEADER: \newcommand{\darkgreen}{green!50!black}
#+LaTeX_HEADER: \newcommand{\darkred}{red!50!black}
#+LaTeX_HEADER: \definecolor{gray}{gray}{0.5}
# ## change the color of the links
#+LaTeX_HEADER: \hypersetup{
#+LaTeX_HEADER:  citecolor=[rgb]{0,0.5,0},
#+LaTeX_HEADER:  urlcolor=[rgb]{0,0,0.5},
#+LaTeX_HEADER:  linkcolor=[rgb]{0,0,0.5},
#+LaTeX_HEADER: }
** Font
# https://tex.stackexchange.com/questions/25249/how-do-i-use-a-particular-font-for-a-small-section-of-text-in-my-document
#+LaTeX_HEADER: \newenvironment{note}{\small \color{gray}\fontfamily{lmtt}\selectfont}{\par}
#+LaTeX_HEADER: \newenvironment{activity}{\color{orange}\fontfamily{qzc}\selectfont}{\par}
** Symbols
# ## valid and cross symbols
#+LaTeX_HEADER: \RequirePackage{pifont}
#+LaTeX_HEADER: \RequirePackage{relsize}
#+LaTeX_HEADER: \newcommand{\Cross}{{\raisebox{-0.5ex}%
#+LaTeX_HEADER:		{\relsize{1.5}\ding{56}}}\hspace{1pt} }
#+LaTeX_HEADER: \newcommand{\Valid}{{\raisebox{-0.5ex}%
#+LaTeX_HEADER:		{\relsize{1.5}\ding{52}}}\hspace{1pt} }
#+LaTeX_HEADER: \newcommand{\CrossR}{ \textcolor{red}{\Cross} }
#+LaTeX_HEADER: \newcommand{\ValidV}{ \textcolor{green}{\Valid} }
# ## warning symbol
#+LaTeX_HEADER: \usepackage{stackengine}
#+LaTeX_HEADER: \usepackage{scalerel}
#+LaTeX_HEADER: \newcommand\Warning[1][3ex]{%
#+LaTeX_HEADER:   \renewcommand\stacktype{L}%
#+LaTeX_HEADER:   \scaleto{\stackon[1.3pt]{\color{red}$\triangle$}{\tiny\bfseries !}}{#1}%
#+LaTeX_HEADER:   \xspace
#+LaTeX_HEADER: }

** Code
:PROPERTIES:
:ID: 2ec77c4b-f83d-4612-9a89-a96ba1b7bf70
:END:
# Documentation at https://org-babel.readthedocs.io/en/latest/header-args/#results
# :tangle (yes/no/filename) extract source code with org-babel-tangle-file, see http://orgmode.org/manual/Extracting-source-code.html 
# :cache (yes/no)
# :eval (yes/no/never)
# :results (value/output/silent/graphics/raw/latex)
# :export (code/results/none/both)
#+PROPERTY: header-args :session *R* :tangle yes :cache no ## extra argument need to be on the same line as :session *R*
# Code display:
#+LATEX_HEADER: \RequirePackage{fancyvrb}
#+LATEX_HEADER: \DefineVerbatimEnvironment{verbatim}{Verbatim}{fontsize=\small,formatcom = {\color[rgb]{0.5,0,0}}}
# ## change font size input (global change)
# ## doc: https://ctan.math.illinois.edu/macros/latex/contrib/listings/listings.pdf
# #+LATEX_HEADER: \newskip kipamount    kipamount =6pt plus 0pt minus 6pt
# #+LATEX_HEADER: \lstdefinestyle{code-tiny}{basicstyle=\ttfamily\tiny, aboveskip =  kipamount, belowskip =  kipamount}
# #+LATEX_HEADER: \lstset{style=code-tiny}
# ## change font size input (local change, put just before BEGIN_SRC)
# ## #+ATTR_LATEX: :options basicstyle=\ttfamily\scriptsize
# ## change font size output (global change)
# ## \RecustomVerbatimEnvironment{verbatim}{Verbatim}{fontsize=\tiny,formatcom = {\color[rgb]{0.5,0,0}}}
** Rlogo
#+LATEX_HEADER:\definecolor{grayR}{HTML}{8A8990}
#+LATEX_HEADER:\definecolor{grayL}{HTML}{C4C7C9}
#+LATEX_HEADER:\definecolor{blueM}{HTML}{1F63B5}   
#+LATEX_HEADER: \newcommand{\Rlogo}[1][0.07]{
#+LATEX_HEADER: \begin{tikzpicture}[scale=#1]
#+LATEX_HEADER: \shade [right color=grayR,left color=grayL,shading angle=60] 
#+LATEX_HEADER: (-3.55,0.3) .. controls (-3.55,1.75) 
#+LATEX_HEADER: and (-1.9,2.7) .. (0,2.7) .. controls (2.05,2.7)  
#+LATEX_HEADER: and (3.5,1.6) .. (3.5,0.3) .. controls (3.5,-1.2) 
#+LATEX_HEADER: and (1.55,-2) .. (0,-2) .. controls (-2.3,-2) 
#+LATEX_HEADER: and (-3.55,-0.75) .. cycle;
#+LATEX_HEADER: 
#+LATEX_HEADER: \fill[white] 
#+LATEX_HEADER: (-2.15,0.2) .. controls (-2.15,1.2) 
#+LATEX_HEADER: and (-0.7,1.8) .. (0.5,1.8) .. controls (2.2,1.8) 
#+LATEX_HEADER: and (3.1,1.2) .. (3.1,0.2) .. controls (3.1,-0.75) 
#+LATEX_HEADER: and (2.4,-1.45) .. (0.5,-1.45) .. controls (-1.1,-1.45) 
#+LATEX_HEADER: and (-2.15,-0.7) .. cycle;
#+LATEX_HEADER: 
#+LATEX_HEADER: \fill[blueM] 
#+LATEX_HEADER: (1.75,1.25) -- (-0.65,1.25) -- (-0.65,-2.75) -- (0.55,-2.75) -- (0.55,-1.15) -- 
#+LATEX_HEADER: (0.95,-1.15)  .. controls (1.15,-1.15) 
#+LATEX_HEADER: and (1.5,-1.9) .. (1.9,-2.75) -- (3.25,-2.75)  .. controls (2.2,-1) 
#+LATEX_HEADER: and (2.5,-1.2) .. (1.8,-0.95) .. controls (2.6,-0.9) 
#+LATEX_HEADER: and (2.85,-0.35) .. (2.85,0.2) .. controls (2.85,0.7) 
#+LATEX_HEADER: and (2.5,1.2) .. cycle;
#+LATEX_HEADER: 
#+LATEX_HEADER: \fill[white]  (1.4,0.4) -- (0.55,0.4) -- (0.55,-0.3) -- (1.4,-0.3).. controls (1.75,-0.3) 
#+LATEX_HEADER: and (1.75,0.4) .. cycle;
#+LATEX_HEADER: 
#+LATEX_HEADER: \end{tikzpicture}
#+LATEX_HEADER: }
** Image and graphs
#+LATEX_HEADER: \RequirePackage{epstopdf} % to be able to convert .eps to .pdf image files
#+LATEX_HEADER: \RequirePackage{capt-of} % 
#+LATEX_HEADER: \RequirePackage{caption} % newlines in graphics
#+LaTeX_HEADER: \RequirePackage{tikz-cd} % graph
# ## https://tools.ietf.org/doc/texlive-doc/latex/tikz-cd/tikz-cd-doc.pdf
** Table
#+LATEX_HEADER: \RequirePackage{booktabs} % for nice lines in table (e.g. toprule, bottomrule, midrule, cmidrule)
** Inline latex
# @@latex:any arbitrary LaTeX code@@
** Algorithm
#+LATEX_HEADER: \RequirePackage{amsmath}
#+LATEX_HEADER: \RequirePackage{algorithm}
#+LATEX_HEADER: \RequirePackage[noend]{algpseudocode}
** Math
#+LATEX_HEADER: \RequirePackage{dsfont}
#+LATEX_HEADER: \RequirePackage{amsmath,stmaryrd,graphicx}
#+LATEX_HEADER: \RequirePackage{prodint} % product integral symbol (\PRODI)
# ## lemma
# #+LaTeX_HEADER: \RequirePackage{amsthm}
# #+LaTeX_HEADER: \newtheorem{theorem}{Theorem}
# #+LaTeX_HEADER: \newtheorem{lemma}[theorem]{Lemma}
*** Template for shortcut
#+LATEX_HEADER: \usepackage{ifthen}
#+LATEX_HEADER: \usepackage{xifthen}
#+LATEX_HEADER: \usepackage{xargs}
#+LATEX_HEADER: \usepackage{xspace}
#+LATEX_HEADER: \newcommand\defOperator[7]{%
#+LATEX_HEADER:	\ifthenelse{\isempty{#2}}{
#+LATEX_HEADER:		\ifthenelse{\isempty{#1}}{#7{#3}#4}{#7{#3}#4 \left#5 #1 \right#6}
#+LATEX_HEADER:	}{
#+LATEX_HEADER:	\ifthenelse{\isempty{#1}}{#7{#3}#4_{#2}}{#7{#3}#4_{#1}\left#5 #2 \right#6}
#+LATEX_HEADER: }
#+LATEX_HEADER: }
#+LATEX_HEADER: \newcommand\defUOperator[5]{%
#+LATEX_HEADER: \ifthenelse{\isempty{#1}}{
#+LATEX_HEADER:		#5\left#3 #2 \right#4
#+LATEX_HEADER: }{
#+LATEX_HEADER:	\ifthenelse{\isempty{#2}}{\underset{#1}{\operatornamewithlimits{#5}}}{
#+LATEX_HEADER:		\underset{#1}{\operatornamewithlimits{#5}}\left#3 #2 \right#4}
#+LATEX_HEADER: }
#+LATEX_HEADER: }
#+LATEX_HEADER: \newcommand{\defBoldVar}[2]{	
#+LATEX_HEADER:	\ifthenelse{\equal{#2}{T}}{\boldsymbol{#1}}{\mathbf{#1}}
#+LATEX_HEADER: }
**** Probability
#+LATEX_HEADER: \newcommandx\Esp[2][1=,2=]{\defOperator{#1}{#2}{E}{}{\lbrack}{\rbrack}{\mathbb}}
#+LATEX_HEADER: \newcommandx\Prob[2][1=,2=]{\defOperator{#1}{#2}{P}{}{\lbrack}{\rbrack}{\mathbb}}
#+LATEX_HEADER: \newcommandx\Qrob[2][1=,2=]{\defOperator{#1}{#2}{Q}{}{\lbrack}{\rbrack}{\mathbb}}
#+LATEX_HEADER: \newcommandx\Var[2][1=,2=]{\defOperator{#1}{#2}{V}{ar}{\lbrack}{\rbrack}{\mathbb}}
#+LATEX_HEADER: \newcommandx\Cov[2][1=,2=]{\defOperator{#1}{#2}{C}{ov}{\lbrack}{\rbrack}{\mathbb}}
#+LATEX_HEADER: \newcommandx\Binom[2][1=,2=]{\defOperator{#1}{#2}{B}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\Gaus[2][1=,2=]{\defOperator{#1}{#2}{N}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\Wishart[2][1=,2=]{\defOperator{#1}{#2}{W}{ishart}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\Likelihood[2][1=,2=]{\defOperator{#1}{#2}{L}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\logLikelihood[2][1=,2=]{\defOperator{#1}{#2}{\ell}{}{(}{)}{}}
#+LATEX_HEADER: \newcommandx\Information[2][1=,2=]{\defOperator{#1}{#2}{I}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\Hessian[2][1=,2=]{\defOperator{#1}{#2}{H}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\Score[2][1=,2=]{\defOperator{#1}{#2}{S}{}{(}{)}{\mathcal}}
**** Operators
#+LATEX_HEADER: \newcommandx\Vois[2][1=,2=]{\defOperator{#1}{#2}{V}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\IF[2][1=,2=]{\defOperator{#1}{#2}{IF}{}{(}{)}{\mathcal}}
#+LATEX_HEADER: \newcommandx\Ind[1][1=]{\defOperator{}{#1}{1}{}{(}{)}{\mathds}}
#+LATEX_HEADER: \newcommandx\Max[2][1=,2=]{\defUOperator{#1}{#2}{(}{)}{min}}
#+LATEX_HEADER: \newcommandx\Min[2][1=,2=]{\defUOperator{#1}{#2}{(}{)}{max}}
#+LATEX_HEADER: \newcommandx\argMax[2][1=,2=]{\defUOperator{#1}{#2}{(}{)}{argmax}}
#+LATEX_HEADER: \newcommandx\argMin[2][1=,2=]{\defUOperator{#1}{#2}{(}{)}{argmin}}
#+LATEX_HEADER: \newcommandx\cvD[2][1=D,2=n \rightarrow \infty]{\xrightarrow[#2]{#1}}
#+LATEX_HEADER: \newcommandx\Hypothesis[2][1=,2=]{
#+LATEX_HEADER:         \ifthenelse{\isempty{#1}}{
#+LATEX_HEADER:         \mathcal{H}
#+LATEX_HEADER:         }{
#+LATEX_HEADER: 	\ifthenelse{\isempty{#2}}{
#+LATEX_HEADER: 		\mathcal{H}_{#1}
#+LATEX_HEADER: 	}{
#+LATEX_HEADER: 	\mathcal{H}^{(#2)}_{#1}
#+LATEX_HEADER:         }
#+LATEX_HEADER:         }
#+LATEX_HEADER: }
#+LATEX_HEADER: \newcommandx\dpartial[4][1=,2=,3=,4=\partial]{
#+LATEX_HEADER: 	\ifthenelse{\isempty{#3}}{
#+LATEX_HEADER: 		\frac{#4 #1}{#4 #2}
#+LATEX_HEADER: 	}{
#+LATEX_HEADER: 	\left.\frac{#4 #1}{#4 #2}\right\rvert_{#3}
#+LATEX_HEADER: }
#+LATEX_HEADER: }
#+LATEX_HEADER: \newcommandx\dTpartial[3][1=,2=,3=]{\dpartial[#1][#2][#3][d]}
#+LATEX_HEADER: \newcommandx\ddpartial[3][1=,2=,3=]{
#+LATEX_HEADER: 	\ifthenelse{\isempty{#3}}{
#+LATEX_HEADER: 		\frac{\partial^{2} #1}{\partial #2^2}
#+LATEX_HEADER: 	}{
#+LATEX_HEADER: 	\frac{\partial^2 #1}{\partial #2\partial #3}
#+LATEX_HEADER: }
#+LATEX_HEADER: } 
**** General math
#+LATEX_HEADER: \newcommand\Real{\mathbb{R}}
#+LATEX_HEADER: \newcommand\Rational{\mathbb{Q}}
#+LATEX_HEADER: \newcommand\Natural{\mathbb{N}}
#+LATEX_HEADER: \newcommand\trans[1]{{#1}^\intercal}%\newcommand\trans[1]{{\vphantom{#1}}^\top{#1}}
#+LATEX_HEADER: \newcommand{\independent}{\mathrel{\text{\scalebox{1.5}{$\perp\mkern-10mu\perp$}}}}
#+LaTeX_HEADER: \newcommand\half{\frac{1}{2}}
#+LaTeX_HEADER: \newcommand\normMax[1]{\left|\left|#1\right|\right|_{max}}
#+LaTeX_HEADER: \newcommand\normTwo[1]{\left|\left|#1\right|\right|_{2}}
#+LATEX_HEADER: \newcommand\Veta{\boldsymbol{\eta}}

** Notations

#+LaTeX_HEADER:\newcommand{\Model}{\mathcal{M}}
#+LaTeX_HEADER:\newcommand{\ModelHat}{\widehat{\mathcal{M}}}

#+LaTeX_HEADER:\newcommand{\param}{\Theta}
#+LaTeX_HEADER:\newcommand{\paramHat}{\widehat{\param}}
#+LaTeX_HEADER:\newcommand{\paramCon}{\widetilde{\param}}

#+LaTeX_HEADER:\newcommand{\Vparam}{\boldsymbol{\param}}
#+LaTeX_HEADER:\newcommand{\VparamT}{\Vparam_0}
#+LaTeX_HEADER:\newcommand{\VparamHat}{\boldsymbol{\paramHat}}
#+LaTeX_HEADER:\newcommand{\VparamCon}{\boldsymbol{\paramCon}}

#+LaTeX_HEADER:\newcommand{\X}{X}
#+LaTeX_HEADER:\newcommand{\x}{x}
#+LaTeX_HEADER:\newcommand{\VX}{\boldsymbol{X}}
#+LaTeX_HEADER:\newcommand{\Vx}{\boldsymbol{x}}

#+LaTeX_HEADER:\newcommand{\Y}{Y}
#+LaTeX_HEADER:\newcommand{\y}{y}
#+LaTeX_HEADER:\newcommand{\VY}{\boldsymbol{Y}}
#+LaTeX_HEADER:\newcommand{\Vy}{\boldsymbol{y}}
#+LaTeX_HEADER:\newcommand{\Vvarepsilon}{\boldsymbol{\varepsilon}}


