# Meta-analyzing correlations

#### Brenton Wiernik

#### 2024-06-19

Note: This vignette is a work in progress.

This vignette will walk you through estimating barebones meta-analyses of correlations between multiple constructs. For more vignettes, see the psychmeta overview.

## Getting Started

To begin, you will need your meta-analytic data sheet for analysis. We recommend the rio package for importing data to R. For an introduction to rio, see `vignette("rio", "rio")`.

psychmeta assumes that your data are in “long” format, with each row corresponding to one effect size. For example, this is the format used in this data frame:

    knitr::kable(data_r_meas_multi[1:10,])

sample_id | moderator | x_name | y_name | n | rxyi | rxxi | ryyi | citekey
--- | --- | --- | --- | --- | --- | --- | --- | ---
1 | 1 | X | Y | 416 | 0.49 | 0.79 | 0.77 | Watson2005
1 | 1 | X | Z | 416 | 0.40 | 0.79 | 0.77 | Watson2005
1 | 1 | Y | Z | 416 | 0.36 | 0.77 | 0.77 | Watson2005
2 | 1 | X | Y | 241 | 0.54 | 0.82 | 0.84 | Watson2005
2 | 1 | X | Z | 241 | 0.56 | 0.82 | 0.89 | Watson2005
2 | 1 | Y | Z | 241 | 0.62 | 0.84 | 0.89 | Watson2005
3 | 1 | X | Y | 479 | 0.34 | 0.73 | 0.87 | Zellars2006
3 | 1 | X | Z | 479 | 0.40 | 0.73 | 0.79 | Zellars2006
3 | 1 | Y | Z | 479 | 0.53 | 0.87 | 0.79 | Zellars2006
4 | 1 | X | Y | 167 | 0.37 | 0.78 | 0.79 | Bandura1980

In this table, - `sample_id` contains labels indicating the sample each effect size is drawn from; - `moderator` is a moderator variable, each row containing the effect size’s level for that moderator; - `x_name` and `y_name` are columns indicating the variables/constructs being related in the effect size; - `n` is the sample size for the effect size; - `rxyi` is the effect size (the correlation between the two constructs/variables); - `rxxi` and `ryyi` are the sample reliability values for the measures of the `x_name` and `y_name` variables, respectively; - `citekey` contains the citations keys for each study (used to generate bibliographies of included studies).

You can see this data set includes correlations among three variables: `X`, `Y`, and `Z`, and that each sample contributes several effect sizes, one each for for different pairs of variables/constructs.

If your data are in a different format, you can use the `reshape_wide2long()` function to reshape it.

## Estimating a Barebones Meta-Analysis

Let’s assume your data frame is called `coding_sheet`.

    coding_sheet <- data_r_meas_multi

    head(coding_sheet)
    #>   sample_id moderator x_name y_name   n rxyi rxxi ryyi    citekey
    #> 1         1         1      X      Y 416 0.49 0.79 0.77 Watson2005
    #> 2         1         1      X      Z 416 0.40 0.79 0.77 Watson2005
    #> 3         1         1      Y      Z 416 0.36 0.77 0.77 Watson2005
    #> 4         2         1      X      Y 241 0.54 0.82 0.84 Watson2005
    #> 5         2         1      X      Z 241 0.56 0.82 0.89 Watson2005
    #> 6         2         1      Y      Z 241 0.62 0.84 0.89 Watson2005

The primary function to conduct meta-analyses of correlations is `ma_r()`. To conduct barebones meta-analyses, run:

    ma_res <- ma_r(rxyi = rxyi,
                   n = n,
                   construct_x = x_name,
                   construct_y = y_name,
                   sample_id = sample_id,
                   moderators = moderator,
                   data = coding_sheet
                   )
    #>  **** Running ma_r: Meta-analysis of correlations ****

- `data` is your data frame.
- `rxyi`, `n`, `construct_x`, `construct_y`, `sample_id`, and `moderators` are the names of the columns in your data frame that contain the appropriate values.
  - `rxyi` is the correlation effect sizes;
  - `n` is the sample sizes;
  - `construct_x` and `construct_y` are the labels for the variables/constructs being correlated;
  - `sample_id` is the sample identification labels;
  - `moderators` is a vector of moderator variable names for the meta-analyses.
  - Column names can be provided either with quotes (e.g., `"rxyi"`, `"n"`) or without (e.g., `rxyi`, `n`).

To conduct a barebones meta-analysis, at minimum, `n` and `rxyi` are needed.

### Modeling Options

- By default, correlations are weighted by sample size. You can specify alternative weights using the `wt_type` argument.

- Random-effects variance (τ2 or SDres2) is estimated using the Hunter-Schmidt estimator, computed using the unbiased sample variance estimator (i.e., dividing by \(k-1\) rather than \(k\)). To use the maximum-likelihood estimator instead, specify `var_unbiased = FALSE`.

- Barebones results are corrected for the small-sample bias in the correlation coefficient. To disable this correction, specify `correct_bias = FALSE`.

- By default, confidence and credibility intervals are constructed using a t distribution with \(k-1\) degrees of freedom. To use a normal distribution instead, specify, `conf_method = "norm"` and `cred_method = "norm"`. To customize the coverage levels for these intervals, use the `conf_level` and `cred_level` arguments.

## The psychmeta Meta-Analysis Object

A psychmeta meta-analsyis object is a data frame, with each row being a meta-analysis or subanalysis and each column containing information about or results from that analysis. For example, the results of the analysis above look like this:

    ma_res
    #> Overview tibble of psychmeta meta-analysis of correlations
    #> ----------------------------------------------------------------------
    #> # A tibble: 9 × 8
    #>   analysis_id pair_id construct_x construct_y analysis_type    moderator  meta_tables      escalc
    #> *       <int>   <int> <fct>       <fct>       <fct>            <fct>      <named list>     <named list>
    #> 1           1       1 X           Y           Overall          All Levels <named list [3]> <named list [4]>
    #> 2           2       1 X           Y           Simple Moderator 1          <named list [3]> <named list [4]>
    #> 3           3       1 X           Y           Simple Moderator 2          <named list [3]> <named list [4]>
    #> 4           4       2 X           Z           Overall          All Levels <named list [3]> <named list [4]>
    #> 5           5       2 X           Z           Simple Moderator 1          <named list [3]> <named list [4]>
    #> 6           6       2 X           Z           Simple Moderator 2          <named list [3]> <named list [4]>
    #> 7           7       3 Y           Z           Overall          All Levels <named list [3]> <named list [4]>
    #> 8           8       3 Y           Z           Simple Moderator 1          <named list [3]> <named list [4]>
    #> 9           9       3 Y           Z           Simple Moderator 2          <named list [3]> <named list [4]>
    #>
    #> To extract results, try summary() or the get_stuff functions (run ?get_stuff for help).

Each row corresponds to a different pair of variables/constructs (X-Y; X-Z; Y-Z) and level of the moderator variable (overall/all levels pooled together; moderator = 1; moderator = 2).

- `analysis_id` is a numeric label for each analysis;

- `pair_id` is a numeric label for each pair of variables/constructs (X-Y; X-Z; Y-Z);

- `construct_x` and `construct_y` indicate which variables/constructs are being meta-analyzed.

- `analysis_type` indicates the type of analysis.

  - “Overall” means an overall meta-analysis, pooling across all moderator levels.

  - “Simple Moderator” means a subgroup moderator analysis of only studies with the specified levels of the moderator variable(s) in the next column(s).

    (See below for how to conduct meta-analyses with multiple moderator variables or with continuous moderators)

- `meta_tables` contains the principal meta-analysis results tables.

- `escalc` contains tables of effect sizes, sampling error variances, weights, residuals, and other data. These tables can be used for follow-up analyses or with the metafor package for additional meta-analysis techniques.

## Viewing Results Summaries

To view meta-anlaysis results tables, use the `summary()` function:

    summary(ma_res)
    #> Bare-bones meta-analysis results
    #> ----------------------------------------------------------------------
    #>   analysis_id pair_id construct_x construct_y    analysis_type  moderator  k     N mean_r   sd_r   se_r sd_res CI_LL_95 CI_UL_95 CR_LL_80 CR_UL_80
    #> 1           1       1           X           Y          Overall All Levels 40 11927  0.317 0.1249 0.0198 0.1135    0.277    0.357    0.169    0.465
    #> 2           2       1           X           Y Simple Moderator          1 20  5623  0.397 0.0886 0.0198 0.0729    0.356    0.439    0.300    0.494
    #> 3           3       1           X           Y Simple Moderator          2 20  6304  0.245 0.1086 0.0243 0.0948    0.194    0.296    0.119    0.371
    #> 4           4       2           X           Z          Overall All Levels 40 11927  0.324 0.1288 0.0204 0.1179    0.282    0.365    0.170    0.477
    #> 5           5       2           X           Z Simple Moderator          1 20  5623  0.422 0.0922 0.0206 0.0780    0.379    0.465    0.319    0.526
    #> 6           6       2           X           Z Simple Moderator          2 20  6304  0.236 0.0861 0.0193 0.0677    0.195    0.276    0.146    0.326
    #> 7           7       3           Y           Z          Overall All Levels 40 11927  0.311 0.1369 0.0217 0.1265    0.268    0.355    0.146    0.476
    #> 8           8       3           Y           Z Simple Moderator          1 20  5623  0.405 0.1100 0.0246 0.0979    0.354    0.457    0.275    0.535
    #> 9           9       3           Y           Z Simple Moderator          2 20  6304  0.228 0.0997 0.0223 0.0841    0.181    0.274    0.116    0.339
    #>
    #>
    #> Information available in the meta-analysis object includes:
    #>  - meta_tables   [ access using get_metatab() ]
    #>  - escalc        [ access using get_escalc() ]

In this table,

- `analysis_id`, `pair_id`, `construct_x`, `construct_y`, and the moderator columns are defined as above.
- `k` is the number of effect sizes contributing to each meta-analysis. `N` is the total sample size contributing to each meta-analysis.
- `mean_r` is the weighted mean correlation.
- `sd_r` is the weighted observed standard deviation of correlations.
- `se_r` is the standard error of `mean_r`.
- `sd_res` is the estimated random-effects standard deviation (residual SD of correlations after accounting for sampling error).
- `CI_LL_95` and `CI_UL_95` are the upper and lower bounds of the confidence interval for `mean_r`; the number indicates the coverage level (default: 95%).
- `CR_LL_80` and `CR_UL_80` are the upper and lower bounds of the credibility interval for the estimated population distribution; the number indicates the coverage level (default: 80%).

To view additional results, such as observed variance (`var_r`) or standard deviation of sampling errors (`sd_e`), use the `get_metatab()` function and select the appropriate columns:

    names(get_metatab(ma_res))
    #>  [1] "analysis_id"   "pair_id"       "construct_x"   "construct_y"   "analysis_type" "moderator"     "k"             "N"             "mean_r"
    #> [10] "var_r"         "var_e"         "var_res"       "sd_r"          "se_r"          "sd_e"          "sd_res"        "CI_LL_95"      "CI_UL_95"
    #> [19] "CR_LL_80"      "CR_UL_80"

    get_metatab(ma_res)$var_r
    #> [1] 0.015607018 0.007854615 0.011804107 0.016601343 0.008502949 0.007418860 0.018751347 0.012097853 0.009938716

To view all columns of this table, convert it to a `data.frame` or `tibble`:

    dplyr::as_tibble(get_metatab(ma_res))
    #> # A tibble: 9 × 20
    #>   analysis_id pair_id construct_x construct_y analysis_type   moderator     k     N mean_r   var_r   var_e var_res   sd_r   se_r   sd_e sd_res CI_LL_95 CI_UL_95
    #>         <int>   <int> <fct>       <fct>       <fct>           <fct>     <dbl> <dbl>  <dbl>   <dbl>   <dbl>   <dbl>  <dbl>  <dbl>  <dbl>  <dbl>    <dbl>    <dbl>
    #> 1           1       1 X           Y           Overall         All Leve…    40 11927  0.317 0.0156  0.00273 0.0129  0.125  0.0198 0.0523 0.113     0.277    0.357
    #> 2           2       1 X           Y           Simple Moderat… 1            20  5623  0.397 0.00785 0.00255 0.00531 0.0886 0.0198 0.0504 0.0729    0.356    0.439
    #> 3           3       1 X           Y           Simple Moderat… 2            20  6304  0.245 0.0118  0.00281 0.00899 0.109  0.0243 0.0530 0.0948    0.194    0.296
    #> 4           4       2 X           Z           Overall         All Leve…    40 11927  0.324 0.0166  0.00270 0.0139  0.129  0.0204 0.0520 0.118     0.282    0.365
    #> 5           5       2 X           Z           Simple Moderat… 1            20  5623  0.422 0.00850 0.00242 0.00608 0.0922 0.0206 0.0492 0.0780    0.379    0.465
    #> 6           6       2 X           Z           Simple Moderat… 2            20  6304  0.236 0.00742 0.00284 0.00458 0.0861 0.0193 0.0533 0.0677    0.195    0.276
    #> 7           7       3 Y           Z           Overall         All Leve…    40 11927  0.311 0.0188  0.00275 0.0160  0.137  0.0217 0.0525 0.126     0.268    0.355
    #> 8           8       3 Y           Z           Simple Moderat… 1            20  5623  0.405 0.0121  0.00251 0.00959 0.110  0.0246 0.0501 0.0979    0.354    0.457
    #> 9           9       3 Y           Z           Simple Moderat… 2            20  6304  0.228 0.00994 0.00286 0.00708 0.0997 0.0223 0.0535 0.0841    0.181    0.274
    #> # ℹ 2 more variables: CR_LL_80 <dbl>, CR_UL_80 <dbl>

    as.data.frame(get_metatab(ma_res))
    #>   analysis_id pair_id construct_x construct_y    analysis_type  moderator  k     N    mean_r       var_r       var_e     var_res       sd_r       se_r
    #> 1           1       1           X           Y          Overall All Levels 40 11927 0.3166643 0.015607018 0.002731165 0.012875853 0.12492805 0.01975286
    #> 2           2       1           X           Y Simple Moderator          1 20  5623 0.3971059 0.007854615 0.002545026 0.005309589 0.08862626 0.01981744
    #> 3           3       1           X           Y Simple Moderator          2 20  6304 0.2449125 0.011804107 0.002813166 0.008990941 0.10864671 0.02429414
    #> 4           4       2           X           Z          Overall All Levels 40 11927 0.3235671 0.016601343 0.002704401 0.013896942 0.12884620 0.02037237
    #> 5           5       2           X           Z Simple Moderator          1 20  5623 0.4220597 0.008502949 0.002422999 0.006079950 0.09221143 0.02061910
    #> 6           6       2           X           Z Simple Moderator          2 20  6304 0.2357143 0.007418860 0.002839689 0.004579171 0.08613281 0.01925988
    #> 7           7       3           Y           Z          Overall All Levels 40 11927 0.3113684 0.018751347 0.002751395 0.015999952 0.13693556 0.02165141
    #> 8           8       3           Y           Z Simple Moderator          1 20  5623 0.4053224 0.012097853 0.002505340 0.009592513 0.10999024 0.02459457
    #> 9           9       3           Y           Z Simple Moderator          2 20  6304 0.2275640 0.009938716 0.002862440 0.007076275 0.09969311 0.02229206
    #>         sd_e     sd_res  CI_LL_95  CI_UL_95  CR_LL_80  CR_UL_80
    #> 1 0.05226055 0.11347182 0.2767104 0.3566182 0.1687380 0.4645905
    #> 2 0.05044825 0.07286692 0.3556276 0.4385843 0.3003585 0.4938534
    #> 3 0.05303929 0.09482057 0.1940642 0.2957607 0.1190165 0.3708084
    #> 4 0.05200385 0.11788529 0.2823600 0.3647741 0.1698872 0.4772469
    #> 5 0.04922396 0.07797403 0.3789034 0.4652160 0.3185314 0.5255880
    #> 6 0.05328873 0.06766957 0.1954029 0.2760257 0.1458675 0.3255611
    #> 7 0.05245374 0.12649092 0.2675743 0.3551626 0.1464700 0.4762669
    #> 8 0.05005337 0.09794137 0.3538453 0.4567994 0.2752828 0.5353619
    #> 9 0.05350178 0.08412060 0.1809062 0.2742218 0.1158747 0.3392533

### Moderator Analyses

Results for subgroup analyses for different levels of categorical moderators are shown in the rows of the meta-analysis results table. To estimate confidence intervals for differences between levels or an omnibus ANOVA statistic, use the `anova()` function:

    anova(ma_res)
    #> Warning: There was 1 warning in `mutate()`.
    #> ℹ In argument: `anova = purrr::map(.data$data, .anova.ma_psychmeta, conf_level)`.
    #> ℹ In group 1: `pair_id = 1`, `construct_x = X`, `construct_y = Y`.
    #> Caused by warning:
    #> ! The `x` argument of `as_tibble.matrix()` must have unique column names if `.name_repair` is omitted as of tibble 2.0.0.
    #> ℹ Using compatibility `.name_repair`.
    #> ℹ The deprecated feature was likely used in the psychmeta package.
    #>   Please report the issue at <https://github.com/psychmeta/psychmeta/issues>.
    #>   pair_id construct_x construct_y moderator F value df_num df_denom level_1 level_2 mean_1 mean_2  diff CI_LL_95 CI_UL_95
    #> 1       1           X           Y moderator    23.6      1     36.5       1       2  0.397  0.245 0.152   0.0887    0.216
    #> 2       2           X           Z moderator    43.6      1     37.8       1       2  0.422  0.236 0.186   0.1292    0.243
    #> 3       3           Y           Z moderator    28.7      1     37.6       1       2  0.405  0.228 0.178   0.1106    0.245

### Correcting for Statistical Artifacts

See Artifact corrections](artifact_corrections.html).

## Outputting Results

To output the main meta-analysis results table to RMarkdown, Word, HTML, PDF, or other formats, use the `metabulate()` function. For example, to output the above results to a Word document, run:

    metabulate(ma_res, file = "meta-analysis_results.docx", output_format = "word")

## Follow-Up Analyses

### Plotting

You can add plots for each meta-analysis in `ma_res` using the `plot_forest()` and `plot_funnel()` functions:

    ma_res <- plot_funnel(ma_res)
    #> Funnel plots have been added to 'ma_obj' - use get_plots() to retrieve them.
    ma_res <- plot_forest(ma_res)
    #> Forest plots have been added to 'ma_obj' - use get_plots() to retrieve them.

You can view these plots using the `get_plots()` function. This will return a list of all of the plots in this results. Specify which meta-analysis you want to view plots for by passing its `analysis_id` to `[[`:

    get_plots(ma_res)[["forest"]][[2]]
    #> $moderated
    #> NULL
    #>
    #> $unmoderated
    #> $unmoderated$barebones
    #>
    #> $unmoderated$individual_correction
    #> NULL
    get_plots(ma_res)[["funnel"]][[2]]
    #> $barebones
    #>
    #> $individual_correction
    #> list()
    #>
    #> $artifact_distribution
    #> list()

For forest plots, if you select an “Overall” meta-analysis, it will include plots faceted by moderator levels (`"moderated"`) and not (`"unmoderated"`):

    get_plots(ma_res)[["forest"]][[1]][["moderated"]][["barebones"]]
    get_plots(ma_res)[["forest"]][[1]][["unmoderated"]][["barebones"]]

### Heterogeneity Analyses

psychmeta reports the random-effects standard deviaton (τ or SD_res_) and credibility intervals (`mean_r` ± crit × SDres) in the main meta-analaysis results tables. To view confidence intervals for SD_res_ or additional heterogeneity statistics, use the `heterogeneity()` function:

    ma_res <- heterogeneity(ma_res)
    #> Heterogeneity analyses have been added to 'ma_obj' - use get_heterogeneity() to retrieve them.
    get_heterogeneity(ma_res)[[1]][["barebones"]]
    #>
    #> Heterogeneity results for r
    #> ---------------------------
    #>
    #> Accounted for a total of 17.500% of variance
    #>
    #> Correlation between r values and artifactual perturbations: 0.418
    #>
    #> The reliability of observed effect sizes is: 0.825
    #>
    #>
    #> Random effects variance estimates
    #> ---------------------------------
    #> Hunter-Schmidt method (with k-correction):
    #>   sd_res  (tau):   0.113, SE = 0.016, 95% CI = [0.088, 0.152]
    #>   var_res (tau^2): 0.013, SE = 0.004, 95% CI = [0.008, 0.023]
    #>
    #>   Q statistic: 222.862 (df = 39, p = 0.000)
    #>   H: 2.390   H^2: 5.714   I^2: 82.500
    #>
    #> DerSimonian-Laird method:
    #>   sd_res  (tau):   0.114
    #>   var_res (tau^2): 0.013
    #>
    #>   Q statistic: 223.408
    #>   H: 2.393   H^2: 5.728   I^2: 82.543
    #>
    #> Outlier-robust method (absolute deviation from mean):
    #>   sd_res  (tau_r):   0.121
    #>   var_res (tau_r^2): 0.015
    #>
    #>   Q_r statistic: 78.734
    #>   H_r: 2.498   H_r^2: 6.242   I_r^2: 0.840
    #>
    #> Outlier-robust method (absolute deviation from median):
    #>   sd_res  (tau_m):   0.118
    #>   var_res (tau_m^2): 0.014
    #>
    #>   Q_m statistic: 77.813
    #>   H_m: 2.438   H_m^2: 5.944   I_m^2: 0.832

### Publication Bias and Sensitivity Analyses

psychmeta supports cumulative meta-analysis for publication/small-sample bias detection, leave-1-out sensitivity analyses, and bootstrap confidence intervals using the sensitivity function:

    ma_res <- sensitivity(ma_res)
    #>  **** Computing sensitivity analyses ****
    #> Bootstrapped meta-analyses have been added to 'ma_obj' - use get_bootstrap() to retrieve them.
    #> leave-1-out meta-analyses have been added to 'ma_obj' - use get_leave1out() to retrieve them.
    #> Cumulative meta-analyses have been added to 'ma_obj' - use get_cumulative() to retrieve them.
    get_cumulative(ma_res)[[1]][["barebones"]]
    #> Cumulative meta-analysis results
    #> ----------------------------------------
    #>    study_added  k     N mean_r    var_r   var_e   var_res   sd_r   se_r   sd_e sd_res CI_LL_95 CI_UL_95 CR_LL_80 CR_UL_80
    #> 1           28  1   487  0.300       NA 0.00170        NA     NA 0.0413 0.0413     NA   0.2194    0.381       NA       NA
    #> 2            3  2   966  0.320 0.000802 0.00167 -0.000869 0.0283 0.0200 0.0409 0.0000   0.0658    0.575    0.320    0.320
    #> 3           31  3  1407  0.270 0.008675 0.00184  0.006838 0.0931 0.0538 0.0429 0.0827   0.0387    0.501    0.114    0.426
    #> 4           33  4  1841  0.268 0.005916 0.00188  0.004039 0.0769 0.0385 0.0433 0.0636   0.1453    0.390    0.164    0.372
    #> 5           14  5  2273  0.308 0.013208 0.00181  0.011402 0.1149 0.0514 0.0425 0.1068   0.1655    0.451    0.144    0.472
    #> 6            1  6  2689  0.336 0.015939 0.00176  0.014180 0.1263 0.0515 0.0419 0.1191   0.2039    0.469    0.161    0.512
    #> 7           17  7  3086  0.342 0.013757 0.00177  0.011985 0.1173 0.0443 0.0421 0.1095   0.2336    0.451    0.184    0.500
    #> 8           38  8  3461  0.338 0.012208 0.00182  0.010389 0.1105 0.0391 0.0426 0.1019   0.2452    0.430    0.193    0.482
    #> 9            6  9  3818  0.338 0.010894 0.00185  0.009040 0.1044 0.0348 0.0431 0.0951   0.2576    0.418    0.205    0.471
    #> 10          32 10  4165  0.326 0.011469 0.00192  0.009547 0.1071 0.0339 0.0438 0.0977   0.2498    0.403    0.191    0.462
    #> 11          27 11  4510  0.331 0.010806 0.00194  0.008868 0.1040 0.0313 0.0440 0.0942   0.2614    0.401    0.202    0.460
    #> 12          37 12  4855  0.317 0.012867 0.00200  0.010862 0.1134 0.0327 0.0448 0.1042   0.2449    0.389    0.175    0.459
    #> 13          21 13  5197  0.317 0.011939 0.00203  0.009909 0.1093 0.0303 0.0451 0.0995   0.2505    0.383    0.182    0.452
    #> 14          20 14  5536  0.321 0.011481 0.00204  0.009442 0.1072 0.0286 0.0452 0.0972   0.2592    0.383    0.190    0.452
    #> 15          24 15  5872  0.324 0.010973 0.00205  0.008923 0.1048 0.0270 0.0453 0.0945   0.2665    0.383    0.197    0.452
    #> 16          30 16  6195  0.309 0.015242 0.00212  0.013122 0.1235 0.0309 0.0460 0.1146   0.2428    0.374    0.155    0.462
    #> 17          36 17  6515  0.301 0.015529 0.00216  0.013366 0.1246 0.0302 0.0465 0.1156   0.2373    0.365    0.147    0.456
    #> 18          23 18  6828  0.306 0.015222 0.00217  0.013051 0.1234 0.0291 0.0466 0.1142   0.2445    0.367    0.154    0.458
    #> 19          22 19  7138  0.300 0.015323 0.00221  0.013113 0.1238 0.0284 0.0470 0.1145   0.2403    0.360    0.148    0.452
    #> 20          18 20  7441  0.308 0.016155 0.00221  0.013947 0.1271 0.0284 0.0470 0.1181   0.2483    0.367    0.151    0.465
    #> 21          11 21  7739  0.313 0.016082 0.00222  0.013867 0.1268 0.0277 0.0471 0.1178   0.2548    0.370    0.156    0.469
    #> 22          19 22  8030  0.316 0.015749 0.00223  0.013522 0.1255 0.0268 0.0472 0.1163   0.2601    0.371    0.162    0.470
    #> 23          12 23  8315  0.319 0.015428 0.00224  0.013188 0.1242 0.0259 0.0473 0.1148   0.2649    0.372    0.167    0.470
    #> 24          40 24  8586  0.315 0.015286 0.00227  0.013012 0.1236 0.0252 0.0477 0.1141   0.2630    0.367    0.165    0.466
    #> 25          34 25  8854  0.313 0.014968 0.00230  0.012664 0.1223 0.0245 0.0480 0.1125   0.2624    0.363    0.165    0.461
    #> 26          16 26  9110  0.318 0.015639 0.00231  0.013328 0.1251 0.0245 0.0481 0.1154   0.2680    0.369    0.167    0.470
    #> 27           5 27  9365  0.320 0.015334 0.00233  0.013006 0.1238 0.0238 0.0483 0.1140   0.2715    0.369    0.171    0.470
    #> 28           9 28  9619  0.318 0.015125 0.00236  0.012766 0.1230 0.0232 0.0486 0.1130   0.2704    0.366    0.170    0.467
    #> 29          35 29  9869  0.318 0.014723 0.00238  0.012342 0.1213 0.0225 0.0488 0.1111   0.2720    0.364    0.172    0.464
    #> 30          39 30 10116  0.320 0.014558 0.00240  0.012163 0.1207 0.0220 0.0489 0.1103   0.2754    0.365    0.176    0.465
    #> 31          10 31 10360  0.322 0.014260 0.00241  0.011847 0.1194 0.0214 0.0491 0.1088   0.2778    0.365    0.179    0.464
    #> 32           2 32 10601  0.327 0.015026 0.00242  0.012610 0.1226 0.0217 0.0492 0.1123   0.2824    0.371    0.180    0.474
    #> 33          13 33 10838  0.328 0.014805 0.00243  0.012373 0.1217 0.0212 0.0493 0.1112   0.2851    0.371    0.183    0.474
    #> 34          25 34 11075  0.323 0.015912 0.00247  0.013440 0.1261 0.0216 0.0497 0.1159   0.2787    0.367    0.171    0.474
    #> 35           8 35 11288  0.321 0.015838 0.00250  0.013334 0.1258 0.0213 0.0500 0.1155   0.2773    0.364    0.170    0.471
    #> 36          29 36 11471  0.319 0.015734 0.00254  0.013193 0.1254 0.0209 0.0504 0.1149   0.2765    0.361    0.169    0.469
    #> 37           4 37 11638  0.320 0.015536 0.00257  0.012964 0.1246 0.0205 0.0507 0.1139   0.2782    0.361    0.171    0.468
    #> 38          15 38 11786  0.318 0.015541 0.00261  0.012928 0.1247 0.0202 0.0511 0.1137   0.2771    0.359    0.170    0.466
    #> 39          26 39 11916  0.317 0.015602 0.00266  0.012942 0.1249 0.0200 0.0516 0.1138   0.2760    0.357    0.168    0.465
    #> 40           7 40 11927  0.317 0.015607 0.00273  0.012876 0.1249 0.0198 0.0523 0.1135   0.2767    0.357    0.169    0.465
    #>
    #> See the 'plots' list for data visualizations.
    get_cumulative(ma_res)[[1]][["barebones"]][["plots"]]
    #> $mean_plot
    #> Warning: Removed 1 rows containing missing values (`geom_segment()`).
    #> Warning: Removed 1 rows containing missing values (`geom_point()`).
    #> Removed 1 rows containing missing values (`geom_point()`).
    #>
    #> $sd_plot
    #> Warning: Removed 1 rows containing missing values (`geom_point()`).
    #> Warning: Removed 1 row containing missing values (`geom_line()`).

    get_bootstrap(ma_res)[[1]][["barebones"]]
    #> Bootstrapped meta-analysis results
    #> ----------------------------------------
    #>          boot_mean boot_var CI_LL_95 CI_UL_95
    #> k         4.00e+01 0.00e+00 4.00e+01 4.00e+01
    #> N         1.19e+04 3.23e+05 1.08e+04 1.30e+04
    #> mean_r    3.17e-01 3.95e-04 2.78e-01 3.55e-01
    #> var_r     1.51e-02 9.26e-06 1.10e-02 2.38e-02
    #> var_e     2.73e-03 2.52e-08 2.47e-03 3.12e-03
    #> var_res   1.24e-02 9.05e-06 8.35e-03 2.13e-02
    #> sd_r      1.22e-01 1.55e-04 1.05e-01 1.54e-01
    #> se_r      1.93e-02 3.86e-06 1.66e-02 2.44e-02
    #> sd_e      5.22e-02 2.29e-06 4.97e-02 5.58e-02
    #> sd_res    1.10e-01 1.86e-04 9.13e-02 1.46e-01
    #> CI_LL_95  2.78e-01 4.59e-04 2.34e-01 3.17e-01
    #> CI_UL_95  3.56e-01 3.62e-04 3.20e-01 3.93e-01
    #> CR_LL_80  1.73e-01 9.11e-04 1.05e-01 2.23e-01
    #> CR_UL_80  4.61e-01 5.11e-04 4.25e-01 5.14e-01
    #>
    #> See list item 'boot_data' for meta-analysis results from each bootstrap iteration