Title:	Optimal Confidence Intervals for Visual Testing
Version:	0.7
Description:	Identifies the optimal confidence level to represent the results of a set of pairwise tests as suggested by Armstrong and Poirier (2025) <doi:10.1017/pan.2024.24>.
Depends:	R (≥ 4.1.0)
Imports:	dplyr, emmeans, ggplot2, ggtext, HDInterval, multcomp, tidyr
Suggests:	carData, collapse, factorplot (≥ 1.3), forcats, ggnewscale, ggsignif, knitr, lme4, marginaleffects, multcompView, mvtnorm, patchwork, psre, rmarkdown, sandwich, testthat (≥ 3.0.0), wooldridge
VignetteBuilder:	knitr
License:	MIT + file LICENSE
Encoding:	UTF-8
RoxygenNote:	7.3.2
Config/testthat/edition:	3
NeedsCompilation:	no
Packaged:	2026-02-22 00:37:53 UTC; david
Author:	Dave Armstrong [aut, cre], William Poirier [aut]
Maintainer:	Dave Armstrong <davearmstrong.ps@gmail.com>
Repository:	CRAN
Date/Publication:	2026-03-03 10:00:31 UTC

Calculate z-score for Confidence Interval Overlap

Description

Calculates the z-score required such that confidence intervals do not overlap under the null hypothesis withe a specified probability.

Usage

gen_z(b, v, alpha = 0.05, df = Inf, ...)

Arguments

Value

A list with two elements: ave_z: A data frame with one row for each estimate in b and the following variables:

• vij: observation number

• s_zb: standard deviation of the z-scores across all pairs of intervals containing that estimate.

• min_zb, max_zb: The minimum and maximum z-scores for the pairs of intervals containing that estimate.

• zb: The mean z-score for the pairs of intervals containing that estimate.

• ci: The confidence level corresponding to zb. all_z: A data frame with one row for each pair of estimates in b and the following variables:

• i, j: The indices of the two estimates in the pair.

• s_i, s_j: The standard errors of the two estimates in the pair.

• theta: The ratio of the standard errors of the two estimates.

• rho: The correlation between the two estimates.

• zb: The z-score for the pair of estimates.

• ci : The confidence level corresponding to zb.

• olap_ave The probability that the two intervals do not overlap under the null hypothesis.

• olap_84 The probability that two 84% confidence intervals for the estimates in the pair would not overlap under the null hypothesis.

References

Harvey Goldstein and Michael J.R. Healy. (1995) "The Graphical Presentation of A Collection of Means." Journal of the Royal Statistical Society, Series A 158(1): 175-177 doi:10.2307/2983411. David Afshartous and Richard A. Preston. (2010) "Confidence Intervals for Dependent Data: Equating Non-overlap with Statistical Significance." Computational Statistics and Data Analysis 54: 2296-2305 doi:10.1016/j.csda.2010.04.011

Examples

data(mtcars)
mod <- lm(mpg ~ wt + hp + disp + vs, data=mtcars)
gen_z(coef(mod), vcov(mod))

Forest plot points with precision-weighted squares and summary diamonds

Description

geom_forestpoint() draws the central markers in a forest plot: non-summary rows are rendered as squares whose area is controlled by the ggplot2 size aesthetic (typically proportional to precision), while summary rows are rendered as diamonds whose width reflects the confidence interval and whose height is derived from that width and bounded by row spacing.

Usage

geom_forestpoint(
  mapping = NULL,
  data = NULL,
  stat = "identity",
  position = "identity",
  ...,
  diamond_aspect = 1,
  diamond_row_frac = 0.4,
  diamond_min_frac = 0.06,
  na.rm = FALSE,
  show.legend = NA,
  inherit.aes = TRUE
)

Arguments

Details

This geom is designed to work with standard ggplot2 scales (e.g., scale_size_area()) and pairs naturally with geom_linerange() for confidence intervals and geom_foreststripe() for background striping.

For non-summary rows, the square side length is derived from the ggplot2 size aesthetic (in mm units), so users can control point sizing using standard size scales such as ggplot2::scale_size_area().

For summary rows, the diamond width is determined by the supplied confidence interval (xmin/xmax), and the height is computed as a bounded function of that width. The diamond height does not use the size aesthetic.

Value

A ggplot2 layer object that can be added to a plot.

Draw alternating row stripes for forest plots and forest tables

Description

geom_foreststripe() adds alternating horizontal background bands (“zebra striping”) to forest plots or forest tables. The stripes are drawn using the y-axis scale (rather than the data itself), which makes the geom robust to missing rows, summary rows, and faceting.

Usage

geom_foreststripe(
  mapping = NULL,
  data = NULL,
  position = "identity",
  ...,
  n_cols,
  col_gap = 1,
  start = 2L,
  fill = "grey92",
  colour = NA,
  na.rm = FALSE,
  show.legend = FALSE,
  inherit.aes = TRUE
)

Arguments

Details

This geom is designed to work with both the forest figure panel (estimates and confidence intervals) and the forest table panel produced by geom_foresttable(). Because stripes are computed from the panel scales, the two panels remain visually aligned when composed with patchwork.

The stripes are computed from the panel’s y range rather than the data rows. For discrete y scales, this corresponds to the integer row positions used by ggplot2. Every other row is selected starting at start.

Horizontal extents are derived from the panel x range, allowing the stripes to span both table columns and forest-plot panels without requiring explicit xmin/xmax aesthetics.

Value

A ggplot2 layer object that can be added to a plot.

See Also

geom_foresttable(), geom_forestpoint(), gg_forest()

Render a forest-plot table panel as a ggplot2 layer

Description

geom_foresttable() builds a “table” panel for forest plots by expanding the input data to long form (one row per forest row × table column) and drawing formatted cell text at fixed x positions. Column headers are typically added with scale_x_foresttable() (top axis tick labels), which makes the table align cleanly with a forest plot when composing with patchwork.

Usage

geom_foresttable(
  mapping = NULL,
  data = NULL,
  position = "identity",
  ...,
  cols,
  fmt = NULL,
  fmt_default = as.character,
  col_gap = 1,
  col_align = "left",
  col_nudge = 0,
  na.rm = FALSE,
  show.legend = FALSE,
  inherit.aes = TRUE
)

Arguments

Details

The cols argument controls which columns are printed. Formatting can be customized via fmt, a named list of functions (one per column) that convert cell values to character strings.

geom_foresttable() uses ggplot2's feature where the data argument may be a function: when data = NULL, this layer supplies a data function that receives the plot data and expands it to long form while preserving all original columns (including faceting variables and the y mapping). This makes the geom compatible with faceting and grouping while keeping the user-facing API simple.

Column positions are computed as seq_along(cols) * col_gap + col_nudge. Horizontal justification is determined by col_align (left/center/right).

Value

A ggplot2 layer object that can be added to a plot.

See Also

scale_x_foresttable(), gg_forest()

Get Letters for Multiple Comparisons

Description

Gets the letter matrix for a compact letter display. This can be passed to the letter_plot() function from the psre package to produce plots of confidence intervals with a letter display.

Usage

get_letters(x = NULL, ...)

Arguments

Value

A logical index indicating which estimates are in which letter group.

Extract representative difficulty levels from a VizTest result

Description

get_viztest_levels() selects one or more representative stimulus levels from a VizTest result object based on empirical difficulty and agreement. Levels are chosen from rows with the maximum agreement probability (psame) and can be returned individually (e.g., lowest, highest) or collectively.

Usage

get_viztest_levels(
  x,
  method = c("easiest", "all", "lowest", "highest", "middle"),
  ...
)

Arguments

Details

The function first subsets x$tab to rows achieving the maximum value of psame (ignoring missing values). From this subset:

• "lowest" returns the smallest level.

• "highest" returns the largest level.

• "middle" returns the median level.

• "easiest" returns the level with the maximum easy value.

• "all" returns a named vector containing all four selections.

Value

A numeric value when method is one of "lowest", "highest", "middle", or "easiest". When method = "all", a named numeric vector with elements lowest, highest, middle, and easiest.

See Also

VizTest

Examples

data(mtcars)
mtcars$cyl <- as.factor(mtcars$cyl)
mtcars$hp <- scale(mtcars$hp)
mtcars$wt <- scale(mtcars$wt)
mod <- lm(qsec ~ hp + wt + cyl, data=mtcars)
v <- viztest(mod)
v

get_viztest_levels(v, "easiest")

Build a paired forest plot + companion table (patchwork-ready)

Description

gg_forest() creates two aligned ggplot objects: (1) a forest plot with confidence intervals and weighted points (including summary diamonds), and (2) a “table” rendered as text in a ggplot panel with column headers on the top axis. The returned objects share the same y values so they can be combined with patchwork. This uses lower level functions in the package like geom_forestpooint, geom_foreststripe, geom_foresttable and scale_x_foresttable that could be used to customize the look of the forest plot further.

Usage

gg_forest(
  data,
  y,
  x,
  data_cols,
  xmin_std,
  xmax_std,
  size_prop = 1,
  max_size = 15,
  is_summary = FALSE,
  xmin_ici = NULL,
  xmax_ici = NULL,
  vline = NULL,
  ci_colors = c(ici = "gray65", std = "black"),
  use_log_scale = FALSE,
  stripe_table = TRUE,
  stripe_figure = TRUE,
  start_stripe = 3,
  col_nudge = 0,
  table_format_list = NULL,
  col_align = NULL,
  table_header_size = 16,
  table_text_size = 4,
  fig_xlab = "Estimate",
  ...
)

Arguments

Details

Column arguments are provided as strings and are evaluated safely using .data[[...]].

Value

An object of class "gg_forest": a list with two ggplot objects (forest, table) suitable for composition with patchwork.

See Also

geom_forestpoint(), geom_foresttable(), geom_foreststripe(), scale_x_foresttable()

Examples

# Load Packages
library(emmeans)
library(VizTest)
library(dplyr)

# Use built-in Esophageal Cancer Data
data(esoph)

# Aggregate data by age group
ag_data <- aggregate(esoph[,c("ncases", "ncontrols")], list(age = esoph$age), sum)

# Turn counts into integerss (not required, but makes printing nicer)
ag_data$ncases <- as.integer(ag_data$ncases)
ag_data$ncontrols <- as.integer(ag_data$ncontrols)

# Make age into unordered factor
ag_data$age <- factor(as.character(ag_data$age), 
                      levels=levels(esoph$age))
                      
# Estimate model of prevalence by age and overall (the summary model)                       
model1 <- glm(cbind(ncases, ncontrols) ~ age,
              data = ag_data, family = binomial())
model_sum <- glm(cbind(ncases, ncontrols) ~ 1,
                 data = ag_data, family = binomial())

# Make data frame of results for plotting using emmeans
fit <- emmeans(model1, "age")
fit_ci <- confint(fit)

# add in original count data
ag_data <- cbind(fit, ag_data[,c("ncases", "ncontrols")])

# turn coefficients and confidence intervals into odds ratio scale 
ag_data$or <- exp(ag_data$emmean)
ag_data$lower <- exp(ag_data$asymp.LCL)
ag_data$upper <- exp(ag_data$asymp.UCL)
# Make summary data frame that we can use for plotting                  
fit_sum <- data.frame(age= "Summary", emmean = coef(model_sum), 
  SE = unname(sqrt(vcov(model_sum))), or = exp(coef(model_sum)))
sum_ci <- confint(model_sum)
fit_sum$lower <- exp(sum_ci[1])
fit_sum$upper <- exp(sum_ci[2])
fit_sum$ncases <- sum(ag_data$ncases)
fit_sum$ncontrols <- sum(ag_data$ncontrols)
rownames(fit_sum) <- NULL

# Find the optimal visual testing intervals
viztest(fit, include_zero=FALSE, make_plot=FALSE, test_level = .05)

# Add inferential CIs to data (not for summary, though)
fit_ici <- confint(fit, level = .75)
ag_data$lower_ici <- exp(fit_ici$asymp.LCL)
ag_data$upper_ici <- exp(fit_ici$asymp.UCL)

# bind together the age-specific and summary data frames for plotting
ag_data <- dplyr::bind_rows(ag_data, fit_sum)

# identify summary row
ag_data$is_sum <- ag_data$age == "Summary"

# add point-size weight
ag_data$pt_size <- 1/ag_data$SE^2

# make age_label such that ages plot smallest at top and summary at bottom
ag_data$age_label <- factor(ag_data$age, levels=rev(ag_data$age))

# Make gg forest plot
out <- gg_forest(ag_data, 
    y = "age_label", 
    x = "or", 
    xmin_std = "lower", 
    xmax_std = "upper", 
    xmin_ici = "lower_ici", 
    xmax_ici = "upper_ici",
    size_prop = "pt_size", 
    is_summary = "is_sum", 
    use_log_scale = TRUE, 
    data_cols = c("Age" = "age_label", 
                   "Controls" = "ncontrols", 
                   "Cases"="ncases", 
                   "OR" = "or"), 
    max_size=5,
    table_header_size = 16, 
    table_text_size = 5, 
    col_nudge=c(-.085, 0,0,0), 
    diamond_aspect=15, 
    diamond_row_frac = .9)
   
# print plot
plot(out, widths=1, 1)

Make Annotations for Significance Brackets

Description

Makes a list of annotations for significance brakcets produced by the geom_signif() function from the ggsignif package. The annotations are added for pairs of estimates whose confidence intervals overlap, but the estimates are nonetheless significantly different from each other.

Usage

make_annotations(
  obj,
  type = c("auto", "significant", "insignificant", "discrepancies"),
  tol = 0,
  nudge = NULL,
  ...
)

Arguments

Examples

data(chickwts)
chick_mod <- lm(weight ~ feed, data=chickwts)
library(marginaleffects)
chick_preds <- avg_predictions(chick_mod, variables="feed")
b <- coef(chick_preds)
names(b) <- chick_preds$feed
v <- vcov(chick_preds)
chick_vt_data <- make_vt_data(b, v)
chick_vt <- viztest(chick_vt_data, test_level = 0.0001, include_zero=FALSE)
chick_vt

make_annotations(chick_vt, type="discrepancies")

Make Template for Pairwise Significance Input

Description

Provides a template for producing a binary vector indicating whether each pair of estimates has a significant difference.

Usage

make_diff_template(
  estimates,
  include_zero = TRUE,
  include_intercept = FALSE,
  ...
)

Arguments

Details

The viztest() function uses a normal difference of means test to identify whether there is a significant difference or not. While this test could be done with adjustments for multiplicity or robust standard errors of all different kinds, there may be times when the user would prefer to identify the significant differences manually. The viztest() function internally reorders the estimates from largest to smallest so this function does that and then prints the pairs that will correspond with the visual testing grid search being done by viztest().

Please note that the include_zero and include_intercept arguments should be set the same here as they are in your call to viztest(). If they are not, viztest() will stop because the results from the comparison of confidence intervals will have different dimensions than the differences that are manually provides.

Value

A two-column data frame containing the names of the larger and smaller parameters in the appropriate order. This can be used to identify the appropriate order in which to specify the sig_diffs argument to viztest().

Examples

make_diff_template(estimates = c(e1 = 2, e2 = 1, e3 = 3))

Make custom visual testing data

Description

Makes custom visual testing objects that can be used as input to the viztest() function. This is useful in the case where coef() and vcov() do not function as expected on objects of interest, where the user wants to intervene with some modification to the usual estimates or (more likely) variance-covariance matrix or where normal theory tests may not be as useful (e.g., in the case of simulations of non-normal values). The examples section below shows how this could be leveraged to use a heteroskedasticity-consistent covariance matrix in the test rather than the one returned by lm().

Usage

make_vt_data(
  estimates,
  variances = NULL,
  type = c("est_var", "sim"),
  tol = 1e-08,
  ...
)

Arguments

Value

1. If the input is a vector of parameter estimates and a variance-covariance matrix, then a list with estimates and a variance-covariance matrix of class "vtcustom" is returned. In this case, the functionms coef.vtcustom() and vcov.vtcustom() are used to extract the coefficients and variance-covariance matrix in a way that will work with viztest.default().

2. If the input is a matrix of simulation draws, an object of class "vtsim" that has a single element - the data giving the draws from the simulation is returned. In this case, viztest.vtsim() does the relevant testing.

Examples

data(mtcars)
mtcars$cyl <- as.factor(mtcars$cyl)
mtcars$hp <- scale(mtcars$hp)
mtcars$wt <- scale(mtcars$wt)
mod <- lm(qsec ~ hp + wt + cyl, data=mtcars)
V <- sandwich::vcovHC(mod, "HC3")
vtdat <- make_vt_data(coef(mod), V)
viztest(vtdat, 
        test_level = .025, 
        include_intercept = FALSE, 
        include_zero = FALSE)

Plotting method for gg_forest objects.

Description

Uses patchwork to combine the forest plot and table components.

Usage

## S3 method for class 'gg_forest'
plot(x, ..., widths = c(1.4, 1))

Arguments

Value

A ggplot object combining the forest plot and table components.

Plot Method for viztest Objects

Description

Plots the output of viztest objects with optional reference lines

Usage

## S3 method for class 'viztest'
plot(
  x,
  ...,
  add_test_level = TRUE,
  ref_lines = "none",
  viz_diff_thresh = 0.02,
  make_plot = TRUE,
  level = c("ce", "max", "min", "median"),
  trans = I,
  est_point_args = list(color = "black", size = 2),
  opt_ci_args = list(),
  test_ci_args = list(),
  ref_line_args = list(color = "gray75", linetype = 3),
  scale_linewidth_args = list(values = c(3.5, 0.5)),
  scale_color_args = list(values = c("gray75", "black")),
  overall_theme = theme_bw,
  theme_arg = list(legend.position = "top", plot.caption = element_textbox_simple(width =
    unit(1, "npc"), halign = 0, margin = margin(1, 0, 0, 0, "lines"))),
  remove_caption = FALSE
)

Arguments

Details

The ref_lines argument identifies what reference lines will be plotted in the figure. For any particular stimulus, the reference lines run along the upper bound of the stimulus from the stimulus location to the most distant stimulus with overlapping confidence intervals. When ref_lines = "all", all lines are plotted, though in displays with many stimuli, this can make for a messy graph. When "ref_lines = ambiguous" is specified, then only the ones that help discriminate in cases where the result might be visually difficult to discern are plotted. A comparison is determined to be visually difficult if the upper bound of the stimulus in question is within viz_diff_thresh times the difference between the smallest lower bound and the largest upper bound. If ref_lines = "non", then none of the reference lines are plotted. Alternatively, you can specify the names of stimuli whose reference lines will be plotted. These should be the same as the names in the data. The viztest() function returns an object est, which contains the data that are used as input to this function. The variable vbl in The est data frame contains the stimulus names.

Value

By default, a ggplot is returned. If make_plot = FALSE, the data for the plot are returned, but the plot is not constructed. If the data are returned, the following variables are in the dataset:

• vbl - The name of the parameter.

• est - The parameter estimate

• se - The standard error of the estimate

• lwr, upr - The inferential confidence bounds being used

• lwr_add, upr_add - The confidence intervals that come from add_level.

• label - Factor giving the parameter names

• stim_start, stim_end - y-axis bounds of the reference line

• bound_start, bound_end - x-axis values for reference lines

• ambiguous - Logical vector indicating whether the comparison is considered "ambiguous".

Examples

data(mtcars)
mod2 <- lm(mpg ~ as.factor(cyl) + vs + am + as.factor(gear), data = mtcars)
v <- viztest(mod2)
plot(v, ref_lines="ambiguous") + ggplot2::theme_classic()

Print Method for viztest Objects

Description

Prints a summary of the results from the viztest() function.

Usage

## S3 method for class 'viztest'
print(x, ..., best = TRUE, missed_tests = TRUE, level = NULL)

Arguments

Details

The results are printed in such a way that the range of optional levels is produced including the range along with two candidates for the best levels to use - middle and easiest.

Prints the results from the viztest function

Value

Printed results that give the level(s) that correspond most closely with the pairwise test results. The values returned are the smallest, largest, middle and easiest. By default this function also reports the tests that are not captured by the (non-)overlaps in confidence intervals when each different level is used.

Reorder a factor for a forest plot

Description

Orders levels of x by bvar (via fn), with "Summary" optionally pinned to the bottom (or top).

Usage

reorder_forest(
  x,
  bvar,
  fn = mean,
  descending = TRUE,
  summary_bottom = TRUE,
  ...
)

Arguments

X scale for forest tables with column headers on the top axis

Description

scale_x_foresttable() constructs a continuous x scale designed for use with geom_foresttable(). It places table column headers on the top axis tick labels, sets appropriate breaks and limits based on the requested columns, and ensures consistent spacing so that table panels align cleanly with forest plots when composed using patchwork.

Usage

scale_x_foresttable(
  cols,
  col_labels = NULL,
  col_gap = 1,
  position = "top",
  expand = ggplot2::expansion(mult = c(0, 0)),
  limits = NULL,
  ...
)

Arguments

Details

The scale uses numeric x positions internally (one per column), which allows precise control over column spacing and alignment.

This scale is typically paired with geom_foresttable() and a void theme (theme_void()) to create a table-like panel whose headers are rendered as axis labels. Because headers are true axis labels, ggplot2 can align the table panel perfectly with a forest plot panel when combining plots.

Value

A ggplot2 scale object suitable for addition to a plot.

See Also

geom_foresttable(), gg_forest()

Calculate Correspondence Between Pairwise Test and CI Overlaps

Description

viztest() does a grid search over range_levels to find the confidence level(s) such that the (non-)overlaps in confidence intervals corresponds as closely as possible with the results of pairwise tests. To the extent that a level is found that accounts for all pairwise tests, confidence bounds at this level can be added to coefficient or marginal effects plots to enable readers to reliably identify estimates that are statistically different from each other.

Usage

viztest(
  obj,
  test_level = 0.05,
  range_levels = c(0.25, 0.99),
  level_increment = 0.01,
  adjust = c("none", "holm", "hochberg", "hommel", "bonferroni", "BH", "BY", "fdr"),
  cifun = c("quantile", "hdi"),
  include_intercept = FALSE,
  include_zero = TRUE,
  sig_diffs = NULL,
  tol = 1e-08,
  ...
)

Arguments

Details

The algorithm first calculates results of a set of pairwise tests. For objects with estimates and a variance-covariance matrix, normal theory tests are calculated. Optionally, these tests can be subjected to a multiplicity adjustment. In the case of simulation results, something akin to p-values are calculated by identifying the probability that one estimate is larger than another. To mimic the way we use p-values in the frequentist case, we subtract the probability of difference from 1, such that smaller values indicate more confidence in the difference. The algorithm then performs a grid search over range_levels at increments of level_increment. For each candidate level, the confidence intervals for all parameters are calculated. For each pair of estimates, it identifies whether the confidence intervals (or credible intervals if the input is a matrix of Bayesian simulation draws) overlaps. For each candidate level, it calculates the proportion of times where differences are significant/credible and confidence/credible intervals do not overlap or differences are not significant/credible and the intervals do overlap. The main idea is to find the level(s) such that the (non-)overlaps perfectly correspond with whether the differences are significant.

If such a level can be found, a visual inspection of confidence or credible intervals at that level will identify whether a pair of estimates is statistically different or not.

While most of the parameters are straightforward, the sig_diffs argument must be specified such that the stimuli are in order from highest to lowest. This is most easily done by using make_diff_template() to identify the appropriate order of the comparisons.

Value

A list (of class "viztest") with the following elements:

1. tab: a data frame with results from the grid search. The data frame has four variables: level - is the confidence level used in the grid search; psame - the proportion of (non-)overlaps that match the normal theory tests; pdiff - the proportion of pairwise tests that are statistically significant; easy - the ease with which the comparisons are made.

2. pw_tests: A logical vector indicating which tests are significantly significant.

3. ci_tests: A logical vector indicating whether the confidence intervals are disjoint (TRUE) or overlap (FALSE).

4. combs: The pairwise combinations of stimuli used in the test. Note, the stimuli are reordered from largest to smallest, so the numbers do not represent the position in the original ordering.

5. param_names: A vector of the names of the parameters reordered by size - largest to smallest.

6. L: The lower confidence bounds from the grid search.

7. U: The upper confidence bounds from the grid search.

8. est: A data frame with the variables vbl - the parameter name; est - the parameter estimate; se - the parameter standard error.

9. call: model call

References

David A. Armstrong II and William Poirier. "Decoupling Visualization and Testing when Presenting Confidence Intervals" Political Analysis doi:10.1017/pan.2024.24.

Examples

data(mtcars)
mtcars$cyl <- as.factor(mtcars$cyl)
mtcars$hp <- scale(mtcars$hp)
mtcars$wt <- scale(mtcars$wt)
mod <- lm(qsec ~ hp + wt + cyl, data=mtcars)
viztest(mod)