This is a major release that modernizes the user-facing API, refines the internal computation, and adds planning options. It is the first CRAN release since 1.2.1.
The bias and uncertainty correction and the sample size computations
follow the same method as 1.2.1. The new options default to what 1.2.1
assumed: the regression planners’ random_predictors
defaults to FALSE (predictors fixed by design) and the
within-subjects and split-plot planners’ epsilon defaults
to 1 (sphericity), so a plan that uses neither is computed in the frame
1.2.1 used. For every documented example that 1.2.1 can also compute,
the rounded (three-decimal) adjusted noncentrality parameter matches the
1.2.1 value, except in two examples that set
alpha_prior = 1. The main numerical change is the engine
that finds the noncentrality parameter (see “Engine” below), which
shifts some planned sample sizes, most by a few units, and makes them
more accurate. Three smaller deliberate changes also move plans. An
alpha_prior of 1 is computed as exactly 1, where 1.x
computed with .999 in its place, which leaves most such plans unchanged
and makes the rest slightly smaller (see “No Publication Filter” below).
And, described under “Bug Fixes”, the general between-subjects
function’s df_denominator argument, silently ignored in
1.x, now functions, and the independent t planner rounds the
harmonic mean of two prior group sizes in integer arithmetic, which
changes the plan (to a smaller, correct one) for pairs whose harmonic
mean is a whole number. One request that 1.x answered now stops: a
desired_power at or below alpha_planned, for
which 1.x returned the smallest sample size it considers, is not
allowable (also under “Bug Fixes”). Every plan also gains one row, its
planning effect size (see “Sampled Predictors, Sphericity, and the
Planning Effect Size” below).
stats::uniroot(), replacing the search over the fixed grid
seq(0, 100, by = step) used in 1.x. Two consequences:
alpha_prior = 1, described in the next section), but the
extra precision shifts some planned sample sizes. For the documented
unequal-n replication example
(ss_buc_independent_t(t_observed = 3, n = c(50, 55), assurance = .90)),
the necessary sample size moves from 1482 (the value BUCSS 1.x reported
under its grid search) to 1485, which is the smallest sample size that
actually reaches the target power. Across the developer characterization
grid, leaving out the cases the old cap affected, the
df_denominator change, and the harmonic-mean correction,
about 94% of sample sizes are unchanged (the plans with
alpha_prior = 1 included); most of the rest move by a few
units and none by more than 2 percent. The exception lies close to the
assurance ceiling, beyond all but one case in that grid, where the
corrected parameter is small beside the 1.x search’s resolution of .001:
within about 25 times that resolution on the F scale (50 times
on the t scale) a 1.x sample size can differ from the 2.0.0 one
by more than 2 percent, and within about .001 of the ceiling by 10
percent or more, in either direction. Among the documented examples,
eight planned sample sizes move: 1482 to 1485
(ss_buc_independent_t); 1904 to 1902
(ss_buc_rm_anova); 968 to 969
(ss_buc_mixed_anova); 1491 to 1489
(ss_buc_mixed_anova_general); 3963 to 3960
(ss_buc_reg_joint); and, in the tables of the vignette
“Assurance and the Publication Bias Correction”, 5069 to 5042
(ss_buc_independent_t(t_observed = 3, n = 20, assurance = .90))
and, from ss_buc_paired_t() with
alpha_prior = 1 and assurance = .80, 377 to
376 and 915 to 910 (for a prior t from 30 pairs with a
two-sided p value of .10 and of .15; in these two the rounded
parameter moves as well, from .793 to .794 and from .508 to .509). Each
new value is the smallest sample size that actually attains the target
power at the exact noncentrality parameter; they are pinned in the
package’s regression tests.step argument and any grid resolution setting are gone,
since root finding has no resolution parameter.alpha_prior of 1 Is Exactly 1An alpha_prior of 1 is computed as exactly 1. The
method’s article describes the setting: “If publication bias is thought
not to apply, [alpha_P] can be set equal to 1” (Anderson, Kelley, &
Maxwell, 2017, p. 1552, where alpha_P is the level
alpha_prior names). With no publication filter every result
would have been published, so the publication critical value is 0, the
power term that truncates the likelihood is 1, the prior statistic has
its ordinary distribution, and the largest supportable assurance is
exactly 1 - p. BUCSS 1.x computed with .999 in place of 1, that
is, with a filter that drops results whose p value is above
.999. No value is substituted for 1 any longer.
How far plans move. Computing with exactly 1 rather than .999
raises the adjusted noncentrality parameter a little, so a plan with
alpha_prior = 1 is the same size or smaller, never larger.
Across the 2,172 such plans of the developer characterization grid (all
eighteen planners), computed both ways, the parameter rises by a median
of about 0.002 percent and at most 1 percent; 95 percent of the planned
sample sizes are unchanged, and the other 103 are smaller, most by one
participant (79 of them), the largest moves being about 1 percent of a
very large plan (36,146 to 35,787) and one participant in a small one
(64 to 63, or 1.6 percent). Against BUCSS 1.2.1 on the grid described
under “Engine”, 829 of the 855 comparable plans with
alpha_prior = 1 (97 percent) have the sample size 1.2.1
gave, and the other 26 are one or two participants smaller, at most 1.6
percent (64 to 63 per group); none is larger. The rounded
(three-decimal) parameter differs from the 1.2.1 value in 182 of the
855, each higher by .001 (at most 0.2 percent). Outside that grid, a
paired t significant at .01 from 3,000 pairs, planned with
alpha_prior = 1 at an assurance of .80, needs
7,823 pairs where 1.2.1 gave 7,825. Plans with alpha_prior
below 1 are unaffected.
alpha_prior also accepts the string
"No Publication Filter" as a synonym for 1, so that a call
can say what the value means. It is read without regard to case, with
spaces, underscores, or hyphens between the words; the result echoes it
as an alpha_prior of 1; the deprecated functions’
alpha.prior accepts it too; and any other string is refused
with a message naming the range (above 0, at most 1) and the
string.
The exported functions are renamed to the ss_buc_*
prefix, so the names make clear that these planners apply the bias and
uncertainty correction rather than ordinary power analysis. The
between-subjects ANOVA planner is split into a one-way and a two-way
function, and the two t test planners gain effect size
aliases:
| 1.x | 2.0.0 |
|---|---|
ss.power.it |
ss_buc_independent_t (alias
ss_buc_smd) |
ss.power.dt |
ss_buc_paired_t (alias
ss_buc_smd_paired) |
ss.power.ba |
ss_buc_one_way_anova,
ss_buc_factorial_anova |
ss.power.ba.general |
ss_buc_factorial_anova_general |
ss.power.wa |
ss_buc_rm_anova |
ss.power.wa.general |
ss_buc_rm_anova_general |
ss.power.spa |
ss_buc_mixed_anova |
ss.power.spa.general |
ss_buc_mixed_anova_general |
ss.power.reg1 |
ss_buc_reg_coef |
ss.power.reg.all |
ss_buc_R2 |
ss.power.reg.joint |
ss_buc_reg_joint |
Arguments are renamed to snake_case (for example
t.observed becomes t_observed,
alpha.prior becomes alpha_prior,
df.numerator becomes df_numerator), the 1.x
power argument is now desired_power (matching
the DMAR package’s planners and the
desired_power row it echoes), and effect
values are snake_case ("factor.A" becomes
"factor_A", "between.only" becomes
"between_only", and so on).
The functions now return a tidy object of class
bucss_plan rather than a printed two-element list, shaped
like the result tables of the sibling package DMAR. It is a
data.frame with a character term column and a
numeric value column. The design results come first: the
necessary sample size, named for its unit
(necessary_n_per_group, necessary_n_per_cell,
or necessary_N); total_N, the implied total,
for per-group and per-cell designs; actual_power, the power
of the plan returned; and ncp_adjusted, whose scale (that
of the prior study’s t, F, or chi square statistic)
each help page states. Rows echoing the planning inputs follow (a
length-2 prior n echoes as
n_1/n_2), so the assumptions travel with the
result through subsetting and CSV export. Only non-numeric metadata (the
design label, the effect tested, the unit) plus the assurance ceiling
and the planned test’s degrees of freedom travel on attributes.
The dot-named 1.x functions (ss.power.it,
ss.power.dt, ss.power.ba,
ss.power.ba.general, ss.power.wa,
ss.power.wa.general, ss.power.spa,
ss.power.spa.general, ss.power.reg1,
ss.power.reg.all, ss.power.reg.joint) still
work. Each accepts the 1.x argument names, translates them, and calls
the corresponding ss_buc_* function, so scripts written for
BUCSS 1.x continue to run. Calling one issues a deprecation warning that
names its replacement, once per function per session; the
ss_buc_* functions themselves never issue it.
ss.power.ba dispatches to ss_buc_one_way_anova
when levels.B is NULL and to
ss_buc_factorial_anova otherwise; the dotted
effect values are translated to snake_case; and the 1.x
step argument is accepted and ignored.
The deprecated functions return a bucss_plan object,
but a [[ method preserves the 1.x positional extraction:
result[[1]] is the necessary sample size and
result[[2]] is the adjusted noncentrality parameter, the
same two values the 1.x functions returned as an unnamed list. Column
and name access (result$value,
result[["value"]]) are unaffected.
Six designs join the twelve that 1.2.1 covered. Two re-present computations the package already made (the one-sample t and the analysis of covariance); four extend the correction to statistics or distributions it did not previously handle.
ss_buc_correlation() plans from a prior Pearson
correlation (or its t). A correlation study samples both
variables, so its test statistic is not noncentral F:
conditional on the sampled predictor it is, but the noncentrality
parameter is itself random, proportional to the sampled spread of the
predictor. The marginal distribution is a chi square mixture of
noncentral F distributions, and it is more dispersed. The
planner uses that mixture, which has an exact closed form (the Poisson
weights of the noncentral F become negative binomial weights),
for both the correction and the planned study’s power, so the returned
sample size is exact for bivariate normal data. Treating the statistic
as noncentral F, which is correct only when the predictor is
fixed by design, overstates the planned study’s power by roughly
1/n for a plan of n participants (about .01 at
n = 100 and .05 at n = 20) and returns a sample size
typically 2 to 5 participants light. When the predictor really is fixed,
the analysis is a regression slope and ss_buc_reg_coef()
with p = 1 and random_predictors = FALSE is
the planner for it; with random_predictors = TRUE that call
returns exactly this planner’s result. The largest supportable assurance
is the same either way, because at a zero effect the mixture is the
central F distribution.
ss_buc_one_sample_t() plans a one-sample t
test. This is exactly the computation ss_buc_paired_t()
performs, reported with a one-sample design label and a total sample
size instead of a number of pairs.
ss_buc_ancova() plans an analysis of covariance from
the number of covariates, rather than requiring the user to work out the
error degrees of freedom for
ss_buc_factorial_anova_general(). With no covariates it
reproduces the corresponding ANOVA planner exactly. The omnibus group
effect is the default; a single-degree-of-freedom contrast among the
adjusted means is available through
df_numerator = 1.
ss_buc_manova() plans a two-group multivariate
comparison (Hotelling’s T squared), taking either the
F or the T squared. This is the multivariate case
whose hypothesis has rank one and therefore an exact noncentral
F; hypotheses of higher rank have an eigenvalue spectrum rather
than a single noncentrality parameter and are outside the
method.
ss_buc_chisq_diff() plans from a nested model chi
square difference (likelihood ratio) test, the test used for a
constrained path in a structural equation model. It is not for
an omnibus model fit chi square, where a publishable result is a small
statistic and the selection region is inverted, and the function
refuses that case rather than only warning about it.
That is why it takes both models’ degrees of freedom,
df_full and df_restricted, rather than only
their difference: an omnibus fit chi square is the comparison of a model
against the saturated model, which has zero degrees of freedom, so
df_full = 0 is its signature. Given only the difference,
nothing distinguishes a fit test from a genuine comparison of two
substantive models and the mistake would pass silently. (A
just-identified full model also has zero degrees of freedom; the help
page and the error say how to plan that case.) The help page states the
finite-sample liberality of the likelihood ratio test for a named
model.
ss_buc_welch_t() plans a Welch (unequal variance)
t test from the two prior group sizes and the groups’ relative
spread. Only the ratio of the standard deviations enters the
computation, but a paper usually reports the group standard deviations
or variances rather than their ratio, so the spread may be given any of
three ways: sd_ratio, or sd_1 and
sd_2, or var_1 and var_2. Exactly
one form is accepted per call, and the form given is the one echoed back
in the result. A Welch statistic is not exactly noncentral t,
so this planner is an approximation, and its help page says which
assumption it needs and suggests re-running across a range of ratios. A
minimum planned sample size of 10 per group is imposed because of
estimation accuracy issues at smaller sample sizes: below 10 per group
the approximation overstates the real Welch test’s power, badly so for
the smallest, while from 10 per group it is within .006 of the real
test’s power for every plan returned at an alpha_planned of
.05. At a stricter alpha_planned it overstates the power of
the smallest plans by more (at 10 per group by up to about .012 at .01
and .025 at .001) and is within .006 again from 14 and 20 per group,
respectively.
Added plot.bucss_plan(), so plot() on
any plan draws its assurance ceiling. The method re-runs that plan’s own
planner across a range of assurance values and draws two panels sharing
an assurance axis: the adjusted noncentrality parameter falling to zero
at the ceiling, and the necessary sample size it implies growing
asymptotically on a logarithmic scale. The ceiling is marked, the plan
itself is marked, and the region beyond the ceiling is shaded and
labeled as one where no plan exists.
The point of the figure is the shape. Assurance does not degrade
gracefully as it rises, it hits a wall, and that is the hardest thing
about the method to convey in prose. For a prior independent t
test with t = 3.00 and 20 per group, the plan runs from 25 per
group at assurance = .50 to over 65,000 just short of the
ceiling at .905, and is undefined above it.
The grid of assurance values is spaced evenly in the logarithm of the remaining headroom rather than in assurance, so roughly half the plotted points fall in the last few hundredths below the ceiling. Spacing evenly in assurance would draw the wall as a single jump between two grid points and lose the shape.
The plotted values are returned invisibly as an ordinary
data.frame, one row per assurance, so the figure can be
redrawn with any other graphics system. The method is written in base
graphics; graphics joins generics and
stats in Imports, and no plotting package is
required.
Added ss_buc_sensitivity(), a Monte Carlo companion
that takes a plan from any ss_buc_* function and exercises
the claim the method actually makes. It reads the design and the
planning inputs off the plan, simulates prior studies of that design
from a true effect the user names, discards the ones a literature would
not have published, runs the same planner on each survivor, and reports
how often a plan built this way reaches the target power. There is
deliberately no default for true_ncp: defaulting to the
corrected estimate would ask the method to grade its own
answer.
The output distinguishes two attainment rates, and the
distinction settles a question the method’s documentation had left
implicit. attainment counts only the prior studies for
which a plan was issued, which is what a user experiences;
attainment_with_refusals counts a refusal as attaining,
because declining to plan is the limiting case of an arbitrarily
conservative plan, and it is the rate the assurance guarantee is about.
When the refusal rate is high the first sits well below
assurance, which is the visible price of the method
refusing rather than guessing rather than a defect.
The package-wide verification ships with the package. The
simulation harness that validated the planners, twenty-four checks
covering assurance calibration across every sampling family, raw-data
audits of the correlation planner, the Welch planner, and the regression
coefficient planner with sampled predictors, and the finite-sample
behavior behind the chi square caveat, is installed under
inst/tools/ and reruns in a few seconds (under a minute in
its full mode) with
source(system.file("tools", "verify_by_simulation.R", package = "BUCSS")).
It reports the version and location of the BUCSS it verified.
ss_buc_sensitivity() reports the Monte Carlo
standard error of both attainment rates. Only the planner’s own refusals
count as refusals; any other error stops the audit with its message, and
the planner’s warnings are summarized once rather than repeated for
every simulated study (the warning for an assurance below .5, which the
plan already raised when it was built, is not repeated). A plan given as
Hotelling’s T squared can be audited like any other.
Neither leaves a trace in the session.
ss_buc_sensitivity()’s seed defaults to
NULL, which draws from the caller’s own random number
stream; a supplied seed is set through withr::local_seed(),
so the caller’s generator state and kind are restored on exit (or
removed again, if the caller had none), and the package never reads or
writes the random number state itself. withr (>= 3.0.3;
earlier versions of local_seed() did not restore anything
when called from inside a function) joins Imports. The
harness runs entirely inside local() with the same seed
handling, so sourcing it creates or overwrites nothing in the global
environment, leaves the caller’s random number state as it was, and
reports a failed check as an error rather than quitting the
session.
The three regression planners, ss_buc_reg_coef(),
ss_buc_R2(), and ss_buc_reg_joint(), take a
random_predictors argument (default FALSE, the
fixed-predictor frame of 1.2.1) for predictors sampled along with the
outcome, as is typical of observational research in the behavioral
sciences (Kelley & Maxwell, 2003). Whether the predictors are fixed
by design or sampled changes neither the estimates nor the distribution
of the test statistic when the effect is zero, but it changes the
statistic’s distribution, and so the power, when the effect is not zero
(Sampson, 1974; Gatsonis & Sampson, 1989). With
random_predictors = TRUE the noncentrality parameter is
itself random, proportional to the sampled spread of the tested
predictors once the other predictors are accounted for (chi square on
N - 1 - (p - q) degrees of freedom when
q of p predictors are tested), so the statistic
follows a chi square mixture of noncentral F distributions. The
mixture has an exact closed form (the Poisson weights of the noncentral
F become negative binomial weights), used for both the
correction and the planned study’s power; it is the distribution
ss_buc_correlation() uses: with one predictor each of the
three returns exactly what ss_buc_correlation() returns for
the same prior study (ss_buc_reg_coef(p = 1) for the same
t and N, and ss_buc_R2(p = 1) and
ss_buc_reg_joint(p = 1, p_joint = 1) for an
F_observed equal to t squared), with the planning
effect size stated in each planner’s own metric. Treating sampled
predictors as fixed overstates the planned study’s power, by about
1/n for a plan of n participants with one predictor
and about 4/n for a coefficient among eight (1/n to
1.4/n for the squared multiple correlation). Which frame plans
the larger study depends on the other predictors and on how far the plan
outgrows the prior study; the help pages give the pattern. The largest
supportable assurance is the same in both frames. The frame is echoed as
a random_predictors row (1 or 0) on every regression plan,
named in the printed footer, and followed by
ss_buc_sensitivity() and plot(). The
deprecated ss.power.reg1(),
ss.power.reg.all(), and ss.power.reg.joint()
keep the fixed-predictor computation of 1.x.
The within-subjects and split-plot planners,
ss_buc_rm_anova(), ss_buc_rm_anova_general(),
ss_buc_mixed_anova(), and
ss_buc_mixed_anova_general(), take an epsilon
argument (the last argument, default 1) for a prior study reported with
Greenhouse-Geisser or Huynh-Feldt corrected degrees of freedom:
epsilon is the reported numerator degrees of freedom
divided by the uncorrected ones. The prior study’s F and the
planned study’s power are both treated under the approximation of Muller
and Barton (1989), with the degrees of freedom and the noncentrality
parameter multiplied by epsilon (the convention of
DMAR’s ss_power_rm_anova() and
ss_power_split_plot_anova()), and ncp_adjusted
stays on the sphericity scale. It applies to within-subjects effects and
to the interaction of a between-subjects and a within-subjects factor,
each with its own lower bound; the between-subjects effect of a
split-plot design involves no sphericity, so a value other than 1 is
ignored there, with a warning. The correction is an approximation, and
the help pages report its accuracy in simulations of raw data: at an
assurance of .80, plans reached the target power in .70 to
.80 of replications for trend-like effects and in .92 to .99 for effects
in the least variable contrasts, against .26 to .64 and .70 to .87,
respectively, when the departure was ignored. The default,
epsilon = 1, reproduces the plans that assume sphericity
exactly. Every plan for an effect with a within-subjects component
echoes epsilon as a row (1 under sphericity), and
ss_buc_sensitivity(), plot(), and
planning_sentence() follow it.
Every plan reports its planning effect size, an adjusted effect
size: the value, in the design’s usual effect size metric, used to plan
the sample size given the situation at hand (the prior study’s result,
the publication bias assumed through alpha_prior, and the
uncertainty allowed for through assurance). It is
ncp_adjusted restated in that metric, as a row directly
after ncp_adjusted, and it is what the plan is built on,
not an estimate of the population effect to report: above an assurance
of .5 it is deliberately conservative (a lower confidence limit), and at
.5 it is the median unbiased estimate. The rows are
smd_planning (the independent t planner and its
alias ss_buc_smd(), and the Welch t planner, which
standardizes by the root mean square of the two standard deviations,
following Bonett, 2008), dz_planning (the paired t
planner and its alias; Cohen’s d_z, the mean of the paired
differences divided by their standard deviation, where the z
names the difference score, which Cohen, 1988, writes Z = X - Y),
std_mean_planning (the one-sample t),
f_planning (Cohen’s f relative to the effect’s own
error term, for all eight analysis of variance and covariance planners,
so that f squared over 1 plus f squared is the
population partial eta squared for the effect), f2_planning
(Cohen’s f squared, for a regression coefficient and for a
subset of predictors), R2_planning (the squared multiple
correlation), rho_planning (the correlation),
D2_planning (the squared Mahalanobis distance, for the
two-group multivariate comparison), and rmsea_d_planning
(RMSEA_D, with D for difference: the root mean square error of
approximation of the chi square difference test between the two nested
models, as distinct from the RMSEA of either model’s fit, following
Savalei, Brace, & Fouladi, 2024). actual_power is
exactly the ordinary power of a study of the returned sample size when
the population effect is the planning effect size, in the plan’s own
frame (with sampled predictors, the random-predictor power; with
epsilon below 1, the Muller and Barton approximation, the
effect staying on the sphericity scale). For an unbalanced prior study
the row belongs to the same balanced design as
ncp_adjusted. planning_sentence() names it in
a second sentence and glance() gives it a column;
tidy() and the 1.x [[1]]/[[2]]
positions are unchanged. For the within-subjects and split-plot
planners, f_planning is not the raw-score f that
repeated measures power functions such as DMAR’s
ss_power_rm_anova() take; the help pages give the
conversion.
The independent and paired t test planners are available
under both test-specific names (ss_buc_independent_t,
ss_buc_paired_t) and effect size names
(ss_buc_smd, ss_buc_smd_paired) for the
standardized mean difference. The two names in each pair are the same
function.
Added print.bucss_plan(), which prints the aligned
term/value table in the DMAR
display style (whole numbers clean, other values at
options(bucss.digits) significant figures, with more where
fewer would show a value below 1 as 1, stored values never rounded) and
closes with factual footer lines naming the design, for the regression
planners the predictor frame, the unit the sample size is counted in,
and the largest assurance the prior result supports. A
knitr::knit_print method renders the same table as a kable
in R Markdown.
The effect argument also accepts case-insensitive
shorthand in the two-way between-subjects, within-subjects, and
split-plot planners, for example "A" for
"factor_A", "AxB" for
"interaction", and "bs"/"ws" for
"between"/"within". The snake_case values
remain the documented forms and the result always records the canonical
one.
Added tidy() and glance() methods for
bucss_plan results (from the generics
package), with the same division of labor as DMAR’s
planners: tidy() is the compact estimate view
(term = "sample_size", estimate, and
power), and glance() is the one-row wide view
that begins with the same three columns and then has every quantity and
echoed planning input as a column, plus the design metadata and the
planned study’s test degrees of freedom (df_effect, and
df_error where the test has a denominator). Both stop with
an explanation, rather than reporting one plan’s numbers for another,
when given several plans stacked into one object or a plan with one of
its rows removed.
Every planner reports actual_power, the statistical
power of the plan it reports: the power of a study of the returned
sample size when the effect is the returned adjusted noncentrality
parameter (equivalently, the planning effect size). Because a sample
size must be a whole number, it is never below the requested
desired_power and is usually a little above it. Each help
page states this definition.
Added planning_sentence(), which turns a result into
the sentence an author writes in a manuscript’s planning section (the
analog of DMAR’s results_sentence()). The
sentence names only the corrections the plan actually made: the
publication bias correction, with its assumed threshold, when
alpha_prior is below 1, and the uncertainty correction when
assurance is above .5, and it notes a departure from
sphericity when epsilon is below 1. A second sentence names
the planning effect size and its metric.
The zero-noncentrality error now says plainly that re-running at
a lower assurance is not free: the plan it returns carries that lower
assurance, not the one originally requested, so the long run proportion
of replications reaching the target power falls accordingly. Raising
alpha_prior is a different kind of remedy, changing an
assumption about the prior study’s literature rather than lowering the
guarantee. The assurance vignette develops the distinction.
When the correction drives the corrected noncentrality parameter
to zero, the error reports the largest assurance the prior result can
support (the closed-form ceiling 1 - p/alpha_prior, where
p is the prior study’s p value) and states whether
lowering assurance to at or below that ceiling can reach a usable plan,
or whether raising alpha_prior is the only remaining lever.
The message clarifies that alpha_prior is the analyst’s
assumption about the publication threshold the prior study’s literature
faced, not a change to the prior study itself. Each planner’s
documentation states this ceiling and illustrates it with a runnable
try() example.
On a successful call, the result also reports the largest
assurance the prior study can support and, for per-group and per-cell
designs, the implied total sample size (total_N). Both are
shown by print.bucss_plan() and travel on the
object.
Added an inst/CITATION file.
citation("BUCSS") returns the package reference, so that
the version and CRAN source are credited, together with the four
articles behind the methods: Anderson, Kelley, and Maxwell (2017),
Anderson and Kelley (2024), Anderson and Maxwell (2017), and the
Anderson
(2021) regression tutorial.
Added a vignette on planning replication studies, which walks through why a literature built from underpowered studies inflates effect sizes and how the bias and uncertainty correction repairs the resulting sample size plan.
Added a vignette on planning a correlation study, which explains
why a correlation is not a fixed-predictor regression slope, when
ss_buc_correlation() and ss_buc_reg_coef()
each apply, and how to audit a correlation plan with
ss_buc_sensitivity() and read its two attainment
rates.
A negative observed t is now accepted by every planner
that takes a t (ss_buc_independent_t(),
ss_buc_paired_t(), ss_buc_one_sample_t(),
ss_buc_welch_t(), ss_buc_reg_coef(), and
ss_buc_correlation()), each of which plans from its
magnitude. The publication rule the correction assumes is two-sided and
the truncated likelihood is symmetric in the sign of t, so
t_observed = -3 is the same evidence as
t_observed = 3; the sign records which group was subtracted
from which (or the direction of the paired difference, or the
coefficient’s coding), never the strength of the prior result. In 1.x
and the earlier 2.0.0 drafts the two t-test planners rejected
any negative t as “nonsignificant”, however large its
magnitude, with advice (raise alpha_prior) that could not
help, while ss.power.reg1/ss_buc_reg_coef
silently accepted the same input; the three planners now agree. The
signed value is still echoed in the stored planning inputs, and a
genuinely nonsignificant magnitude is still rejected. Scripts (and the
designingexperiments.com apps) that previously received an error for a
significant negative t now receive the plan its magnitude
implies; no result that was previously returned has changed.
An assurance or power of exactly 0 or 1
(after the percentage coercion, so also power = 100) is now
rejected with a clear error. In 1.x an assurance of 0 and a
power of 0 or 100 percent were accepted and silently
returned meaningless results: an assurance of 0 has no
solution, so the reported noncentrality parameter was an artifact of the
internal search bound, and a target power of 100 percent is
unattainable, so the returned sample size was where the computed power
crossed 1 by floating-point error. An assurance of 100
percent (assurance = 100) already stopped in 1.x, but with
the misleading message that the corrected noncentrality parameter was
zero. The documented rule that an assurance or
power of exactly 1 is read as 1 percent is
unchanged.
A desired_power (1.x power) at or below
alpha_planned is not allowable and now stops, with an error
that says so and names both values. A test’s power is never below its
Type I error rate, so every sample size meets such a target and no plan
is meaningful; 1.x returned the smallest sample size its search
considers (2 per group, for instance), with no notice. The check follows
the percentage reading, so desired_power = 1, which is read
as 1 percent, stops unless alpha_planned is below .01, and
the message then says how the value was read. The deprecated functions
stop in the same way.
When the two prior group sizes given to
ss_buc_independent_t() (or ss.power.it) have a
harmonic mean that is a whole number, the planner no longer plans from a
spurious second rounding one unit off. The mean was computed in floating
point, where a whole number can land a hair away from itself: with
t_observed = 3 and the other arguments at their defaults,
n = c(49, 49) planned 222 per group where
n = 49 plans 219, and n = c(10, 15), whose
harmonic mean is exactly 12, planned 177 where 12 plans 154. The mean is
now rounded in integer arithmetic. Only such inputs change, and the
change is in the direction of a smaller, correct plan.
For a prior study whose group sizes are unequal, the
between-subjects and split-plot planners (and the independent t
and MANOVA planners) reduce it to a balanced design rounded both down
and up. When a statistic is significant at the prior study’s own degrees
of freedom but not at the rounded-down design, the error now says
exactly that, rather than calling the prior result nonsignificant, and
names an alpha_prior at which a plan can be made. And when
the correction reaches zero, the error names the smaller of the two
rounded designs’ ceilings, so re-running at the assurance it names now
produces a plan; before, it could name the larger one, which failed
again.
ss.power.ba and ss.power.wa refuse
effect = "factor.B" (as well as the interaction) when
levels.B is not given. 1.2.1 refused only the interaction,
and silently returned the Factor A plan for
"factor.B".
ss_buc_factorial_anova_general() and
ss_buc_ancova() reject a df_numerator larger
than cells - 1, and
ss_buc_mixed_anova_general() rejects numerator degrees of
freedom no design of its kind can have and warns that
df_num_within is ignored except for the between-within
interaction. Each used to return an ordinary-looking plan. The 1.x
wrappers ss.power.ba.general and
ss.power.spa.general inherit the same checks, so such 1.x
calls now stop with an explanation (the wrapper passes
df.num.within on only for the interaction, as 1.x used it,
so 1.x calls that supply it with another effect do not warn).
df_denominator now functions in
ss_buc_factorial_anova_general(). In 1.x it was accepted
but silently ignored, and the documented example supplied an impossible
value (df_denominator = 117 with N = 120 and
cells = 6, for which the maximum residual degrees of
freedom is 120 - 6 = 114). The difference
(N - cells) - df_denominator is now treated as the number
of nuisance parameters (covariates or blocking factors) and carried
through to the planned study: supplying
df_denominator = N - cells reproduces the no-nuisance case,
while a smaller value correctly calls for a larger sample. Values
greater than N - cells are rejected.
The error shown when N is omitted in
ss_buc_reg_coef(), ss_buc_R2(), and
ss_buc_reg_joint() incorrectly described N as
the number of pairs. It now correctly describes N as the
total sample size of the previous study.
The nonsignificance error in ss_buc_R2() and
ss_buc_reg_joint() referred to t.observed.
These functions test an F statistic, so the message now
refers to F_observed.
Omitting effect in the between-subjects,
within-subjects, and split-plot functions no longer fails with “the
condition has length > 1” under R >= 4.2. effect now
defaults to its first choice.
ss_buc_rm_anova() no longer silently returns the
single-factor result when effect is "factor_B"
or "interaction" but levels_B is not supplied;
it now stops and explains that levels_B is required for
those effects.
The planners give clearer errors for more input mistakes: a
missing structural argument (cells,
df_numerator, num_groups,
df_num_within, p, or p_joint) and
a prior N too small for the design (fewer than two
observations per cell or group) are now reported with an informative
message.
ss_buc_independent_t() now echoes the prior study’s
sample size (n or N, whichever was supplied)
in the result’s planning-input rows, and hence in the printed table, as
every other planner already did, so a saved or printed result can be
traced back to the prior study.
Every planner now validates that its observed statistic is a
single finite number and that its design counts (n,
N, levels, cells, num_groups,
p, p_joint, prior degrees of freedom) are
single whole numbers, naming the offending argument. In 1.x a fractional
count could flow through the search and return a non-integer “sample
size” (ss.power.reg1(t.observed = 3, N = 150, p = 2.5)
returned 624.5), and NA, infinite, or vector inputs died
inside base R with “missing value where TRUE/FALSE needed” or “the
condition has length > 1”. A within-subjects N of 1, or
an independent-samples total N below 4, is likewise
rejected up front instead of reaching
qt()/qf() with zero degrees of
freedom.
A last round of guards. A malformed effect (two
values included, of which the first was used without notice), a Welch
spread or an alpha_planned too extreme to compute with, and
a missing first argument to ss_buc_sensitivity() or
planning_sentence() each get a clear error naming the
argument; ss_buc_independent_t() reads
n = NULL with N as N alone;
ss_buc_manova() no longer raises a stray warning beside an
error; a nonsignificance refusal at alpha_prior = 1 no
longer advises raising alpha_prior; a printed value below 1
is never shown as 1; the 1.x positional [[ applies only to
a single plan; and loading the package enforces the withr
(>= 3.0.3) minimum, through an import in
NAMESPACE.
Added the reference Maxwell, Delaney, and Kelley (2027, Designing Experiments and Analyzing Data: A Model Comparison Perspective, 4th ed.) to the package references, DOIs for Anderson and Maxwell (2017) and Taylor and Muller (1996), and ORCIDs for all three authors.
The package help page, every planner’s assurance
argument, and
vignette("understanding-assurance", package = "BUCSS")
spell out how assurance is used. It is the long run proportion of
planned studies whose power at the true effect reaches
desired_power, when each is planned from a prior study
published under the rule alpha_prior describes, and the
planners deliver it by planning from the lower limit of a one-sided
confidence interval at level assurance, computed from the
publication-truncated distribution of the prior statistic. The term
follows Kelley and Maxwell (2003), who used assurance in sample size
planning for the probability that a confidence interval will be no wider
than desired; that reference joins every planner’s help page.
Safeguard power (Perugini, Gallucci, & Costantini, 2014)
guards against a prior effect size that is too large by planning for the
lower limit of a two-sided 60% confidence interval around it rather than
for the effect size itself; it allows for uncertainty but not for
publication bias. It is shown to be a special case of the method: with
alpha_prior = 1 and assurance = .80, the
adjusted noncentrality parameter is the lower limit of the two-sided 60%
confidence interval that safeguard power plans from, and in general a
two-sided confidence level C corresponds to
assurance = (1 + C)/2. For F and chi square
statistics the correspondence is exact. The package help page and the
assurance vignette explain how, and where the two differ (for the
t tests and other effects on one degree of freedom, where the
interval here bounds the size of the effect whatever its sign; for an
unbalanced prior design, read as a balanced one; and for the Welch
planner’s minimum): every difference makes the plan here the same as the
safeguard plan or larger, never smaller (for the regression planners,
within the frame random_predictors names).
tests/testthat/test-safeguard-power.R pins the
equivalence.
Every planner’s alpha_prior argument and Details say
plainly what the value means: the significance level a prior result had
to reach to be published, so the correction assumes you see only results
with p below it. alpha_prior = 1 means no
publication filter, since no p value exceeds 1, so there is no
publication bias to correct and only the correction for uncertainty, set
by assurance, remains. The help says that the value may
also be written "No Publication Filter", and cites
Anderson, Kelley, and Maxwell (2017, p. 1552), who note that the level
can be set equal to 1 when publication bias is thought not to apply. The
desired_power argument’s help says that it must exceed
alpha_planned.
Source citations join the help pages of the designs added in 2.0.0, each in the sentence it supports: Welch (1947) and Satterthwaite (1946) for the Welch test and its degrees of freedom; Hotelling (1931) for T squared; Fisher (1928) for the sampling distribution of the multiple correlation coefficient, on the correlation page and where the regression pages describe sampled predictors; and Steiger, Shapiro, and Browne (1985) and Satorra and Saris (1985) for the asymptotic distribution of chi square difference statistics and the power of the likelihood ratio test. Maxwell, Delaney, and Kelley (2027) is cited in the text of every help page that lists it.
The regression planners’ help pages describe both predictor frames, what treating sampled predictors as fixed costs, and which frame plans the larger study (see “Sampled Predictors, Sphericity, and the Planning Effect Size” above). The regression and chi square difference pages also state where the planned sample size search starts (a total of p + 3, and the number of constraints plus 2), so no plan is smaller than that.
The package help page and the Getting Started vignette list all eighteen planners.
The noncentrality search, the truncated-likelihood construction,
and the planned sample size search each live in a single internal helper
(.solve_ncp_assurance,
.tm_f/.tm_t, and
.smallest_n_for_power), shared by every planner, so the
numerical core is maintained in one place.
The zero-noncentrality and nonsignificance stops carry the
condition classes bucss_zero_ncp and
bucss_nonsignificant, so code that loops over plans
(including plot() and ss_buc_sensitivity())
can catch a refusal without catching anything else. plot()
now draws the plans of a prior result whose ceiling is below .5 instead
of treating the warning for such an assurance as a refusal.
R’s warning that a noncentral t probability “may not have been computed to full precision”, which the root finder triggered with large observed t statistics, is muffled where it arises; the solved parameter agrees with independent quadrature in every such case examined.
The planned sample size search stops with a message rather than
looping forever once the answer exceeds about 4.5e15 (2^52). Doubles
hold every whole number exactly only up to 2^53, beyond which the
bisection could not finish; answers below the bound are found exactly.
An assurance a hair below the ceiling gets there, as does an implausibly
large prior sample size or an alpha_planned too small to
compute with, and the message names each. The random-predictor series
(the correlation planner’s, and the regression planners’ with
random_predictors = TRUE) is one internal function, summed
over a window around its mode, so it stays exact for a strong
correlation in a very large prior study (an r of .9 with an
N in the hundreds of thousands), where a fixed number of terms
would have dropped most of its mass.
Seeds are set through withr (>= 3.0.3), and R
3.6.0 is now the stated minimum, which the package’s delayed S3
registration and its dependencies already required.
The planned sample size search brackets by doubling and then
bisects instead of stepping by one, so its cost is logarithmic in the
answer. Requests whose planned sample size runs to the millions (an
assurance just below the prior’s ceiling) now return
instantly instead of iterating for minutes, and a huge answer can no
longer overflow the integer step counter. Power is monotone in the
sample size, so the returned value is the identical integer the
incremental search produced (verified case for case on the developer
characterization grid).
Factored the shared planning-input validation into a single
internal helper, and moved the duplicated @references,
planning-argument, and @return documentation into
man-roxygen templates, so each is maintained in one place
across every planner. The stats functions are imported
individually rather than wholesale, an inst/WORDLIST
curates the domain terms for the spell check enabled by
Language: en-US, and a GitHub Actions workflow runs R CMD
check across platforms.
Added a testthat (edition 3) test suite: regression
oracles that pin the computed outputs, object-structure smoke tests,
input-validation tests, coercion tests, and tests confirming each
deprecated dotted name forwards to its ss_buc_*
replacement. The new options have their own files
(test-random-predictors.R, test-epsilon.R,
test-effect-rows.R, and
test-safeguard-power.R), which check each against an
independent computation: the random-predictor mixture against quadrature
and against the correlation planner, epsilon against the
corrected-degrees-of-freedom power, the planning effect size against the
ordinary power of the returned sample size, and safeguard power against
confidence limits computed directly.
test-no-publication-filter.R checks every planner at
alpha_prior = 1 against the statistic’s ordinary
distribution, computed in base R.