Corrected the heteroskedasticity-robust Ichimura
index-coefficient covariance returned by
vcov(npindex(..., gradients = TRUE)) when the model has
more than one free index coefficient. Link-gradient and residual weights
are now applied observation by observation. Fits with one free
coefficient remain exactly unchanged, as do beta, bandwidth, objective,
fitted-value, residual, and gradient results. Serial and MPI results
remain exactly equivalent, with no payload change.
Repaired finite-support bounded continuous-kernel normalization for all supported Gaussian, Epanechnikov, uniform, and truncated-Gaussian kernels. The centered normalization now avoids tail cancellation, uses the platform C99 error function where appropriate, and preserves the uniform-density limit as the bandwidth tends to infinity.
Replaced the bounded distribution operator with the mathematically correct observation-centered truncated-kernel CDF. Analytic centered interval primitives and hoisted invariant work preserve exact support endpoints, numerical accuracy, the finite-support large-bandwidth uniform limit, and serial/MPI numerical equivalence.
Corrected transform.bounds = TRUE initialization for
regression initial starts and multistarts and for later
conditional-distribution multistarts. External bandwidth starts are now
inverse-mapped before transformed Powell search; the public default
remains FALSE and MPI payloads are unchanged.
Clarified optimizer summaries so objective-cache lookups, NOMAD
point lookups, family-native R-optim refinement, and total
function-evaluation accounting are labelled consistently without
implying that their denominators are interchangeable.
Clarified bandwidth-object replay documentation across estimator families. Stored bandwidth objects retain search and fit metadata but do not silently materialize or expand training data; callers must provide data again where the documented replay route requires it.
Hardened MPI lifecycle and protocol handling. Bootstrap fanout timeout or failure now reports a hard failure and instructs the user to restart R before further MPI-backed computation; attach close resets pool state, profile communicator metadata remains distinct, and internal protocol tags are allocated portably.
Standardized the public nomad.opts contract across
supported bandwidth selectors and hardened native NOMAD callback cleanup
and allocation guards without changing MPI message payloads.
Refreshed MPICH build and runtime guidance for current MacPorts layouts and documented the supported session, attach, and profile/manual-broadcast launch modes. Open MPI remains outside the validated 0.70-5 backend matrix.
Reworked npindexbw() / npindex()
internals after 0.70-4. Ichimura and Klein-Spady single-index objectives
now reuse the established npreg leave-one-out backend where
applicable, preserving the public estimator contract while materially
improving high-dimensional and local-polynomial objective evaluation.
MPI-aware objective services now keep the root-owned optimizer and
worker evaluation protocols explicit.
Repaired MPI single-index NOMAD/Powell handoff behavior for
npindex(..., nomad = TRUE) and related bandwidth searches.
The Powell refinement stage now uses the same one-active-protocol
discipline as the validated MPI search routes, avoiding worker/master
desynchronization during the NOMAD-to-Powell transition.
Distributed post-CV npindex() fitting, evaluation,
gradients, and related components across owned evaluation rows in active
MPI sessions while keeping the full training data available where kernel
smoothing requires it. Focused large-sample checks showed the expected
scaling improvement and preserved serial-equivalent numerical
results.
Added user documentation for npindex() optimizer
choice and beta interpretation. The documentation now gives practical
guidance on when derivative-free Nelder-Mead remains a reasonable
low-dimensional default and when BFGS is useful for higher-dimensional
index searches, and it explains the relative interpretation of
normalized single-index beta coefficients.
Hardened single-index fit, evaluation, plotting, service, and summary behavior. Bounded continuous-kernel options are now carried consistently through objective, fit, evaluation, variance, and bootstrap routes; Klein-Spady confusion-matrix output is guarded against out-of-range fitted values; and large-bandwidth shortcuts are restricted to kernel/order combinations whose constant-weight approximation was validated.
Repaired single-index plot bootstrap memory usage. Wild-bootstrap
self-maps now avoid public-facing O(n^2) kernel-weight
allocations for the ordinary large-sample plot route, while preserving
public fitted/plotting contracts and MPI row ownership.
Repaired generalized-nearest-neighbor local-polynomial derivative
ownership across npreghat() and related public routes.
Mixed-degree local-polynomial fits now route available derivative
components through the correct owner, preserve
apply == H %*% y contracts, and report unavailable
derivative components consistently rather than silently applying the
wrong operator.
Repaired local-constant derivative ownership in
npreghat() so degree-zero local-polynomial derivative
requests use the analytic local-constant derivative contract only when
that contract is mathematically available. Scalar, matrix, multi-column,
serial-equivalent, and active-pool error routes were validated
separately.
Improved conditional mixed-degree local-polynomial gradients for
npcdens() / npcdist() and public
conditional-gradient accessors. The partial-availability contract now
matches the regression-family policy: available components are returned,
unavailable components are represented as NA, and
incoherent metadata fails clearly.
Repaired MPI conditional-density native derivative workspace handling so no-gradient and gradient calls can be mixed safely in one session without stale native derivative workspace state affecting later calls.
Repaired exact nearest-neighbor bootstrap parity in MPI conditional-density and conditional-distribution plot routes. Adaptive and generalized nearest-neighbor exact bootstrap paths now rebuild bandwidth state on the expanded resample consistently with the serial package.
Repaired formula/data reentry contracts across density,
distribution, conditional density, conditional distribution,
single-index, smooth coefficient, partially linear, quantile,
conditional-mode, copula, and significance-test routes. Explicit
estimator data= now overrides stored bandwidth-object data
where that public call shape is supported, formula newdata
is validated against fitted RHS variables, and direct formula calls with
numeric smoothing parameters no longer misroute formula objects as
native data before autodispatch.
Hardened MPI autodispatch argument transport and bandwidth fingerprints. Named arguments materialized through S3 dispatch are preserved before worker fanout, and bandwidth-object reuse checks are stronger against stale remote object reuse.
Repaired unconditional density and distribution edge contracts. Ordered kernel-code selection is now consistent between conditional bandwidth selection and fitting where normalization is required, all-NA input is rejected before native calls, training and evaluation omission metadata are retained separately where needed, categorical zero-bandwidth standard-error handling is consistent across categorical configurations, and unsupported bandwidth-selection method codes fail clearly.
Clarified normal-reference bandwidth documentation for density and distribution routes. The rule-of-thumb formulas are documented as fast exploratory Silverman-style heuristics, not production substitutes for cross-validation or likelihood-based selection.
Hardened npcdistbw() normal-reference method
handling so the R method code and native C method code stay aligned,
avoiding accidental fall-through to an unintended bandwidth-selection
branch.
Repaired and hardened public estimator contracts found during
adversarial audits of npscoef, npplreg,
npqreg, nplsqreg, npconmode,
npcopula, npreg, npcdens,
npcdist, npudens, and npudist.
Repairs include npscoef iterated backfit behavior and MPI
service-task handling, partially-linear fit reentry, quantile
inversion/clamping contracts, conditional-mode probability/tie handling
and orchestrator localization, copula sample reentry, regression
tree/large-h predicate alignment, and density/distribution
formula/native argument consistency.
Plot-bootstrap memory hardening now covers single-index,
conditional-mode, and partially-linear plot routes. The default plot
evaluation grids remain linear in the training sample size, and explicit
neval == ntrain style requests remain
user-controlled.
Native shadow-object and cache lifecycles were hardened. Conditional-density shadow pointers, native objective-cache state, and regression large-h / large-lambda caches are cleared at the appropriate top-level lifecycle boundary so pointer-keyed helper state cannot leak across independent calls, datasets, or MPI sessions.
npksum() numeric-bandwidth dispatch now constructs
data-aware bandwidth objects for mixed continuous/categorical data
instead of falling back to a default-typed object. The serial and MPI
packages now share the same public behavior for numeric smoothing
parameters.
The local-polynomial regression CVKS low-support objective path now uses a named bandwidth-method code and aligns the R and C routing contracts, reducing the risk of drift between search metadata and native objective handling.
Expanded focused tests, demos, benchmarks, and release-protocol sentinels across the public exported surface. The release protocol now requires an explicit public-exported-surface inventory, estimator-family sentinels, documentation/demo/benchmark smoke coverage, MPI scaling canaries, and installed package proof before release-ready claims.
Hardened proactive C cleanup paths by aligning LAPACK helper
out-of-memory cleanup with the serial package, clearing extended
nearest-neighbor alias state in the shared estimator cleanup helper,
routing selected bandwidth-constructor allocation and unsupported-method
failures through existing cleanup labels, and guarding large
np_kernelsum() allocation-size products before calling the
legacy alloc_vecd(int) allocator.
Added nomad = "auto" for local-polynomial degree
searches, preserving the serial package policy while keeping MPI
autodispatch and NOMAD shadow protocols explicit. Small one-dimensional
degree lattices use exhaustive Powell-style search; larger or explicitly
requested NOMAD surfaces continue to use NOMAD.
Repaired npRmpi fit-wrapper NOMAD guards after
normalized NOMAD control handling so nomad = TRUE and
nomad = "auto" routes stay on the intended local/native
path rather than being accidentally autodispatched.
Bandwidth and fit summaries now report cumulative search diagnostics more clearly. NOMAD cache output distinguishes repeated point lookups avoided by NOMAD from true objective computations; Powell summaries expose repeated objective lookups avoided by the package-side cache; hybrid NOMAD+Powell and exhaustive Powell timing labels are reported consistently.
Base-graphics plot scaling and legends have been hardened for
multi-panel displays. Plots now honor active
mfrow/cex behavior more consistently, use
role-appropriate legend sizes, and draw factor legends with point glyphs
matching the plotted estimates.
Fixed-bandwidth bias-corrected bootstrap plot intervals have been reworked across the supported serial-equivalent and MPI-aware plot families. Bias-corrected centers and intervals now share a common centering contract for regression, unconditional density and distribution, conditional density and distribution, single-index, partially linear, and smooth-coefficient plots where supported.
Pair/block/geometric bootstrap intervals in regression-style
routes now use smooth-bootstrap bias correction when
center = "bias-corrected" is requested; wild-bootstrap
regression intervals retain the standard wild-bootstrap correction.
Density and distribution routes use perturbation-based smooth-bootstrap
bias correction. Gaussian, uniform, and second-order Epanechnikov
perturbation kernels are supported; higher-order signed perturbation
kernels fail closed.
Bias-corrected plot support now covers mixed-data unconditional
density and distribution, mixed-data conditional density and
distribution, and conditional gradient displays.
Non-fixed/adaptive/generalized bandwidth bias correction and empirically
unsupported npqreg bias correction remain fail-closed with
explicit messages.
Bias-corrected plot payloads returned by
output = "data" and output = "both" now
consistently expose fitted values, bias-corrected values, gradients,
gradient bias corrections, and interval payloads with the same centering
contract used for rendering.
Derivative-order validation has been tightened across plot/gradient routes so unsupported derivative requests fail early or are represented consistently rather than silently plotting a lower-order derivative.
Single-index formula dispatch with explicit bandwidths has been repaired in MPI-aware routes, and single-index bias-corrected plot centers, output payloads, and legends have been tightened.
The native loadNamespace("crs") call construction
used by CRS-backed native search routes is now protected, resolving the
RCHK protection finding without changing the native NOMAD search
contract.
Added MPI-aware nplsqreg()/nplsqregbw()
support for location-scale quantile regression, including formula/data
and bandwidth-object workflows, scalar/vector tau,
prediction, residual extraction, summaries, and plot routes built on the
shared quantile plotting engine.
Supported MPI MADS/NOMAD-backed bandwidth-search routes now use
the final native crs NOMAD C API rather than the retired
legacy snomadr() fallback. The runtime dependency on
crs is now declared in Imports, while
LinkingTo remains for the native header.
Native NOMAD routes now preserve progress best-record reporting, expose cache/evaluation diagnostics, honor explicit start and option controls, and reject unsupported or indeterminate cache-off settings before solver entry. Inadmissible GLP degree candidates are guarded before expensive evaluator work in serial-equivalent and MPI-dispatched routes.
npindexbw(..., method = "ichimura", regtype = c("ll", "lp"))
now reuses the established local-polynomial regression objective
evaluator, and MPI autodispatch uses a rank-0-driven objective service
for the fixed-degree and NOMAD degree-search Ichimura local-polynomial
routes. Focused sentinel runs preserved payloads while restoring useful
scaling for the formerly flat local-linear and local-polynomial
single-index rows.
MPI fanout coverage has been extended for computationally heavy bootstrap workloads in specification, dependence, distribution-equality, quantile, and symmetry tests, and plot-bootstrap RNG streams now restore the serial-equivalent final state after MPI fanout.
The shipped npplreg attach-mode demo now explicitly
finalizes the master rank after successful attach shutdown, hardening
release-sentinel teardown without changing estimator or runtime
defaults.
MPI auto-dispatch for nplsqreg() now materializes
named method-level ... arguments before worker fanout,
preserving user-supplied scale and option values that
arrive through S3 ..n placeholders.
options(np.tree = "auto") is now the default tree
mode. In auto mode, continuous kd-tree routes are enabled only for
bounded-support continuous kernels ("epanechnikov" and
"uniform"); np.tree = TRUE remains the
explicit force-on override and np.tree = FALSE remains the
force-off diagnostic path.
Powell bandwidth searches now expose package-side
repeated-candidate objective caching through
options(np.objective.cache = TRUE/FALSE). The cache remains
enabled by default and is scoped to one bandwidth solve, so it can reuse
exact candidates across Powell restarts without carrying state across
datasets or later calls. Continuous-only generalized/adaptive
nearest-neighbor routes also retain their integer nearest-neighbor
objective cache under the same switch. The option is synchronized to MPI
workers in autodispatch sessions; NOMAD solver caching and extended-NN
distance reuse remain separate mechanisms.
Continuous large-bandwidth shortcut evaluations can now be
disabled with options(np.largeh = FALSE), and discrete
near-upper-bandwidth shortcut evaluations can now be disabled with
options(np.largelambda = FALSE). Both remain enabled by
default and are synchronized to MPI workers in autodispatch sessions.
These switches are intended for diagnostic timing and reproducibility
studies that need to separate tree effects from large-bandwidth and
large-lambda fast paths without changing the canonical dense/tree
objective machinery.
Local-polynomial regression cross-validation now uses a leaner
hot symmetric weighted-sum loop. Fixed-bandwidth
npregbw(..., regtype = "lp", bwmethod = "cv.ls")
objective probes in active MPI sessions match serial np
objective values to numerical precision while substantially reducing
local-polynomial CV evaluation time.
Shared weighted outer-product accumulation in
npksum() now uses a guarded BLAS dgemm route
when the operation is dense, non-permuted, and memory-bounded. Focused
fixed-bandwidth probes preserve serial/MPI objective parity while
substantially accelerating high-basis local-polynomial regression and
smooth-coefficient objective rows; small and scalar routes remain on the
established loop path.
Unconditional density least-squares cross-validation now uses a
leaner fixed-bandwidth Gaussian convolution loop. Fixed-bandwidth
npudensbw(..., bwmethod = "cv.ls") objective probes
preserve objective values exactly in the focused validation rows while
materially reducing the convolution portion of the objective
calculation. Conditional-density least-squares objective probes inherit
the same fixed-bandwidth Gaussian convolution improvement.
Non-Gaussian scalar-bandwidth convolution helpers now hoist the response bandwidth power outside the inner loop, improving fixed-bandwidth least-squares density cross-validation with compact-support kernels while preserving objective values exactly in focused probes.
Continuous-kernel vector helpers now reuse the loop-invariant signed inverse bandwidth scale inside their inner loops. Focused density, conditional density, and regression objective probes preserve serial/MPI objective parity while reducing repeated scaling work in shared C hot paths.
Conditional density and conditional distribution least-squares cross-validation now use a size-aware row-block policy for local-polynomial objective evaluation. The accepted route keeps the bounded-quadrature cap unchanged, bounds transient memory by sample size, and preserves objective values to numerical precision while materially reducing evaluator overhead for fixed-bandwidth CVLS probes in serial and MPI sessions.
Local-polynomial conditional density maximum-likelihood
cross-validation now uses the same bounded-memory block machinery for
fixed and generalized nearest-neighbor bandwidths. Focused
npcdensbw(..., bwmethod = "cv.ml", regtype = "lp") probes
preserve objective values and selected bandwidths to numerical precision
in serial and MPI sessions while reducing objective and full-search
runtime.
Large-sample categorical-only regression now uses the MPI-safe
profile-compressed route under
options(np.categorical.compress = TRUE), which is enabled
by default. This categorical route is independent of
options(np.tree). Repeated predictor profiles are
compressed before bandwidth search, fitting, prediction/evaluation,
standard errors, hat-helper use, and plot bootstrap helpers, preserving
the established dense-route numerical contract while reducing repeated
work.
Categorical-only unconditional density routes now use the same
profile-compression idea when
options(np.categorical.compress = TRUE) is enabled. The
fixed-bandwidth fit/evaluation route preserves dense-route
fitted/evaluation values while avoiding repeated computation over
identical categorical profiles, and the bandwidth-search route now uses
the same compressed support representation for all-categorical data. As
with other flat categorical search surfaces, selected smoothing
parameters may drift by optimizer-path amounts while preserving the
objective scale. Very fast compressed routes may remain overhead-floor
limited, so MPI acceleration is most useful once the uncompressed work
would be genuinely long-running.
Categorical-only conditional density and conditional distribution
bandwidth searches now honor
options(np.categorical.compress = TRUE). The promoted route
preserves the objective value to numerical precision while allowing
harmless optimizer-path drift in selected smoothing parameters,
especially near upper-bound or large-bandwidth regions where the
objective is flat.
Ordered-only unconditional distribution bandwidth search and
fit/evaluation routes also use profile compression when
options(np.categorical.compress = TRUE) is enabled. The
bandwidth-search route preserves the objective value to numerical
precision while allowing harmless optimizer-path drift in selected
smoothing parameters; fitted distribution values and standard errors are
preserved while avoiding repeated computation over identical ordered
profiles.
Fixed-bandwidth local-constant npscoef() fits now
use categorical-profile compression when all Z variables
are categorical and options(np.categorical.compress = TRUE)
is enabled. The route preserves fitted means, coefficient surfaces,
asymptotic mean standard errors, and coefficient/gradient standard
errors for training and evaluation fits while avoiding repeated work
over duplicate Z profiles. The corresponding
npscoefhat(output = "apply") path and count-based
plot-bootstrap helper use the same profile compression without changing
the explicit full-matrix output = "matrix"
contract.
Internal categorical-profile and large-bandwidth caches are now
cleared at the relevant top-level density, distribution,
conditional-density, conditional-distribution, and regression cleanup
points. These caches are keyed by call-local row pointers, so clearing
them per .Call prevents stale same-process state from
leaking across unrelated data sets or MPI dispatch modes.
Fixed npcdens() and npcdist() formula
calls with explicit numeric smoothing parameters, such as
npcdist(y ~ x, data = dat, bws = c(.25,.25)), so
npRmpi preserves the established
formula-to-bandwidth-object rewrite before MPI autodispatch.
Hardened the npudist() formula route so formula
calls are handled before MPI autodispatch.
npplreg() now activates the already validated
categorical regression compression path for its internal all-categorical
Z regressions when
options(np.categorical.compress = TRUE) is enabled, without
requiring users to request continuous kd-tree acceleration through
options(np.tree).
Formula variables whose names contain dots, such as
y.irr ~ x, are no longer mistaken for the formula wildcard
. in conditional density and conditional distribution
bandwidth routes. The conditional-density bandwidth formula route also
now expands the actual wildcard form y ~ . using the
supplied data frame, matching the conditional-distribution
route.
Fixed MPI conditional-density and conditional-distribution NOMAD
degree-search routes so Powell refinement and promoted wrappers such as
npconmode() reach the intended bandwidth-object
construction path rather than the pre-search autodispatch preflight used
by non-degree-search routes.
npqreg() is now a fully fledged MPI-aware
quantile-regression front end. It supports the formula/data workflow,
internally computes npcdistbw() bandwidths when a bandwidth
object is not supplied, accepts scalar or vector tau,
reuses selected bandwidths for additional quantiles in
plot(), and exposes the usual S3 surface:
fitted(), predict(),
predict(..., se.fit=TRUE), se(),
gradients(), summary(), print(),
quantile(), and plot().
npqreg() prediction now honors the standard
newdata workflow while preserving native exdat
precedence for compatibility with existing npRmpi call
surfaces. Formula-based prediction validates that new data contain the
required right-hand-side variables.
npqreg() plotting has been expanded for vector
quantiles, level/gradient displays, ordered predictors, user-specified
legends, and object-fed plotting of additional tau values
without recomputing cross-validation. The fixed-bandwidth gradient path
now uses the MPI-aware helper route.
npconmode() is now a first-class conditional-mode
estimator. It supports formula/data and bandwidth-object workflows,
forwards bandwidth-selection options to npcdensbw(),
propagates local polynomial and NOMAD metadata, and exposes
fitted(), predict(), summary(),
print(), gradients(), and plot()
methods.
npconmode() now supports optional class-probability
matrices and level-specific probability gradients. For
non-local-constant fits, probabilities are normalized to be non-negative
and to sum to one across the discrete response support before modal
classification.
npconmode() now fails early for non-categorical
responses and validates formula-based newdata against the
original right-hand-side variables.
npconmode() plotting now supports object-fed
class-probability slices and two-dimensional probability surfaces,
optional rgl rendering, and probability-level asymptotic
intervals where defined. Surface bootstrap intervals for class
probabilities remain intentionally deferred.
npcopula() is now a first-class copula estimator. It
supports formula/data and bandwidth-object workflows, automatic
two-dimensional probability grids, explicit u evaluation
grids, and ordinary extractable object components including
$bws.
npcopula() now provides fitted(),
predict(), predict(..., se.fit=TRUE),
se(), summary(), print(),
as.data.frame(), and richer plot() methods.
Plotting supports base persp, image, and
optional rgl rendering, with asymptotic and MPI-fanned
bootstrap intervals for copula surfaces where defined.
npcopula() explicit-grid evaluation now uses the
direct estimator route, preserving numerical results while avoiding the
severe runtime growth of the previous expanded-grid path when users
request larger probability grids.
The automatic local-polynomial NOMAD controls have been split
into explicit restart toggles: powell.remin for Powell
restarts and nomad.remin for the second NOMAD hot start.
This preserves the Powell Numerical Recipes restart default while
allowing NOMAD hot starts to be controlled separately.
Deprecated legacy remin remains accepted by
npregbw() and npreg() with a warning and is
mapped to the modern powell.remin/nomad.remin
controls where appropriate, preserving downstream compatibility while
documenting the new spelling.
Hat-operator helpers now support an additional constraint-oriented output route for objects needed by shape-constrained quadratic programming workflows, avoiding reimplementation of local-polynomial hat-matrix construction in user examples.
Local-polynomial derivative support has been broadened across the
conditional estimator family. npreg(),
npcdens(), and npcdist() now honor
gradient.order more consistently for fitted, evaluated,
predicted, and plotted objects when the selected polynomial degree is
high enough, including vector derivative orders over continuous
predictors and tensor/additive/Bernstein local-polynomial bases. The MPI
implementation dispatches the corresponding conditional hat-apply helper
work across the active worker pool where applicable.
Core and semiparametric S3 prediction paths have been hardened
around newdata, native evaluation-argument precedence,
formula RHS validation, and se.fit handling while
preserving npRmpi route independence.
Front-end/bandwidth argument hygiene has been tightened so
estimator-only controls such as proper are not forwarded
into bandwidth selectors that do not accept them.
MPI lifecycle and plotting routes received additional hardening,
including soft npRmpi.quit() behavior, local object-fed
plot computation where required, and explicit fanout of applicable
bootstrap workloads.
Documentation has been refreshed for the promoted
npqreg(), npconmode(), and
npcopula() workflows, including the local-polynomial NOMAD
route, probability/gradient outputs, plot controls, and examples that
use the streamlined interfaces.
The pre-release validation suite was expanded with focused hostile argument tests, S3 contract tests, installed/tarball proof scripts, route-aware MPI probes, and serial/MPI parity checks for the newly promoted estimator families.
The default multistart cap for bandwidth selection now follows
min(2, p) across the mirrored estimator families, replacing
the older min(5, p) cap. This includes automatic LP
degree-search calls when search.engine="nomad" or
"nomad+powell" and nmulti is not supplied
explicitly.
The univariate boundary density helper
npuniden.boundary() now defaults to
nmulti=1.
The empirical studies supporting this mirror change are
documented in np-master/benchmarks/validation/, with a
summary note kept in this repository’s
benchmarks/validation/ folder.
LP-capable front ends now accept nomad=TRUE as a
documented convenience preset for the recommended automatic NOMAD
local-polynomial route, mirroring the serial package defaults and
help-page guidance.