% Generated by roxygen2: do not edit by hand
% Please edit documentation in R/utils.R
\name{DataPreprocessing}
\alias{DataPreprocessing}
\alias{preprocessTrainData}
\alias{preprocessPredictionData}
\alias{savePreprocessorToJson}
\alias{savePreprocessorToJsonString}
\alias{createPreprocessorFromJson}
\alias{createPreprocessorFromJsonString}
\title{Data Preprocessing Routines}
\usage{
preprocessTrainData(input_data)

preprocessPredictionData(input_data, metadata)

savePreprocessorToJson(object)

savePreprocessorToJsonString(object)

createPreprocessorFromJson(json_object)

createPreprocessorFromJsonString(json_string)
}
\arguments{
\item{input_data}{Covariates, provided as either a dataframe or a matrix}

\item{metadata}{List containing information on variables, including train set
categories for categorical variables}

\item{object}{List containing information on variables, including train set
categories for categorical variables}

\item{json_object}{in-memory wrapper around JSON C++ object containing covariate preprocessor metadata}

\item{json_string}{in-memory JSON string containing covariate preprocessor metadata}
}
\value{
\code{preprocessTrainData} returns a list with transformed matrix data and a "metadata" list with details on the preprocessing procedures applied.
\code{preprocessPredictionData} returns a matrix reflecting the data transformations specified in the provided metadata list.

\code{savePreprocessorToJson} return an object of type \code{CppJson}.
\code{savePreprocessorToJsonString} returns a string dump of the preprocessor's JSON representation.

\code{createPreprocessorFromJson} and \code{createPreprocessorFromJsonString} both return metadata lists.
}
\description{
The functions in this group are designed to handle data preprocessing for stochastic forest models.
For example, factor-valued columns in data frames are either one-hot encoded or converted to integer indices
before the dataframe is converted to a standard matrix format for sampling.
This preprocessing routine defines a set of "steps" that must be repeated on out-of-sample datasets before
predictions can be obtained from a sampling model.

\code{preprocessTrainData} preprocesses covariates for the forest sampler routines, depending on the input type.
DataFrames will be preprocessed based on their column types (numeric columns are not modified, ordered factors are
integer coded, and unordered factors are one-hot encoded). Matrices are unmodified (assuming all columns are numeric).
This function also records and returns a "metadata" list with preprocessing details to ensure that other datasets can be preprocessing identically.

\code{preprocessPredictionData} preprocesses covariates for the forest sampler routines, based on the steps outlined in a metadata list produced by \code{preprocessTrainData}.

These procedures are handled internally in the \code{\link[=bart]{bart()}} and \code{\link[=bcf]{bcf()}} functions, but they are provided in \code{stochtree} as convenience functions for users writing custom samplers.
Furthermore, while R lists can be serialized to \code{RDS} format, we offer a number of JSON serialization routines for the metadata list produced by \code{preprocessTrainData} for
consistency with the broader serialization approach of stochtree (see \link{BARTSerialization} and \link{BCFSerialization}).

Following the API for serializing \code{bartmodel} and \code{bcfmodel} objects, we can convert metadata to JSON or JSON strings via
\code{savePreprocessorToJson} and \code{savePreprocessorToJsonString}. Similarly, we can reload a metadata list from JSON or JSON strings
via \code{createPreprocessorFromJson} and \code{createPreprocessorFromJsonString}.
}
\examples{
# Check that running the same data through `preprocessTrainData`
# and `preprocessPredictionData` yields the same result
n <- 100
x1 <- rnorm(n)
x2 <- factor(sample(1:3, n, replace = TRUE), ordered = TRUE)
x3 <- factor(sample(1:3, n, replace = TRUE), ordered = FALSE)
df1 <- data.frame(x1 = x1, x2 = x2, x3 = x3)
df2 <- data.frame(x1 = x1, x2 = x2, x3 = x3)
preprocess_train_list <- preprocessTrainData(df1)
df1_process <- preprocess_train_list$data
df1_metadata <- preprocess_train_list$metadata
df2_process <- preprocessPredictionData(df2, df1_metadata)
all.equal(df1_process, df2_process)

# Save to in-memory JSON
metadata_json <- savePreprocessorToJson(df1_metadata)
# Save to JSON string
metadata_json_string <- savePreprocessorToJsonString(df1_metadata)

# Reload metadata list from in-memory JSON object
metadata_roundtrip <- createPreprocessorFromJson(metadata_json)
# Reload metadata list from JSON string
metadata_roundtrip <- createPreprocessorFromJsonString(metadata_json_string)

cov_df <- data.frame(x1 = 1:5, x2 = 5:1, x3 = 6:10)
metadata <- list(num_ordered_cat_vars = 0, num_unordered_cat_vars = 0,
                 num_numeric_vars = 3, numeric_vars = c("x1", "x2", "x3"))
X_preprocessed <- preprocessPredictionData(cov_df, metadata)
}
