# Sample datasets

## Overview

This vignette describes various scenarios of creating sample datasets that fit the preferences and needs of the users; and this for the different models. The functions in the package PINstimation use two types of datasets: (1) A sequence of daily buys and sells (2) A high-frequency trading data. This is the reason why only two sample datasets are preloaded with the package, namely `dailytrades`, and `hfdata`. The users can also generate simulation data using the function `generatedata_mpin()` for PIN, and MPIN models; and the function `generatedata_adjpin()` for the ADJPIN model. Below we provide some scenarios of creating sample datasets both for the PIN, MPIN, and ADJPIN models.

## Sample datasets for the PIN model

The PIN model is an multilayer PIN model with a single information layer. We can, therefore, use the function `generatedata_mpin()`, in order to generate sample data for the PIN model. Generically, this is done as follows:

    generatedata_mpin(..., layers=1)

If the user would like to create a sample dataset for infrequently traded stock, she can specify low values or ranges for the trade intensity rates. For instance, let’s assume that the user suspects that an infrequently-traded stock has an average of uninformed trading intensity for buys and sells between `300` and `500`. They generate a single sample dataset for this scenario as follows:

    pindata <- generatedata_mpin(layers=1, ranges = list(eps.b=c(300, 500), eps.s=c(300,500)), verbose = FALSE)

The details of the generated sample dataset can be displayed with the following code

    show(pindata)

    ## ----------------------------------
    ## Data series successfully generated
    ## ----------------------------------
    ## Simulation model     : MPIN model
    ## Number of layers : 1 layer(s)
    ## Number of trading days   : 60 days
    ## ----------------------------------
    ## Type object@data to get the simulated data
    ##
    ##  Data simulation
    ##
    ## ===========  ==============  ============  =============
    ## Variables    Theoretical.    Empirical.    Aggregates.
    ## ===========  ==============  ============  =============
    ## alpha        0.462853        0.4           0.4
    ## delta        0.635296        0.625         0.625
    ## mu           259             260.27        260.27
    ## eps.b        465             464.98        464.98
    ## eps.s        413             417.64        417.64
    ## ----
    ## Likelihood   -               (602.573)     (602.573)
    ## mpin         -               0.105508      0.105508
    ## ===========  ==============  ============  =============
    ##
    ## -------
    ## Running time: 0.009 seconds

You access the sequences of buys, and sells through the slot `@data` of the object `pindata`.

    show(pindata@data[1:10, ])

    ##      b   s
    ## 1  495 429
    ## 2  455 424
    ## 3  494 673
    ## 4  471 713
    ## 5  493 630
    ## 6  742 396
    ## 7  493 451
    ## 8  482 630
    ## 9  484 397
    ## 10 444 435

You can, now use the dataset object `pindata` to check the accuracy of the different estimation functions. You can do that by comparing the actual parameters of the sample datasets to the estimated parameters of the estimation functions. Let us start with displaying the actual parameters of the sample datasets. These can be accessed through the slot `@empiricals` of the dataset object, which stores the empirical parameters computed from the sequences of buys and sells generated. Please refer to the documentation of `generatedata_mpin()` for more information.

    actual <- unlist(pindata@empiricals)
    show(actual)

    ##    alpha    delta       mu    eps.b    eps.s
    ##   0.4000   0.6250 260.2712 464.9804 417.6444

Estimate the PIN model using the function `pin_ea()`, and display the estimated parameters

    model <- pin_ea(data=pindata@data, verbose = FALSE)
    estimates <- model@parameters
    show(estimates)

    ##       alpha       delta          mu       eps.b       eps.s
    ##   0.3999997   0.6250055 260.2848636 465.3148049 417.3043139

Now calculate the absolute errors of the estimation method.

    errors <- abs(actual - estimates)
    show(errors)

    ##        alpha        delta           mu        eps.b        eps.s
    ## 2.803387e-07 5.455089e-06 1.362178e-02 3.344128e-01 3.401306e-01

## Sample datasets for the MPIN model

In contrast to the PIN model, the number of information layers is free. We can, therefore, use the function `generatedata_mpin()` with the desired number of information layers, in order to generate sample data for the MPIN model. We can also skip specifying the number of layers, and the default setting will be used: the number of layers will be randomly selected from the integer set from `1` to `5`. Generically, this is done as follows:

    generatedata_mpin(...)

If the user would like to create a sample dataset for frequently traded stock with two information layers, she can set the argument layers to 2, and specify high values or ranges for the trade intensity rates. For instance, let’s assume that the user suspects that a frequently-traded stock has an average of uninformed trading intensity for buys and sells between `12000` and `15000`. They generate a single sample dataset for this scenario as follows:

    mpindata <- generatedata_mpin(layers=2, ranges = list(eps.b=c(12000, 15000), eps.s=c(12000,15000)), verbose = FALSE)

The details of the generated sample dataset can be displayed with the following code

    show(mpindata)

    ## ----------------------------------
    ## Data series successfully generated
    ## ----------------------------------
    ## Simulation model     : MPIN model
    ## Number of layers : 2 layer(s)
    ## Number of trading days   : 60 days
    ## ----------------------------------
    ## Type object@data to get the simulated data
    ##
    ##  Data simulation
    ##
    ## ===========  ==================  ==================  =============
    ## Variables    Theoretical.        Empirical.          Aggregates.
    ## ===========  ==================  ==================  =============
    ## alpha        0.401875, 0.177702  0.383333, 0.183333  0.566667
    ## delta        0.130201, 0.433840  0.130435, 0.272727  0.176471
    ## mu           1263, 2546          1248.28, 2531.68    1663.5
    ## eps.b        13664               13677.59            13677.59
    ## eps.s        12714               12718.57            12718.57
    ## ----
    ## Likelihood   -                   (813.651)           (813.651)
    ## mpin         -                   0.03448             0.03448
    ## ===========  ==================  ==================  =============
    ##
    ## -------
    ## Running time: 0.004 seconds

You access the sequences of buys, and sells through the slot `@data` of the object `pindata`.

    show(mpindata@data[1:10, ])

    ##        b     s
    ## 1  13791 13909
    ## 2  14853 12806
    ## 3  13862 12723
    ## 4  13623 12623
    ## 5  13702 12707
    ## 6  15018 12837
    ## 7  14947 12936
    ## 8  14762 12594
    ## 9  13706 12556
    ## 10 14806 12828

You can, now use the dataset object `mpindata` to check the accuracy of the different estimation functions, namely `mpin_ml()`, and `mpin_ecm()`. You can do that by comparing the empirical PIN value derived from the sample dataset to the estimated PIN value of the estimation functions. Let us start with displaying the empirical PIN value obtained from the sample dataset. This value can be accessed through the slot `@emp.pin` of the dataset object, which stores the empirical PIN value computed from the sequences of buys and sells generated. Please refer to the documentation of `generatedata_mpin()` for more information.

    actualmpin <- unlist(mpindata@emp.pin)
    show(actualmpin)

    ##       MPIN
    ## 0.03448024

Estimate the MPIN model using the functions `mpin_ml()`, and `mpin_ecm`, and display the estimated MPIN values.

    model_ml <- mpin_ml(data=mpindata@data, verbose = FALSE)
    model_ecm <- mpin_ecm(data=mpindata@data, verbose = FALSE)
    mlmpin <- model_ml@mpin
    ecmpin <- model_ecm@mpin
    estimates <- setNames(c(mlmpin, ecmpin), c("ML", "ECM"))
    show(estimates)

    ##         ML        ECM
    ## 0.03454762 0.03454334

Now calculate the absolute errors of both estimation methods.

    errors <- abs(actualmpin - estimates)
    show(errors)

    ##           ML          ECM
    ## 6.738051e-05 6.309936e-05

The function `generatedata_mpin()` can generate a `data.series` object that contains a collection of `dataset` objects. For instance, the user can generate a collection of 10 datasets, whose data sequences span 60 days, and contain 3 layers, and use it to check the accuracy of the MPIN estimation.

    size <- 10
    collection <- generatedata_mpin(series = size, layers = 3, verbose = FALSE)
    show(collection)

    ## ----------------------------------
    ## Simulated data successfully generated
    ## ----------------------------------
    ## Simulation model     : MPIN model
    ## Number of layers : 3 layer(s)
    ## Number of datasets   : 10 datasets
    ## Number of trading days   : 60 days
    ## ----------------------------------
    ## Type object@datasets to access the list of dataset objects
    ##
    ##  Data simulation
    ##
    ## -------
    ## Running time: 0.048 seconds

    accuracy <- devmpin <- 0
    for (i in 1:size) {
        sdata <- collection@datasets[[i]]
        model <- mpin_ml(sdata@data, xtraclusters = 3, verbose=FALSE)
        accuracy <- accuracy + (sdata@layers == model@layers)
        devmpin <- devmpin + abs(sdata@emp.pin - model@mpin)

    }
    cat('The accuracy of layer detection: ', paste0(accuracy*(100/size),"%.\n"), sep="")

    ## The accuracy of layer detection: 90%.

    cat('The average error in MPIN estimates: ', devmpin/size, ".\n", sep="")

    ## The average error in MPIN estimates: 0.002195456.

## Sample datasets for the ADJPIN model

The AdjPIN model is an extension of the PIN model that includes the possibility of liquidity shocks. To obtain a sample dataset distributed according to the assumptions of the AdjPIN model, users can use the function `generatedata_adjpin()`. Generically, this is done as follows:

    generatedata_adjpin(...)

If the user desires to create 10 sample datasets for frequently traded stock, they can specify high values or ranges for the trade intensity rates. For instance, let’s assume that the user suspects that a frequently-traded stock has an average of uninformed trading intensity for buys and sells between `10000` and `15000`.

    adjpindatasets <- generatedata_adjpin(series = 10, ranges = list(eps.b=c(10000, 15000), eps.s=c(10000,15000)), verbose = FALSE)

The details of the generated sample data series can be displayed with the following code:

    show(adjpindatasets)

    ## ----------------------------------
    ## Simulated data successfully generated
    ## ----------------------------------
    ## Simulation model     : AdjPIN model
    ## Model Restrictions   : Unrestricted model
    ## Number of datasets   : 10 datasets
    ## Number of trading days   : 60 days
    ## ----------------------------------
    ## Type object@datasets to access the list of dataset objects
    ##
    ##  Data simulation
    ##
    ## -------
    ## Running time: 0.151 seconds

You access the first dataset from `adjpindata` using this code:

    adjpindata <- adjpindatasets@datasets[[1]]
    show(adjpindata)

    ## ----------------------------------
    ## Data series successfully generated
    ## ----------------------------------
    ## Simulation model     : AdjPIN model
    ## Model Restrictions   : Unrestricted model
    ## Number of trading days   : 60 days
    ## ----------------------------------
    ## Type object@data to get the simulated data
    ##
    ##  Data simulation
    ##
    ## ===========  ==============  ============
    ## Variables    Theoretical.    Empirical.
    ## ===========  ==============  ============
    ## alpha        0.624592        0.7
    ## delta        0.718889        0.571429
    ## theta        0.118567        0.166667
    ## theta'       0.606026        0.571429
    ## ----
    ## eps.b        11844           11853.15
    ## eps.s        10496           10495.63
    ## mu.b         57445           57373.47
    ## mu.s         65080           65020.57
    ## d.b          10031           9968.69
    ## d.s          10622           10612.54
    ## ----
    ## Likelihood                   (890.219)
    ## adjPIN       0.558           0.578
    ## PSOS         0.124           0.124
    ## ===========  ==============  ============
    ##
    ## -------
    ## Running time: 0.018 seconds

You can, now use the dataset object `adjpindata` to check the accuracy of the different estimation functions, namely MLE, and ECM algorithms. You can do that by comparing the empirical adjpin, and psos values derived from the sample dataset to the estimated adjpin, and psos values obtained from the estimation functions. Let us start with displaying the empirical adjpin, and psos values obtained from the sample dataset. These values can be accessed through the slot `@emp.pin` of the dataset object, which stores the empirical adjpin/psos value computed from the sequences of buys and sells generated. Please refer to the documentation of `generatedata_adjpin()` for more information.

    actualpins <- unlist(adjpindata@emp.pin)
    show(actualpins)

    ##    adjpin      psos
    ## 0.5775748 0.1237669

Estimate the AdjPIN model using `adjpin(method="ML")`, and `adjpin(method="ECM"`, and display the estimated adjpin/psos values.

    model_ml <- adjpin(data=adjpindata@data, method = "ML", verbose = FALSE)
    model_ecm <- adjpin(data=adjpindata@data, method = "ECM", verbose = FALSE)
    mlpins <- c(model_ml@adjpin, model_ml@psos)
    ecmpins <- c(model_ecm@adjpin, model_ecm@psos)
    estimates <- rbind(mlpins, ecmpins)
    colnames(estimates) <- c("adjpin", "psos")
    rownames(estimates) <- c("ML", "ECM")
    show(estimates)

    ##        adjpin      psos
    ## ML  0.5773024 0.1238676
    ## ECM 0.5773015 0.1238636

Now calculate the absolute errors of both estimation methods.

    errors <- abs(estimates - rbind(actualpins, actualpins))
    show(errors)

    ##           adjpin         psos
    ## ML  0.0002723645 1.006566e-04
    ## ECM 0.0002732715 9.671608e-05

## Getting help

If you encounter a clear bug, please file an issue with a minimal reproducible example on GitHub.