# Embedding R/exams Exercises as Forms in R/Markdown or Quarto Documents

## Overview

The package `exams2forms` provides several building blocks for embedding exercises written with the R package exams (also known as R/exams) in interactive documents or quizzes written with rmarkdown or quarto.

The idea is that the dynamic exercises in R/exams' Rmd (R/Markdown) or Rnw (R/LaTeX) format can also be reused in HTML documents, web pages, or online books. This facilitates their use for self-paced learning and self-assessment without the need for a learning management system etc. (By default the correct answers are obfuscated in the documents so that they are not obvious when inspecting the HTML source code.)

For (summative) assessments the same dynamic exercises could then be exported to different learning management systems or employed in written exams.

All R/exams exercise types are supported:

- Single-choice (schoice).
- Multiple-choice (mchoice).
- Numeric (num).
- Text (string).
- Cloze combining all of the previous elements (cloze).

Many of the ideas as well as the code in the package have been adapted from the webexercises package, authored by Dale Barr and Lisa DeBruine.

## First examples

As a quick demonstration for R/exams exercises embedded into an HTML document, the two examples from the First Steps tutorial are included below: The single-choice exercise swisscapital and the numeric exercise deriv, both in three random variations.

What is the seat of the federal authorities in Switzerland (i.e., the de facto capital)?

Geneva Lausanne St. Gallen Bern Vaduz

There is no de jure capital but the de facto capital and seat of the federal authorities is Bern.

- False
- False
- False
- True
- False

What is the seat of the federal authorities in Switzerland (i.e., the de facto capital)?

Lausanne Basel Geneva St. Gallen Bern

There is no de jure capital but the de facto capital and seat of the federal authorities is Bern.

- False
- False
- False
- False
- True

What is the seat of the federal authorities in Switzerland (i.e., the de facto capital)?

Bern Geneva St. Gallen Basel Vaduz

There is no de jure capital but the de facto capital and seat of the federal authorities is Bern.

- True
- False
- False
- False
- False

What is the derivative of \(f(x) = x^{2} e^{3.2 x}\), evaluated at \(x = 0.59\)?

Using the product rule for \(f(x) = g(x) \cdot h(x)\), where \(g(x) := x^{2}\) and \(h(x) := e^{3.2 x}\), we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 2 x^{2 - 1} \cdot e^{3.2 x} + x^{2} \cdot e^{3.2 x} \cdot 3.2 \\ &= e^{3.2 x} \cdot(2 x^1 + 3.2 x^{2}) \\ &= e^{3.2 x} \cdot x^1 \cdot (2 + 3.2 x). \end{aligned} \] Evaluated at \(x = 0.59\), the answer is \[ e^{3.2 \cdot 0.59} \cdot 0.59^1 \cdot (2 + 3.2 \cdot 0.59) = 15.153964. \] Thus, rounded to two digits we have \(f'(0.59) = 15.15\).

What is the derivative of \(f(x) = x^{4} e^{3.5 x}\), evaluated at \(x = 0.65\)?

Using the product rule for \(f(x) = g(x) \cdot h(x)\), where \(g(x) := x^{4}\) and \(h(x) := e^{3.5 x}\), we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 4 x^{4 - 1} \cdot e^{3.5 x} + x^{4} \cdot e^{3.5 x} \cdot 3.5 \\ &= e^{3.5 x} \cdot(4 x^3 + 3.5 x^{4}) \\ &= e^{3.5 x} \cdot x^3 \cdot (4 + 3.5 x). \end{aligned} \] Evaluated at \(x = 0.65\), the answer is \[ e^{3.5 \cdot 0.65} \cdot 0.65^3 \cdot (4 + 3.5 \cdot 0.65) = 16.763849. \] Thus, rounded to two digits we have \(f'(0.65) = 16.76\).

What is the derivative of \(f(x) = x^{7} e^{2.8 x}\), evaluated at \(x = 0.72\)?

Using the product rule for \(f(x) = g(x) \cdot h(x)\), where \(g(x) := x^{7}\) and \(h(x) := e^{2.8 x}\), we obtain \[ \begin{aligned} f'(x) &= [g(x) \cdot h(x)]' = g'(x) \cdot h(x) + g(x) \cdot h'(x) \\ &= 7 x^{7 - 1} \cdot e^{2.8 x} + x^{7} \cdot e^{2.8 x} \cdot 2.8 \\ &= e^{2.8 x} \cdot(7 x^6 + 2.8 x^{7}) \\ &= e^{2.8 x} \cdot x^6 \cdot (7 + 2.8 x). \end{aligned} \] Evaluated at \(x = 0.72\), the answer is \[ e^{2.8 \cdot 0.72} \cdot 0.72^6 \cdot (7 + 2.8 \cdot 0.72) = 9.430757. \] Thus, rounded to two digits we have \(f'(0.72) = 9.43\).

In addition to the question and the interaction element, there are three buttons providing the following functionality.

Button | Function
--- | ---
✓ | Check the answer and display whether it is correct or not. When clicked, the symbol is toggled and ↰ is displayed, which can be clicked to hide the solution again.
? | Display the full correct solution explanation.
↺ | Switch to the next question.

Inclusion of the solution explanation is optional and the next question button is only displayed if there is more than one random variation of a question. The icons and mouseover text can also be adapted (see below).

To set up a similar standalone file with these two exercises, the `exams2webquiz()` interface from the `exams2forms` package can be used:

    library("exams2forms")
    exams2webquiz(c("swisscapital.Rmd", "deriv.Rmd"), n = 3)

## More elaborate examples

To showcase some more exercise types, the following examples from the R/exams package are used: capitals (multiple-choice), function (string/text), fruit (numeric with table and images), lm2 (cloze containing string, multiple-choice, numeric, and single-choice elements as well as an embedded data file).

Which of the following cities are the capital of the corresponding country?

Tokyo (Japan)Warsaw (Poland)Auckland (New Zealand)Astana (Kazakhstan)Zürich (Switzerland)

- True. Tokyo is the capital of Japan.
- True. Warsaw is the capital of Poland.
- False. The capital of New Zealand is Wellington.
- True. Astana is the capital of Kazakhstan.
- False. The de facto capital of Switzerland is Bern.

What is the name of the R function for negative binomial regression?

`glm.nb` is the R function for negative binomial regression. See `?glm.nb` for the corresponding manual page.

Given the following information:

| \(+\) |  | \(+\) |  | = | \(143\)
| \(+\) |  | \(+\) |  | = | \(437\)
| \(+\) |  | \(+\) |  | = | \(433\)

Compute:

| \(+\) |  | \(+\) |  | = | \(\text{?}\)

The information provided can be interpreted as the price for three fruit baskets with different combinations of the three fruits. This corresponds to a system of linear equations where the price of the three fruits is the vector of unknowns \(x\):

\(x_1 =\) |  | \(x_2 =\) |  | \(x_3 =\) |

The system of linear equations is then: \[ \begin{aligned} \left( \begin{array}{rrr} 1 & 2 & 0 \\ 2 & 0 & 1 \\ 0 & 2 & 1 \end{array} \right) \cdot \left( \begin{array}{r} x_1 \\ x_2 \\ x_3 \end{array} \right) & = & \left( \begin{array}{r} 143 \\ 437 \\ 433 \end{array} \right) \end{aligned} \] This can be solved using any solution algorithm, e.g., elimination: \[ x_1 = 49, \, x_2 = 47, \, x_3 = 339. \] Based on the three prices for the different fruits it is straightforward to compute the total price of the fourth fruit basket via:

| \(+\) |  | \(+\) |  | = |
\(x_1\) | \(+\) | \(x_2\) | \(+\) | \(x_3\) | = |
\(49\) | \(+\) | \(47\) | \(+\) | \(339\) | = | \(435\)

Theory: Consider a linear regression of `y` on `x`. It is usually estimated with which estimation technique (three-letter abbreviation)?

This estimator yields the best linear unbiased estimator (BLUE) under the assumptions of the Gauss-Markov theorem. Which of the following properties are required for the errors of the linear regression model under these assumptions?

independent zero expectation normally distributed identically distributed homoscedastic

Application: Using the data provided in linreg.csv estimate a linear regression of `y` on `x`. What are the estimated parameters?

Intercept:

Slope:

In terms of significance at 5% level:

Theory: Linear regression models are typically estimated by ordinary least squares (OLS). The Gauss-Markov theorem establishes certain optimality properties: Namely, if the errors have expectation zero, constant variance (homoscedastic), no autocorrelation and the regressors are exogenous and not linearly dependent, the OLS estimator is the best linear unbiased estimator (BLUE).

Application: The estimated coefficients along with their significances are reported in the summary of the fitted regression model, showing that `x` and `y` are not significantly correlated (at 5% level).

    Call:
    lm(formula = y ~ x, data = d)

    Residuals:
         Min       1Q   Median       3Q      Max
    -0.59846 -0.17142 -0.01458  0.13740  0.68566

    Coefficients:
                Estimate Std. Error t value Pr(>|t|)
    (Intercept)  0.03055    0.02507   1.219    0.226
    x           -0.03080    0.04702  -0.655    0.514

    Residual standard error: 0.2504 on 98 degrees of freedom
    Multiple R-squared:  0.00436,   Adjusted R-squared:  -0.005799
    F-statistic: 0.4292 on 1 and 98 DF,  p-value: 0.5139

Code: The analysis can be replicated in R using the following code.

    ## data
    d <- read.csv("linreg.csv")
    ## regression
    m <- lm(y ~ x, data = d)
    summary(m)
    ## visualization
    plot(y ~ x, data = d)
    abline(m)

Again, the `exams2webquiz()` function can be used to set up a standalone file based on the same exercises:

    exams2webquiz(c("capitals.Rmd", "function.Rmd", "fruit.Rmd", "lm2.Rmd"))

## Building blocks

To accomplish the functionality demonstrated above, the package provides the following contents:

- `exams2forms()`: Main workhorse function from the package. Like other `exams2xyz()` interfaces this takes a vector or list of exercise files and returns Markdown text, including HTML snippets, than can be included in `rmarkdown` or `quarto` documents. This includes questions, suitable interactions for the different types of answers (with correct solutions also suitably encoded in the HTML), and optionally full solution explanations.
- `webex.css`, `webex.js`: CSS (Cascading Style Sheets) and Javascript files shipped within the package and providing the code necessary for the exercise and quiz display and processing the different types of interactions.
- `webquiz()`: Small wrapper function that creates a `knitr::html_document()` but includes the CSS and Javascript files above.
- `exams2webquiz()`: Convenience interface that combines all of the above elements. It sets up a `webquiz()` document in which `exams2forms()` is used to embed the specified exercises, calls `rmarkdown::render()` to process it, and by default displays it in the browser. This is most useful for quickly trying out how R/exams exercises can work in HTML documents.
- `forms_num()`, `forms_string()`, `forms_schoice()`, `forms_mchoice()`: Helper functions for just embedding the user interactions for the different types of exercises. Typically not called directly by the user.

## Demo files

While `exams2webquiz()` is convenient for quickly setting up an HTML document containing certain exercises, further customizations are typicallly needed for more elaborate documents. To demonstrate how this works the `exams2forms` package provides two demo `rmarkdown` files which can also be downloaded here:

- Download quiz.Rmd.
- Download questions.Rmd.

The first file `quiz.Rmd` renders a number of different exercises into a quiz using a single `exams2forms()` call, indicating that the `results` should be included `"asis"`:

    ---
    title: "R/exams quiz"
    output: exams2forms::webquiz
    ---

    ```{r setup, include = FALSE}
    ## package and list of various exercises
    library("exams2forms")
    exm <- list(
      c("swisscapital.Rmd", "capitals.Rmd"),
      "deriv.Rnw",
      "deriv2.Rnw",
      "fruit.Rmd",
      "boxplots.Rmd",
      "ttest.Rmd",
      "function.Rmd",
      "lm2.Rnw",
      "fourfold2.Rmd"
    )
    ```

    ```{r quiz, echo = FALSE, message = FALSE, results = "asis"}
    exams2forms(exm, n = 3)
    ```

The `questions.Rmd` file looks similar but contains different sections, each with a single question set up via `exams2forms()`.

Both files can be rendered to HTML via `rmarkdown::render()` or by clicking the knit button after opening the files in RStudio etc.

## Setup and customization

When setting up a more elaborate document or even a full webpage or online book with `rmarkdown` or `quarto`, then the simple `webquiz()` HTML document provided by `exams2forms` is probably not sufficient. In this case, it is best to take the CSS and Javascript files from the package or download them here:

- Download webex.css.
- Download webex.js.

The files can then be placed in the same folder as the `rmarkdown` or `quarto` project. They can also be adapted relatively easily by changing the definitions of colors, icons, text, etc. in the first few lines of each file.

To include the CSS and Javascript in an `rmarkown` project, the YAML header should include:

    output:
      html_document:
        css: webex.css
        includes:
          after_body: webex.js

Similarly, in a `quarto` project the YAML header or the `_quarto.yml` file should include:

    format:
      html:
        css: webex.css
        include-after-body: webex.js

## Standalone interaction forms

While it is not the primary focus of the `exams2forms` package, it is also possible to directly include interaction forms in documents like in the `webexercises` package, i.e., without setting up full R/exams exercises. A few simple examples are inlcuded below for numeric, text, single-choice, and multiple-choice interactions (both using drop-down interactions here), respectively.

What is the answer to the ultimate question of life, the universe, and everything?

Which superhero is the secret identity of Bruce Wayne?

Which of the following villains is not an adversary of Batman?

Which of the following characters are romantic interests of Spider-Man?

- Mary Jane Watson
- Pepper Potts
- Selina Kyle
- Gwen Stacy

The corresponding code snippets included in the inline code chunks are:

- `forms_num(42, width = 10)`
- `forms_string("Batman", width = 20, usecase = FALSE)`
- `forms_schoice(c("Bane", "Riddler", "Thanos", "Poison Ivy"), c(FALSE, FALSE, TRUE, FALSE), display = "dropdown")`
- `forms_mchoice(c("Mary Jane Watson", "Pepper Potts", "Selina Kyle", "Gwen Stacy"), c(TRUE, FALSE, FALSE, TRUE), display = "dropdown")`