Quick Start

Data Science Looks at Discrimination

DSLD provides statistical, graphical, and tabular methods for exploring possible discrimination and bias. It supports two related questions:

We use Y for the outcome, S for a sensitive attribute, and X for other predictors. Many functions wrap base R, qeML, or fairml methods and add group comparisons or fairness diagnostics. Their first arguments usually identify the data frame, outcome column (yName), and sensitive column (sName or sNames). Use summary() and predict() for the supported fitted-model classes.

Analytical operations

Group comparisons with confounder adjustment

The lsa dataset, supplied by qeML, contains law-school admissions and outcome variables. We use LSAT score as Y, race as S, and the remaining columns as covariates. Some covariates are measured after the LSAT, so this is an illustration of the software rather than a causal model of test-score differences.

data(lsa, package = "qeML")
names(lsa)
#>  [1] "age"      "decile1"  "decile3"  "fam_inc"  "lsat"     "ugpa"    
#>  [7] "gender"   "race1"    "cluster"  "fulltime" "bar"
lsat_model <- dsldLinear(lsa, "lsat", "race1")
summary(lsat_model)
#> $`Summary Coefficients`
#>      Covariate    Estimate StandardError       PValue
#> 1  (Intercept) 31.98578856   0.448435264 0.000000e+00
#> 2          age  0.02082458   0.005841758 3.641634e-04
#> 3      decile1  0.12754812   0.020946536 1.134602e-09
#> 4      decile3  0.21495015   0.020918737 0.000000e+00
#> 5      fam_inc  0.30085804   0.035953051 0.000000e+00
#> 6         ugpa -0.27817274   0.080430542 5.430993e-04
#> 7   gendermale  0.51377385   0.060037102 0.000000e+00
#> 8   race1black -4.74826307   0.198088318 0.000000e+00
#> 9    race1hisp -2.00145969   0.203504412 0.000000e+00
#> 10  race1other -0.86803104   0.262528590 9.449471e-04
#> 11  race1white  1.24708760   0.154627086 6.661338e-16
#> 12    cluster2 -5.10668358   0.119798362 0.000000e+00
#> 13    cluster3 -2.43613709   0.074744210 0.000000e+00
#> 14    cluster4  1.21094567   0.088478368 0.000000e+00
#> 15    cluster5  3.79427535   0.124476695 0.000000e+00
#> 16    cluster6 -5.53216090   0.210750853 0.000000e+00
#> 17   fulltime2 -1.38882076   0.116212777 0.000000e+00
#> 18     barTRUE  1.74973262   0.102818692 0.000000e+00
#> 
#> $`Sensitive Factor Level Comparisons`
#>    Factors Compared Estimates Standard Errors       P-Value
#> 1     asian - black  4.748263       0.1980883 5.658131e-127
#> 2      asian - hisp  2.001460       0.2035044  7.959167e-23
#> 3     asian - other  0.868031       0.2625286  9.449471e-04
#> 4     asian - white -1.247088       0.1546271  7.315779e-16
#> 5      black - hisp -2.746803       0.1863750  3.672659e-49
#> 6     black - other -3.880232       0.2515488  1.105293e-53
#> 7     black - white -5.995351       0.1409991  0.000000e+00
#> 8      hisp - other -1.133429       0.2562971  9.764506e-06
#> 9      hisp - white -3.248547       0.1457509 4.792757e-110
#> 10    other - white -2.115119       0.2194472  5.504309e-22

The summary reports model coefficients and pairwise sensitive-group differences, including their standard errors. Factor columns are represented using the fitted model’s contrasts. For two groups, the comparison is the second factor level minus the first; for more groups, the labels state each difference’s direction.

For example, the family-income coefficient describes the adjusted change in expected LSAT score per income-category step. The race coefficients and group comparisons describe adjusted associations under this model. Their practical size and uncertainty matter alongside their p-values; these associations alone do not establish discrimination or its cause.

Setting interactions = TRUE fits a separate model for each sensitive group. Supply sComparisonPts to compare these fitted models at specific predictor values. The repository README demonstrates this with wage income and age. useSandwich = TRUE requests a sandwich covariance estimate for linear models.

Fair prediction

A predictor can carry information about S even when S is omitted from the model. Fair-prediction methods limit this influence through constraints or feature deweighting. Evaluate prediction error and fairness diagnostics together: a stronger constraint does not guarantee improvement in every measure.

The bundled compas1 data include recidivism outcomes and demographic variables. We illustrate dsldFgrrm(), a wrapper for fairml’s generalized ridge method, using a reproducible subset of two racial groups. unfairness lies in (0, 1]; smaller values impose a stronger fairness constraint under the selected definition. Always identify the positive outcome using yesYVal.

data(compas1, package = "dsld")
records <- droplevels(subset(compas1, race %in% c("African-American", "Caucasian")))
records <- records[, c("age", "priors_count", "sex", "two_year_recid", "race")]
records <- records[sample.int(nrow(records), 800), ]
train_rows <- sample.int(nrow(records), 600)
training <- records[train_rows, ]
testing <- records[-train_rows, ]
test_x <- testing[, setdiff(names(testing), "two_year_recid")]

baseline <- dsldFgrrm(training, "two_year_recid", "race", unfairness = 1,
                      yesYVal = "Yes")
constrained <- dsldFgrrm(training, "two_year_recid", "race", unfairness = 0.1,
                         yesYVal = "Yes")
baseline_result <- predict(baseline, test_x)
constrained_result <- predict(constrained, test_x)
data.frame(
  model = c("Baseline", "Constrained"),
  test_error = c(
    mean((baseline_result$preds > 0.5) != (testing$two_year_recid == "Yes")),
    mean((constrained_result$preds > 0.5) != (testing$two_year_recid == "Yes"))
  )
)
#>         model test_error
#> 1    Baseline      0.330
#> 2 Constrained      0.335
constrained_result$correlations
#>                  feature correlation
#> 1        race==Caucasian -0.02603029
#> 2 race==African-American  0.02603029

These probabilities describe the specified positive outcome. Correlations measure association with sensitive attributes; their absolute magnitudes are useful for comparison, but zero correlation is not a general fairness guarantee. The historical fields trainAcc and testAcc contain errors, not accuracy: mean absolute error for regression and misclassification rate for classification.

Graphical and tabular operations

Conditional disparity

dsldConditDisparity() plots an outcome against a numeric predictor, separately by sensitive group. It can restrict rows through condits and omit groups with fewer than minS rows. The binary plotting interface uses the first outcome factor level as its positive class.

law_sample <- lsa[sample.int(nrow(lsa), 1500), ]
levels(law_sample$bar)
#> [1] "FALSE" "TRUE"
dsldConditDisparity(law_sample, "bar", "race1", "lsat", minS = 50)

Inspect the outcome coding, group sample sizes, and uncertainty before drawing substantive conclusions from the curves.

Group frequencies and possible confounders

The bundled svcensus data describe workers in engineering occupations in Silicon Valley. Compare education and occupation distributions by gender:

data(svcensus, package = "dsld")
names(svcensus)
#> [1] "age"     "educ"    "occ"     "wageinc" "wkswrkd" "gender"
dsldFrequencyByS(svcensus, cName = "educ", sName = "gender")
#>        Frequency of 14 Frequency of 16 Frequency of zzzOther
#> female       0.2068052      0.02098615             0.7722086
#> male         0.2177579      0.04110130             0.7411408
dsldFrequencyByS(svcensus, cName = "occ", sName = "gender")
#>        Frequency of 100 Frequency of 101 Frequency of 102 Frequency of 106
#> female        0.3117359        0.2349226        0.3274246       0.04258354
#> male          0.2016862        0.2203267        0.3433671       0.01923330
#>        Frequency of 140 Frequency of 141
#> female       0.02587612       0.05745721
#> male         0.04446055       0.17092610
tapply(svcensus$wageinc, svcensus$occ, mean)
#>      100      101      102      106      140      141 
#> 50396.47 51373.53 68797.72 53639.86 67019.26 69494.44

Columns follow the factor levels shown in the table labels. Education codes 14 and 16 represent master’s and doctoral degrees. Differences in occupation frequencies and mean wages can motivate examining occupation as a possible confounder. Selecting adjustment variables still requires substantive reasoning; similar group frequencies alone do not rule a variable in or out.

dsldDensityByS() compares numeric distributions. dsldConfounders() applies frequency or density comparisons to the other columns in a data frame.

Frequency-based parallel coordinates

Parallel coordinates place each variable on a separate vertical axis and join the values belonging to one observation. dsldFreqPCoord() can display the most typical patterns within each sensitive group, using nearest-neighbor density estimates. The axes are centered and scaled.

plot_data <- law_sample[, c("fam_inc", "ugpa", "gender", "lsat", "race1")]
dsldFreqPCoord(plot_data, m = 25, sName = "race1", k = 15)

This view can reveal multivariate patterns that separate plots miss. Its density and sampling settings affect which observations appear; it does not summarize every member of a group.

Comparing feature sets

dsldTakeALookAround() compares predictor subsets C. It reports:

The same held-out rows are used for all feature sets. Lower (c) means S is more predictable from C, which can indicate stronger proxy information. Interpret these errors relative to suitable outcome baselines.

wages <- svcensus[sample.int(nrow(svcensus), 800),
                   c("age", "educ", "wkswrkd", "gender", "wageinc")]
dsldTakeALookAround(wages, "wageinc", "gender", maxFeatureSetSize = 2, holdout = 100)
#> Warning in `[<-.factor`(`*tmp*`, ri, value = 3L): invalid factor level, NA
#> generated
#> Warning in `[<-.factor`(`*tmp*`, ri, value = 3L): invalid factor level, NA
#> generated
#>   Feature Names        a        b   c
#> 1           age 31517.44 30329.07 0.3
#> 2          educ 30587.71 29712.81 0.3
#> 3       wkswrkd 26729.35 26384.64 0.3
#> 4      age,educ 30126.36 29465.02 0.3
#> 5   age,wkswrkd 26667.38 26223.17 0.3
#> 6  educ,wkswrkd 27085.18 26776.01 0.3

Python interface

Install the R package first, then run python -m pip install ./inst from the repository root. The Python guide includes installation instructions and three notebooks covering data conversion, analysis, fair prediction, and graphics.

import rpy2.robjects as ro
from rpy2.robjects.packages import importr
from dsldPy import dsldPyLinear, dsldPyLinearSummary

bundled = ro.Environment()
importr("utils").data("svcensus", package="dsld", envir=bundled)
model = dsldPyLinear(bundled["svcensus"], "wageinc", "gender")
dsldPyLinearSummary(model)

Associated textbook and function reference

The free companion textbook explains the statistical concepts using DSLD examples. For the categorized function reference, run vignette("Function_List", package = "dsld").