Metafrontier Malmquist Productivity Index

Motivation

Standard Malmquist productivity indices measure productivity change over time by decomposing it into efficiency change and technical change. When groups of firms face different restrictions of a common metatechnology, however, this decomposition misses an important dimension: changes in the technology gap between a group’s frontier and the global best practice.

The metafrontier Malmquist TFP index of O’Donnell, Rao, and Battese (2008) extends the standard index with a three-way decomposition that separates within-group dynamics from cross-group technology convergence or divergence.

The three-way decomposition

For a firm observed at times \(s\) and \(t\), the metafrontier Malmquist index decomposes as:

\[M^* = TEC \times TGC \times TC^*\]

where:

TEC (Technical Efficiency Change) \(= TE^{group}_t / TE^{group}_s\) measures whether the firm moved closer to or further from its own group’s frontier.
TGC (Technology Gap Change) \(= TGR_t / TGR_s\) measures whether the group’s frontier moved closer to or further from the metafrontier. A value above 1 indicates the group is catching up technologically.
TC* (Metafrontier Technical Change) measures the shift of the global production possibility frontier itself.

Values above 1 indicate improvement; below 1 indicate deterioration.

Simulating panel data

The simulate_metafrontier() function generates cross-sectional data. For panel data, we call it repeatedly with varying parameters to create time-varying technology gaps:

set.seed(42)
panels <- lapply(1:4, function(t) {
  sim <- simulate_metafrontier(
    n_groups = 2,
    n_per_group = 50,
    beta_meta = c(1.0, 0.5, 0.3),
    tech_gap = c(0, 0.3 + 0.03 * t),  # G2 falls behind over time
    sigma_u = c(0.2, 0.3),
    sigma_v = 0.15,
    seed = 42 + t
  )
  sim$data$time <- t
  sim$data$id <- seq_len(nrow(sim$data))
  sim$data
})
panel_data <- do.call(rbind, panels)

table(panel_data$group, panel_data$time)
#>     
#>       1  2  3  4
#>   G1 50 50 50 50
#>   G2 50 50 50 50

In this simulation, Group G1 operates at the metafrontier (zero technology gap), while G2 has an increasing gap over time. We would expect TGC < 1 for G2 (falling behind) and TGC \(\approx\) 1 for G1.

Computing the index

malm <- malmquist_meta(
  log_y ~ log_x1 + log_x2,
  data = panel_data,
  group = "group",
  time = "time",
  id = "id",
  orientation = "output",
  rts = "crs"
)

malm
#> 
#> Metafrontier Malmquist TFP Index
#> ================================
#> Method:       dea 
#> Orientation:  output 
#> RTS:          crs 
#> Groups:       G1, G2 
#> Periods:      1 -> 2 -> 3 -> 4 
#> Observations: 300 
#> 
#> Mean decomposition (M* = TEC x TGC x TC*):
#>   MPI  = 1.145 
#>   TEC  = 1.123 
#>   TGC  = 1.127 
#>   TC*  = 1.002

The id argument identifies firms across periods. Firms not observed in both periods of a consecutive pair are dropped from that comparison, and malmquist_meta() warns about how many observations were dropped.

Detailed results

The summary() method provides group-level and period-level breakdowns:

summary(malm)
#> 
#> Metafrontier Malmquist TFP Index Summary
#> =========================================
#> 
#> Call:
#> malmquist_meta(formula = log_y ~ log_x1 + log_x2, data = panel_data, 
#>     group = "group", time = "time", id = "id", orientation = "output", 
#>     rts = "crs")
#> 
#> Method:      dea 
#> Orientation: output 
#> RTS:         crs 
#> Groups:      G1, G2 
#> Periods:     1 -> 2 -> 3 -> 4 
#> Observations: 300 
#> 
#> Overall means:
#>   MPI  = 1.145 
#>   TEC  = 1.123 
#>   TGC  = 1.127 
#>   TC*  = 1.002 
#> 
#> --- Three-Way Decomposition by Group ---
#> M* = TEC x TGC x TC*
#> 
#> Group: G1 (n = 150 )
#>    MPI    TEC    TGC     TC 
#> 1.1806 1.2388 0.9833 1.0273 
#> 
#> Group: G2 (n = 150 )
#>    MPI    TEC    TGC     TC 
#> 1.1096 1.0082 1.2703 0.9767 
#> 
#> --- By Period ---
#> 
#> Period 1 -> 2 
#>    MPI    TEC    TGC     TC 
#> 1.3082 1.0360 1.3314 0.9567 
#> 
#> Period 2 -> 3 
#>    MPI    TEC    TGC     TC 
#> 1.0250 1.5288 1.0757 0.6300 
#> 
#> Period 3 -> 4 
#>    MPI    TEC    TGC     TC 
#> 1.1021 0.8056 0.9732 1.4194 
#> 
#> --- Technology Gap Ratios ---
#> 
#> Group: G1 
#>   Mean TGR (from): 0.987 
#>   Mean TGR (to):   0.9687 
#>   Mean TGC:        0.9833 
#> 
#> Group: G2 
#>   Mean TGR (from): 0.7011 
#>   Mean TGR (to):   0.8234 
#>   Mean TGC:        1.27

Interpreting the decomposition

The main results table contains one row per firm per consecutive period pair:

head(malm$malmquist, 10)
#>    id group period_from period_to       MPI       TEC       TGC        TC
#> 1   1    G1           1         2 2.5584365 2.7812965 1.0000000 0.9198719
#> 2   2    G1           1         2 1.6371319 2.2260475 0.8271005 0.8891826
#> 3   3    G1           1         2 0.5330948 0.6482012 0.9164504 0.8973991
#> 4   4    G1           1         2 1.0177377 1.1385028 0.9851380 0.9074124
#> 5   5    G1           1         2 2.6412214 2.3940135 1.0000000 1.1032609
#> 6   6    G1           1         2 1.3723118 1.5701415 1.0000000 0.8740052
#> 7   7    G1           1         2 0.6848389 0.8406678 1.0000000 0.8146368
#> 8   8    G1           1         2 1.7332279 1.8166178 1.0000000 0.9540961
#> 9   9    G1           1         2 1.0869101 1.1994245 1.0000000 0.9061930
#> 10 10    G1           1         2 1.3215333 1.7610854 0.8459739 0.8870349

Each row reports:

Column	Meaning
`MPI`	Metafrontier Malmquist TFP index (\(M^* = TEC \times TGC \times TC^*\))
`TEC`	Within-group efficiency change
`TGC`	Technology gap change
`TC`	Metafrontier technical change

The identity can be verified:

m <- malm$malmquist
complete <- complete.cases(m[, c("MPI", "TEC", "TGC", "TC")])
all.equal(m$MPI[complete], m$TEC[complete] * m$TGC[complete] * m$TC[complete])
#> [1] TRUE

Within-group vs metafrontier Malmquist

The object also stores the standard within-group Malmquist decomposition and the metafrontier-level decomposition:

# Within-group: MPI_group = EC_group x TC_group
head(malm$group_malmquist)
#>   id group period_from period_to MPI_group  EC_group  TC_group
#> 1  1    G1           1         2  6.164525 2.7812965 2.2164215
#> 2  2    G1           1         2  1.757668 2.2260475 0.7895912
#> 3  3    G1           1         2  0.556865 0.6482012 0.8590929
#> 4  4    G1           1         2  1.018773 1.1385028 0.8948361
#> 5  5    G1           1         2  2.641221 2.3940135 1.1032609
#> 6  6    G1           1         2  1.325650 1.5701415 0.8442872

# Metafrontier: MPI_meta = EC_meta x TC_meta
head(malm$meta_malmquist)
#>   id group period_from period_to  MPI_meta   EC_meta   TC_meta
#> 1  1    G1           1         2 2.5584365 2.7812965 0.9198719
#> 2  2    G1           1         2 1.6371319 1.8411649 0.8891826
#> 3  3    G1           1         2 0.5330948 0.5940443 0.8973991
#> 4  4    G1           1         2 1.0177377 1.1215823 0.9074124
#> 5  5    G1           1         2 2.6412214 2.3940135 1.1032609
#> 6  6    G1           1         2 1.3723118 1.5701415 0.8740052

The within-group index captures only efficiency change and frontier shift within the group. The metafrontier index additionally accounts for whether the group is converging toward or diverging from the global best practice.

Technology gap dynamics

The TGR at each period endpoint is stored in the tgr component:

tgr_df <- malm$tgr

# Mean TGR by group and period
aggregate(cbind(TGR_from, TGR_to) ~ group, data = tgr_df, FUN = mean)
#>   group  TGR_from    TGR_to
#> 1    G1 0.9869568 0.9686967
#> 2    G2 0.7010526 0.8233954

For G2, we expect TGR to decline over time (increasing technology gap). The TGC column confirms this:

aggregate(TGC ~ group, data = tgr_df, FUN = mean)
#>   group       TGC
#> 1    G1 0.9832828
#> 2    G2 1.2702686

Returns to scale assumptions

The rts argument controls the DEA technology assumption. Under variable returns to scale, scale effects are netted out:

malm_vrs <- malmquist_meta(
  log_y ~ log_x1 + log_x2,
  data = panel_data,
  group = "group",
  time = "time",
  id = "id",
  rts = "vrs"
)
#> Warning: 68 of 1200 cross-period DEA programs were infeasible (rts = "vrs");
#> the affected TC and MPI values are NA and are excluded from reported means.

# Compare mean MPI under CRS vs VRS
data.frame(
  CRS = colMeans(malm$malmquist[, c("MPI", "TEC", "TGC", "TC")],
                 na.rm = TRUE),
  VRS = colMeans(malm_vrs$malmquist[, c("MPI", "TEC", "TGC", "TC")],
                 na.rm = TRUE)
)
#>          CRS       VRS
#> MPI 1.145084 1.0121064
#> TEC 1.123475 1.0049310
#> TGC 1.126776 1.0142854
#> TC  1.002026 0.9998827

Using real-world panel data

The plm package provides Produc, a panel of 48 US states over 1970–1986 with a built-in region grouping variable. This is a natural candidate for metafrontier Malmquist analysis:

library(plm)
data("Produc", package = "plm")

malm_us <- malmquist_meta(
  gsp ~ pc + emp,
  data = Produc,
  group = "region",
  time = "year",
  id = "state",
  rts = "crs"
)
summary(malm_us)

Similarly, sfaR::utility provides electric utility data with a binary regu variable (retail-access vs. traditionally regulated states) over 1986–1996, where regu = 1 indicates that the state had enacted retail-access (restructuring) legislation:

library(sfaR)
data("utility", package = "sfaR")

malm_util <- malmquist_meta(
  y ~ k + labor + fuel,
  data = utility,
  group = "regu",
  time = "year",
  id = "firm",
  rts = "vrs"
)
summary(malm_util)

Caveats

A few limitations should be kept in mind when interpreting the results.

Interpretation of the index

The Malmquist index, and by extension its metafrontier version, is not a proper index in the sense of O’Donnell (2012): it is not multiplicatively complete, and its binary comparisons are not transitive, so chained comparisons across several periods depend on the path taken. Interpretation should therefore be confined to binary period-to-period comparisons; avoid cumulating the indices into a multi-period productivity level series.

Cross-period infeasibility

Under rts = "vrs", "drs", "irs", or "fdh", the cross-period distance functions (evaluating a period-\(t\) observation against the period-\(s\) frontier) may have no feasible solution. Infeasible cases are returned as NA (never Inf), counted, and reported, so summary statistics over the remaining observations should be read with the reduced sample in mind. Under rts = "crs" the programmes are always feasible.

Firm matching

Observations are matched across consecutive periods via the id variable. Firms observed in only one period of a pair are dropped from that comparison, and malmquist_meta() warns about the number of dropped observations. With substantial attrition or entry, the index is computed on the surviving matched sample only.

SFA-based index

With method = "sfa", the metafrontier at each period is formed as the pointwise maximum of the fitted group frontiers, an approximation that the function announces with a message. For exact decompositions, prefer the default method = "dea".

References

O’Donnell, C.J. (2012). An aggregate quantity framework for measuring and decomposing productivity change. Journal of Productivity Analysis, 38(3), 255–272.
O’Donnell, C.J., Rao, D.S.P. and Battese, G.E. (2008). Metafrontier frameworks for the study of firm-level efficiencies and technology ratios. Empirical Economics, 34(2), 231–255.
Battese, G.E., Rao, D.S.P. and O’Donnell, C.J. (2004). A metafrontier production function for estimation of technical efficiencies and technology gaps for firms operating under different technologies. Journal of Productivity Analysis, 21(1), 91–103.