Matches only

The case match-stream was first built for: greedy 1:1 nearest-neighbor matching on the propensity score, without replacement, within exact strata, with calipers on the score and on two covariates, and no outcome. The result is the pairs, written to Parquet.

The propensity score is the fitted probability of treatment from a logit of d on x1 + x2 + x3. The model absorbs a fixed effect (| i), so it is fitted out of core by feglm_stream. | is never used for exact strata: those go in exact=. To match on the covariates themselves instead, see Mahalanobis matching; for the other methods, see Choosing a matching method.

import tempfile
from pathlib import Path

import polars as pl

from match_stream import match_nn, simulate_matching

pl.Config.set_tbl_hide_dataframe_shape(True)
df = simulate_matching(100_000)

m = match_nn(
    "d ~ x1 + x2 + x3", df,
    exact=["r"],                                  # match within r
    score_model="d ~ x1 + x2 + x3 | i",                # logit with i absorbed
    caliper={"ps": 0.2, "x1": 0.25, "x2": 0.25},  # ps in SDs of the score; x1, x2 raw
    match_order="largest",                            # the default for a score
    id="id", verbose=False)
m.summary()
print(m.counts())

out = Path(tempfile.mkdtemp()) / "pairs.parquet"
m.matches().sink_parquet(out)                     # lazy: streams to disk
print(f"\n{pl.scan_parquet(out).select(pl.len()).collect().item():,} pairs written")
print(pl.read_parquet(out).head(3))

Output:

match-stream: nn, 1:1 without replacement, estimand ATT, order largest
  distance: ps (score model: d ~ x1 + x2 + x3 | i, logit)
  exact: r (5 strata)
  calipers: ps 0.2 SD, x1 0.25, x2 0.25
               available     matched   unmatched   discarded
  treated         25,497      24,610         887           0
  control         74,503      24,610      49,893           0
{'treated': 25497, 'control': 74503, 'matched_treated': 24610, 'matched_control': 24610, 'unmatched_treated': 887, 'unmatched_control': 49893, 'discarded_treated': 0, 'discarded_control': 0, 'pairs': 24610, 'dropped_missing': 0}

24,610 pairs written
┌─────────────┬─────────────┬────────────┬────────────┬──────┬──────────┬──────────┐
│ treated_row ┆ control_row ┆ treated_id ┆ control_id ┆ rank ┆ distance ┆ subclass │
│ ---         ┆ ---         ┆ ---        ┆ ---        ┆ ---  ┆ ---      ┆ ---      │
│ i64         ┆ i64         ┆ i64        ┆ i64        ┆ i32  ┆ f64      ┆ i64      │
╞═════════════╪═════════════╪════════════╪════════════╪══════╪══════════╪══════════╡
│ 10          ┆ 49531       ┆ 11         ┆ 49532      ┆ 1    ┆ 0.001206 ┆ 1        │
│ 15          ┆ 18737       ┆ 16         ┆ 18738      ┆ 1    ┆ 0.000139 ┆ 2        │
│ 19          ┆ 86024       ┆ 20         ┆ 86025      ┆ 1    ┆ 0.000214 ┆ 3        │
└─────────────┴─────────────┴────────────┴────────────┴──────┴──────────┴──────────┘

Notes.

  • Calipers. The score caliper is in standard deviations of the score over all units; covariate calipers are in raw units. Write (width, "sd") or (width, "raw") to change a caliper’s units.

  • Order. Treated units are matched in match_order. The default is “largest” for a score: the highest-scoring units choose first.

  • Covariate calipers. A control that fails one is skipped and the next nearest tried; the treated unit is not dropped.

  • Output. matches() is a LazyFrame, so sink_parquet streams it.