Matching

Three functions, one per kind of matching. They share the data and formula arguments, exact strata, the score, memory and thread options, and profile=True.

match_stream.match_nn(fml, data, *, exact=None, distance='ps', score=None, score_model=None, score_link='logit', n_matches=1, replace=False, match_order=None, seed=None, caliper=None, focal='treated', id=None, keep_ties=False, tie_tolerance=0.0, min_matches=1, match_in_rounds=True, discard='none', reestimate_score=False, mahalanobis_vars=None, memory='auto', workdir=None, max_memory_gb=None, memory_reserve=None, n_threads=None, profile=False, verbose=True)[source]

Nearest-neighbor matching, out of core: each focal unit takes its nearest other-arm units, on a propensity score or a multivariate distance.

Matching only: no outcome is needed, and the result holds the pairs and weights. Where conventions differ between matching packages, the default is R’s MatchIt’s; “Conventions and other implementations” in the documentation lists the differences.

Parameters:

  • fml (str) – "treat ~ x1 + x2": the treatment (0/1 or boolean) and the matching covariates (column names). The covariates are the default score model and the balance table’s rows. Exact strata do not go here: | means absorbed fixed effects in this package family, so it is allowed only in score_model.

  • data (str, Path, polars.LazyFrame or polars.DataFrame) – A Parquet path or glob (never fully loaded), or a frame.

  • exact (column name or list of names, optional) – Exact-matching strata: units match only within cells of these columns. "a^b" is the same as ["a", "b"].

  • distance ({"ps", "logit_ps", "mahalanobis", "ivariance", "euclidean"}) – What to match on. "ps": the propensity score match-stream fits (score_model, score_link); "logit_ps": its linear predictor; "mahalanobis", "ivariance" (each covariate scaled by its variance) or "euclidean": the covariates themselves. To match on a score column already in the data, give score.

  • score (str, optional) – A score column already in the data to match on, instead of fitting one.

  • score_model (str, optional) – The score model, a formula for hdfe-stream’s feglm_stream, which absorbs fixed effects after |: "treat ~ x1 + x2 | firm". Default: a logit of the treatment on the matching covariates.

  • score_link ({"logit", "probit"}) – The score model’s link.

  • n_matches (int, default 1) – Matches per focal unit.

  • replace (bool, default False) – Match with replacement. Without replacement, matching is greedy in the order match_order.

  • match_order ({"largest", "smallest", "closest", "random", "data"}, optional) – The order focal units are matched in: by descending or ascending score, by ascending pair distance, at random (seed), or in data order. Default: “largest” (“smallest” when the controls are focal) for a score, “data” for a multivariate distance (where largest/smallest do not apply).

  • seed (int, optional) – Seed for match_order="random".

  • caliper (float or dict, optional) – A number is a caliper on the score, in standard deviations of the score over all units. A dict names calipers: {"ps": 0.2, "age": 3}; covariate calipers are in raw units. To change a caliper’s units give (width, "sd") or (width, "raw"): {"ps": 0.2, "income": (0.5, "sd")}. A pair is allowed when every gap is at most its caliper. A control that fails a covariate caliper is skipped and the next nearest is tried; the focal unit is not dropped.

  • focal ({"treated", "control"}) – The units that choose: treated units are matched to controls (the ATT’s match), or controls to treated units (the ATC’s).

  • id (str, optional) – A unit id column, carried into the match table and the weights.

  • keep_ties (bool, default False) – With replacement, keep every control at exactly the k-th nearest distance (otherwise exactly k are kept, the ones with the larger score first). Each focal unit’s matches then share its weight.

  • tie_tolerance (float, default 0) – With ties: a control is tied when its squared distance, divided by the score’s variance, is within this of the k-th’s. The default, 0, keeps exact ties only.

  • min_matches (int, default 1) – With n_matches > 1: drop a focal unit that ends with fewer than this many matches (a caliper or exact strata can leave it short). The default keeps every focal unit with at least one match; min_matches=n_matches keeps only fully matched units. Matches tied at the k-th distance (keep_ties) count. Applied after matching: without replacement, the controls a dropped unit took are not offered to other units (the greedy pass is not rerun).

  • match_in_rounds (bool, default True) – Without replacement and n_matches > 1: match in rounds (every focal unit takes its 1st match, then those matched take their 2nd, …; each round in match_order). False: one pass, each focal unit taking its k matches at its turn; with match_order="closest" that is all pairs in ascending distance, a unit taking controls until it has k. Identical for n_matches=1.

  • discard ({"none", "both", "control", "treated"} or a column name) – Common support: before matching, drop the units whose score lies outside the other arm’s range (“treated”: treated units outside the controls’ range; “control”: the reverse; “both”: either, outside the overlap of the two ranges; a unit at a bound is kept). A column name drops the units where it is true. Discarded units are not matched, and are counted apart (counts, the balance table’s “Discarded” row); calipers in standard deviations use the units kept.

  • reestimate_score (bool, default False) – With discard, refit the score model on the units kept and match on the new score.

  • mahalanobis_vars (list of column names, optional) – Match on the Mahalanobis distance between these covariates, within the caliper on the score: the score still orders the matching (match_order), carries the caliper and decides discard. The covariance is pooled within the treatment groups, over the units kept.

  • memory ({"auto", "in_memory", "by_stratum"}) – How the narrow columns (tens of bytes a unit) are held while matching: all strata at once (fastest), or a batch of whole strata at a time, read back from a Parquet copy on disk (memory bounded by the largest stratum and the batch). Both give the same matches. “auto” takes “in_memory” when it fits the memory headroom (stream-limits).

  • workdir (path, optional) – Scratch directory for the score model’s files and the by-stratum copy. Default: $HDFE_STREAM_WORKDIR, else the system temporary directory.

  • max_memory_gb (float, optional) – The memory limit and the share kept in reserve that memory="auto" plans with (stream-limits’ MemoryBudget; default: the limits it detects, including a PBS job’s).

  • memory_reserve (float, optional) – The memory limit and the share kept in reserve that memory="auto" plans with (stream-limits’ MemoryBudget; default: the limits it detects, including a PBS job’s).

  • n_threads (int, optional) – Threads for matching and for effect(): numba’s, BLAS’s and the k-d trees’ (stream-limits’ thread_scope). Default: as they are (every core). Polars’ threads are fixed when polars is imported: set STREAM_LIMITS_MAX_THREADS in the job script.

  • profile (bool, default False) – Return a Profile (hdfe-stream’s) of the memory and disk the match would need, from a scan of the data, without matching (see match_stream.profile).

  • verbose (bool, default True) – Print the score model’s progress (hdfe-stream’s feglm_stream).

Returns:

MatchSet, or a Profile with profile=True

match_stream.match_kernel(fml, data, *, exact=None, distance='ps', score=None, score_model=None, score_link='logit', kernel='epan', ridge=None, bandwidth='pair_distance', focal='treated', id=None, memory='auto', workdir=None, max_memory_gb=None, memory_reserve=None, n_threads=None, profile=False, verbose=True)[source]

Kernel or ridge matching on the score, out of core: each focal unit is compared with a weighted average of the other arm’s units of its stratum within one bandwidth of its score.

Matching only: no outcome is needed, and the result holds the matched units and their weights; there is no match table. Parameters not described here are as in match_nn.

Parameters:

  • fml – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • data – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • exact – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • distance – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • score – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • score_model – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • score_link – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • focal – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • id – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • memory – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • workdir – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • max_memory_gb – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • memory_reserve – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • n_threads – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • profile – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • verbose – As in match_nn. distance is "ps" or "logit_ps" here (kernel matching is on a score), or give score.

  • kernel ({"epan", "rectangle", "triangle", "biweight", "triweight"}) – The kernel, with support +/- 1 (“uniform” is “rectangle”).

  • ridge (None, True or float) – None: kernel matching. True: ridge matching, Frolich’s regularized local-linear weights, with the kernel’s ridge parameter (Seifert and Gasser’s: 5/16 for “epan”). A number: ridge matching with that parameter; 0 is local-linear matching.

  • bandwidth (float, "pair_distance", "cross_validation", PairDistance or CrossValidation) – The kernel’s half-width. A number is in score units. "pair_distance" (the default): 1.5 times the 90th percentile of the nonzero 1:1 nearest-neighbor distances; PairDistance(factor=, quantile=) changes those two numbers. "cross_validation": the bandwidth that predicts best a unit’s outcome from the others (or, without an outcome, that moves the matched mean of the score least); CrossValidation(outcome=, grid=, weighted=, penalty=) sets the outcome, the bandwidths tried and the rest. The criterion without an outcome forms the pairs within each bandwidth tried, so it costs more than the others on large strata. A bandwidth is chosen for the focal arm at hand.

Returns:

KernelSet, or a Profile with profile=True

match_stream.match_cem(fml, data, *, cutpoints, exact=None, focal='treated', id=None, workdir=None, max_memory_gb=None, memory_reserve=None, n_threads=None, profile=False, verbose=True)[source]

Coarsened exact matching, out of core: units match exactly on binned covariates (and the exact columns); cells with both arms are kept, and weighted so the matched controls sum to the matched treated units.

Parameters:

  • fml – As in match_nn.

  • data – As in match_nn.

  • exact – As in match_nn.

  • focal – As in match_nn.

  • id – As in match_nn.

  • workdir – As in match_nn.

  • max_memory_gb – As in match_nn.

  • memory_reserve – As in match_nn.

  • n_threads – As in match_nn.

  • profile – As in match_nn.

  • verbose – As in match_nn.

  • cutpoints (int or dict) – The bins of each matching covariate: a number of equal-width bins over its range, or a list of cut points; an int applies to every covariate. Required.

Returns:

CEMSet, or a Profile with profile=True

Kernel bandwidth rules

match_kernel(bandwidth=) takes a number, the name of a rule, or one of these objects to change a rule’s settings.

class match_stream.PairDistance[source]

The bandwidth from the distances of 1:1 nearest-neighbor matching.

factor times the quantile quantile of the nonzero distances between each focal unit and its nearest other-arm unit (with replacement, pooled over strata). The default, 1.5 times the 90th percentile, is narrow: focal units with no other-arm unit within it are unmatched.

__init__(factor=1.5, quantile=0.9)
classmethod __new__(*args, **kwargs)
class match_stream.CrossValidation[source]

The bandwidth chosen by leave-one-out cross-validation.

Parameters:

  • outcome (str, optional) – Predict this outcome column of each other-arm unit from the rest and choose the bandwidth with the smallest error. Without it the criterion is the balance of the score: how little leaving out any one other-arm unit moves the matched mean of the score.

  • grid (sequence of float, optional) – The bandwidths tried. Default: the pair-distance bandwidth times 1.5^k for k = -4..4, extended and refined around the minimum.

  • weighted (bool, default False) – Weight the outcome’s errors by the units’ matching weights. Needs outcome.

  • penalty (None or "support_loss") – None judges every bandwidth on the same units. “support_loss” judges each bandwidth on the units it can evaluate, times a penalty for the units it loses to common support (and for bandwidths above the score’s standard deviation), searching outward from the pair-distance bandwidth (or over grid).

__init__(outcome=None, grid=None, weighted=False, penalty=None)
classmethod __new__(*args, **kwargs)