Short version: if your dataset comes with replicate weights, use them. If you ignore them, your standard errors will often be fantasy, usually too small. Then your p-values start lying. Politely, but still.

Why replicate weights exist

In complex surveys (and large-scale assessments), the data are rarely a simple random sample. You get stratification, clustering, unequal selection probabilities, nonresponse adjustments, calibration. All useful, but they break the “vanilla” formulas for standard errors.

Replicate weights are the practical workaround: instead of deriving a custom variance formula for every estimator, the dataset provides many alternate weight sets. You recompute your statistic under each set and look at how much it wiggles.

The core idea

Compute your estimate once using the full weight, call it θ. Then compute it again for each replicate weight, θ_r, for r = 1..R.

Variance(θ) ≈  Σ  c_r * (θ_r - θ)^2
               r

The only thing that changes across BRR, Fay, jackknife, bootstrap is the scaling constant(s) c_r and how the replicate weights were built.

BRR and Fay’s BRR

BRR (Balanced Repeated Replication) is common when the sample design can be represented as many half-samples. A common variance estimator is:

Var(θ) = (1/R) * Σ (θ_r - θ)^2

Fay’s method is a BRR variant that makes replicates “less extreme” (more stable) using a Fay factor f. You must use the factor your data documentation specifies.

Var(θ) = 1 / (R * (1 - f)^2) * Σ (θ_r - θ)^2

Jackknife (JK1, JK2) in plain language

Jackknife replicates are usually “leave-one-group-out” or “leave-one-PSU-out” ideas, depending on the design. A common form is:

Var(θ) = (R - 1)/R * Σ (θ_r - θ)^2

Details vary by provider (JK1 vs JK2, stratified vs not). Again, the dataset documentation decides the constants.

Common gotchas that mess up inference

• Using full weights for point estimates but ignoring replicates for SEs. You end up mixing a complex-design estimator with a simple-design SE.
• Forgetting the scale factors. BRR vs Fay vs JK are not interchangeable. Wrong scaling, wrong SEs.
• Subsetting incorrectly. In many workflows you keep all cases, then do domain estimation. Blindly dropping cases can distort variance estimation if you do it the wrong way.
• Using replicate weights but the wrong statistic. Example: your software silently changes the model or estimator due to missingness or weight handling. Always sanity-check.

A minimal workflow you can trust

1. Compute θ with the full weight.
2. Compute θ_r for each replicate weight.
3. Use the correct replicate-variance formula for your method (BRR, Fay, JK, bootstrap).
4. Use the right degrees of freedom (often tied to number of replicates or PSUs).

Quick R pseudocode

# Pseudocode: shape depends on your package + data layout
theta  <- estimate(data, w = w_full)

thetas <- sapply(1:R, function(r) estimate(data, w = w_rep[, r]))

# BRR:
V  <- (1/R) * sum((thetas - theta)^2)

# Fay BRR (with fay = f):
V  <- (1 / (R * (1 - f)^2)) * sum((thetas - theta)^2)

SE <- sqrt(V)

Closing thought

Replicate weights look annoying at first, but they’re basically a cheat code: the data provider already did the hard design work, and you just need to follow the recipe. Do that, and your inference stops hallucinating confidence.