The Middle Matrix

Introduction

A prior post on this blog recommended HC3 standard errors for the reduced-form markup equation estimated on transfer pricing benchmark sets. It stated the estimator and left two questions unanswered, both of the kind that surface under examination rather than during drafting.

The first is what the middle matrix of the sandwich actually is, and whether it is the hat matrix under another name. It is not. The two are of different orders and serve different purposes, and the only connection between them is that the diagonal of one supply the weights used to build the other. The second is whether HC0, the unadjusted member of the family, is simply the conventional OLS standard error. It is not that either, although the relation between them is exact and is worth setting out, because it establishes that the classical formula is a constrained special case of the robust one rather than a rival to it.

An economist who cannot say what the middle matrix is has not understood the estimator he is relying on, and an adverse expert will establish that on the record. What follows is the algebra, in a consistent notation.

Notation and Design

Let i=1,\ldots,n index comparables and t=1,\ldots,T index years, with T=3. The number of observations is N=nT, and k is the number of regressors, k=2 in the markup equation. The design matrix X is of order N by k.

The distinction between minuscule n and majuscule N is not decorative. Benchmark sets in transfer pricing run from about five to fifteen comparables, so n is small; with three years of reported data for each, N runs from fifteen to forty-five. Long and Ervin (2000) recommend HC3 whenever the sample is below roughly 250, and that threshold is stated in observations. The relevant comparison is therefore N against 250, not n against 250. The recommendation holds with room to spare on either reading, but only the statement in N is verifiable.

The model is:

Y=Xb+u,
\qquad
E(u)=0,
\qquad
E(uu')=\Omega
\\[8pt]
\Omega=\operatorname{diag}(\sigma_{it}^2)

where the variances \sigma_{it}^2 are not assumed equal across comparables. That assumption, and only that assumption, is what the correction relaxes.

The Hat Matrix

Stacking the N=nT observations, the hat matrix is:

H=X(X'X)^{-1}X'

H is of order N\times N, symmetric, and idempotent, with \operatorname{tr}(H)=k. It projects observed revenue onto the column space of X, so that the fitted values are \hat{Y}=HY. Its diagonal element is:

h_{it}=x_{it}'(X'X)^{-1}x_{it},
\qquad
0\leq h_{it}\leq1,
\qquad
\sum h_{it}=k

This is the leverage of comparable-year (i,t): literally the partial derivative of \hat{Y}_{it} with respect to Y_{it}, the weight the observation places on its own fitted value. Average leverage is k/N. A comparable-year with h_{it} well above k/N is one the fitted line is pulled toward. Note that the leverage is written h_{it} rather than h_{ii}, because here i indexes comparables and the repeated-index form would be read as a within-comparable quantity.

The complement I-H is the residual maker, since e=(I-H)Y=(I-H)u. From it follows the identity that makes leverage relevant to a variance estimator at all: under homoskedasticity:

E[e_{it}^2]=\sigma^2(1-h_{it})

Least squares residuals are systematically smaller than the errors they estimate, and most severely so where leverage is highest. This is a property of the arithmetic, not of the data.

The matrix and its name are due to Tukey (1972). Hoaglin and Welsch (1978) is the standard exposition of its properties and of the diagnostic use of its diagonal, and supplies the working rule that a leverage exceeding 2k/N marks an observation worth inspecting. In a benchmark set of ten comparables over three years, 2k/N is about 0.13, and the comparable-years above it are the ones the fitted line is being drawn toward. Identifying them is not an idle exercise: they are the observations whose residuals least squares understates most, and therefore the ones the correction acts on.

The Middle Matrix

Write the covariance estimator in sandwich form:

\operatorname{Var}(\hat{b})
=
(X'X)^{-1}S(X'X)^{-1}
\\[8pt]
S
=
X'WX
=
\sum_i\sum_t
w_{it}e_{it}^2x_{it}x_{it}'

S is the middle matrix, of order k\times k. It is not a projection, it is not idempotent, and it is not dimensionless: it carries the units of X^2Y^2. It estimates the covariance matrix of the score X'u, which is the quantity the classical formula misstates when the error variances differ. That this can be estimated without specifying the form of the unequal variances, requiring only that they exist, is the result of White (1980), following Eicker (1967) and Huber (1967).

The contrast with the hat matrix is complete:

The four estimators in general use differ only in the weight w_{it}:

HC2 follows directly from the identity above: dividing the squared residual by (1-h_{it}) restores it, in expectation, to \sigma^2. HC3 divides by the square, deliberately overshooting. The overshoot is not an error but the point: it is what makes the estimator err toward the wider interval at the sample sizes prevailing in transfer pricing. HC1, HC2, and HC3 are due to MacKinnon and White (1985); the HC0 through HC3 labels are the nomenclature of Davidson and MacKinnon (1993).

The squaring is not arbitrary. HC3 is a computationally convenient approximation to the jackknife variance estimator, in which each observation is deleted in turn and the coefficient re-estimated on the remainder. Deletion is exactly what the hat matrix diagonal measures, which is why the same quantity appears in a diagnostic tool and in a variance estimator. The jackknife is Tukey (1958); Miller (1974) is the review.

Is HC0 the Same as the Unadjusted OLS Standard Error?

No, but the relation is exact, and stating it disposes of the objection that a robust standard error is an alien construction imported to widen a range.

The classical formula is:

\operatorname{Var}_{OLS}(\hat{b})
=
s^2(X'X)^{-1},
\qquad
s^2=\frac{e'e}{N-k}

The sandwich reduces to it identically. Substitute the common estimate e'e/N for every e_{it}^2 in the HC1 middle matrix:

\sum_i\sum_t
\left(\frac{e'e}{N}\right)
x_{it}x_{it}'
=
\frac{e'e}{N}X'X

so that:

HC1\rightarrow
\frac{N}{N-k}
(X'X)^{-1}
\left(\frac{e'e}{N}\right)
(X'X)
(X'X)^{-1}
=
s^2(X'X)^{-1}

The conventional OLS standard error is therefore the special case of the sandwich in which every comparable-year is assigned the same squared residual. HC0 differs from it in two respects only: each observation retains its own e_{it}^2, and the degrees-of-freedom scale N/(N-k) is omitted, which HC1 restores. In any finite sample HC0 and the OLS standard error are numerically different. Under homoskedasticity both are consistent for the same limit, which is precisely White’s (1980) point.

The concession is the persuasive part. The classical standard error is not a competing estimator to be argued against; it is a constrained version of the robust one, and the constraint it imposes is a single common variance across comparables that may span two orders of magnitude in operating expense. That constraint is what the benchmark data contradict.

A Note on Symbols

In econometrics the letter M conventionally denotes the residual maker I-H, which is of order N\times N. Using M for the meat of the sandwich, which is of order k\times k, invites a reader to look for one matrix and find another. The middle matrix is written S here for that reason. The appendix of a report is exactly where an adverse expert hunts for a symbol collision, and the cost of avoiding one is a keystroke.

What the Correction Does Not Address

HC3 corrects for unequal variances across observations. It does not correct for dependence among them. In the design considered here each comparable contributes T=3 years, so the N=nT observations are not independent: the residuals of a given comparable are correlated across its own years. A cluster-robust estimator is the textbook response to that dependence, but with n between five and fifteen the number of clusters is far below the range in which the cluster estimator is reliable, and the small-cluster corrections proposed in the literature are exotic enough that requiring them of a corporate tax department would exceed what the documentation standards contemplate.

The defensible course is to state the limitation rather than to mis-correct it. A pooled HC3 estimate, reported as such, with the panel structure disclosed, is stronger than a cluster-robust estimate computed on fifteen clusters and presented without qualification. An acknowledged limitation is not a concession; it is what distinguishes an analysis from an advocacy document.

Conclusion

Three things follow from the algebra above.

The middle matrix and the hat matrix are distinct objects. The hat matrix is built from X alone and describes geometry: which observations pull the fitted line toward themselves. The middle matrix is built from X and the residuals and describes uncertainty: how much the score varies. Leverage enters the second only because the identity E[e_{it}^2]=\sigma^2(1-h_{it}) tells us the residuals understate the errors, and by exactly how much.

The correction touches nothing but the standard error. The estimated markup is unchanged by the choice among HC0, HC1, HC2, and HC3, because the choice affects only S and not the bread of the sandwich. Nothing about the point estimate is being argued.

And the conventional standard error is not a separate method. It is the sandwich with every squared residual replaced by their common average. Whoever prefers it is not declining to use a robust estimator; he is using one under a restriction he has not tested. The burden of that restriction belongs where it falls.

References

Belsley, D. A., E. Kuh, and R. E. Welsch (1980). Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York: Wiley. The book-length treatment of leverage and influence.

Davidson, R., and J. G. MacKinnon (1993). Estimation and Inference in Econometrics. New York: Oxford University Press. The source of the HC0 through HC3 nomenclature, which is often misattributed to MacKinnon and White (1985).

Eicker, F. (1963). “Asymptotic Normality and Consistency of the Least Squares Estimators for Families of Linear Regressions.” The Annals of Mathematical Statistics 34(2), 447-456. DOI 10.1214/aoms/1177704156. Asymptotic normality of least squares when the error distributions are not identical across observations.

Eicker, F. (1967). “Limit Theorems for Regression with Unequal and Dependent Errors.” In L. M. Le Cam and J. Neyman (eds.), Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. Berkeley: University of California Press, 59-82. The closer antecedent to the sandwich variance itself.

Greene, W. (2018). Econometric Analysis. Chapter on the classical linear regression model, section on robust estimation of asymptotic covariance matrices. Cited to establish that the estimator is textbook material rather than a specialist construction. Section and page vary by edition and should be verified against the edition relied upon.

Hoaglin, D. C., and R. E. Welsch (1978). “The Hat Matrix in Regression and ANOVA.” The American Statistician 32(1), 17-22 (correction at 32, 146). The standard exposition of the hat matrix, its diagonal as a measure of leverage, and the 2k/N working rule.

Huber, P. J. (1967). “The Behavior of Maximum Likelihood Estimates Under Nonstandard Conditions.” In L. M. Le Cam and J. Neyman (eds.), Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics. Berkeley: University of California Press, 221-233. The general sandwich form.

Long, J. S., and L. H. Ervin (2000). “Using Heteroscedasticity Consistent Standard Errors in the Linear Regression Model.” The American Statistician 54(3), 217-224. The applied recommendation: HC3 as the default whenever the sample is below roughly 250 observations.

MacKinnon, J. G., and H. White (1985). “Some Heteroskedasticity-Consistent Covariance Matrix Estimators with Improved Finite Sample Properties.” Journal of Econometrics 29(3), 305-325. Introduces the three finite-sample alternatives to White’s original estimator and shows by simulation that the unadjusted form is unreliable in small samples. Attributed source of HC3.

Miller, R. G. (1974). “The Jackknife – A Review.” Biometrika 61(1), 1-15. The review of the jackknife; a source for the estimator HC3 standard errors.

Tukey, J. W. (1958). “Bias and Confidence in Not-Quite Large Samples.” The Annals of Mathematical Statistics 29(2), 614. The origin of the jackknife. Cited for priority only: the item is a one-paragraph abstract, and there is no fuller text published behind it.

Tukey, J. W. (1972). “Some Graphical and Semigraphical Displays.” In T. A. Bancroft (ed.), Statistical Papers in Honor of George W. Snedecor. Ames: Iowa State University Press, 293-316. Introduces the hat matrix and its name.

Velleman, P. F., and R. E. Welsch (1981). “Efficient Computing of Regression Diagnostics.” The American Statistician 35(4), 234-242. Reconciles the conflicting terminology of the diagnostic literature and gives the computing formulae.

White, H. (1980). “A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity.” Econometrica 48(4), 817-838. The source of the heteroskedasticity-consistent sandwich in econometrics, namely HC0, the unadjusted form from which HC1 through HC3 are derived. Establishes that valid standard errors are obtainable without specifying the form of the unequal variances, only their presence.