Correlation, Regression, and the Intercept: Why the Aphorism Misses Its Target

1. The Complaint

A commentator objected that stating unit price is proportional — or log-proportional — to unit labor cost (ULC) confuses correlation with causation. The objection deserves a serious answer, because the aphorism it invokes is usually deployed as if self-evidently dispositive. What follows is the algebra that shows where the aphorism is misleading.

2. Two Formulas, One Object

Let $X$ and $Y$ be $N$ paired observations, centered from the mean as $xᵢ = Xᵢ - X̄$ and $yᵢ = Yᵢ - Ȳ$ . Define the three sums of squares and cross-products:

\quad S_{XX}=\sum x_i^2, \qquad S_{YY}=\sum y_i^2, \qquad S_{XY}=\sum x_i y_i

The regression slope of $Y$ on $X$ , the reverse regression slope of $X$ on $Y$ , and the Pearson correlation coefficient are, respectively:

\quad b=\frac{S_{XY}}{S_{XX}}, \qquad b'=\frac{S_{XY}}{S_{YY}}, \qquad r=\frac{S_{XY}}{\sqrt{S_{XX}S_{YY}}}

These are not three independent quantities. Multiply the two slope coefficients:

\quad b\,b' = \left(\frac{S_{XY}}{S_{XX}}\right) \left(\frac{S_{XY}}{S_{YY}}\right) = \frac{S_{XY}^2}{S_{XX}S_{YY}} = r^2

Above, I use majuscule notation close to Weisberg (1985), p. 9. The regression coefficients $b$ and $b’$ are the same relationship measured from opposite ends – $Y$ per unit of $X$ and $X$ per unit of $Y$ – so multiplying them cancels the units and leaves a pure number. That number is $r²$ . Take the square root to restore the sign and get $r$ itself; not a separate statistic, but the geometric mean of the two slopes one gets depending on which variable is treated as dependent.

So, $r = sign(SXY)\cdot\sqrt(b\cdotb')$ : the correlation coefficient is the signed geometric mean of the two regression slopes. This is also the familiar coefficient of determination in the $Y$ -on- $X$ regression, $r² = 1 - SSE/SST$ . Same number, two derivations. Correlation coefficient and regression slope are algebraic siblings, not competing epistemologies. Regression analysis has substituted correlation analysis in applied economics for the reasons I discuss herein.

3. Where They Diverge: Direction

The two regression slopes coincide, in the sense that $b = 1/b'$ , only in the degenerate case $r² = 1$ :

\quad \frac{b}{1/b'} = bb' = r^2 \le 1

Whenever the fit is imperfect $(r² < 1)$ , regressing $Y$ on $X$ and inverting a regression of $X$ on $Y$ give different lines. That wedge, of size $1 - r²$ , is the algebraic signature of asserting a direction. Correlation analysis refuses to choose between the two directions; it reports only their geometric mean. A regression model, by positing $Y = f(X)$ , chooses one direction from theory — and the choice is substantive because it changes the answer whenever the relationship is not deterministic; that is, the relationship is subject to random errors.

4. Where They Diverge: The Intercept

This is the sharper divergence, and the one the aphorism does not engage. Consider the linear model:

\quad Y_i = a+bX_i

The intercept is recovered from the uncentered means:

\quad \hat{a} = \bar{Y} - \hat{b}\,\bar{X}

Now translate the data: $Yᵢ \to Yᵢ + c$ for any constant c. Then $Ȳ \to Ȳ + c$ , but the centered from the mean deviation $yᵢ = Yᵢ - Ȳ$ is unchanged. Since $r$ is a function of $xᵢ$ and $yᵢ$ alone, $r$ is invariant under this translation: it cannot see the intercept. But the intercept shifts by exactly $c$ . The intercept is not a function of the quantities from which $r$ is built — it is annihilated by the centering from the mean operation before the correlation coefficient is ever computed. This is not a matter of convention or emphasis; it is an algebraic fact about what centering discards.

5. Why the Intercept Is Not Trivial

The intercept governs whether a ratio-based summary of the $X$ and $Y$ relationship is trustworthy or reliable. Define the ratio $mᵢ = Yᵢ / Xᵢ = b + a/Xᵢ$ . Averaging over $i$ , gives:

\quad \bar{m} = b + \frac{a}{H}, \qquad \frac{1}{H} = \frac{1}{N} \sum_{i=1}^{N} \frac{1}{X_i}

$H$ is the harmonic mean of the denominator $X$ . This is an algebraic identity, not an asymptotic approximation: the mean ratio equals the true slope b plus a bias term $a/H$ , and the bias vanishes only when $a = 0$ . The same mechanism biases the quartiles of $mᵢ$ — each quantile of the ratio inherits an offset $a/Q₁₋ₚ(X)$ from the corresponding quantile of $X$ , by rank-order inversion of the monotone map $m(X) = b + a/X$ .

Correlation analysis, having discarded the intercept in the act of centering the data from the mean, has nothing to say about whether this bias is present. Only the regression analysis — because it retains the intercept and can test its significance away from zero — can answer the question that matters for any ratio-based profit indicator, royalty rate, or price-cost relationship: is the naive ratio an unbiased estimate of the true marginal relationship, or is it displaced by a structural, non-vanishing amount? Correlation analysis is equivalent to measurement without theory, which is not admissible in science. See Koopmans (1947).

6. What the Aphorism Gets Right — and Where It Misleads

Specifying $Y = f(X)$ and scouting the shape of the scatterplot with a nearest-neighbor LOWESS fit is a modeling choice, not an identification strategy. Cost-plus pricing theory supplies the rationale for this $X$ determines $Y$ direction – instead of simple correlates. See Gandolfo (1980), p. 49, equation 3.41 [“Eq. (3.41) indicates the price that entrepreneurs have to fix if they want to obtain a given share of real national income.”] See also Kalecki (1965), pp. 11-14.

But this is a different and much narrower objection than “correlation is not causation,” and it is not the one the aphorism, as generally deployed, makes. The aphorism’s usual work is to imply that any regression coefficient is merely descriptive, indistinguishable in evidentiary weight from a raw Pearson r. This implication is false, and falsifiable by the algebra above: $r$ and $b$ are siblings through $r² = b\cdotb'$ , but they diverge on the intercept — that a regression tests and a correlation coefficient cannot even express. They also diverge on the choice of dependent and independent variables. Repeating the aphorism does not substitute for testing $â$ or for causal modeling.

References

Aitken, A. (1935). “On Least Squares and Linear Combinations of Observations.” Proceedings of the Royal Society of Edinburgh, 55, 42–48. [Canonical OLS/GLS reference.]

Fisher, R. (1925). Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd, p. 47. [Origin of the statistical test t = 1 convenience threshold underlying the 68% confidence convention.]

Gandolfo, G. (1980, 2nd, revised edition). Economic Dynamics: Methods and Models. Amsterdam: North-Holland. [Mostly macroeconomics.]

JCGM 100:2008. Evaluation of Measurement Data — Guide to the Expression of Uncertainty in Measurement (GUM). Joint Committee for Guides in Metrology. [k = 1 standard-uncertainty convention; critiqued here for implicitly assuming the intercept a = 0.]

Kalecki, M. (1965, revised 2nd edition). Theory of Economic Dynamics. London: George Allen & Unwin. [Chapter 1, “Cost and Prices.”]

Koopmans, T. (1947). “Measurement without Theory.” Review of Economics and Statistics, 29, 3, August, 161-172. [Critique of Burns and Mitchell’s NBER business-cycle methodology.]

Weisberg, S. (1985, 2nd edition). Applied Linear Regression. New York: John Wiley & Sons. [Written by a statistician showing a major concern with residual error analysis.]

Appendix A: Deriving $r² = 1 - SSE/SST$ from $S_{XX}$ , $S_{YY}$ , $S_{XY}$

Section 2 asserts that $r²$ , defined as sign $(S_{XY})·√(b·b′)$ squared, is “also” the familiar coefficient of determination, $r² = 1 - SSE/SST$ . Stated that way, it reads as two separate facts sharing a symbol. It is one fact. Both quantities reduce to the same three sufficient statistics, $S_{XX}$ , $S_{YY}$ , and $S_{XY}$ , defined in Section 2.

By definition, the total sum of squares in the $Y$ -on- $X$ regression is the sum of squared deviations of $Y$ from its own mean:

\quad SST = \sum (Y_i-\bar{Y})^2 = \sum y_i^2 = S_{YY}

This is not a coincidence of notation; $SST$ is the identical object already defined as $S_{YY}$ in Section 2, since $yᵢ \equiv Yᵢ - Ȳ$ by construction.

The residual sum of squares from the fitted line $Ŷᵢ = â + b̂Xᵢ$ is:

\quad SSE = S_{YY} - bS_{XY} = S_{YY} - \frac{S_{XY}^2}{S_{XX}} = S_{YY}(1-r^2)

using $b = S_{XY}/S_{XX}$ in the second step, and $r² = S_{XY²}/(S_{XX} S_{YY})$ in the third. Dividing through by $SST = S_{YY}$ gives:

\quad \frac{SSE}{SST} = 1-r^2 \;\Longrightarrow\; r^2 = 1-\frac{SSE}{SST}

derived from the identical $S_{XX}$ , $S_{YY}$ , $S_{XY}$ that produced $b\cdotb'$ in Section 2. Correlation, the slope product, and goodness-of-fit are one algebraic object viewed three ways: none of $r$ , $b\cdotb'$ , or $R²$ can carry more information than the other two, because all three reduce to the same three sufficient statistics.

Appendix B: The Sign Operator

Section 2 writes $r = sign(S_{XY})·√(b·b′)$ . Here sign(·) denotes the signum function:

\quad \operatorname{sign}(z)= \begin{cases} 1, & z>0,\\ 0, & z=0,\\ -1, & z<0. \end{cases}

The operator is required, not decorative. The square root in $\sqrt(b\cdotb')$ is, by convention, non-negative, regardless of the sign of $S_{XY}$ . Yet $r$ must be able to take either sign, since correlation records the direction of co-movement as well as its strength. Multiplying by sign $(S_{XY})$ restores that sign without altering magnitude, because:

\quad bb' = \frac{S_{XY}^2}{S_{XX}S_{YY}}

depends only on $S_{XY²}$ and has therefore already lost the sign information that $S_{XY}$ itself carried. The sign recaptures the information destroyed by squaring inside $b\cdotb'$ ; sign(·) is the explicit mechanism that reattaches it from outside the square root.

Correlation, Regression, and the Intercept: Why the Aphorism Misses Its Target

Terms of use

By registering with EdgarStat®, you agree not to: