how-to · 5 min read
Standard errors and weighted fits
A fitted slope or coefficient is an estimate, not a fact: repeat the experiment and it moves. Reporting the value on its own hides that variability, while a standard error and confidence interval show how much the data actually pins the parameter down.
This guide explains the residual standard error, the coefficient covariance matrix and the t-based intervals used by the experimental data plotter, and how weighting by 1/σ² changes both the estimate and its uncertainty. A three-point example keeps the arithmetic visible.
Residual standard error is the starting point
A fit leaves a residual at every point: the measured y minus the value the fitted curve predicts there. Squaring and summing those residuals gives the sum of squared errors (SSE), and dividing by the residual degrees of freedom n − p, where p is the number of coefficients (p = degree + 1), turns it into the residual variance s². The square root is the residual standard error, in the same units as y.
That quantity is the data's remaining spread around the fitted curve, and it sets the scale of every coefficient uncertainty. A weighted fit uses weighted residuals instead, so sqrt(weighted SSE/(n − p)) is the square root of the reduced chi-square — a goodness-of-fit scale, not a plain spread in y units. A fit with no residual degrees of freedom (n = p) has no residual variance to estimate, so the standard errors cannot be inferred from the data.
The covariance matrix turns residuals into coefficient errors
For an unweighted polynomial fit, the coefficient covariance matrix is Cov = s²·(XᵀX)⁻¹, where the design matrix X holds one row of powers of x per point. The standard error of a coefficient is the square root of the matching diagonal entry, and the off-diagonal entries describe how the estimates move together; for a straight line, a steeper slope usually pairs with a lower intercept.
When the y uncertainties are known, the weighted fit uses Cov = (XᵀWX)⁻¹ scaled by max(1, χ²_red), with W holding the weights 1/σ² and χ²_red the weighted SSE divided by n − p. The fitter states which convention it used, because the two produce different numbers from the same data.
A two-sided 95% confidence interval is estimate ± t(df)·SE. The critical value comes from the t distribution with df = n − p residual degrees of freedom; it is large when there are few degrees of freedom (12.706 at df = 1, 4.303 at df = 2) and approaches 1.96 for large samples.
Why R² alone is not fit uncertainty
R² measures the fraction of the variation in y explained by the fitted curve. It says nothing directly about how well the slope itself is determined, and it rises whenever a coefficient is added, even if the new coefficient only follows noise. A high R² can sit on top of an interval so wide that the slope is indistinguishable from zero.
The three-point example makes the point. For (1, 2), (2, 3) and (3, 6) the unweighted fit is y = 2x − 0.3333 with R² = 0.9231, which sounds respectable. The residual standard error is 0.8165, but with p = 2 and n = 3 there is only df = 1, so t = 12.706. The slope standard error is 0.5774 and its 95% interval is roughly [−5.34, 9.34]; the intercept standard error is 1.2472 and its interval is roughly [−16.18, 15.51]. Both intervals include zero, so three points cannot pin down either parameter, however tidy R² looks.
Weighted fits when σ is known
Weighting by 1/σ² gives precise points more influence, so the fitted line is pulled toward them. With σ = 1, 1 and 0.5 for the same three points, the weighted fit is y = 2.1429x − 0.5238, a steeper slope and a lower intercept than the unweighted y = 2x − 0.3333. The weighted χ²_red is 0.762, below 1, so the conservative scale max(1, χ²_red) is 1 and the covariance is just (XᵀWX)⁻¹.
The weighted slope standard error is 0.5345 with a 95% interval of roughly [−4.65, 8.93], and the weighted intercept standard error is 1.3973 with an interval of roughly [−18.28, 17.23]. Weighting changed the estimates and slightly tightened the slope error, but one residual degree of freedom still leaves both intervals wide and overlapping zero.
Weighting is only justified when each σ is known and the errors are independent. If the weights are guessed, or if the points share a systematic error, the weighted fit can look more precise without being more accurate; comparison between weighted and unweighted results should stay within the same convention.
Reporting fit uncertainty honestly
Report the estimate with its standard error, the 95% interval, the residual degrees of freedom, the number of points and the weighting convention, and keep the fit equation alongside them. Include the residual standard error and adjusted R² when comparing models with different numbers of coefficients, and say plainly when the interval is wide.
The intervals above assume independent errors, that the polynomial model is the right shape and that the x values are exact. If the errors are correlated, if the curve is misspecified or if x itself carries meaningful uncertainty, the reported standard errors and intervals understate the real uncertainty, and a more careful model is needed.
- Start from the residual standard error and state the residual degrees of freedom.
- Quote each coefficient as estimate ± standard error with a 95% interval.
- Name the covariance convention: s²(XᵀX)⁻¹ or scaled (XᵀWX)⁻¹.
- Weight by 1/σ² only when the uncertainties are known and independent.
- Treat R² as explained variation, not as the uncertainty of the fit.