Fixed Points for Fitted Lines

For many best-fit methods for lines in the plane, each observation possesses an associated point in the plane, or at infinity for particular observations. These associated points are called “fixed points,” because properly chosen replacement points for the originally selected observation create other best-fit lines that contain the same fixed point. Simple linear regression and orthogonal least-squares regression are investigated as examples. The phenomenon has been previously noted for simple regression, where the properly selected replacement points are related to regression diagnostics and the determination of outliers. Orthogonal regression lines are fundamental to principal component analysis. Mechanical devices for simple linear regression and for orthogonal regression are presented. The devices show the dependence of the best-fitting lines, which contain the fixed point, on the new replacement points.