Skip to main content

Linear Regression with scikit-learn

This example uses scikit-learn's bundled diabetes regression dataset, so it does not download data. The target is a quantitative disease-progression measure one year after baseline. It is an educational benchmark, not a diagnostic model.

1. Keep the test set isolated​

Split before fitting anything. LinearRegression does not require scaling for its predictions, so this example has no learned preprocessing step. If you add imputation, feature selection, polynomial features, or scaling, put those steps in a Pipeline so each cross-validation fold learns them from its own training partition.

from sklearn.datasets import load_diabetes
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, root_mean_squared_error, r2_score
from sklearn.model_selection import train_test_split

X, y = load_diabetes(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42
)

model = LinearRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

print(f"MAE: {mean_absolute_error(y_test, y_pred):.2f}")
print(f"RMSE: {root_mean_squared_error(y_test, y_pred):.2f}")
print(f"R²: {r2_score(y_test, y_pred):.3f}")
print(f"Intercept: {model.intercept_:.2f}")

MAE reports a typical absolute error in target units. RMSE weights larger residuals more strongly. Test-set R2R^2 compares the fitted model with the baseline that predicts the test-set mean and may be negative for a poor model.

2. Interpret coefficients carefully​

A coefficient is the fitted change in the prediction for a one-unit change in that feature while the other model inputs are held fixed. That is an association conditional on the included features, not a causal effect. Correlated features can make individual coefficients unstable, and the meaning of a one-unit change depends on the feature's units.

Use residual plots and domain knowledge to look for nonlinearity, unequal error variance, dependence, influential observations, or omitted structure. Evaluate uncertainty with methods appropriate to the sampling design; a good predictive score does not validate inferential assumptions.

3. Predict new observations​

Supply the same columns, units, and semantics used during training. A Pipeline is the safest way to guarantee that learned transformations are reused. Predictions far outside the training range are extrapolations and can be unreliable even when in-range test metrics look reasonable.

Takeaways​

  • Split before learned preprocessing and reserve the test set for final evaluation.
  • Compare several metrics with a meaningful baseline.
  • Treat coefficients as conditional associations unless the study design supports causal inference.
  • Check residual behavior and avoid unsupported extrapolation.