Lesson 11 · Statistics for data analysts
Simple Linear Regression Explained | Statistics #11
Moving from comparing groups to modeling a real relationship directly. Real mtcars data: predicting fuel economy from weight. Let's get into it. Open a new…
- CourseStatistics for data analysts
- Lesson11 of 15
- Video13 min
- FormatJupyter notebook · 9 code cells
- Data1 dataset
What you'll learn
Datasets used in this lesson
Save these next to the notebook. In Google Colab, upload them with the 📁 icon on the left first.
- mtcars.csv preview1.7 KB
📓 Full notebook
Download .ipynbStatistics for Data Analysts, Video 11: Simple Linear Regression#
- Video eleven of the 15-part series: moving from comparing groups to modeling a real relationship directly.
- Real mtcars data: predicting fuel economy from weight.
- Let's get into it.
Before You Start#
- Open a new Jupyter Notebook in VS Code and select your Python interpreter as the kernel.
- You'll need pandas, NumPy, SciPy, and Matplotlib.
- Place mtcars.csv in the same folder as this notebook.
import pandas as pd
import numpy as np
from scipy import stats
import matplotlib.pyplot as plt
mtcars = pd.read_csv('mtcars.csv')
print(mtcars[['model', 'wt', 'mpg']].head())
Part 1: Fitting the Model#
x = mtcars['wt']
y = mtcars['mpg']
result = stats.linregress(x, y)
print(f'Real slope: {result.slope:.3f}')
print(f'Real intercept: {result.intercept:.3f}')
print(f'Real p-value: {result.pvalue:.2e}')
The real slope, about -5.34, means: for every additional 1,000 pounds of real weight, this model predicts about 5.34 fewer real miles per gallon. The real intercept, about 37.3, is the model's predicted mpg at zero weight, mathematically necessary for the line but not a meaningful real car. The real p-value is far below 0.05: strong evidence the real relationship isn't just noise.
plt.figure(figsize=(8, 5))
plt.scatter(x, y, color='steelblue', label='Real cars')
x_line = np.linspace(x.min(), x.max(), 100)
plt.plot(x_line, result.intercept + result.slope * x_line, color='darkred', linewidth=2, label='Fitted line')
plt.title('Real mtcars: Weight vs. MPG')
plt.xlabel('Weight (1000 lbs)')
plt.ylabel('Miles per Gallon')
plt.legend()
plt.show()
Part 2: R-Squared, and What It Actually Measures#
r_squared = result.rvalue ** 2
print(f'Real correlation (r): {result.rvalue:.4f}')
print(f'Real R-squared: {r_squared:.4f}')
print(f'Interpretation: weight explains about {r_squared:.1%} of the real variance in mpg')
Part 3: Predictions and Prediction Intervals#
n = len(x)
yhat = result.intercept + result.slope * x
residuals = y - yhat
mse = np.sum(residuals ** 2) / (n - 2)
se_resid = np.sqrt(mse)
x_mean = x.mean()
sxx = np.sum((x - x_mean) ** 2)
print(f'Real residual standard error: {se_resid:.3f}')
x0 = 3.0
pred_mpg = result.intercept + result.slope * x0
se_pred = se_resid * np.sqrt(1 + 1 / n + (x0 - x_mean) ** 2 / sxx)
t_crit = stats.t.ppf(0.975, df=n - 2)
pi_low, pi_high = pred_mpg - t_crit * se_pred, pred_mpg + t_crit * se_pred
print(f'For a real 3,000 lb car, predicted mpg: {pred_mpg:.2f}')
print(f'Real 95% prediction interval: ({pi_low:.2f}, {pi_high:.2f})')
Part 4: Residual Diagnostics#
fig, axes = plt.subplots(1, 2, figsize=(11, 4))
axes[0].scatter(yhat, residuals, color='steelblue')
axes[0].axhline(0, color='darkred', linestyle='--')
axes[0].set_title('Real Residuals vs. Fitted Values')
axes[0].set_xlabel('Fitted mpg')
axes[0].set_ylabel('Residual')
stats.probplot(residuals, dist='norm', plot=axes[1])
axes[1].set_title('Q-Q Plot of Real Residuals')
plt.tight_layout()
plt.show()
shapiro_stat, shapiro_p = stats.shapiro(residuals)
print(f'Real Shapiro-Wilk test on residuals: statistic={shapiro_stat:.4f}, p-value={shapiro_p:.4f}')
Part 5: Two Cautions - Causation and Extrapolation#
print(f'Real weight range in this data: {x.min():.2f} to {x.max():.2f} (thousands of lbs)')
x_extreme = 8.0
pred_extreme = result.intercept + result.slope * x_extreme
print(f'Extrapolated prediction at {x_extreme} thousand lbs: {pred_extreme:.2f} mpg')
And on causation: this real regression shows heavier cars genuinely tend to get worse real mileage, but it doesn't prove weight causes lower mpg in isolation. Heavier real cars in this dataset also tend to have bigger real engines and more real horsepower, and any of those could be doing some or all of the causal work. Untangling that requires more than one predictor at a time, exactly what video twelve's multiple regression adds.
Wrap-Up: What You Learned#
- Fitting a simple linear regression on real mtcars data, and interpreting the slope and intercept in real units.
- R-squared as the proportion of real variance explained, not a measure of correctness or causation.
- Prediction intervals, quantifying genuine real uncertainty around a single new prediction.
- Residual diagnostics: checking the real fitted-vs-residual plot and a Q-Q plot before trusting the model's other numbers.
- Two real cautions: extrapolating beyond the observed real data range produces nonsense, and a strong real fit alone never proves causation.
- Video twelve extends all of this to multiple predictors at once, including multicollinearity and a full statsmodels regression summary. Subscribe so it lands automatically see you there.
🧠 Test yourself
Pick an answer to check it. Your best score is saved in this browser.
In
stats.linregress(x, y), a slope of -5.34 for weight (1000 lbs) vs mpg means...The slope is the predicted change in y per one-unit change in x.
What does R-squared measure?
R-squared = r**2, the share of variance in y explained by x.
Compared with a confidence interval for the mean response, a prediction interval for one new car is...
The
1 +term in the standard error accounts for the scatter of single observations.What would a clear curve or funnel shape in a residuals-vs-fitted plot suggest?
Residuals should look like random noise around zero; patterns signal problems like non-linearity.
Why is predicting mpg at 8,000 lbs (outside the observed 1.5-5.4 range) risky?
Extrapolating beyond observed data can produce nonsense.
💻 Practice exercises
Try each one in the notebook or in your own editor before opening the solution.
Fit
stats.linregressfor x=[1,2,3,4,5,6], y=[2.1,4.2,5.9,8.1,9.8,12.2]. Print slope, intercept and R-squared.💡 Hint
Use result.rvalue ** 2 for R-squared.
✅ Show solution
from scipy import stats x = [1,2,3,4,5,6] y = [2.1,4.2,5.9,8.1,9.8,12.2] r = stats.linregress(x, y) print(r.slope, r.intercept, r.rvalue ** 2)
Using the same data, predict y for x = 7 and compute the residuals for every point. Print the sum of squared residuals.
💡 Hint
Fitted value = intercept + slope * x.
✅ Show solution
import numpy as np from scipy import stats x = np.array([1,2,3,4,5,6]) y = np.array([2.1,4.2,5.9,8.1,9.8,12.2]) r = stats.linregress(x, y) print('pred at 7:', r.intercept + r.slope * 7) resid = y - (r.intercept + r.slope * x) print('residuals:', resid.round(3)) print('SSE:', (resid ** 2).sum())
Using the same data, compute a 95% prediction interval at x0 = 7 using the formula from the lesson (residual SE with n-2 degrees of freedom).
💡 Hint
se_pred = se_resid * sqrt(1 + 1/n + (x0 - x_mean)**2 / sxx).
✅ Show solution
import numpy as np from scipy import stats x = np.array([1,2,3,4,5,6]) y = np.array([2.1,4.2,5.9,8.1,9.8,12.2]) r = stats.linregress(x, y) n = len(x) resid = y - (r.intercept + r.slope * x) se_resid = np.sqrt((resid ** 2).sum() / (n - 2)) sxx = ((x - x.mean()) ** 2).sum() x0 = 7 pred = r.intercept + r.slope * x0 se_pred = se_resid * np.sqrt(1 + 1/n + (x0 - x.mean()) ** 2 / sxx) t_crit = stats.t.ppf(0.975, df=n - 2) print(pred - t_crit * se_pred, pred + t_crit * se_pred)
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



