Lesson 6 · Scikit-learn deep dive
Scikit-learn Tutorial #6: Pipelines
Chaining preprocessing and a model into one real estimator. Pipeline, makepipeline, named steps, and avoiding data leakage. Let's get into it. Manually…
- CourseScikit-learn deep dive
- Lesson6 of 18
- Video15 min
- FormatJupyter notebook · 10 code cells
What you'll learn
- Why Pipelines - the Leakage Problem
- Building a Pipeline with Named Steps
- makepipeline - Auto-Naming
- fit/predict/score - Treating the Whole Thing as One Estimator
- Accessing Steps - namedsteps, Indexing, Slicing
- Pipeline Parameter Naming - stepparam
- Chaining Multiple Preprocessing Steps
- Avoiding Leakage with crossvalscore
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynbScikit-learn Deep-Dive, Video 6: Pipelines#
- Video six of the eighteen-part series: chaining preprocessing and a model into one real estimator.
- Pipeline, make_pipeline, named steps, and avoiding data leakage.
- Let's get into it.
Part 1: Why Pipelines - the Leakage Problem#
import numpy as np
from sklearn.datasets import load_wine
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
X, y = load_wine(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)
Part 2: Building a Pipeline with Named Steps#
from sklearn.pipeline import Pipeline
pipe = Pipeline([
('scaler', StandardScaler()),
('model', LogisticRegression(max_iter=1000))
])
pipe.fit(X_train, y_train)
print(pipe.named_steps.keys())
Part 3: make_pipeline - Auto-Naming#
from sklearn.pipeline import make_pipeline
auto_pipe = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
print(list(auto_pipe.named_steps.keys()))
auto_pipe.fit(X_train, y_train)
print(round(auto_pipe.score(X_test, y_test), 3))
Part 4: fit/predict/score - Treating the Whole Thing as One Estimator#
predictions = pipe.predict(X_test)
print(predictions[:10])
print(y_test[:10])
print(round(pipe.score(X_test, y_test), 3))
Part 5: Accessing Steps - named_steps, Indexing, Slicing#
fitted_scaler = pipe.named_steps['scaler']
print(fitted_scaler.mean_.round(2))
print(pipe[0])
print(pipe[-1].coef_.shape)
print(pipe[:1])
Part 6: Pipeline Parameter Naming - step__param#
print(sorted(pipe.get_params().keys())[:6])
pipe.set_params(model__C=0.5)
print(pipe.named_steps['model'].C)
Part 7: Chaining Multiple Preprocessing Steps#
from sklearn.impute import SimpleImputer
X_with_gaps = X_train.copy()
X_with_gaps[0, 0] = np.nan
full_pipe = Pipeline([
('imputer', SimpleImputer(strategy='mean')),
('scaler', StandardScaler()),
('model', LogisticRegression(max_iter=1000))
])
full_pipe.fit(X_with_gaps, y_train)
print(round(full_pipe.score(X_test, y_test), 3))
Part 8: Avoiding Leakage with cross_val_score#
from sklearn.model_selection import cross_val_score
scores = cross_val_score(pipe, X, y, cv=5)
print(scores.round(3))
print(round(scores.mean(), 3))
Part 9: The Wrong Way vs the Right Way, Concretely#
leaky_scaler = StandardScaler().fit(X)
X_leaky = leaky_scaler.transform(X)
from sklearn.linear_model import LogisticRegression as LR2
leaky_scores = cross_val_score(LR2(max_iter=1000), X_leaky, y, cv=5)
safe_scores = cross_val_score(pipe, X, y, cv=5)
print(round(leaky_scores.mean(), 3), round(safe_scores.mean(), 3))
Part 10: A Real Pattern - a Reusable build_pipeline Function#
def build_pipeline(model, impute_strategy='mean'):
return Pipeline([
('imputer', SimpleImputer(strategy=impute_strategy)),
('scaler', StandardScaler()),
('model', model)
])
candidate_pipe = build_pipeline(LogisticRegression(max_iter=1000))
candidate_scores = cross_val_score(candidate_pipe, X, y, cv=5)
print(round(candidate_scores.mean(), 3))
Wrap-Up: What You Learned#
- Manually fitting and transforming preprocessing steps separately invites leakage from accidentally re-fitting on test data.
- Pipeline bundles named steps, transformers then a final estimator, into one object with a single fit call.
- make_pipeline builds the same structure with auto-generated step names.
- A fitted pipeline exposes the same fit/predict/score interface as any single estimator.
- named_steps, indexing, and slicing all reach individual fitted steps for inspection.
- step__param naming reaches parameters deep inside a pipeline, the syntax GridSearchCV relies on.
- Pipelines can chain multiple preprocessing steps, like an imputer then a scaler, before the final model.
- Passing a whole pipeline into cross_val_score keeps every fold's preprocessing correctly isolated.
- Scaling the full dataset before splitting is the classic leakage mistake; pipelines make it structurally impossible.
- That wraps up pipelines. Next up: ColumnTransformer, applying different preprocessing per column type.
🧠 Test yourself
Pick an answer to check it. Your best score is saved in this browser.
What is the main problem a Pipeline solves, according to Part 1?
Fitting a scaler separately by hand makes it easy to accidentally refit on test data. A Pipeline keeps fit and transform in the right order.
What does this code print?
auto_pipe = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000)) print(list(auto_pipe.named_steps.keys()))
make_pipeline names each step after its lowercase class name. With Pipeline you choose the names yourself.
A pipeline has steps named 'scaler' and 'model'. Which line changes the logistic regression C value to 0.5?
Parameters inside a pipeline use step name, two underscores, then the parameter. GridSearchCV relies on the same syntax.
What is the difference between these two lines?
leaky = cross_val_score(LogisticRegression(max_iter=1000), StandardScaler().fit_transform(X), y, cv=5) safe = cross_val_score(pipe, X, y, cv=5)
Scaling the whole dataset first lets validation data influence the scaler. Passing the pipeline keeps each fold's preprocessing isolated.
After
pipe.fit(X_train, y_train), what doespipe[-1].coef_.shapeaccess?Indexing a pipeline returns its steps, and -1 is the last one, the model. pipe[0] would be the scaler and pipe[:1] a smaller pipeline.
💻 Practice exercises
Try each one in the notebook or in your own editor before opening the solution.
Build a Pipeline with a StandardScaler and LogisticRegression, fit it on the wine data's training split, and print the test accuracy and the step names.
💡 Hint
Pipeline takes a list of (name, estimator) tuples.
✅ Show solution
import matplotlib matplotlib.use('Agg') from sklearn.datasets import load_wine from sklearn.model_selection import train_test_split from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression X, y = load_wine(return_X_y=True) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0) pipe = Pipeline([ ('scaler', StandardScaler()), ('model', LogisticRegression(max_iter=1000)), ]) pipe.fit(X_train, y_train) print('steps:', list(pipe.named_steps.keys())) print('test accuracy:', round(pipe.score(X_test, y_test), 3))
Put a SimpleImputer first in a pipeline, punch a few NaN holes into the training data, then fit and score. Print the scaler's learned mean for the first feature.
💡 Hint
Use make_pipeline or Pipeline, then pipe.named_steps['standardscaler'].mean_ if you used make_pipeline.
✅ Show solution
import matplotlib matplotlib.use('Agg') import numpy as np from sklearn.datasets import load_wine from sklearn.model_selection import train_test_split from sklearn.pipeline import make_pipeline from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression X, y = load_wine(return_X_y=True) X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=0) X_gaps = X_train.copy() X_gaps[0, 0] = np.nan X_gaps[5, 3] = np.nan X_gaps[10, 7] = np.nan pipe = make_pipeline(SimpleImputer(strategy='mean'), StandardScaler(), LogisticRegression(max_iter=1000)) pipe.fit(X_gaps, y_train) print('test accuracy:', round(pipe.score(X_test, y_test), 3)) print('scaler mean of feature 0:', round(pipe.named_steps['standardscaler'].mean_[0], 3))
Write a build_pipeline(model) helper like the lesson's. Then use cross_val_score to compare LogisticRegression and KNeighborsClassifier on the wine data, and compare the leaky approach of scaling everything first against the pipeline for KNN.
💡 Hint
The leaky version calls StandardScaler().fit_transform(X) before cross_val_score.
✅ Show solution
import matplotlib matplotlib.use('Agg') from sklearn.datasets import load_wine from sklearn.model_selection import cross_val_score from sklearn.pipeline import Pipeline from sklearn.impute import SimpleImputer from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression from sklearn.neighbors import KNeighborsClassifier X, y = load_wine(return_X_y=True) def build_pipeline(model, impute_strategy='mean'): return Pipeline([ ('imputer', SimpleImputer(strategy=impute_strategy)), ('scaler', StandardScaler()), ('model', model), ]) candidates = { 'logreg': LogisticRegression(max_iter=1000), 'knn': KNeighborsClassifier(n_neighbors=5), } for name, model in candidates.items(): scores = cross_val_score(build_pipeline(model), X, y, cv=5) print(name, 'pipeline CV mean:', round(scores.mean(), 3)) leaky_X = StandardScaler().fit_transform(X) leaky = cross_val_score(KNeighborsClassifier(n_neighbors=5), leaky_X, y, cv=5) print('knn leaky CV mean:', round(leaky.mean(), 3))
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



