Lesson 3 · Scikit-learn deep dive

Scikit-learn Tutorial #3: Preprocessing — Encoders

Video three of the eighteen-part series: turning categorical data into genuine numbers a model can use. OneHotEncoder, OrdinalEncoder, LabelEncoder, and…

⬇ Download notebook Open in Colab ↗

CourseScikit-learn deep dive
Lesson3 of 18
Video16 min
FormatJupyter notebook · 10 code cells

scikit-learn NumPy

What you'll learn

Why Categorical Data Needs Encoding
OneHotEncoder Basics
OneHotEncoder - Handling Unknown Categories
OneHotEncoder - the drop Parameter
OrdinalEncoder - When Order Genuinely Matters
OrdinalEncoder - Handling Unknown Categories
LabelEncoder - for Target Labels Only
LabelBinarizer - One-vs-Rest Style Target Encoding

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Scikit-learn Deep-Dive, Video 3: Preprocessing - Encoders#

Video three of the eighteen-part series: turning categorical data into genuine numbers a model can use.
OneHotEncoder, OrdinalEncoder, LabelEncoder, and LabelBinarizer.
Let's get into it.

Part 1: Why Categorical Data Needs Encoding#

import numpy as np
colors = np.array(['red', 'green', 'blue', 'green', 'red']).reshape(-1, 1)
sizes = np.array(['small', 'medium', 'large', 'medium', 'small']).reshape(-1, 1)
print(colors.ravel())
print(sizes.ravel())

['red' 'green' 'blue' 'green' 'red']
['small' 'medium' 'large' 'medium' 'small']

Part 2: OneHotEncoder Basics#

from sklearn.preprocessing import OneHotEncoder
ohe = OneHotEncoder(sparse_output=False)
colors_encoded = ohe.fit_transform(colors)
print(colors_encoded)
print(ohe.categories_)
print(ohe.get_feature_names_out(['color']))

[[0. 0. 1.]
 [0. 1. 0.]
 [1. 0. 0.]
 [0. 1. 0.]
 [0. 0. 1.]]
[array(['blue', 'green', 'red'], dtype='<U5')]
['color_blue' 'color_green' 'color_red']

Part 3: OneHotEncoder - Handling Unknown Categories#

ohe_strict = OneHotEncoder(sparse_output=False)
ohe_strict.fit(colors)
new_colors = np.array(['purple']).reshape(-1, 1)
try:
    ohe_strict.transform(new_colors)
except ValueError as e:
    print('caught: unknown category rejected')
ohe_safe = OneHotEncoder(sparse_output=False, handle_unknown='ignore')
ohe_safe.fit(colors)
print(ohe_safe.transform(new_colors))

caught: unknown category rejected
[[0. 0. 0.]]

Part 4: OneHotEncoder - the drop Parameter#

ohe_full = OneHotEncoder(sparse_output=False)
ohe_dropped = OneHotEncoder(sparse_output=False, drop='first')
full_result = ohe_full.fit_transform(colors)
dropped_result = ohe_dropped.fit_transform(colors)
print(full_result.shape[1])
print(dropped_result.shape[1])
print(ohe_dropped.get_feature_names_out(['color']))

3
2
['color_green' 'color_red']

Part 5: OrdinalEncoder - When Order Genuinely Matters#

from sklearn.preprocessing import OrdinalEncoder
size_order = [['small', 'medium', 'large']]
oe = OrdinalEncoder(categories=size_order)
sizes_encoded = oe.fit_transform(sizes)
print(sizes_encoded.ravel())
print(oe.categories_)

[0. 1. 2. 1. 0.]
[array(['small', 'medium', 'large'], dtype=object)]

Part 6: OrdinalEncoder - Handling Unknown Categories#

oe_safe = OrdinalEncoder(categories=size_order, handle_unknown='use_encoded_value', unknown_value=-1)
oe_safe.fit(sizes)
new_sizes = np.array(['extra-large']).reshape(-1, 1)
print(oe_safe.transform(new_sizes))

[[-1.]]

Part 7: LabelEncoder - for Target Labels Only#

from sklearn.preprocessing import LabelEncoder
target_labels = np.array(['cat', 'dog', 'bird', 'cat', 'dog'])
le = LabelEncoder()
encoded_labels = le.fit_transform(target_labels)
print(encoded_labels)
print(le.classes_)
print(le.inverse_transform(encoded_labels))

[1 2 0 1 2]
['bird' 'cat' 'dog']
['cat' 'dog' 'bird' 'cat' 'dog']

Part 8: LabelBinarizer - One-vs-Rest Style Target Encoding#

from sklearn.preprocessing import LabelBinarizer
lb = LabelBinarizer()
binarized = lb.fit_transform(target_labels)
print(binarized)
print(lb.classes_)
print(lb.inverse_transform(binarized))

[[0 1 0]
 [0 0 1]
 [1 0 0]
 [0 1 0]
 [0 0 1]]
['bird' 'cat' 'dog']
['cat' 'dog' 'bird' 'cat' 'dog']

Part 9: get_feature_names_out() - Naming Encoded Columns#

combined = np.hstack([colors, sizes])
ohe_multi = OneHotEncoder(sparse_output=False)
encoded_multi = ohe_multi.fit_transform(combined)
print(encoded_multi.shape)
print(ohe_multi.get_feature_names_out(['color', 'size']))

(5, 6)
['color_blue' 'color_green' 'color_red' 'size_large' 'size_medium'
 'size_small']

Part 10: A Real Pattern - Encoding Mixed Categorical Data End to End#

def encode_mixed_categoricals(nominal_col, ordinal_col, ordinal_order):
    ohe = OneHotEncoder(sparse_output=False, handle_unknown='ignore')
    oe = OrdinalEncoder(categories=[ordinal_order], handle_unknown='use_encoded_value', unknown_value=-1)
    nominal_encoded = ohe.fit_transform(nominal_col)
    ordinal_encoded = oe.fit_transform(ordinal_col)
    return np.hstack([nominal_encoded, ordinal_encoded])
final_features = encode_mixed_categoricals(colors, sizes, ['small', 'medium', 'large'])
print(final_features.shape)
print(final_features[0])

(5, 4)
[0. 0. 1. 0.]

Wrap-Up: What You Learned#

Almost every estimator requires numeric input; the right encoding choice depends on whether categories have a meaningful order.
OneHotEncoder creates one binary column per category, correct for unordered data since it implies no false numeric ordering.
handle_unknown='ignore' lets OneHotEncoder gracefully handle categories never seen during training, instead of raising.
drop='first' removes one redundant column per feature, avoiding perfect multicollinearity for linear models.
OrdinalEncoder maps categories to a single integer in an explicit, meaningful order; wrong for genuinely unordered categories.
OrdinalEncoder needs an explicit unknown_value, since there's no all-zeros equivalent for a single integer column.
LabelEncoder is for the target y specifically, expects 1-D input, and has no unknown-category handling.
LabelBinarizer turns a multiclass target into a one-hot-style binary matrix for algorithms/metrics that expect that shape.
get_feature_names_out() reliably names every output column, essential once multiple columns are encoded together.
A real pattern: encoding a genuinely mixed dataset, unordered with OneHotEncoder, ordered with OrdinalEncoder, then combined.
That wraps up encoders. Next up: handling missing data with SimpleImputer, KNNImputer, and IterativeImputer.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help