Python library centre

Essential PyTorch Training Techniques for Effective Neural Network Development

PyTorch is a popular Python library for deep learning. It is used to build and train neural networks. PyTorch is known for its flexibility and ease of use.…

⬇ Download notebook Open in Colab ↗

CoursePython library centre
Video21 min
FormatJupyter notebook · 19 code cells

PyTorchNumPy

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Introduction to PyTorch#

PyTorch is a popular Python library for deep learning.
It is used to build and train neural networks.
PyTorch is known for its flexibility and ease of use.
You can use PyTorch for research, production, and prototyping.
PyTorch has real world uses in image recognition, natural language processing, and more.

import warnings; warnings.filterwarnings('ignore')
import sys
# To install PyTorch on Windows, run this command in your terminal:
# pip install torch torchvision torchaudio
import torch
import torchvision
import torchaudio

Core PyTorch Objects and Concepts#

PyTorch uses Tensors, similar to arrays.
Computations can run on the CPU or GPU.
Neural networks in PyTorch use the nn.Module class.
Autograd automatically computes gradients for training.

import torch

# Make a simple tensor
x = torch.tensor([1, 2, 3, 4])
print(x)

# Tensors can be multi-dimensional
y = torch.tensor([[1, 2], [3, 4]])
print(y)
print('Shape:', y.shape)

# Move tensor to GPU if available
if torch.cuda.is_available():
    x = x.to('cuda')
    print('Tensor is on GPU now!')
else:
    print('No GPU found. Using CPU.')

Beginner PyTorch Examples#

Create tensors from lists and numpy arrays.
Do simple math using tensors.
Convert tensors to numpy and back.

import numpy as np

# Create a numpy array and turn it into a tensor
arr = np.array([5, 6, 7])
t = torch.from_numpy(arr)
print(t)

# Add two tensors
a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])
c = a + b
print(c)

# Convert tensor back to numpy array
numpy_array = c.numpy()
print(numpy_array)

Intermediate PyTorch Examples#

Create random tensors.
Use autograd to calculate gradients.
Use basic neural network layers.

# Create a random tensor of shape (2, 3)
rand_tensor = torch.randn(2, 3)
print(rand_tensor)

# Using autograd for gradients
x = torch.tensor([2.0, 3.0], requires_grad=True)
y = x ** 2 + 2 * x + 1
y_sum = y.sum()
y_sum.backward()
print(x.grad)

import torch.nn as nn

# Create a linear layer
layer = nn.Linear(4, 2)

input_data = torch.randn(1, 4)
output = layer(input_data)
print(output)

Advanced PyTorch Examples#

Use the Dataset and DataLoader for batching data.
Build and train a custom neural network.

from torch.utils.data import Dataset, DataLoader

# Define a simple custom dataset
class MyDataset(Dataset):
    def __init__(self):
        self.data = torch.arange(10)

    def __len__(self):
        return len(self.data)

    def __getitem__(self, idx):
        return self.data[idx]

dataset = MyDataset()
loader = DataLoader(dataset, batch_size=3)

for batch in loader:
    print(batch)

import torch.nn.functional as F

# Build a custom neural network
class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc1 = nn.Linear(4, 3)
        self.fc2 = nn.Linear(3, 1)

    def forward(self, x):
        x = F.relu(self.fc1(x))
        x = self.fc2(x)
        return x

model = Net()
sample = torch.randn(2, 4)
result = model(sample)
print(result)

Error Handling and Debugging#

PyTorch will show errors if tensor sizes do not match.
Use print() to check intermediate values.
Use try/except to catch errors.

# Example of a tensor shape error
a = torch.randn(2, 3)
b = torch.randn(4, 3)
try:
    c = a + b
except RuntimeError as e:
    print('Error:', e)

# Print to debug inside the network
class DebugNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.fc = nn.Linear(2, 2)

    def forward(self, x):
        print('Input:', x)
        out = self.fc(x)
        print('Output:', out)
        return out

debug_model = DebugNet()
inp = torch.randn(1, 2)
debug_model(inp)

Best Practices in PyTorch#

Always check tensor shapes as you code.
Put your model in eval() mode when evaluating.
Use with torch.no_grad() to save memory when not training.
Save and load model weights using torch.save and torch.load.

# Set your model to eval() mode
model.eval()
with torch.no_grad():
    x = torch.randn(2, 4)
    out = model(x)
print(out)

# Save and load PyTorch model weights
torch.save(model.state_dict(), 'my_model.pth')

new_model = Net()
new_model.load_state_dict(torch.load('my_model.pth'))
new_model.eval()
print('Model loaded and ready for inference!')

PyTorch Mini Project: Linear Regression#

We will fit a line to simple data using PyTorch.
This is similar to y = mx + b.
We will create fake data, make a model, train it, and see the result.

# Make some fake data points
X = torch.linspace(0, 1, steps=10).unsqueeze(1)
y = 2 * X + 1 + 0.1 * torch.randn(10, 1)
print('X:', X.squeeze())
print('y:', y.squeeze())

# Make a simple linear regression model
linreg = nn.Linear(1, 1)
optimizer = torch.optim.SGD(linreg.parameters(), lr=0.2)
loss_fn = nn.MSELoss()

# Train for 100 steps
for i in range(100):
    pred = linreg(X)
    loss = loss_fn(pred, y)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    if i % 20 == 0:
        print(f'Step {i}, Loss: {loss.item():.4f}')

print('Trained parameters:', list(linreg.parameters()))

# Test the model
test_x = torch.tensor([[0.5]])
with torch.no_grad():
    test_y = linreg(test_x)
print('Predicted y for x=0.5:', test_y.item())

Thanks for Learning PyTorch!#

Please subscribe if you find this helpful.
Watch our other Python tutorials on the channel.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help