Mathew K Analytics

Lesson 35 · Probability and Statistics in python

Bayesian Estimation Explained: Using SciPy and PyMC for Practical Inference in Python

Welcome! In this hands-on tutorial, you will explore probability, statistics, and Bayesian thinking using Python. No experience needed. By the end, youll…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Probability & Bayesian Estimation in Python: Complete Beginner Lesson#

Welcome! In this hands-on tutorial, you will explore probability, statistics, and Bayesian thinking using Python. No experience needed.

By the end, youll understand concepts like probability, mean, distributions, and you will try beginner-friendly Bayesian estimation with real data.

Let us start our journey!

import warnings
from statsmodels.tools.sm_exceptions import ConvergenceWarning, ValueWarning
warnings.filterwarnings("ignore", category=ValueWarning)
warnings.filterwarnings("ignore", category=ConvergenceWarning)
warnings.filterwarnings("ignore", category=RuntimeWarning)

# Import basic libraries for the whole notebook
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
np.random.seed(42)

What is Probability?#

Probability simply means how likely something is to happen. It is always a number between 0 (impossible) and 1 (certain).

Let us try simulating a coin flip.

# Simulate a fair coin flip 10 times
np.random.seed(1)
outcomes = np.random.choice(["Heads", "Tails"], size=10)
print("Coin flips:", outcomes)
Coin flips: ['Tails' 'Tails' 'Heads' 'Heads' 'Tails' 'Tails' 'Tails' 'Tails' 'Tails'
 'Heads']
# Count how many times we got heads
heads = np.sum(outcomes == "Heads")
tails = np.sum(outcomes == "Tails")
print("Heads:", heads, ", Tails:", tails)
Heads: 3 , Tails: 7

Dice Rolling: A Simple Discrete Probability#

Let us roll a six-sided die 12 times and see the results.

# Simulate rolling a die 12 times
np.random.seed(2)
die_rolls = np.random.randint(1, 7, size=12)
print("Die rolls:", die_rolls)
Die rolls: [1 6 1 4 3 4 1 3 2 4 6 3]
# Count frequency of each face
values, counts = np.unique(die_rolls, return_counts=True)
for val, c in zip(values, counts):
    print(f"Number {val}: {c} times")
    
Number 1: 3 times
Number 2: 1 times
Number 3: 3 times
Number 4: 3 times
Number 6: 2 times

Descriptive Statistics: Mean and Standard Deviation#

Descriptive statistics help you summarize a list of numbers, like your die rolls.

# Calculate mean and standard deviation
mean_value = np.mean(die_rolls)
std_value = np.std(die_rolls)
print("Mean:", mean_value)
print("Standard deviation:", std_value)
Mean: 3.1666666666666665
Standard deviation: 1.674979270186815

Handling Missing Data#

Missing data happens often in real datasets. Let us see how to spot it and deal with it safely.

# Create data with a missing value
data = pd.Series([5, 3, np.nan, 7, 6, np.nan])
print("Data:")
print(data)

# Find missing entries
missing = data.isnull()
print("Missing values:")
print(missing)
Data:
0    5.0
1    3.0
2    NaN
3    7.0
4    6.0
5    NaN
dtype: float64
Missing values:
0    False
1    False
2     True
3    False
4    False
5     True
dtype: bool
# Fill missing data with the mean
filled = data.fillna(data.mean())
print("After filling missing values:")
print(filled)
After filling missing values:
0    5.00
1    3.00
2    5.25
3    7.00
4    6.00
5    5.25
dtype: float64

Discrete Distributions: Bernoulli, Binomial, Poisson#

Discrete distributions deal with outcomes you can count: successes, arrivals, events.

from scipy.stats import bernoulli, binom, poisson

# Bernoulli: Single coin flip (success/failure)
probs = bernoulli.pmf([0, 1], p=0.5)
print("Bernoulli PMF (p=0.5):", probs)

# Binomial: 10 trials, p=0.5, probability of 6 successes
binom_p = binom.pmf(6, n=10, p=0.5)
print("Binomial (n=10, p=0.5) probability of 6 successes:", binom_p)

# Poisson: Probability of 3 arrivals, average rate 2
poisson_p = poisson.pmf(3, mu=2)
print("Poisson (mu=2) probability of 3 events:", poisson_p)
Bernoulli PMF (p=0.5): [0.5 0.5]
Binomial (n=10, p=0.5) probability of 6 successes: 0.2050781249999999
Poisson (mu=2) probability of 3 events: 0.18044704431548356

Continuous Distributions: Normal, Exponential#

Continuous distributions model things that can take on any value in a range, like heights and waiting times.

from scipy.stats import norm, expon

# Normal distribution (bell curve)
x = np.linspace(-3, 3, 100)
y = norm.pdf(x, loc=0, scale=1)
plt.figure(figsize=(6, 2))
plt.plot(x, y, label="Normal PDF")
plt.title("Normal Distribution")
plt.legend()
plt.show()

# Exponential: Models waiting times
x2 = np.linspace(0, 6, 100)
y2 = expon.pdf(x2, scale=2)
plt.figure(figsize=(6, 2))
plt.plot(x2, y2, color="orange", label="Exponential PDF")
plt.title("Exponential Distribution")
plt.legend()
plt.show()
No description has been provided for this image
No description has been provided for this image
# Data setup: Load Titanic dataset, use seaborn version or fallback dataset if not available
try:
    titanic = sns.load_dataset('titanic')
    print('Shape:', titanic.shape)
    print(titanic.head())
except Exception:
    # Fallback: Use penguins if Titanic is not available
    titanic = sns.load_dataset('penguins')
    print('Titanic dataset not found. Fallback to penguins dataset.')
    print('Shape:', titanic.shape)
    print(titanic.head())
    
Shape: (891, 15)
   survived  pclass     sex   age  sibsp  parch     fare embarked  class  \
0         0       3    male  22.0      1      0   7.2500        S  Third   
1         1       1  female  38.0      1      0  71.2833        C  First   
2         1       3  female  26.0      0      0   7.9250        S  Third   
3         1       1  female  35.0      1      0  53.1000        S  First   
4         0       3    male  35.0      0      0   8.0500        S  Third   

     who  adult_male deck  embark_town alive  alone  
0    man        True  NaN  Southampton    no  False  
1  woman       False    C    Cherbourg   yes  False  
2  woman       False  NaN  Southampton   yes   True  
3  woman       False    C  Southampton   yes  False  
4    man        True  NaN  Southampton    no   True  
 

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.