Lesson 35 · Python for Banking and Finance
Automating Banking and Finance Workflows with Python Job Scheduling
Learn how to automate repetitive financial tasks using Python job scheduling. See why scheduling is crucial for real-time banking operations. Understand…
- CoursePython for Banking and Finance
- Lesson35 of 24
- Video19 min
- FormatJupyter notebook · 15 code cells
What you'll learn
Data
No separate download needed — the notebook creates or downloads everything it uses.
📓 Full notebook
Download .ipynbScheduling Python Jobs: Real-World Banking Example#
- Learn how to automate repetitive financial tasks using Python job scheduling.
- See why scheduling is crucial for real-time banking operations.
- Understand common data and time pitfalls.
- Build real-world job schedulers for banking using pandas and schedule.
import pandas as pd
import numpy as np
import schedule
import time
import warnings
warnings.filterwarnings('ignore')
Core Data Concepts in Job Scheduling#
- Banking data is often time-stamped and processed in batches.
- Dataframes can represent transaction logs to be processed.
- Scheduling errors usually occur because of time zone mismatches or missed intervals.
# Beginner Example 1: Synthetic Banking Transactions
np.random.seed(42)
n_transactions = 1000
n_customers = 200
df = pd.DataFrame({
'transaction_id': range(1, n_transactions + 1),
'customer_id': np.random.choice([f'CUST_{i:04d}' for i in range(1, n_customers + 1)], n_transactions),
'amount': np.round(np.random.normal(150, 60, n_transactions), 2),
'transaction_type': np.random.choice(['Debit', 'Credit'], n_transactions),
'channel': np.random.choice(['ATM', 'Online', 'Branch', 'POS'], n_transactions),
'date': pd.date_range(start='2024-01-01', periods=n_transactions, freq='h')
})
print(df.shape)
print(df.head(3))
# Beginner Example 2: Load schedule and define a simple job
def simple_job():
print('Banking job executed!')
schedule.every(10).seconds.do(simple_job)
print('Scheduled a simple job to run every 10 seconds.')
# Beginner Example 3: Run the simple scheduler loop (demo only)
import threading
def run_scheduler(duration=25):
start = time.time()
while time.time() - start < duration:
schedule.run_pending()
time.sleep(1)
print('Starting the scheduler for 25 seconds...')
run_scheduler(25)
# Intermediate Example 1: Schedule a job to summarize daily transaction totals
def summarize_daily_transactions():
daily = df.resample('D', on='date').amount.sum()
print(f'Total daily transaction amount: {daily.iloc[-1]:.2f}')
schedule.clear()
schedule.every().day.at('10:00').do(summarize_daily_transactions)
print('Scheduled summary job for 10:00 AM daily.')
# Intermediate Example 2: Parameterized scheduled job by transaction channel
def summarize_by_channel(channel):
daily = df[df['channel']==channel].resample('D', on='date').amount.sum()
print(f"Total daily amount for {channel}: {daily.iloc[-1]:.2f}")
schedule.clear()
for ch in df['channel'].unique():
schedule.every().day.at('10:05').do(summarize_by_channel, ch)
print('Scheduled separate summaries for each transaction channel at 10:05.')
# Intermediate Example 3: Job that flags high-value transactions for review
def flag_high_value():
review = df[(df['amount'] > 500) & (df['date'].dt.date == pd.Timestamp.now().date())]
print(f'Flagged {len(review)} high-value transactions today.')
schedule.clear()
schedule.every().day.at('15:00').do(flag_high_value)
print('Scheduled high-value transaction screening at 3 PM daily.')
Advanced Scheduling in Banking#
- Jobs can run in background threads to avoid blocking main systems.
- Parameterized, dynamic jobs build flexibility.
- Common errors: jobs not being called, intervals overlapping, or missed deadlines.
# Advanced Example 1: Run the schedule loop in the background using threading
def start_scheduler_thread():
stop_event = threading.Event()
def loop():
while not stop_event.is_set():
schedule.run_pending()
time.sleep(1)
t = threading.Thread(target=loop, daemon=True)
t.start()
return stop_event
stop_handle = start_scheduler_thread()
print('Scheduler now runs in a background thread. Use stop_handle.set() to stop.')
# Advanced Example 2: Dynamic scheduling based on file or data event
def new_data_trigger():
print('New transactions data received. Starting reconciliation.')
schedule.every().day.at('18:00').do(new_data_trigger)
print('Dynamic job registered to run daily at 6 PM.')
# Advanced Example 3: Schedule jobs with conditional parameters
def conditional_summary(hour):
if hour < 12:
print('Morning: Send dashboard updates.')
else:
print('Afternoon: Trigger batch settlements.')
from datetime import datetime
current_hour = datetime.now().hour
schedule.every().day.at('11:00').do(conditional_summary, hour=11)
schedule.every().day.at('16:00').do(conditional_summary, hour=16)
print('Set up conditional scheduler based on time of day.')
Error Handling and Debugging#
- Common scheduling errors include misaligned intervals and exceptions inside jobs.
- Always log exceptions for future analysis.
- Use print statements and logs to confirm schedules triggered.
# Example: Handle exceptions in scheduled banking jobs
def robust_job():
try:
# Simulate risky job
raise ValueError('Database unavailable!')
except Exception as ex:
print(f'Error occurred: {ex}')
schedule.clear()
schedule.every(15).seconds.do(robust_job)
print('Scheduled robust job with error handling.')
# Debug print all active scheduled jobs
for job in schedule.jobs:
print(job)
# Remove a job safely by function reference
def cleanup_fun():
print('This job should be removed after first run.')
schedule.every().minute.do(cleanup_fun)
schedule.clear(cleanup_fun)
print('The job has been scheduled, then removed immediately.')
Patterns and Best Practices#
- Use function names, not lambda, for easier removal.
- Always wrap job code in try-except.
- Schedule jobs off-business hours to avoid load spikes.
- Test with small intervals, then scale up.
# Example: Wrap all scheduled jobs in a robust wrapper for logging
def job_wrapper(func):
def safe(*args, **kwargs):
try:
return func(*args, **kwargs)
except Exception as ex:
print(f'ERROR: {ex}')
return safe
def log_settlement():
print('Settlement job runs safely!')
schedule.clear()
schedule.every(20).seconds.do(job_wrapper(log_settlement))
print('Scheduled all jobs with a robust wrapper.')
# End-to-End: Banking Summary Scheduler and Output to File
def write_daily_summary():
latest = df[df['date'].dt.date == pd.Timestamp.now().date()]
day_sum = latest['amount'].sum()
with open('banking_daily_total.txt', 'w') as f:
f.write(f"Total for {pd.Timestamp.now().date()}: {day_sum:.2f}\n")
print('Summary written to banking_daily_total.txt.')
schedule.clear()
schedule.every().day.at('20:00').do(write_daily_summary)
# Demo write to file now instead of waiting until 8 PM
write_daily_summary()
Next steps and Excercises#
- Try scheduling an alert email when large deposits arrive.
- Connect in your real customer/account datasets.
- Join us on YouTube for job monitoring with dashboards!
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



