Mathew K Analytics

Lesson 35 · Python for Banking and Finance

Automating Banking and Finance Workflows with Python Job Scheduling

Learn how to automate repetitive financial tasks using Python job scheduling. See why scheduling is crucial for real-time banking operations. Understand…

⬇ Download notebookOpen in Colab ↗

What you'll learn

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Scheduling Python Jobs: Real-World Banking Example#

  • Learn how to automate repetitive financial tasks using Python job scheduling.
  • See why scheduling is crucial for real-time banking operations.
  • Understand common data and time pitfalls.
  • Build real-world job schedulers for banking using pandas and schedule.
import pandas as pd
import numpy as np
import schedule
import time
import warnings
warnings.filterwarnings('ignore')

Core Data Concepts in Job Scheduling#

  • Banking data is often time-stamped and processed in batches.
  • Dataframes can represent transaction logs to be processed.
  • Scheduling errors usually occur because of time zone mismatches or missed intervals.
# Beginner Example 1: Synthetic Banking Transactions
np.random.seed(42)
n_transactions = 1000
n_customers = 200
df = pd.DataFrame({
    'transaction_id': range(1, n_transactions + 1),
    'customer_id': np.random.choice([f'CUST_{i:04d}' for i in range(1, n_customers + 1)], n_transactions),
    'amount': np.round(np.random.normal(150, 60, n_transactions), 2),
    'transaction_type': np.random.choice(['Debit', 'Credit'], n_transactions),
    'channel': np.random.choice(['ATM', 'Online', 'Branch', 'POS'], n_transactions),
    'date': pd.date_range(start='2024-01-01', periods=n_transactions, freq='h')
})
print(df.shape)
print(df.head(3))
(1000, 6)
   transaction_id customer_id  amount transaction_type channel  \
0               1   CUST_0103  238.77           Credit     ATM   
1               2   CUST_0180  269.17           Credit     POS   
2               3   CUST_0093   58.62           Credit  Online   

                 date  
0 2024-01-01 00:00:00  
1 2024-01-01 01:00:00  
2 2024-01-01 02:00:00  
# Beginner Example 2: Load schedule and define a simple job
def simple_job():
    print('Banking job executed!')

schedule.every(10).seconds.do(simple_job)
print('Scheduled a simple job to run every 10 seconds.')
Scheduled a simple job to run every 10 seconds.
# Beginner Example 3: Run the simple scheduler loop (demo only)
import threading
def run_scheduler(duration=25):
    start = time.time()
    while time.time() - start < duration:
        schedule.run_pending()
        time.sleep(1)

print('Starting the scheduler for 25 seconds...')
run_scheduler(25)
Starting the scheduler for 25 seconds...
Banking job executed!
Banking job executed!
Banking job executed!
# Intermediate Example 1: Schedule a job to summarize daily transaction totals
def summarize_daily_transactions():
    daily = df.resample('D', on='date').amount.sum()
    print(f'Total daily transaction amount: {daily.iloc[-1]:.2f}')

schedule.clear()
schedule.every().day.at('10:00').do(summarize_daily_transactions)
print('Scheduled summary job for 10:00 AM daily.')
Scheduled summary job for 10:00 AM daily.
# Intermediate Example 2: Parameterized scheduled job by transaction channel
def summarize_by_channel(channel):
    daily = df[df['channel']==channel].resample('D', on='date').amount.sum()
    print(f"Total daily amount for {channel}: {daily.iloc[-1]:.2f}")

schedule.clear()
for ch in df['channel'].unique():
    schedule.every().day.at('10:05').do(summarize_by_channel, ch)
print('Scheduled separate summaries for each transaction channel at 10:05.')
Scheduled separate summaries for each transaction channel at 10:05.
# Intermediate Example 3: Job that flags high-value transactions for review
def flag_high_value():
    review = df[(df['amount'] > 500) & (df['date'].dt.date == pd.Timestamp.now().date())]
    print(f'Flagged {len(review)} high-value transactions today.')

schedule.clear()
schedule.every().day.at('15:00').do(flag_high_value)
print('Scheduled high-value transaction screening at 3 PM daily.')
Scheduled high-value transaction screening at 3 PM daily.

Advanced Scheduling in Banking#

  • Jobs can run in background threads to avoid blocking main systems.
  • Parameterized, dynamic jobs build flexibility.
  • Common errors: jobs not being called, intervals overlapping, or missed deadlines.
# Advanced Example 1: Run the schedule loop in the background using threading
def start_scheduler_thread():
    stop_event = threading.Event()
    def loop():
        while not stop_event.is_set():
            schedule.run_pending()
            time.sleep(1)
    t = threading.Thread(target=loop, daemon=True)
    t.start()
    return stop_event

stop_handle = start_scheduler_thread()
print('Scheduler now runs in a background thread. Use stop_handle.set() to stop.')
Scheduler now runs in a background thread. Use stop_handle.set() to stop.
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
This job should be removed after first run.
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Error occurred: Database unavailable!
Settlement job runs safely!
Settlement job runs safely!
Settlement job runs safely!
Settlement job runs safely!
# Advanced Example 2: Dynamic scheduling based on file or data event
def new_data_trigger():
    print('New transactions data received. Starting reconciliation.')
schedule.every().day.at('18:00').do(new_data_trigger)
print('Dynamic job registered to run daily at 6 PM.')
Dynamic job registered to run daily at 6 PM.
# Advanced Example 3: Schedule jobs with conditional parameters
def conditional_summary(hour):
    if hour < 12:
        print('Morning: Send dashboard updates.')
    else:
        print('Afternoon: Trigger batch settlements.')

from datetime import datetime
current_hour = datetime.now().hour
schedule.every().day.at('11:00').do(conditional_summary, hour=11)
schedule.every().day.at('16:00').do(conditional_summary, hour=16)
print('Set up conditional scheduler based on time of day.')
Set up conditional scheduler based on time of day.

Error Handling and Debugging#

  • Common scheduling errors include misaligned intervals and exceptions inside jobs.
  • Always log exceptions for future analysis.
  • Use print statements and logs to confirm schedules triggered.
# Example: Handle exceptions in scheduled banking jobs
def robust_job():
    try:
        # Simulate risky job
        raise ValueError('Database unavailable!')
    except Exception as ex:
        print(f'Error occurred: {ex}')

schedule.clear()
schedule.every(15).seconds.do(robust_job)
print('Scheduled robust job with error handling.')
Scheduled robust job with error handling.
# Debug print all active scheduled jobs
for job in schedule.jobs:
    print(job)
Job(interval=15, unit=seconds, do=robust_job, args=(), kwargs={})
# Remove a job safely by function reference
def cleanup_fun():
    print('This job should be removed after first run.')

schedule.every().minute.do(cleanup_fun)
schedule.clear(cleanup_fun)
print('The job has been scheduled, then removed immediately.')
The job has been scheduled, then removed immediately.

Patterns and Best Practices#

  • Use function names, not lambda, for easier removal.
  • Always wrap job code in try-except.
  • Schedule jobs off-business hours to avoid load spikes.
  • Test with small intervals, then scale up.
# Example: Wrap all scheduled jobs in a robust wrapper for logging
def job_wrapper(func):
    def safe(*args, **kwargs):
        try:
            return func(*args, **kwargs)
        except Exception as ex:
            print(f'ERROR: {ex}')
    return safe

def log_settlement():
    print('Settlement job runs safely!')

schedule.clear()
schedule.every(20).seconds.do(job_wrapper(log_settlement))
print('Scheduled all jobs with a robust wrapper.')
Scheduled all jobs with a robust wrapper.
# End-to-End: Banking Summary Scheduler and Output to File
def write_daily_summary():
    latest = df[df['date'].dt.date == pd.Timestamp.now().date()]
    day_sum = latest['amount'].sum()
    with open('banking_daily_total.txt', 'w') as f:
        f.write(f"Total for {pd.Timestamp.now().date()}: {day_sum:.2f}\n")
    print('Summary written to banking_daily_total.txt.')

schedule.clear()
schedule.every().day.at('20:00').do(write_daily_summary)

# Demo write to file now instead of waiting until 8 PM
write_daily_summary()
Summary written to banking_daily_total.txt.

Next steps and Excercises#

  • Try scheduling an alert email when large deposits arrive.
  • Connect in your real customer/account datasets.
  • Join us on YouTube for job monitoring with dashboards!

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.