Lesson 10 · Python standard library deep dive

Python subprocess Explained: Run External Commands Safely | Standard Library #10

Video ten of the twenty-five-part series: subprocess, for launching and controlling external programs from Python. Running commands, capturing output,…

⬇ Download notebook Open in Colab ↗

CoursePython standard library deep dive
Lesson10 of 24
Video16 min
FormatJupyter notebook · 13 code cells

subprocess

Data

No separate download needed — the notebook creates or downloads everything it uses.

📓 Full notebook

Download .ipynb

Python Standard Library Deep-Dive, Video 10: subprocess#

Video ten of the twenty-five-part series: subprocess, for launching and controlling external programs from Python.
Running commands, capturing output, checking return codes, timeouts, environment variables, and Popen.
Let's get into it.

Part 1: What subprocess Offers#

import subprocess
import sys
result = subprocess.run([sys.executable, '-c', 'print(2 + 2)'])
print(result)

CompletedProcess(args=['c:\\Users\\makmw\\AppData\\Local\\Programs\\Python\\Python312\\python.exe', '-c', 'print(2 + 2)'], returncode=0)

Part 2: subprocess.run() Basics#

command = [sys.executable, '-c', 'print("hello from a subprocess")']
result = subprocess.run(command)
print(type(result))
print(result.args)
print(result.returncode)

<class 'subprocess.CompletedProcess'>
['c:\\Users\\makmw\\AppData\\Local\\Programs\\Python\\Python312\\python.exe', '-c', 'print("hello from a subprocess")']
0

Part 3: Capturing Output#

command = [sys.executable, '-c', 'print("to stdout")']
result = subprocess.run(command, capture_output=True, text=True)
print(repr(result.stdout))
print(repr(result.stderr))
print(type(result.stdout))

'to stdout\n'
''
<class 'str'>

script = 'import sys; print("normal output"); print("error output", file=sys.stderr)'
command = [sys.executable, '-c', script]
result = subprocess.run(command, capture_output=True, text=True)
print('STDOUT:', result.stdout.strip())
print('STDERR:', result.stderr.strip())

STDOUT: normal output
STDERR: error output

Part 4: Checking Return Codes and check=True#

failing_command = [sys.executable, '-c', 'import sys; sys.exit(1)']
result = subprocess.run(failing_command)
print(result.returncode)
if result.returncode != 0:
    print('Command failed')

1
Command failed

try:
    subprocess.run(failing_command, check=True)
except subprocess.CalledProcessError as e:
    print(f'Caught: {e}')
    print(e.returncode)

Caught: Command '['c:\\Users\\makmw\\AppData\\Local\\Programs\\Python\\Python312\\python.exe', '-c', 'import sys; sys.exit(1)']' returned non-zero exit status 1.
1

Part 5: Passing Input to a Process#

script = 'name = input(); print(f"Hello, {name}!")'
command = [sys.executable, '-c', script]
result = subprocess.run(command, input='Ana\n', capture_output=True, text=True)
print(result.stdout.strip())

Hello, Ana!

Part 6: Shell vs No Shell#

result = subprocess.run('echo hello from the shell', shell=True, capture_output=True, text=True)
print(result.stdout.strip())
user_input = 'safe text; rm -rf something'
safe_result = subprocess.run([sys.executable, '-c', 'import sys; print(sys.argv[1])', user_input], capture_output=True, text=True)
print(safe_result.stdout.strip())

hello from the shell
safe text; rm -rf something

Part 7: Timeouts#

slow_command = [sys.executable, '-c', 'import time; time.sleep(5)']
try:
    subprocess.run(slow_command, timeout=1)
except subprocess.TimeoutExpired as e:
    print(f'Caught: process ran longer than {e.timeout} seconds')

Caught: process ran longer than 1.0 seconds

Part 8: Environment Variables for Subprocesses#

import os
custom_env = os.environ.copy()
custom_env['MY_APP_MODE'] = 'testing'
script = 'import os; print(os.environ.get("MY_APP_MODE"))'
command = [sys.executable, '-c', script]
result = subprocess.run(command, env=custom_env, capture_output=True, text=True)
print(result.stdout.strip())

testing

Part 9: Popen for Advanced Control#

process = subprocess.Popen(
    [sys.executable, '-c', 'print("running via Popen")'],
    stdout=subprocess.PIPE,
    text=True
)
print('Popen call returned immediately')
stdout, stderr = process.communicate()
print(stdout.strip())
print(process.returncode)

Popen call returned immediately
running via Popen
0

Part 10: Common Patterns#

with open('demo_helper_script.py', 'w') as f:
    f.write('print("processed successfully")')
result = subprocess.run(
    [sys.executable, 'demo_helper_script.py'],
    capture_output=True,
    text=True
)
print(result.stdout.strip())

processed successfully

def run_command(command, timeout=10):
    try:
        result = subprocess.run(
            command, capture_output=True, text=True,
            timeout=timeout, check=True
        )
        return True, result.stdout
    except subprocess.CalledProcessError as e:
        return False, e.stderr
    except subprocess.TimeoutExpired:
        return False, 'Command timed out'
success, output = run_command([sys.executable, '-c', 'print("all good")'])
print(success, output.strip())
success, output = run_command([sys.executable, '-c', 'import sys; sys.exit(1)'])
print(success, output)

True all good
False

Wrap-Up: What You Learned#

subprocess.run launches an external command and waits for it to finish.
Prefer the list form over shell=True whenever possible; it avoids shell injection entirely.
capture_output and text control whether and how output is captured as strings.
check=True raises CalledProcessError on failure instead of a silent non-zero return code.
input feeds data to a subprocess's stdin; timeout guards against a hung process.
env controls the subprocess's environment variables; copy os.environ first to preserve the rest.
Popen offers non-blocking, lower-level control that run itself is built on top of.
Two real patterns: running an external script safely, and a reusable success/failure wrapper.
That wraps up subprocess. Next up: threading and multiprocessing, for concurrency.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help