Lesson 21 · Python standard library deep dive

Python shutil, tempfile & glob Explained: File Operations | Standard Library #21

Video twenty-one of the twenty-five-part series: shutil, tempfile, and glob, real file system utilities. Copying, moving, archiving, temporary files and…

⬇ Download notebook ⬇ Notebook + data (.zip)Open in Colab ↗

CoursePython standard library deep dive
Lesson21 of 24
Video17 min
FormatJupyter notebook · 11 code cells
Data5 datasets

Datasets used in this lesson

Save these next to the notebook. In Google Colab, upload them with the 📁 icon on the left first.

📓 Full notebook

Download .ipynb

Python Standard Library Deep-Dive, Video 21: shutil, tempfile, glob#

Video twenty-one of the twenty-five-part series: shutil, tempfile, and glob, real file system utilities.
Copying, moving, archiving, temporary files and directories, and pattern-based file matching.
Let's get into it.

Part 1: shutil.copy(), copy2(), copyfile()#

import shutil, os
os.makedirs('demo_src', exist_ok=True)
with open('demo_src/report.txt', 'w') as f:
    f.write('quarterly figures\n')
shutil.copy('demo_src/report.txt', 'demo_src/report_copy.txt')
shutil.copy2('demo_src/report.txt', 'demo_src/report_copy2.txt')
print(sorted(os.listdir('demo_src')))
with open('demo_src/report_copy.txt') as f:
    print(f.read())

['report.txt', 'report_copy.txt', 'report_copy2.txt']
quarterly figures

Part 2: shutil.copytree(), rmtree(), move()#

os.makedirs('demo_src/subdir', exist_ok=True)
with open('demo_src/subdir/nested.txt', 'w') as f:
    f.write('nested content\n')
shutil.copytree('demo_src', 'demo_backup')
print(sorted(os.listdir('demo_backup')))
print(sorted(os.listdir('demo_backup/subdir')))

['report.txt', 'report_copy.txt', 'report_copy2.txt', 'subdir']
['nested.txt']

shutil.move('demo_backup/report_copy.txt', 'demo_backup/renamed_report.txt')
print(sorted(os.listdir('demo_backup')))
shutil.rmtree('demo_backup')
print(os.path.exists('demo_backup'))

['renamed_report.txt', 'report.txt', 'report_copy2.txt', 'subdir']
False

Part 3: shutil.disk_usage() and shutil.which()#

usage = shutil.disk_usage('.')
print(usage.total > 0)
print(usage.free > 0)
python_path = shutil.which('python3')
print(python_path is not None)
print(shutil.which('a_command_that_genuinely_does_not_exist'))

True
True
True
None

Part 4: shutil.make_archive() and unpack_archive()#

archive_path = shutil.make_archive('demo_archive', 'zip', 'demo_src')
print(archive_path)
print(os.path.exists(archive_path))
shutil.unpack_archive(archive_path, 'demo_extracted')
print(sorted(os.listdir('demo_extracted')))

c:\PythonTraining\Full_videos\Notebooks\demo_archive.zip
True
['report.txt', 'report_copy.txt', 'report_copy2.txt', 'subdir']

Part 5: tempfile.NamedTemporaryFile#

import tempfile
with tempfile.NamedTemporaryFile(mode='w', suffix='.txt', delete=False) as tmp:
    tmp.write('temporary content')
    tmp_path = tmp.name
print(os.path.exists(tmp_path))
with open(tmp_path) as f:
    print(f.read())
os.remove(tmp_path)
print(os.path.exists(tmp_path))

True
temporary content
False

Part 6: tempfile.TemporaryDirectory#

with tempfile.TemporaryDirectory() as tmpdir:
    print(os.path.isdir(tmpdir))
    scratch_file = os.path.join(tmpdir, 'scratch.txt')
    with open(scratch_file, 'w') as f:
        f.write('scratch work')
    print(os.listdir(tmpdir))
    saved_path = tmpdir
print(os.path.exists(saved_path))

True
['scratch.txt']
False

Part 7: tempfile.mkstemp() and mkdtemp()#

fd, path = tempfile.mkstemp(suffix='.log')
print(os.path.exists(path))
os.write(fd, b'raw log line\n')
os.close(fd)
with open(path) as f:
    print(f.read())
os.remove(path)
temp_dir_path = tempfile.mkdtemp()
print(os.path.isdir(temp_dir_path))
os.rmdir(temp_dir_path)

True
raw log line

True

Part 8: glob.glob() Basics#

import glob
for name in ['a.txt', 'b.txt', 'c.csv', 'notes.md']:
    with open(f'demo_src/{name}', 'w') as f:
        f.write('x')
print(sorted(glob.glob('demo_src/*.txt')))
print(sorted(glob.glob('demo_src/*.csv')))
print(sorted(glob.glob('demo_src/*')))

['demo_src\\a.txt', 'demo_src\\b.txt', 'demo_src\\report.txt', 'demo_src\\report_copy.txt', 'demo_src\\report_copy2.txt']
['demo_src\\c.csv']
['demo_src\\a.txt', 'demo_src\\b.txt', 'demo_src\\c.csv', 'demo_src\\notes.md', 'demo_src\\report.txt', 'demo_src\\report_copy.txt', 'demo_src\\report_copy2.txt', 'demo_src\\subdir']

Part 9: Recursive glob() with ** and iglob()#

os.makedirs('demo_src/deep/nested', exist_ok=True)
with open('demo_src/deep/nested/buried.txt', 'w') as f:
    f.write('buried')
all_txt = glob.glob('demo_src/**/*.txt', recursive=True)
print(sorted(all_txt))
lazy_matches = glob.iglob('demo_src/**/*.txt', recursive=True)
print(type(lazy_matches))
print(sorted(list(lazy_matches)))

['demo_src\\a.txt', 'demo_src\\b.txt', 'demo_src\\deep\\nested\\buried.txt', 'demo_src\\report.txt', 'demo_src\\report_copy.txt', 'demo_src\\report_copy2.txt', 'demo_src\\subdir\\nested.txt']
<class 'generator'>
['demo_src\\a.txt', 'demo_src\\b.txt', 'demo_src\\deep\\nested\\buried.txt', 'demo_src\\report.txt', 'demo_src\\report_copy.txt', 'demo_src\\report_copy2.txt', 'demo_src\\subdir\\nested.txt']

Part 10: A Real Pattern - Backup and Cleanup Utility#

def backup_matching_files(source_dir, pattern, archive_name):
    with tempfile.TemporaryDirectory() as staging:
        matches = glob.glob(os.path.join(source_dir, pattern))
        for path in matches:
            shutil.copy2(path, staging)
        archive = shutil.make_archive(archive_name, 'zip', staging)
        return archive, len(matches)
archive_path, count = backup_matching_files('demo_src', '*.txt', 'demo_final_backup')
print(f'archived {count} files to {archive_path}')
print(os.path.exists(archive_path))
shutil.unpack_archive(archive_path, 'demo_final_check')
print(sorted(os.listdir('demo_final_check')))

archived 5 files to c:\PythonTraining\Full_videos\Notebooks\demo_final_backup.zip
True
['a.txt', 'b.txt', 'report.txt', 'report_copy.txt', 'report_copy2.txt']

Wrap-Up: What You Learned#

shutil.copy/copy2/copyfile copy files with increasing levels of metadata preservation.
shutil.copytree/rmtree/move handle whole directory trees; rmtree deletes recursively, so use it carefully.
shutil.disk_usage reports space; shutil.which searches PATH exactly like a shell would.
shutil.make_archive/unpack_archive zip or tar an entire directory tree in one call, and extract it back.
tempfile.NamedTemporaryFile gives a real file with a genuine path, auto-deleted on close by default.
tempfile.TemporaryDirectory gives a real directory, auto-deleted recursively on context-manager exit.
tempfile.mkstemp/mkdtemp are lower-level: they return a path with no automatic cleanup at all.
glob.glob matches files with shell-style wildcards; * within a segment, ? for a single character.
glob.glob(..., recursive=True) with ** matches across any subdirectory depth; iglob returns a lazy generator.
A real pattern: combining glob, tempfile, and shutil into a small backup utility.
That wraps up shutil, tempfile, and glob. Next up: io, string, and textwrap, for text and stream utilities.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.

☕ Buy me a coffee Other ways to help