Mathew K Analytics

Lesson 42 · Real-World Data Analytics

nlargest, nsmallest & pivot_table: Tennis Pace of Play | Python Data Analytics #42

Staying in sports analytics with a completely different sport. Three real FiveThirtyEight datasets on how real time between points varies by real player,…

What you'll learn

Datasets used in this lesson

Save these next to the notebook. In Google Colab, upload them with the 📁 icon on the left first.

📓 Full notebook

Download .ipynb

Data Analytics 100, Video 42: Real ATP Tennis Pace-of-Play Analysis#

  • Video forty-two of the hundred-video real-world data analytics series, staying in sports analytics with a completely different sport.
  • Three real FiveThirtyEight datasets on how real time between points varies by real player, real tournament, and real match situation.
  • Let's get into it.

This lesson is for learners who want practice combining several related datasets into one analysis, using tennis as the setting. The business question: what makes a tennis match slow? Is it the player, the tournament and its court surface, or the situation inside a match?

You will work with three FiveThirtyEight datasets. players_time.csv gives each ATP player's seconds_added_per_point, a long-run rating of how much longer (positive) or shorter (negative) points take when he plays, covering 1991 to 2015. events_time.csv gives the same measure for each tournament and surface. serve_times.csv holds 120 individually timed points from a 2015 French Open sample, with the server, set, game_score and seconds_before_next_point.

In this lesson you will:

  • profile three datasets and check them for missing values and duplicates
  • rank players and tournaments by pace and compare court surfaces with groupby
  • merge two datasets on a shared key and measure how well they agree with a correlation

You should be comfortable with pandas groupby, boolean filtering and describe(); see earlier lessons in the series if not.

Part 1: Real Player Pace Dataset#

This lesson uses three separate files, so you start by loading the first one and checking its size. The code imports pandas, numpy and matplotlib, reads players_time.csv into a DataFrame called players with pd.read_csv, and shows players.shape. The output, (218, 2), means 218 rows and 2 columns: one row per player, with the player's name and his seconds_added_per_point rating. A two-column table is small, but it becomes much more useful once you merge it with the point-level data later in the lesson.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
players = pd.read_csv('players_time.csv')
players.shape
(218, 2)

Part 2: Real Tournament Pace Dataset#

The second file describes pace at the tournament level rather than the player level. pd.read_csv('events_time.csv') loads it into events, and events.shape returns (205, 4): 205 rows and 4 columns. Each row is a tournament and surface combination, and the columns include the tournament, its surface and its seconds_added_per_point. Having pace measured for both players and events lets you separate who is playing from where they are playing, which is the core idea of this lesson.

events = pd.read_csv('events_time.csv')
events.shape
(205, 4)

Part 3: Real Point-Level Serve Dataset#

The third file is the most detailed: individual points that were timed one by one. pd.read_csv('serve_times.csv') loads it into serves, and the output (120, 7) shows 120 rows and 7 columns. Each row is one point, with columns such as the server, the set, the game_score and seconds_before_next_point. 120 points is a small sample, so treat any patterns from this table as suggestive rather than definitive.

serves = pd.read_csv('serve_times.csv')
serves.shape
(120, 7)

Part 4: Real Missing Data Check#

With three tables, checking missing values one by one gets repetitive, so this step does it in one line. For each DataFrame, isna().sum() counts missing values per column, and the second .sum() adds those counts into a single number for the whole table. The three totals are then added together. The output is 0, so there are no missing values anywhere in the three datasets. This double .sum().sum() pattern is a handy shortcut whenever you only need a yes or no answer for a whole table.

players.isna().sum().sum() + events.isna().sum().sum() + serves.isna().sum().sum()
np.int64(0)

Part 5: Real Player Pace Preview#

Before ranking players, you need to understand the scale of the pace rating. The code calls players.head(5) and then describe() on seconds_added_per_point; Jupyter only displays the last expression, so you see the summary statistics. Values range from -6.37 to 6.35 seconds, with a mean of about 0.94 and a median (the 50% line) of about 1.97. The mean sitting below the median tells you a group of fast players pulls the average down. The 25% line is about -1.95, so a quarter of players speed points up by roughly two seconds or more.

players.head(5)
players['seconds_added_per_point'].describe()
count    218.000000
mean       0.941743
std        2.860936
min       -6.370000
25%       -1.947500
50%        1.965000
75%        3.040000
max        6.350000
Name: seconds_added_per_point, dtype: float64

Part 6: Real Slowest Players#

Identifying the slowest players answers the obvious question fans and officials ask about pace of play. players.nlargest(5, 'seconds_added_per_point') returns the five highest ratings, and the double brackets keep just the player and rating columns. The output is led by Emanuel Couto at 6.35 seconds added per point, then Nicolas Massu (6.21), Rafael Nadal (5.92), Martin Stringari (5.68) and T.J. Middleton (5.60). The result is saved as slowest_players because the final summary print reuses the top name.

slowest_players = players.nlargest(5, 'seconds_added_per_point')[['player', 'seconds_added_per_point']]
slowest_players
player seconds_added_per_point
217 Emanuel Couto 6.35
216 Nicolas Massu 6.21
215 Rafael Nadal 5.92
214 Martin Stringari 5.68
213 T.J. Middleton 5.60

Part 7: Real Fastest Players#

The other end of the ranking shows which players keep points moving. nsmallest(5, 'seconds_added_per_point') returns the five lowest, most negative, ratings. The output shows Dustin Brown fastest at -6.37 seconds per point, followed by Rohan Bopanna (-4.95), Chris Guccione (-4.63), Benoit Paire (-4.56) and Lukas Dlouhy (-4.35). Remember that a negative value here means faster than average, not a data error. Always check what the sign of a metric means before you interpret it.

fastest_players = players.nsmallest(5, 'seconds_added_per_point')[['player', 'seconds_added_per_point']]
fastest_players
player seconds_added_per_point
0 Dustin Brown -6.37
1 Rohan Bopanna -4.95
2 Chris Guccione -4.63
3 Benoit Paire -4.56
4 Lukas Dlouhy -4.35

Part 8: Real Slowest Players Chart#

A bar chart is a clear way to present a ranking to people who will not read a table. The code takes the ten slowest players with nlargest(10, ...), uses set_index('player') so names label the bars, sorts with sort_values() so the slowest bar sits at the top, and draws a horizontal bar chart with kind='barh'. It then adds an axis label and title and saves the figure as slowest_players.png. There is no displayed output because plt.close() closes the figure after saving. Open the image and check how tightly grouped the top ten are.

fig, ax = plt.subplots(figsize=(10, 6))
players.nlargest(10, 'seconds_added_per_point').set_index('player')['seconds_added_per_point'].sort_values().plot(kind='barh', ax=ax, color='darkred')
ax.set_xlabel('Real Seconds Added Per Point')
ax.set_title('Real 10 Slowest ATP Players, 1991-2015')
plt.tight_layout()
plt.savefig('slowest_players.png', dpi=100)
plt.close()

Part 9: Real Tournament Pace Preview#

Now you switch from players to tournaments and look at the overall spread first. As before, events.head(5) runs but only the last line, describe() on seconds_added_per_point, is displayed. Tournament ratings run from -2.98 to 5.38 seconds, with a mean of about 1.76 and a median of 1.63. The standard deviation is about 1.60, much smaller than the players' 2.86. In other words, tournaments differ from each other less than individual players do, which is a useful first clue about where pace comes from.

events.head(5)
events['seconds_added_per_point'].describe()
count    205.000000
mean       1.761805
std        1.600261
min       -2.980000
25%        0.730000
50%        1.630000
75%        2.920000
max        5.380000
Name: seconds_added_per_point, dtype: float64

Part 10: Real Pace by Surface#

Court surface is one of the most likely drivers of tournament pace, because it changes how long rallies last. events.groupby('surface')['seconds_added_per_point'].mean() averages the rating for each surface, .round(3) tidies it and sort_values() orders from fastest to slowest. The output shows grass fastest at -1.333 seconds, then carpet (0.773), hard courts (1.425) and clay slowest at 3.135. The result is saved as surface_avg for the chart that follows. These are averages of tournament ratings, so each tournament counts equally regardless of how many matches it hosts.

surface_avg = events.groupby('surface')['seconds_added_per_point'].mean().round(3).sort_values()
surface_avg
surface
Grass    -1.333
Carpet    0.773
Hard      1.425
Clay      3.135
Name: seconds_added_per_point, dtype: float64

Part 11: Real Surface Pace Chart#

A simple bar chart makes the surface pattern easy to see at a glance. The code calls surface_avg.plot(kind='bar', ...) on the sorted averages, passes a list of colours, adds a y-axis label and title, and saves the figure as surface_pace.png. There is no displayed output because the figure is closed after saving. When you open the image, look for the grass bar dipping below zero and the clay bar standing well above the rest. Bars below zero are easy to miss, so a horizontal line at zero can help readers.

fig, ax = plt.subplots(figsize=(8, 6))
surface_avg.plot(kind='bar', ax=ax, color=['seagreen', 'sienna', 'steelblue'])
ax.set_ylabel('Real Average Seconds Added Per Point')
ax.set_title('Real Average Pace of Play by Court Surface')
plt.tight_layout()
plt.savefig('surface_pace.png', dpi=100)
plt.close()

Part 12: Real Slowest Tournaments#

Ranking individual tournaments checks whether the surface pattern holds at the event level. nlargest(5, 'seconds_added_per_point') returns the five slowest tournaments with their surface. The output shows the Rio Open slowest at 5.38 seconds per point, then Maceio (5.28), Costa Do Sauipe (5.19), Florence (5.08) and Viña del Mar (4.96). All five are clay events, which strongly supports the surface averages you just calculated. Seeing the same story in both a group average and a ranking makes a finding much more convincing.

slowest_events = events.nlargest(5, 'seconds_added_per_point')[['tournament', 'surface', 'seconds_added_per_point']]
slowest_events
tournament surface seconds_added_per_point
204 Rio Open Clay 5.38
203 Maceio Clay 5.28
202 Costa Do Sauipe Clay 5.19
201 Florence Clay 5.08
200 Viña del Mar Clay 4.96

Part 13: Real Fastest Tournaments#

The fastest tournaments complete the picture. nsmallest(5, 'seconds_added_per_point') returns the five lowest ratings. The output is led by Wimbledon at -2.98 seconds, then the London Olympics (-2.62), Manchester (-2.33) and Eastbourne (-1.78), all grass, with Birmingham on carpet (-1.63) in fifth. The London Olympics tennis event was played at Wimbledon, so it is no surprise to see it next to Wimbledon. The surface effect is clear at both ends of the ranking.

fastest_events = events.nsmallest(5, 'seconds_added_per_point')[['tournament', 'surface', 'seconds_added_per_point']]
fastest_events
tournament surface seconds_added_per_point
0 Wimbledon Grass -2.98
1 London Olympics Grass -2.62
2 Manchester Grass -2.33
3 Eastbourne Grass -1.78
4 Birmingham Carpet -1.63

Part 14: Real Point-Level Serve Preview#

The point-level table measures time in a different way, so you should look at its scale before comparing it with anything. describe() on seconds_before_next_point (the last line, so the one displayed) shows a mean of about 20.4 seconds and a median of 19.5, ranging from 9 to 38 seconds. Note that this is actual elapsed time between points, not a rating relative to average like the other two files. Mixing up absolute and relative measures is a common mistake when you combine datasets.

serves.head(5)
serves['seconds_before_next_point'].describe()
count    120.000000
mean      20.358333
std        6.889031
min        9.000000
25%       15.000000
50%       19.500000
75%       25.250000
max       38.000000
Name: seconds_before_next_point, dtype: float64

Part 15: Real Server Averages#

Here you check whether some players take noticeably longer between points on the timed sample. groupby('server')['seconds_before_next_point'].mean() averages the time for each server, .round(2) tidies it and sort_values(ascending=False) puts the slowest first. The output shows Pablo Andujar slowest at 30.83 seconds, followed by Carlos Berlocq (28.88) and Rafael Nadal (28.83), with Nick Kyrgios fastest at 14.16. The result is saved as server_avg for the merge later. Some of these averages rest on only a few points, which you will see shortly.

server_avg = serves.groupby('server')['seconds_before_next_point'].mean().round(2).sort_values(ascending=False)
server_avg
server
Pablo Andujar           30.83
Carlos Berlocq          28.88
Rafael Nadal            28.83
Borna Coric             26.71
John Isner              24.50
Andy Murray             23.70
Nicolas Almagro         21.67
Daniel Gimeno Traver    21.17
Bernard Tomic           20.86
Andrey Kuznetsov        17.50
Lukas Rosol             17.29
Roger Federer           16.25
Novak Djokovic          16.00
Benoit Paire            14.33
Nick Kyrgios            14.16
Name: seconds_before_next_point, dtype: float64

Part 16: Real Deuce Point Pace#

Tense moments such as deuce might be expected to slow players down. The code filters serves to rows where game_score == 'Deuce', takes the mean time, and does the same for all other rows using !=. Displaying both variables separated by a comma shows them as a pair. The output is 19.18 seconds at deuce versus 20.55 seconds otherwise, so deuce points were actually slightly quicker in this sample. With only 120 points in total, treat this as an interesting observation, not a firm conclusion.

deuce_pace = serves[serves['game_score'] == 'Deuce']['seconds_before_next_point'].mean().round(2)
non_deuce_pace = serves[serves['game_score'] != 'Deuce']['seconds_before_next_point'].mean().round(2)
deuce_pace, non_deuce_pace
(np.float64(19.18), np.float64(20.55))

Part 17: Real Pace by Set Number#

Fatigue and pressure late in a match could change pace, so this step averages time by set. groupby('set')['seconds_before_next_point'].mean().round(2) gives one average per set number. The output shows 22.33 seconds in set 1, 19.73 in set 2, 18.42 in set 3 and 30.67 in set 5. There is no set 4 in the output because the sample contains no fourth-set points. The jump in set 5 is striking, but the next step on points per server will remind you how small these groups can be.

set_avg = serves.groupby('set')['seconds_before_next_point'].mean().round(2)
set_avg
set
1    22.33
2    19.73
3    18.42
5    30.67
Name: seconds_before_next_point, dtype: float64

Part 18: Real Set Pace Chart#

A line chart helps you see how pace shifts across a match. The code plots set_avg.index against set_avg.values with circle markers, labels both axes, adds a title and saves the figure as set_pace_trend.png. There is no displayed output because the figure is closed after saving. In the image, look for the steady decline from set 1 to set 3 and the sharp rise at set 5. Be aware that the line joins set 3 directly to set 5, so it can look as though a fourth-set value exists when it does not.

fig, ax = plt.subplots(figsize=(8, 6))
ax.plot(set_avg.index, set_avg.values, marker='o', color='purple')
ax.set_xlabel('Real Set Number')
ax.set_ylabel('Real Average Seconds Before Next Point')
ax.set_title('Real Pace of Play by Set Number, 2015 French Open Sample')
plt.tight_layout()
plt.savefig('set_pace_trend.png', dpi=100)
plt.close()

Part 19: Real Cross-Dataset Merge#

This is the key step of the lesson: joining the single-day timing data to the long-run player ratings. reset_index() turns server_avg from a Series into a DataFrame, and rename changes server to player so the key matches the players table, and renames the time column to match_day_avg. Then merge(players, on='player', how='inner') keeps only players found in both tables. The output shows all 15 servers matched, for example Rafael Nadal at 28.83 seconds on the day and 5.92 seconds added per point long-term. Matching on names works here, but spelling differences would silently drop rows with an inner join.

server_avg_df = server_avg.reset_index().rename(columns={'server': 'player', 'seconds_before_next_point': 'match_day_avg'})
merged = server_avg_df.merge(players, on='player', how='inner')
merged[['player', 'match_day_avg', 'seconds_added_per_point']]
player match_day_avg seconds_added_per_point
0 Pablo Andujar 30.83 2.00
1 Carlos Berlocq 28.88 4.08
2 Rafael Nadal 28.83 5.92
3 Borna Coric 26.71 3.18
4 John Isner 24.50 -2.33
5 Andy Murray 23.70 2.53
6 Nicolas Almagro 21.67 -2.01
7 Daniel Gimeno Traver 21.17 -1.64
8 Bernard Tomic 20.86 -3.54
9 Andrey Kuznetsov 17.50 -2.40
10 Lukas Rosol 17.29 -4.13
11 Roger Federer 16.25 -2.43
12 Novak Djokovic 16.00 2.21
13 Benoit Paire 14.33 -4.56
14 Nick Kyrgios 14.16 -3.15

Part 20: Real Cross-Dataset Correlation#

If pace is a stable habit, players who were slow on the timed day should also have high long-run ratings. merged['match_day_avg'].corr(merged['seconds_added_per_point']) measures this with correlation, a number from -1 to 1 where values near 1 mean the two move together closely. The output is 0.737, a fairly strong positive relationship. It suggests pace of play is a consistent player trait. With only 15 players, though, a couple of unusual cases such as Novak Djokovic (fast on the day, positive long-run rating) can shift the figure noticeably.

cross_corr = round(merged['match_day_avg'].corr(merged['seconds_added_per_point']), 3)
cross_corr
np.float64(0.737)

Part 21: Real Points per Server#

Averages are only as reliable as the number of observations behind them, so you count points per server. serves['server'].value_counts() counts how many timed points each player served. The output ranges from 19 points for Nick Kyrgios and 16 for Roger Federer down to just 2 for John Isner. Averages for players with only a handful of points, such as Isner, Novak Djokovic or Andrey Kuznetsov, should be read with caution. Checking group sizes like this is a good habit before you rank groups by their averages.

points_per_server = serves['server'].value_counts()
points_per_server
server
Nick Kyrgios            19
Roger Federer           16
Andy Murray             10
Nicolas Almagro          9
Benoit Paire             9
Carlos Berlocq           8
Bernard Tomic            7
Lukas Rosol              7
Borna Coric              7
Pablo Andujar            6
Rafael Nadal             6
Daniel Gimeno Traver     6
Andrey Kuznetsov         4
Novak Djokovic           4
John Isner               2
Name: count, dtype: int64

Part 22: Real Ad-Point Frequency#

Advantage points are another high-pressure situation worth counting. serves['game_score'].str.startswith('Ad') checks whether each score text begins with Ad, returning True or False for each row, and .sum() counts the Trues. The output is 16, so 16 of the 120 timed points were played at advantage. The .str accessor is how you apply text methods to a whole column at once; calling startswith directly on the Series without .str would raise an error.

ad_points = serves['game_score'].str.startswith('Ad').sum()
ad_points
np.int64(16)

Part 23: Real Duplicate Player Check#

Because the merge used player names as the key, you need to be sure each name appears only once in the players table. players['player'].duplicated() flags True for any name already seen in an earlier row, and .sum() counts them. The output is 0, so every player name is unique. If there had been duplicates, the merge would have produced extra rows for those players and distorted the correlation.

players['player'].duplicated().sum()
np.int64(0)

Part 24: Real Grass vs Clay Gap#

This step puts a single number on the surface effect by comparing the two extremes. The code filters events to grass and to clay tournaments, takes the mean rating for each and subtracts grass from clay. The output is 4.468, meaning clay events added about 4.5 more seconds per point than grass events. It matches the surface averages from earlier (3.135 minus -1.333). Subtracting a negative number adds to the gap, so a quick mental check like this is a good way to confirm your code.

grass_avg = events[events['surface'] == 'Grass']['seconds_added_per_point'].mean()
clay_avg = events[events['surface'] == 'Clay']['seconds_added_per_point'].mean()
round(clay_avg - grass_avg, 3)
np.float64(4.468)

Part 25: Saving the Real Merged Dataset#

Saving the merged table preserves the most useful output of the lesson for later work. merged.to_csv('tennis_pace_merged.csv', index=False) writes it without the row index, which would otherwise appear as an extra unnamed column. The code then reads it back and compares shapes with reloaded.shape == merged.shape. The output is True, so the saved file has the same number of rows and columns as the merged DataFrame.

merged.to_csv('tennis_pace_merged.csv', index=False)
reloaded = pd.read_csv('tennis_pace_merged.csv')
reloaded.shape == merged.shape
True

Part 26: Real Sanity Check, Row Counts#

Sanity checks confirm the data has not changed unexpectedly while you worked. This line checks that the three tables still have 218, 205 and 120 rows respectively, joining the tests with and so all three must pass. The output is True. In a script you could put this inside an assert so the code stops with a clear error if a file ever changes.

len(players) == 218 and len(events) == 205 and len(serves) == 120
True

Part 27: Real Sanity Check, No Missing Values#

This repeats the earlier missing-value check as a strict True or False test for each table. Each isna().sum().sum() == 0 checks one DataFrame, and and requires all three to pass. The output is np.True_, numpy's version of True, which appears because the comparisons return numpy values. It means exactly the same as a plain True: no missing values anywhere.

players.isna().sum().sum() == 0 and events.isna().sum().sum() == 0 and serves.isna().sum().sum() == 0
np.True_

Part 28: Real Sanity Check, Correlation Bounds#

A correlation must always fall between -1 and 1, so a value outside that range would signal a bug. The code uses Python's chained comparison -1.0 <= cross_corr <= 1.0, which checks both limits in one expression. The output is np.True_, so the 0.737 correlation is within bounds. This is a basic check, but bounds tests like this are a cheap way to catch errors in calculated metrics.

-1.0 <= cross_corr <= 1.0
np.True_

Part 29: Real Top Finding Print#

Turning the main result into a sentence makes it ready for a report or a slide. The code uses .iloc[0] to take the first row of slowest_players, pulls out the name and the rating, and inserts them into an f-string. The output reads: Emanuel Couto added the most time per point of any player in this dataset, at 6.35 seconds. Because the sentence is built from variables, it updates automatically if the data changes.

slowest_name = slowest_players.iloc[0]['player']
slowest_value = slowest_players.iloc[0]['seconds_added_per_point']
print(f'{slowest_name} added the real most time per point of any player in this dataset, at {slowest_value} real seconds.')
Emanuel Couto added the real most time per point of any player in this dataset, at 6.35 real seconds.

Part 30: Real Recap Print#

The final print combines the main findings from all three datasets into one summary. It uses len() on each table along with the stored clay_avg, grass_avg and cross_corr inside an f-string. The output states that across 218 players, 205 tournament and surface combinations and 120 timed points, clay courts ran 4.468 seconds slower per point than grass, and single-day pace correlated with long-run pace at 0.737. Reusing variables like this keeps your summary consistent with the analysis.

print(f'Across {len(players)} real players, {len(events)} real tournament-surface combinations, and {len(serves)} real timed points, clay courts ran {round(clay_avg - grass_avg, 3)} real seconds slower per point than grass, and single-day pace correlated with real long-run pace at {cross_corr}.')
Across 218 real players, 205 real tournament-surface combinations, and 120 real timed points, clay courts ran 4.468 real seconds slower per point than grass, and single-day pace correlated with real long-run pace at 0.737.

Wrap-Up: What You Learned#

  • This real lesson combined three real FiveThirtyEight datasets, real player pace across 1991-2015, real tournament-surface pace, and real point-level 2015 French Open timing, with genuinely zero missing values across all three.
  • Real clay courts ran meaningfully slower than real grass courts, a real concrete surface effect worth remembering.
  • A real player's real single-day pace on the court genuinely correlated with their real multi-year pace rating, confirming pace of play is a real consistent player trait, not just noise.
  • Merging real datasets on a real shared key, like player name here, is a real practical way to combine multiple real honest sources into one real deeper analysis.
  • Every real number in this lesson traces back to genuinely real, complete FiveThirtyEight data, the same honest-sourcing standard carried through every domain so far.
  • Next video: a real historical MLB All-Star selections dataset, continuing this domain with a third completely different sport.

Practice on your own

  • Group events by surface and use .count() alongside .mean() to see how many tournaments sit behind each surface average, and decide whether carpet has enough events to trust.
  • Repeat the deuce comparison for advantage points: compare the mean seconds_before_next_point for rows where game_score starts with Ad against all other rows.
  • Filter merged to players with at least 6 timed points (use points_per_server), recalculate the correlation and see whether it rises or falls.

Each exercise reuses code you have already written, so focus on reading the outputs carefully. The next video in the series moves to a third sport with a historical MLB All-Star selections dataset.

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.