Mathew K Analytics

Lesson 6 · Real-World Data Analytics

Price Elasticity in Python: Do Discounts Really Pay? | Data Analytics #6

Do discounts actually grow revenue? We measure price elasticity of demand from real historical price changes in retail data, using a log-log regression in…

📓 Full notebook

Download .ipynb

Data Analytics 100, Video 6: Price and Discount Impact Analysis#

  • Video six of the hundred-video real-world data analytics series.
  • Measuring real price elasticity of demand directly from real historical price changes already sitting in the same retail data.
  • Let's get into it.

This lesson is for analysts who already have a cleaned retail transactions table and want to answer a pricing question with it: when this shop lowers a price, do customers buy enough extra units to make it worthwhile? That idea is called price elasticity of demand, the percentage change in quantity sold for a 1% change in price.

You will work with online_retail_clean.csv, the cleaned version of the online retail data built earlier in the series. The columns that matter here are StockCode, Description, UnitPrice, Quantity, InvoiceNo, InvoiceDate, Country and Revenue. No experiment is needed: the same products were sold at several different prices over time, and that natural variation is what you measure.

In this lesson you will:

  • estimate elasticity for one product with a log-log regression using np.polyfit
  • compare discounted and full-price orders on size and revenue
  • loop the method across hundreds of products and rank them by price sensitivity
  • use the fitted curve to suggest a revenue-maximising price, and learn its limits

You should be comfortable with pandas groupby, boolean masks and basic matplotlib; see earlier lessons in the series if not.

Part 1: Where Real Price Variation Comes From#

Before measuring how price affects demand, you need to know the data actually contains price changes to learn from. This step loads the cleaned file with pd.read_csv, and parse_dates=['InvoiceDate'] makes the date column a real datetime so you can use .dt methods later. The output (391150, 9) confirms 391,150 rows and 9 columns. Price variation in this shop comes from ordinary trading: the same item sold at different unit prices on different invoices, which is exactly the raw material for an elasticity estimate.

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
clean = pd.read_csv('online_retail_clean.csv', parse_dates=['InvoiceDate'])
clean.shape
(391150, 9)

Part 2: Real Description Lookup per Product#

A StockCode is fine for joins but meaningless on a chart or in a report, so you want a readable name for each product. The same code can appear with slightly different descriptions, so the code groups by StockCode and uses .agg(lambda s: s.mode().iloc[0]) to keep the most frequent description, taking the first if there is a tie. The output shows the first three, such as 10002 mapping to INFLATABLE POLITICAL GLOBE. This desc_lookup Series is reused throughout the lesson whenever a product needs a name.

desc_lookup = clean.groupby('StockCode')['Description'].agg(lambda s: s.mode().iloc[0])
desc_lookup.head(3)
StockCode
10002    INFLATABLE POLITICAL GLOBE 
10080       GROOVY CACTUS INFLATABLE
10120                   DOGGY RUBBER
Name: Description, dtype: object

Part 3: How Many Real Products Actually Have Price Variation#

Elasticity can only be estimated for products that were sold at more than one price. Here groupby('StockCode')['UnitPrice'].nunique() counts the distinct prices per product, then (price_variation > 1).sum() counts how many have more than one. The output shows 2,623 of 3,659 products, roughly 72%, have some price variation. That is encouraging: most of the catalogue carries a usable pricing signal. Keep in mind that two prices is a very thin basis for a regression, which is why a stricter threshold is used later.

price_variation = clean.groupby('StockCode')['UnitPrice'].nunique()
(price_variation > 1).sum(), len(price_variation)
(np.int64(2623), 3659)

Part 4: Picking One Real Product for a Deep Dive#

Working through one product in detail first lets you check every step by eye before you automate it. The code picks stock code 85099B, looks up its description (that line is not the last expression, so it is not displayed), and filters the full table to just that product with a boolean mask. The output shows 1,615 rows for this product, plenty of transactions to work with. Saving the filtered rows as sub means every later single-product step reads from one consistent slice.

deep_dive_sc = '85099B'
desc_lookup[deep_dive_sc]
sub = clean[clean['StockCode'] == deep_dive_sc]
sub.shape[0]
1615

Part 5: Real Quantity Sold at Each Real Price Point#

To see a demand curve you need total units sold at each price, not individual line items. The code groups sub by UnitPrice and uses named aggregation: TotalQty=('Quantity', 'sum') and Orders=('InvoiceNo', 'nunique'), counting distinct invoices. The output lists eight price points from 1.65 to 4.13 GBP. The pattern is not a smooth curve: 1.79 sold 19,136 units across 135 orders, while 2.08 sold 11,324 units across 1,062 orders, and 4.13 sold only 6 units. Lower prices go with much larger orders, a hint that bulk buyers may be getting lower price tiers.

by_price = sub.groupby('UnitPrice').agg(TotalQty=('Quantity', 'sum'), Orders=('InvoiceNo', 'nunique')).reset_index()
by_price.sort_values('UnitPrice')
UnitPrice TotalQty Orders
0 1.65 9230 65
1 1.74 1500 10
2 1.75 800 5
3 1.79 19136 135
4 1.95 3982 317
5 2.04 100 1
6 2.08 11324 1062
7 4.13 6 5

Part 6: Visualizing Real Price vs Real Quantity#

A scatter plot makes the price and quantity relationship visible before you fit any model, so you can spot outliers or odd shapes. The code uses plt.scatter with price on the x-axis and total quantity on the y-axis, puts the product name in the title via desc_lookup, and saves the figure with plt.savefig rather than showing it. When you open price_vs_quantity_scatter.png, look for the general downward slope and for the lone point at 4.13 GBP far to the right. With only eight points, each one strongly influences the fitted line.

plt.figure(figsize=(8, 6))
plt.scatter(by_price['UnitPrice'], by_price['TotalQty'], s=80, color='darkorange')
plt.xlabel('Real Unit Price (GBP)')
plt.ylabel('Real Total Quantity Sold')
plt.title(f'Real Price vs Quantity for {desc_lookup[deep_dive_sc]}')
plt.tight_layout()
plt.savefig('price_vs_quantity_scatter.png', dpi=120)
plt.close()

Part 7: Real Log-Log Elasticity Regression#

Taking the logarithm of both price and quantity turns a percentage relationship into a straight line, and the slope of that line is the elasticity. The code keeps price points with positive quantity (you cannot take the log of zero or a negative), applies np.log to both columns, then np.polyfit(log_price, log_qty, 1) fits a degree-1 line and returns the slope and the intercept. The output is -7.4: a 1% higher price is associated with about 7.4% fewer units. That is a very strong response, and with only eight points it should be treated as indicative rather than precise.

valid = by_price[by_price['TotalQty'] > 0]
log_price = np.log(valid['UnitPrice'])
log_qty = np.log(valid['TotalQty'])
elasticity, intercept = np.polyfit(log_price, log_qty, 1)
round(elasticity, 2)
np.float64(-7.4)

Part 8: Interpreting the Real Elasticity Value#

A raw number like -7.4 means little to a stakeholder, so this step translates it. The rule of thumb is that if the absolute elasticity is above 1, demand is elastic: quantity moves proportionally more than price, so a price cut can raise revenue. Below 1 it is inelastic and cuts mostly lose margin. The code tests abs(elasticity) > 1 and builds a sentence with an f-string. The output confirms this product is elastic, with a 1% price cut associated with roughly a 7.4% change in units sold. Remember that association is not proof the price caused the change.

is_elastic = abs(elasticity) > 1
print(f'Elasticity of {round(elasticity, 2)} means this product is {"elastic" if is_elastic else "inelastic"}: a 1% price cut is associated with roughly a {round(abs(elasticity), 1)}% change in units sold.')
Elasticity of -7.4 means this product is elastic: a 1% price cut is associated with roughly a 7.4% change in units sold.

Part 9: Defining a Real Regular Price vs Real Discount Price#

The raw data has no discount flag, so you need a working definition. The code takes the most common price with sub['UnitPrice'].mode().iloc[0] as the regular price (that value is not displayed because it is not the last line), copies sub with .copy() to avoid pandas warnings when adding a column, and flags rows priced below it. Calling .mean() on a True/False column gives the share of True values. The output 0.331 means about 33% of this product's rows were sold below the regular price. Note that rows above the regular price count as not discounted.

regular_price = sub['UnitPrice'].mode().iloc[0]
regular_price
sub = sub.copy()
sub['IsDiscounted'] = sub['UnitPrice'] < regular_price
sub['IsDiscounted'].mean().round(3)
np.float64(0.331)

Part 10: Real Average Order Size, Discounted vs Full Price#

If discounts drive demand, discounted orders should be bigger. The code splits sub with the IsDiscounted mask and its opposite ~sub['IsDiscounted'], then averages Quantity for each group. The output shows an average of 64.9 units per discounted line versus 10.5 at full price, more than six times larger. Be careful how you read this: in a wholesale-heavy shop, large orders may be what earns the lower price in the first place, so the direction of cause could run from quantity to price as well as the other way.

avg_qty_discounted = sub[sub['IsDiscounted']]['Quantity'].mean()
avg_qty_full = sub[~sub['IsDiscounted']]['Quantity'].mean()
round(avg_qty_discounted, 1), round(avg_qty_full, 1)
(np.float64(64.9), np.float64(10.5))

Part 11: Did Real Discounting Actually Grow Total Revenue#

Bigger orders only matter if they bring in more money overall. The code sums the Revenue column separately for discounted and full-price rows. The output shows about 61,462 GBP from discounted rows against about 23,579 GBP at full price, so roughly 72% of this product's revenue came at a below-regular price. That looks like a win for discounting, but this comparison cannot tell you what those customers would have spent at full price. It describes where revenue came from, not what a price change would do.

revenue_discounted = sub[sub['IsDiscounted']]['Revenue'].sum()
revenue_full = sub[~sub['IsDiscounted']]['Revenue'].sum()
round(revenue_discounted, 2), round(revenue_full, 2)
(np.float64(61461.84), np.float64(23578.7))

Part 12: Real Revenue per Percent Discount#

Not all discounts are equal, so this step asks whether deeper discounts go with bigger orders. The code computes each discounted row's DiscountPct relative to the regular price, then pd.cut with bins=[0, 10, 20, 30, 100] sorts them into bands, and observed=True hides empty bands. The output shows average quantity of 12.8 for discounts up to 10%, jumping to 142.9 for 10 to 20% and 142.0 for 20 to 30%. No rows fall above 30%. The jump suggests a threshold effect rather than a smooth increase, which is typical of tiered bulk pricing.

discount_rows = sub[sub['IsDiscounted']].copy()
discount_rows['DiscountPct'] = (regular_price - discount_rows['UnitPrice']) / regular_price * 100
discount_rows.groupby(pd.cut(discount_rows['DiscountPct'], bins=[0, 10, 20, 30, 100]), observed=True)['Quantity'].mean().round(1)
DiscountPct
(0, 10]      12.8
(10, 20]    142.9
(20, 30]    142.0
Name: Quantity, dtype: float64

Part 13: Scaling Up to Real Many Products#

One product is a case study; a pricing decision needs the whole catalogue. To keep the estimates trustworthy, the code only keeps products with at least four distinct prices and more than 500 units sold in total. Each condition is built as a boolean Series indexed by StockCode, they are combined with &, and .index returns the matching codes. The output shows 373 candidate products. A common mistake is using and instead of & with pandas Series, which raises an error, and forgetting the brackets around each condition.

qty_by_sc = clean.groupby('StockCode')['Quantity'].sum()
price_var_by_sc = clean.groupby('StockCode')['UnitPrice'].nunique()
candidates = price_var_by_sc[(price_var_by_sc >= 4) & (qty_by_sc > 500)].index
len(candidates)
373

Part 14: Real Elasticity Loop Across Products#

This step repeats the single-product method for every candidate. For each stock code, the loop sums quantity by price, drops non-positive totals, skips products with fewer than four usable price points or no price spread, then fits np.polyfit on the logged values and stores the slope with the product name and number of price points. .astype(float) makes sure np.log gets plain numbers. The output shows 373 results, so every candidate passed the checks. Collecting tuples in a list and building a DataFrame once at the end is faster than appending to a DataFrame inside the loop.

results = []
for sc in candidates:
    s = clean[clean['StockCode'] == sc]
    bp = s.groupby('UnitPrice')['Quantity'].sum()
    bp = bp[bp > 0]
    if len(bp) < 4 or bp.index.to_series().std() == 0:
        continue
    lp, lq = np.log(bp.index.values.astype(float)), np.log(bp.values.astype(float))
    e, _ = np.polyfit(lp, lq, 1)
    results.append((sc, desc_lookup[sc], e, len(bp)))
len(results)
373

Part 15: Real Elasticity Results Table#

A results table lets you sort, filter and share the estimates. The code builds elasticity_df from the list of tuples with named columns, then sorts by Elasticity and shows the five most negative. Only that last expression is displayed. The output lists PANTRY ROLLING PIN at about -8.83, JUMBO BAG PINK POLKADOT at -8.75 and ALARM CLOCK BAKELIKE IVORY at -8.64. Notice the NumPricePoints column: several of these extreme values rest on only four or five price points, so treat them as leads to investigate rather than settled facts.

elasticity_df = pd.DataFrame(results, columns=['StockCode', 'Description', 'Elasticity', 'NumPricePoints'])
elasticity_df.shape[0]
elasticity_df.sort_values('Elasticity').head(5)
StockCode Description Elasticity NumPricePoints
248 22978 PANTRY ROLLING PIN -8.827491 5
140 22386 JUMBO BAG PINK POLKADOT -8.745900 7
206 22730 ALARM CLOCK BAKELIKE IVORY -8.643995 5
49 21754 HOME BUILDING BLOCK WORD -8.480004 4
217 22766 PHOTO FRAME CORNICE -7.958406 4

Part 16: What Fraction Show Real Normal Demand Behavior#

Economic theory says demand should fall as price rises, so most elasticities ought to be negative. The code compares each value with zero and takes .mean() of the boolean result, which gives the share of products with negative elasticity. The output is 89.0%, so the large majority behave as expected. The other 11% show quantity rising with price, which usually signals something else at work, such as seasonal demand, price rises during busy periods or different customer types, rather than a genuine love of higher prices.

pct_negative = (elasticity_df['Elasticity'] < 0).mean()
round(pct_negative * 100, 1)
np.float64(89.0)

Part 17: Real Average Elasticity Across the Catalog#

A single summary figure helps you describe the catalogue as a whole. The code computes both the mean and the median elasticity, but only the median is displayed because it is the last expression. The output median is -2.99, meaning the typical product sells about 3% fewer units for every 1% price increase, comfortably in elastic territory. The median is the better headline here because the extreme values seen earlier, both very negative and strongly positive, would pull a mean around. Print both with print() if you want to compare them.

elasticity_df['Elasticity'].mean().round(2)
elasticity_df['Elasticity'].median().round(2)
np.float64(-2.99)

Part 18: Visualizing the Real Elasticity Distribution#

A histogram shows the full spread of elasticities, not just the average. The code uses plt.hist with 30 bins and adds a dashed red vertical line at -1 with plt.axvline, marking the boundary between elastic and inelastic products. The figure is saved as elasticity_distribution.png. When you open it, look for how much of the distribution sits to the left of the red line, where the bulk of products should be, and for the long tail of positive values on the right that you met in the previous step.

plt.figure(figsize=(9, 5))
plt.hist(elasticity_df['Elasticity'], bins=30, color='seagreen', edgecolor='white')
plt.axvline(-1, color='red', linestyle='--', label='Real elastic/inelastic boundary')
plt.xlabel('Real Estimated Elasticity')
plt.ylabel('Real Number of Products')
plt.title('Real Distribution of Price Elasticity Across the Catalog')
plt.legend()
plt.tight_layout()
plt.savefig('elasticity_distribution.png', dpi=120)
plt.close()

Part 19: Real Most Price-Sensitive Products#

The most price-sensitive products are the best candidates for promotions, because a small price cut is associated with a large jump in units. The code sorts by Elasticity ascending and keeps the top 10, then shows just the description and elasticity. The output runs from PANTRY ROLLING PIN at about -8.83 to PIGGY BANK RETROSPOT at about -7.72, with several jumbo bags and building block words in between. Before acting on this list, check how many price points and transactions sit behind each estimate.

most_elastic = elasticity_df.sort_values('Elasticity').head(10)
most_elastic[['Description', 'Elasticity']]
Description Elasticity
248 PANTRY ROLLING PIN -8.827491
140 JUMBO BAG PINK POLKADOT -8.745900
206 ALARM CLOCK BAKELIKE IVORY -8.643995
49 HOME BUILDING BLOCK WORD -8.480004
217 PHOTO FRAME CORNICE -7.958406
50 LOVE BUILDING BLOCK WORD -7.858437
357 CHILDRENS CUTLERY RETROSPOT RED -7.830124
287 JUMBO BAG 50'S CHRISTMAS -7.753061
150 GUMBALL COAT RACK -7.743492
181 PIGGY BANK RETROSPOT -7.721176

Part 20: Real Least Price-Sensitive Products#

At the other end are products where price cuts would mostly give away margin. The code keeps rows with elasticity above -1, sorts them from highest to lowest with ascending=False, and shows the top 10. The output is dominated by positive values, led by SET OF 20 VINTAGE CHRISTMAS NAPKINS at about 15.29 and JUMBO BAG VINTAGE DOILY at about 10.94. Positive elasticities this large are a warning sign: a seasonal item like Christmas napkins likely had higher prices in peak season when demand was high anyway, so the estimate reflects timing, not true price response.

least_elastic = elasticity_df[elasticity_df['Elasticity'] > -1].sort_values('Elasticity', ascending=False).head(10)
least_elastic[['Description', 'Elasticity']]
Description Elasticity
241 SET OF 20 VINTAGE CHRISTMAS NAPKINS 15.289603
270 JUMBO BAG VINTAGE DOILY 10.939941
259 PANTRY CHOPPING BOARD 6.685920
22 POTTERING IN THE SHED METAL SIGN 6.468089
71 JUMBO STORAGE BAG SKULLS 6.235546
250 SET 2 PANTRY DESIGN TEA TOWELS 5.879909
154 PICNIC BASKET WICKER LARGE 5.257591
295 WALL ART VILLAGE SHOW 4.902284
288 LETTER HOLDER HOME SWEET HOME 4.843712
366 JUMBO BAG BAROQUE BLACK WHITE 4.811251

Part 21: Real Revenue-Maximizing Price for the Deep-Dive Product#

The fitted model can be turned into a pricing suggestion. The code creates 50 evenly spaced prices between the observed minimum and maximum with np.linspace, predicts quantity with np.exp(intercept) * price ** elasticity (the log-log line converted back), multiplies by price to get revenue, and uses np.argmax to find the best one. The output is 1.65 GBP, the lowest price in the range. That is not a coincidence: with an elasticity below -1 predicted revenue always rises as price falls, so the answer will sit at the edge of whatever range you give it.

candidate_prices = np.linspace(valid['UnitPrice'].min(), valid['UnitPrice'].max(), 50)
predicted_qty = np.exp(intercept) * candidate_prices ** elasticity
predicted_revenue = candidate_prices * predicted_qty
optimal_price = candidate_prices[np.argmax(predicted_revenue)]
round(optimal_price, 2)
np.float64(1.65)

Part 22: Visualizing the Real Predicted Revenue Curve#

Plotting the predicted revenue curve shows why the optimum landed where it did. The code draws revenue against the candidate prices with plt.plot and marks the chosen price with a dashed red axvline, then saves revenue_curve_optimal_price.png. When you open it, expect a curve that falls steadily from left to right, with the red line at the far left edge. A real pricing decision would also need costs and margins: maximising revenue is not the same as maximising profit, and selling below cost can raise revenue while losing money.

plt.figure(figsize=(8, 6))
plt.plot(candidate_prices, predicted_revenue, color='purple')
plt.axvline(optimal_price, color='red', linestyle='--', label=f'Real optimal price: {round(optimal_price, 2)}')
plt.xlabel('Real Hypothetical Unit Price (GBP)')
plt.ylabel('Real Predicted Total Revenue')
plt.title(f'Real Predicted Revenue Curve for {desc_lookup[deep_dive_sc]}')
plt.legend()
plt.tight_layout()
plt.savefig('revenue_curve_optimal_price.png', dpi=120)
plt.close()

Part 23: Real Sanity Check on the Elasticity Math#

Whenever you rely on a fitted model, a quick independent check guards against mistakes like a sign error. The code compares the lowest and highest price points in valid, which are in price order from the earlier groupby, computes the percentage change in price and in quantity between them, and tests whether the signs differ with np.sign. The output is True: price went up while quantity went down, which agrees with the negative elasticity from the regression. It is a rough check on two points only, not a replacement for the fit.

manual_pct_price_change = (valid['UnitPrice'].iloc[-1] - valid['UnitPrice'].iloc[0]) / valid['UnitPrice'].iloc[0]
manual_pct_qty_change = (valid['TotalQty'].iloc[-1] - valid['TotalQty'].iloc[0]) / valid['TotalQty'].iloc[0]
np.sign(manual_pct_qty_change) != np.sign(manual_pct_price_change)
np.True_

Part 24: Saving the Real Elasticity Results#

Saving results lets you share them or pick them up in a later lesson without rerunning the loop. The code writes elasticity_df, sorted by elasticity, to product_price_elasticity.csv with index=False so the row index is not saved as an extra column. It then reads the file back and checks the row count matches. The output True confirms nothing was lost. Reloading and checking is a cheap habit that catches problems such as writing to the wrong folder or a truncated file.

elasticity_df.sort_values('Elasticity').to_csv('product_price_elasticity.csv', index=False)
reloaded = pd.read_csv('product_price_elasticity.csv')
reloaded.shape[0] == elasticity_df.shape[0]
True

Part 25: Real Business Takeaway#

This step turns the analysis into a sentence a manager can act on. The code counts products with elasticity below -1 as elastic and the rest as inelastic, then prints a summary. The output reports 303 elastic products, good candidates for discounting, and 70 inelastic ones where discounts mostly give away margin. The 70 includes the products with positive elasticity, which are better described as unreliable estimates than as truly insensitive. A useful next step would be to check margins on the 303 before planning any promotion.

n_elastic = (elasticity_df['Elasticity'] < -1).sum()
n_inelastic = (elasticity_df['Elasticity'] >= -1).sum()
print(f'{n_elastic} real products are genuinely elastic and good discounting candidates; {n_inelastic} are inelastic, where discounting mostly just gives away real margin.')
303 real products are genuinely elastic and good discounting candidates; 70 are inelastic, where discounting mostly just gives away real margin.

Part 26: Real Weekday Effect on Discount Timing#

Knowing when discounted sales happen can help with promotion timing. Back on the deep-dive product, the code extracts the weekday name with .dt.day_name(), groups by it and averages the boolean IsDiscounted flag to get the share of discounted rows per day. The output ranges from 39.1% on Tuesday down to 20.7% on Sunday. Saturday does not appear at all, which tells you this shop recorded no sales of this product on Saturdays. Differences of a few percentage points between weekdays may simply be noise.

sub['Weekday'] = sub['InvoiceDate'].dt.day_name()
sub.groupby('Weekday')['IsDiscounted'].mean().round(3).sort_values(ascending=False)
Weekday
Tuesday      0.391
Monday       0.357
Wednesday    0.344
Friday       0.330
Thursday     0.318
Sunday       0.207
Name: IsDiscounted, dtype: float64

Part 27: Real Country Mix Among Discount Buyers#

Discount buyers may come from particular markets, which matters if you plan targeted promotions. The code filters sub to discounted rows and uses value_counts().head(5) on Country to count rows per country. The output is dominated by the United Kingdom with 483 discounted rows, far ahead of the Netherlands with 14 and Germany and France with 8 each. Because the shop is UK-based and most of its trade is domestic, compare these counts with each country's overall share of sales before concluding that UK buyers favour discounts.

sub[sub['IsDiscounted']]['Country'].value_counts().head(5)
Country
United Kingdom    483
Netherlands        14
Germany             8
France              8
Belgium             4
Name: count, dtype: int64

Part 28: Real Correlation Between Elasticity and Sales Volume#

It is worth asking whether best-selling products are more or less price-sensitive than slow sellers. The code maps each product's total quantity sold onto elasticity_df with .map(volume_lookup), then .corr() builds a correlation matrix and .iloc[0, 1] pulls out the single value between the two columns. The output is -0.074, very close to zero, so there is essentially no linear relationship between sales volume and elasticity. In practice that means you cannot guess a product's price sensitivity from its popularity: you need to estimate it.

volume_lookup = clean.groupby('StockCode')['Quantity'].sum()
elasticity_df['TotalVolume'] = elasticity_df['StockCode'].map(volume_lookup)
elasticity_df[['Elasticity', 'TotalVolume']].corr().iloc[0, 1].round(3)
np.float64(-0.074)

Part 29: Real Price Point Count vs Elasticity Reliability#

Estimates built on few price points are less reliable, so it helps to see whether restricting to better-supported products changes the picture. The code keeps products with at least six price points and computes their mean elasticity. The row count is calculated but not shown, since only the last expression is displayed. The output mean is -3.38, a little more negative than the catalogue median of -2.99 seen earlier, so the better-supported products still look clearly elastic. Print reliable.shape[0] too if you want to see how many products remain.

reliable = elasticity_df[elasticity_df['NumPricePoints'] >= 6]
reliable.shape[0]
reliable['Elasticity'].mean().round(2)
np.float64(-3.38)

Part 30: Real Recap Print#

Ending with a one-line recap gives you a figure to quote in a report or slide. The code prints how many products were analysed and how many came out as elastic, reusing n_elastic from the business takeaway. The output reads 373 products analysed, 303 of them genuinely discount-worthy. If you rerun the notebook on fresh data, this line is the quickest place to see whether the overall picture has shifted.

print(f'Estimated price elasticity for {len(elasticity_df)} real products; {n_elastic} are genuinely discount-worthy.')
Estimated price elasticity for 373 real products; 303 are genuinely discount-worthy.

Wrap-Up: What You Learned#

  • Real price elasticity of demand can genuinely be estimated straight from a real product's own historical price changes, no experiment required.
  • A real log-log regression of quantity against price gives a real slope that is, by definition, the elasticity.
  • An elasticity below negative one means demand is genuinely elastic, discounting pays off in real extra volume; above negative one, discounting mostly gives away real margin.
  • The real most-frequent price makes a reasonable real stand-in regular price when no explicit discount flag exists in the raw data.
  • A real fitted elasticity curve can even suggest a real revenue-maximizing price directly, not just whether to discount at all.
  • Next video: real customer lifetime value modeling, projecting how much each real customer is actually worth going forward.

Practice on your own

  1. Rerun the deep dive (price table, log-log fit and discount share) for PANTRY ROLLING PIN (stock code 22978) and compare its elasticity with the -8.83 from the catalogue loop.
  2. Add a NumOrders count to the loop results and filter elasticity_df to products with at least six price points and 50 orders. Recompute the share of negative elasticities and the median, and see whether the positive outliers disappear.
  3. Split the deep-dive product's rows by month with .dt.month and check how its average UnitPrice and total Quantity move together across the year, to see whether seasonality could be driving the elasticity.

The next video in the series covers customer lifetime value modelling: projecting how much each customer is likely to be worth going forward.

🧠 Test yourself

Pick an answer to check it. Your best score is saved in this browser.

  1. How does the lesson estimate a product's price elasticity from its own price history?

  2. A product has an elasticity of -1.8. What does that mean?

  3. What does this code print?

    import pandas as pd
    prices = pd.Series([5.0, 5.0, 5.0, 4.0, 3.5])
    print(prices.mode().iloc[0])
    
  4. Why does the lesson only fit elasticity for products with several different price points?

  5. A product has an elasticity of -0.4. What is the sensible conclusion?

💻 Practice exercises

Try each one in the notebook or in your own editor before opening the solution.

  1. Create a small DataFrame for one product with 6 price points (UnitPrice) and the total quantity sold at each (TotalQty). Fit a log-log regression with np.polyfit, print the elasticity, and print whether the product is elastic or inelastic.

    💡 Hint

    Take np.log of both columns, then polyfit with degree 1.

    ✅ Show solution
    import numpy as np
    import pandas as pd
    
    by_price = pd.DataFrame({
        'UnitPrice': [1.25, 1.45, 1.65, 1.85, 2.10, 2.50],
        'TotalQty':  [980, 760, 610, 480, 350, 240],
    })
    log_price = np.log(by_price['UnitPrice'])
    log_qty = np.log(by_price['TotalQty'])
    elasticity, intercept = np.polyfit(log_price, log_qty, 1)
    kind = 'elastic' if abs(elasticity) > 1 else 'inelastic'
    print('Elasticity:', round(elasticity, 2))
    print('This product is', kind)
    
  2. Make 14 transaction rows (Quantity, UnitPrice) for one product. Define the regular price as the mode of UnitPrice, flag discounted rows, add Revenue, then print the share of discounted rows, average quantity per row for discounted vs full price, and revenue from each group.

    💡 Hint

    IsDiscounted = UnitPrice < regular_price, then group or filter.

    ✅ Show solution
    import pandas as pd
    
    sub = pd.DataFrame({
        'Quantity': [6, 12, 6, 24, 6, 18, 6, 30, 6, 12, 6, 24, 6, 6],
        'UnitPrice': [2.5, 2.0, 2.5, 1.8, 2.5, 2.0, 2.5, 1.8, 2.5, 2.0, 2.5, 1.8, 2.5, 2.5],
    })
    sub['Revenue'] = sub['Quantity'] * sub['UnitPrice']
    regular_price = sub['UnitPrice'].mode().iloc[0]
    sub['IsDiscounted'] = sub['UnitPrice'] < regular_price
    print('Regular price:', regular_price)
    print('Discounted share:', round(sub['IsDiscounted'].mean(), 3))
    print(sub.groupby('IsDiscounted').agg(AvgQty=('Quantity', 'mean'), Revenue=('Revenue', 'sum')).round(2))
    
  3. Build price/quantity data for 3 products (5 price points each, quantity falling as price rises). Loop over the products, estimate each elasticity with a log-log fit, store results in a DataFrame, label each Elastic or Inelastic, and also find the revenue-maximizing price for the first product from a grid of candidate prices.

    💡 Hint

    Predicted quantity is np.exp(intercept) * price ** elasticity; revenue is price times that.

    ✅ Show solution
    import numpy as np
    import pandas as pd
    
    prices = np.array([1.0, 1.5, 2.0, 2.5, 3.0])
    data = {
        'MUG':    np.array([1000, 540, 330, 220, 160]),   # steep drop: elastic
        'CANDLE': np.array([500, 430, 380, 340, 310]),    # gentle drop: inelastic
        'BAG':    np.array([800, 520, 380, 300, 250]),
    }
    results = []
    fits = {}
    for name, qty in data.items():
        e, intercept = np.polyfit(np.log(prices), np.log(qty), 1)
        fits[name] = (e, intercept)
        results.append((name, round(e, 2), 'Elastic' if e < -1 else 'Inelastic'))
    elasticity_df = pd.DataFrame(results, columns=['Product', 'Elasticity', 'Type'])
    print(elasticity_df)
    
    e, intercept = fits['MUG']
    grid = np.linspace(prices.min(), prices.max(), 50)
    pred_revenue = grid * np.exp(intercept) * grid ** e
    print('Revenue-maximizing price for MUG on this grid:', round(grid[np.argmax(pred_revenue)], 2))
    

Found this useful?

All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.