Lesson 6 · R Programming: Zero to Expert
Advanced R Programming: purrr, R6, data.table & testthat (Full Course) | R 106
Video 6 of R Programming: Zero to Expert, the finale. Functions, environments, OOP, speed, testing and a project. R script: 06advancedr.R. Download it from…
- CourseR Programming: Zero to Expert
- Lesson6 of 6
- Video1 h 12 min
- FormatR script + Jupyter notebook (R) · 115 code cells
What you'll learn
Files for this lesson
Save these in one folder, open the script in RStudio and set that folder as the working directory (Session → Set Working Directory → To Source File Location).
- 06_advanced_r.R21.9 KB
📓 Full notebook
Download .ipynbAdvanced R Programming: Write R Like an Expert#
Video 6 of R Programming: Zero to Expert, the finale.
Functions, environments, OOP, speed, testing and a project.
R script: 06_advanced_r.R. Download it from this page and open it in RStudio, or run this notebook with an R kernel (for example in Google Colab: Runtime → Change runtime type → R).
Quick recap of videos 1 to 5#
scores <- c(72, 88, 95, 61)
grade <- function(x) ifelse(x >= 70, "pass", "fail")
grade(scores)
mean(scores)
table(grade(scores))
Three scores pass and the sixty one fails. The mean is exactly seventy nine, and the table confirms one fail and three passes.
Functions are objects#
You can store them, pass them around and return them.
square <- function(x) x^2
class(square)
formals(square)
body(square)
square
The class is function. formals lists a single argument called x, with no default, which is why its value looks empty. body returns the expression x squared, and printing square shows its full source code.
x <- c(4, 8, 15, 16, 23, 42)
summaries <- list(mean = mean, median = median, sd = sd)
summaries$median(x)
sapply(summaries, function(f) f(x))
The median is fifteen point five. The sapply call gives a named vector: mean eighteen, median fifteen point five, and standard deviation about thirteen point five.
apply_twice <- function(f, x) f(f(x))
apply_twice(square, 3)
apply_twice(sqrt, 16)
Three squared is nine, and nine squared is eighty one. The square root of sixteen is four, and of four is two.
Anonymous functions and the lambda shortcut#
sapply(1:5, function(n) n * 10)
sapply(1:5, \(n) n * 10)
(\(a, b) a + b)(2, 3)
Both sapply calls return ten, twenty, thirty, forty and fifty. The immediately called lambda adds two and three to give five.
Closures and function factories#
A factory returns a new function that remembers its setup.
power_factory <- function(exp) {
function(x) x^exp
}
square2 <- power_factory(2)
cube_it <- power_factory(3)
square2(4)
cube_it(4)
Four squared is sixteen and four cubed is sixty four. Each new function remembered its own exponent.
cube_it
environment(cube_it)
ls(environment(cube_it))
get("exp", envir = environment(cube_it))
Printing cube it shows only x to the power of exp, plus a bytecode line and a hexadecimal environment address. Those addresses will look different on your machine. That environment contains exactly one variable, exp, and its value is three. That is where the memory lives.
Lazy evaluation#
R only evaluates an argument when it is first used.
lazy_demo <- function(a, b) {
a * 2
}
lazy_demo(5, stop("this is never evaluated"))
We get ten. The stop call was never run, because the function never touched b.
adder_factory <- function(n) {
function(x) x + n
}
k <- 5
add_k <- adder_factory(k)
k <- 100
add_k(1)
It returns one hundred and one. The promise for n was only evaluated on this first call, and by then k was already one hundred.
adder_factory2 <- function(n) {
force(n)
function(x) x + n
}
k <- 5
add_k2 <- adder_factory2(k)
k <- 100
add_k2(1)
Now we get six. Because n was forced inside the factory, changing k afterwards made no difference.
Pause and try it: a multiplier factory#
Write make_multiplier(m) returning a function that multiplies by m.
make_multiplier <- function(m) {
force(m)
\(x) x * m
}
double_it <- make_multiplier(2)
triple_it <- make_multiplier(3)
double_it(1:4)
triple_it(1:4)
The doubler gives two, four, six and eight, and the tripler gives three, six, nine and twelve.
Environments: where R finds things#
An environment is a bag of names, plus a link to its parent.
environment()
e <- new.env()
assign("fruit", "mango", envir = e)
e$count <- 3
get("fruit", envir = e)
ls(e)
At the top level we are in R underscore GlobalEnv, the global environment, where all your everyday variables live. get returns mango, and ls shows the two names, count and fruit, in alphabetical order.
exists("fruit")
exists("fruit", envir = e)
mget(c("fruit", "count"), envir = e)
parent.env(e)
fruit does not exist in the global environment, so the first answer is FALSE, but it does exist in e. mget returns a list with fruit equal to mango and count equal to three. And the parent of e is the global environment, because that is where we created it.
search()
We see the global environment first, then the attached packages like stats, graphics, utils and methods, and finally base at the very end. RStudio adds an entry or two of its own.
Lexical scoping#
Functions look up variables where they were DEFINED, not called.
x <- "global x"
show_x <- function() x
use_local <- function() {
x <- "local x"
show_x()
}
use_local()
We get global x. show x was defined at the top level, so it looks in the global environment, ignoring the local x inside the function that called it.
Remembering state with <<-#
<<- assigns in a PARENT environment. Use it with care.
make_counter <- function() {
count <- 0
function() {
count <<- count + 1
count
}
}
counter_a <- make_counter()
counter_b <- make_counter()
counter_a()
counter_a()
counter_b()
Counter a returns one and then two, while counter b returns one. Each counter has its own private environment, so they never interfere.
running <- 0
add_all <- function(v) {
for (val in v) running <<- running + val
}
add_all(1:4)
running
The function printed nothing, yet running is now ten. It changed a global variable behind our backs.
Environments are modified in place#
set_fruit <- function(env) {
env$fruit <- "papaya"
}
set_fruit(e)
e$fruit
Fruit is now papaya. The function changed the original environment, not a copy.
basket <- list(fruit = "mango")
set_fruit_list <- function(lst) {
lst$fruit <- "papaya"
lst
}
new_basket <- set_fruit_list(basket)
basket$fruit
new_basket$fruit
The original basket still holds mango, while the new basket holds papaya. Lists, vectors and data frames are copied when modified. Environments are not.
Pause and try it: a bank account closure#
Write make_account(balance) returning deposit() and balance().
make_account <- function(balance = 0) {
list(
deposit = function(amount) {
balance <<- balance + amount
invisible(balance)
},
balance = function() balance
)
}
acct <- make_account(100)
acct$deposit(50)
acct$balance()
The deposit printed nothing because it returns invisibly, and the balance is now one hundred and fifty.
Functional programming with purrr#
purrr gives type-stable, consistent tools for iteration.
library(tidyverse)
We see the usual attach message listing the core packages, plus a conflicts section. For example, dplyr's filter masks the filter function from stats.
nums <- list(
a = 1:5,
b = c(10, 20),
c = c(2.5, 3.5, 4)
)
map(nums, mean)
We get a list with three elements. a has mean three, b has mean fifteen, and c has mean three point three three.
map_dbl(nums, mean)
map_int(nums, length)
map_chr(nums, \(v) paste("n =", length(v)))
map_lgl(nums, \(v) all(v > 2))
map dbl gives a named numeric vector, three, fifteen and three point three three. The lengths are five, two and three. map chr builds labels like n equals five, and map lgl shows that only b and c have all their values above two.
map_dbl(nums, range)
We get an error saying the result must be length one, not two, and purrr even tells us it failed at index one, the element named a.
This one is meant to fail, so you can see the error message.
Two or more inputs: map2 and pmap#
names_vec <- c("Amina", "Kofi", "Lena")
ages <- c(31, 45, 28)
map2_chr(names_vec, ages, \(n, a) paste(n, "is", a))
We get three sentences: Amina is thirty one, Kofi is forty five, and Lena is twenty eight.
params <- tibble(
n = c(2, 3, 2),
mean = c(0, 50, 100),
sd = c(1, 5, 10)
)
set.seed(6)
pmap(params, \(n, mean, sd) round(rnorm(n, mean, sd), 1))
We get a list of three: two values near zero, three values near fifty, and two values near one hundred, each rounded to one decimal place.
Filtering and combining: keep, discard, reduce#
nums |> keep(\(v) mean(v) > 3)
nums |> discard(\(v) mean(v) > 3)
keep returns b and c, whose means are above three. discard returns only a, whose mean is exactly three, which is not above three.
reduce(1:5, \(acc, x) acc + x)
accumulate(1:5, \(acc, x) acc + x)
reduce gives fifteen, the total of one to five. accumulate shows the running totals one, three, six, ten and fifteen.
t1 <- tibble(id = 1:3, score = c(5, 6, 7))
t2 <- tibble(id = 1:3, team = c("x", "y", "z"))
t3 <- tibble(id = 1:3, active = c(TRUE, FALSE, TRUE))
reduce(list(t1, t2, t3), left_join, by = "id")
We get one tibble with three rows and four columns: id, score, team and active, all matched up by id.
When things fail: safely and possibly#
inputs <- list(10, "oops", 4)
map(inputs, log)
The whole map fails with non numeric argument to mathematical function, and purrr points at index two. We lose the good results too.
This one is meant to fail, so you can see the error message.
safe_log <- safely(log)
results <- map(inputs, safe_log)
results[[2]]
results |> map("error") |> map_lgl(is.null)
For the second element, result is NULL and error holds the simple error object with the message about a non numeric argument. The final check gives TRUE, FALSE, TRUE: elements one and three worked, element two failed.
possible_log <- possibly(log, otherwise = NA_real_)
map_dbl(inputs, possible_log)
We get two point three zero for log of ten, NA for the text, and one point three nine for log of four.
Pause and try it: count words#
Count the words in each sentence with strsplit() and map_int().
sentences <- c("R is fun", "purrr makes loops tidy", "hi")
sentences |>
strsplit(" ") |>
map_int(length)
The answer is three, four and one words.
Conditions: errors, warnings and messages#
R has three main conditions: error, warning and message.
check_age <- function(age) {
if (!is.numeric(age)) stop("age must be a number")
if (age < 0) warning("negative age, using 0")
message("Checking age ", age)
max(age, 0)
}
check_age(30)
We see the message, checking age thirty, and then the returned value, thirty.
check_age(-5)
The message appears, the function returns zero, and R adds a warning message at the end: negative age, using zero.
check_age("ten")
An error: age must be a number. R also tells us which call failed, check age with the string ten.
This one is meant to fail, so you can see the error message.
Handling conditions with tryCatch#
result <- tryCatch(
check_age("ten"),
error = function(e) {
message("Caught: ", conditionMessage(e))
NA
}
)
result
Instead of crashing, we see caught, age must be a number, and result is simply NA.
tryCatch(
check_age(-5),
warning = function(w) paste("Handled:", conditionMessage(w)),
finally = message("Cleanup always runs")
)
We see cleanup always runs, then the string handled, negative age, using zero. Notice what is missing: the checking age message never appeared. When tryCatch catches a condition, it abandons the rest of the function.
withCallingHandlers: handle, then carry on#
withCallingHandlers(
check_age(-5),
warning = function(w) {
message("Logged: ", conditionMessage(w))
invokeRestart("muffleWarning")
}
)
We see the logged warning first, then checking age minus five, and the return value zero. The function ran to completion.
tidy_up <- function() {
on.exit(message("on.exit: tidying up"))
stop("something broke")
}
res <- try(tidy_up(), silent = TRUE)
class(res)
The cleanup message printed even though the function failed. try returned an object of class try error, which we could test for.
Custom condition classes#
validate_price <- function(p) {
if (p < 0) {
stop(errorCondition(
paste("price cannot be negative:", p),
class = "negative_price",
data = list(price = p)
))
}
p
}
tryCatch(validate_price(-1), error = function(e) class(e))
The class vector is negative price, then error, then condition. Our error is still an ordinary error, but it is also something more specific.
tryCatch(
validate_price(-3),
negative_price = function(e) {
message("Fixing negative price ", e$data$price)
abs(e$data$price)
},
error = function(e) NA
)
The specific handler won. We see fixing negative price minus three, and the result is three.
Defensive programming#
Check inputs at the top: stopifnot(), match.arg(), missing().
summarise_scores <- function(x,
method = c("mean", "median",
"trimmed")) {
stopifnot("x must be numeric" = is.numeric(x),
"x must not be empty" = length(x) > 0)
method <- match.arg(method)
switch(method,
mean = mean(x),
median = median(x),
trimmed = mean(x, trim = 0.1)
)
}
summarise_scores(c(50, 60, 70, 100))
summarise_scores(c(50, 60, 70, 100), "med")
The default mean is seventy, and the median is sixty five.
summarise_scores(c(50, 60), "mode")
match arg stops with a clear message: arg should be one of mean, median, trimmed.
This one is meant to fail, so you can see the error message.
summarise_scores("ninety")
We get our own friendly message from stopifnot: x must be numeric.
This one is meant to fail, so you can see the error message.
describe_vec <- function(x, label) {
if (missing(label)) label <- deparse(substitute(x))
cat(label, "has", length(x), "values\n")
}
describe_vec(mtcars$mpg)
describe_vec(1:10, "my numbers")
The first call prints mtcars dollar sign mpg has thirty two values, using our own code as the label. The second uses my numbers, and reports ten values.
Pause and try it: a safe divide#
Write safe_divide(a, b): error if not numeric, warn if b is 0.
safe_divide <- function(a, b) {
stopifnot(is.numeric(a), is.numeric(b))
if (b == 0) {
warning("division by zero, returning NA")
return(NA_real_)
}
a / b
}
safe_divide(10, 4)
tryCatch(safe_divide(1, 0), warning = \(w) -1)
Ten divided by four is two point five, and the zero case was caught and replaced with minus one.
Object-oriented R: S3#
S3 = a class attribute + generic functions that dispatch on it.
f <- factor(c("low", "high", "low"))
typeof(f)
class(f)
unclass(f)
The type is integer, but the class is factor. Without its class, the factor is just the integers two, one, two, with a levels attribute holding high and low.
print
print is just one line: function of x and dots, use method print. Plus the bytecode and the base namespace it lives in.
Constructor, validator and helper#
Example: a temperature vector that knows its unit.
new_temp <- function(x = double(), unit = "C") {
stopifnot(is.double(x), is.character(unit))
structure(x, class = "temperature", unit = unit)
}
validate_temp <- function(t) {
unit <- attr(t, "unit")
if (!unit %in% c("C", "F")) stop("unit must be C or F")
limit <- if (unit == "C") -273.15 else -459.67
if (any(unclass(t) < limit)) stop("below absolute zero")
t
}
temperature <- function(x, unit = "C") {
validate_temp(new_temp(as.double(x), unit))
}
t1 <- temperature(c(21.5, 30, 18))
t1
It works, but the printout is clumsy: the numbers twenty one point five, thirty and eighteen, followed by raw attribute lines for the class and the unit.
temperature(-300)
We get the error below absolute zero, because minus three hundred Celsius is impossible.
This one is meant to fail, so you can see the error message.
Methods: format, print and summary#
format.temperature <- function(x, ...) {
paste(formatC(unclass(x), format = "f", digits = 1),
attr(x, "unit"))
}
print.temperature <- function(x, ...) {
cat("<temperature:", length(x), "readings>\n")
print(format(x), quote = FALSE)
invisible(x)
}
t1
Much better. A header says temperature, three readings, followed by twenty one point five C, thirty point zero C and eighteen point zero C.
summary.temperature <- function(object, ...) {
v <- unclass(object)
u <- attr(object, "unit")
cat("Range:", min(v), "to", max(v), u, "\n")
cat("Mean: ", round(mean(v), 2), u, "\n")
invisible(object)
}
summary(t1)
The range is eighteen to thirty C, and the mean is twenty three point one seven C.
t1[2:3]
We get the plain numbers thirty and eighteen. Square bracket subsetting dropped our class and unit attributes.
`[.temperature` <- function(x, i) {
new_temp(unclass(x)[i], attr(x, "unit"))
}
t1[2:3]
Now subsetting keeps the class: temperature, two readings, thirty point zero C and eighteen point zero C.
Your own generic function#
to_fahrenheit <- function(x, ...) UseMethod("to_fahrenheit")
to_fahrenheit.temperature <- function(x, ...) {
if (attr(x, "unit") == "F") return(x)
temperature(unclass(x) * 9 / 5 + 32, unit = "F")
}
to_fahrenheit.default <- function(x, ...) {
stop("can't convert an object of class ", class(x)[1])
}
to_fahrenheit(t1)
Our three readings become seventy point seven F, eighty six point zero F and sixty four point four F.
Inheritance with NextMethod()#
body_temp <- function(x) {
t <- temperature(x, "C")
class(t) <- c("body_temp", class(t))
t
}
print.body_temp <- function(x, ...) {
NextMethod()
cat("Readings with fever:", sum(unclass(x) >= 38), "\n")
invisible(x)
}
patient <- body_temp(c(36.8, 38.4, 37.1, 39))
patient
inherits(patient, "temperature")
We see the temperature header and four formatted readings, then our extra line, readings with fever, two. And inherits confirms that a body temp is also a temperature.
S4 in brief#
Formal classes with typed slots, validity checks and setMethod().
setClass("Account",
slots = c(owner = "character", balance = "numeric"),
validity = function(object) {
if (object@balance < 0) "balance cannot be negative" else TRUE
}
)
savings <- new("Account", owner = "Amina", balance = 250)
savings@balance
The balance slot holds two hundred and fifty, exactly what we supplied.
new("Account", owner = "Ben", balance = -5)
No. The validity function rejects it: invalid class Account object, balance cannot be negative.
This one is meant to fail, so you can see the error message.
setGeneric("deposit", function(acc, amount) {
standardGeneric("deposit")
})
setMethod("deposit", "Account", function(acc, amount) {
acc@balance <- acc@balance + amount
validObject(acc)
acc
})
setMethod("show", "Account", function(object) {
cat("Account of", object@owner, "| balance:", object@balance, "\n")
})
savings <- deposit(savings, 100)
savings
set generic prints the name deposit. Then our show method prints account of Amina, balance three hundred and fifty.
Copy-on-modify#
Names point to values. R copies a value only when you modify it.
a <- c(1, 2, 3)
b <- a
b[1] <- 100
a
b
a is still one, two, three, while b is one hundred, two, three. Changing b never affects a.
double_first <- function(v) {
v[1] <- v[1] * 2
v
}
out <- double_first(a)
a
out
No. a is still one, two, three, and out is two, two, three. Functions in R cannot change their inputs, they work on a copy.
e1 <- new.env()
e1$value <- 1
e2 <- e1
e2$value <- 999
e1$value
e1 dollar sign value is now nine hundred and ninety nine. There was never a copy, just two names for one environment. That is reference semantics.
R6: classes with reference semantics#
Mutable objects with fields and methods, like Python or Java.
library(R6)
BankAccount <- R6Class("BankAccount",
public = list(
owner = NULL,
balance = 0,
initialize = function(owner, balance = 0) {
self$owner <- owner
self$balance <- balance
},
deposit = function(amount) {
self$balance <- self$balance + amount
invisible(self)
},
withdraw = function(amount) {
if (amount > self$balance) stop("insufficient funds")
self$balance <- self$balance - amount
invisible(self)
},
print = function(...) {
cat("<BankAccount>", self$owner, "|", self$balance, "\n")
invisible(self)
}
)
)
acct1 <- BankAccount$new("Amina", 100)
acct1$deposit(50)$withdraw(30)
acct1
The account belongs to Amina and has a balance of one hundred and twenty: one hundred, plus fifty, minus thirty.
acct2 <- acct1
acct2$deposit(1000)
acct1$balance
The original now has one thousand one hundred and twenty. acct one and acct two are the same object, not copies.
acct3 <- acct1$clone()
acct3$deposit(5)
c(acct1$balance, acct3$balance)
The original stays at eleven hundred and twenty, and the clone has eleven hundred and twenty five. The clone really is separate.
Private fields and active bindings#
Thermostat <- R6Class("Thermostat",
private = list(.celsius = 20),
active = list(
celsius = function(value) {
if (missing(value)) return(private$.celsius)
stopifnot(is.numeric(value), value >= 5, value <= 35)
private$.celsius <- value
},
fahrenheit = function() private$.celsius * 9 / 5 + 32
)
)
th <- Thermostat$new()
th$celsius
th$fahrenheit
th$celsius <- 25
th$fahrenheit
It starts at twenty Celsius, which is sixty eight Fahrenheit. After setting twenty five Celsius, Fahrenheit reads seventy seven.
Inheritance in R6#
SavingsAccount <- R6Class("SavingsAccount",
inherit = BankAccount,
public = list(
rate = 0.05,
add_interest = function() {
self$deposit(self$balance * self$rate)
},
print = function(...) {
super$print()
cat(" interest rate:", self$rate, "\n")
invisible(self)
}
)
)
sav <- SavingsAccount$new("Kofi", 1000)
sav$add_interest()
sav
class(sav)
Kofi's balance is now one thousand and fifty, and the extra line shows the rate of zero point zero five. The class vector is SavingsAccount, BankAccount, then R6. R6 builds its class attribute just like S3 inheritance.
Performance: writing fast R#
Measure with system.time(). Vectorise. Preallocate. Never grow.
set.seed(1)
x <- runif(1e6)
loop_version <- function(x) {
out <- numeric(length(x))
for (i in seq_along(x)) {
out[i] <- if (x[i] > 0.5) x[i]^2 else 0
}
out
}
system.time(r1 <- loop_version(x))
The elapsed column is the one to read, it is wall clock time in seconds. For a million elements, the loop takes a small fraction of a second. Your exact numbers will differ.
system.time(r2 <- ifelse(x > 0.5, x^2, 0))
system.time(r3 <- (x > 0.5) * x^2)
identical(r1, r2)
identical(r1, r3)
Both vectorised versions are noticeably faster, and the arithmetic trick is usually the fastest of all. identical confirms that all three results match exactly.
Never grow an object inside a loop#
grow <- function(n) {
out <- c()
for (i in 1:n) out <- c(out, i^2)
out
}
prealloc <- function(n) {
out <- numeric(n)
for (i in 1:n) out[i] <- i^2
out
}
system.time(grow(3e4))
system.time(prealloc(3e4))
Growing takes around a second, while preallocating takes a few thousandths of a second. That is a difference of a couple of hundred times.
pieces <- lapply(1:2000, \(i) data.frame(id = i, value = i * 2))
system.time({
slow_df <- data.frame()
for (p in pieces) slow_df <- rbind(slow_df, p)
})
system.time(fast_df <- do.call(rbind, pieces))
system.time(tidy_df <- bind_rows(pieces))
nrow(fast_df)
The loop is clearly the slowest, do call is several times faster, and bind rows is fastest. All produce two thousand rows.
Reduce(`+`, 1:5, accumulate = TRUE)
Reduce shows the running totals one, three, six, ten and fifteen, exactly like purrr's accumulate.
data.table: speed for big data#
Syntax: DT[i, j, by] = where, select or compute, grouped by.
library(data.table)
data table lists the functions it masks: date helpers like month, year and wday from lubridate, between, first and last from dplyr, and transpose from purrr.
set.seed(2024)
n <- 1e6
DT <- data.table(
id = 1:n,
region = sample(c("North", "South", "East", "West"), n, TRUE),
product = sample(LETTERS[1:5], n, replace = TRUE),
units = rpois(n, 3),
price = round(runif(n, 5, 50), 2)
)
dim(DT)
head(DT, 3)
We have one million rows and five columns. head shows the first three transactions, each with its id, region, product, units and price.
DT[region == "North" & units > 5, .N]
DT[, .(revenue = sum(units * price)), by = region]
About twenty one thousand North transactions sold more than five units. The second result has one row per region, each with total revenue of roughly twenty million.
DT[, revenue := units * price]
DT[order(-revenue)][1:3, .(id, region, revenue)]
The top transactions each brought in several hundred, from many units at a price near fifty.
system.time(
DT[, .(total = sum(revenue)), by = .(region, product)]
)
tb <- as_tibble(DT)
system.time(
tb |>
group_by(region, product) |>
summarise(total = sum(revenue), .groups = "drop")
)
Both are quick on a million rows, but data table is usually the faster of the two. The gap grows dramatically with tens of millions of rows.
setkey(DT, region)
key(DT)
DT["East", .(orders = .N, avg_units = mean(units))]
The key is region. About a quarter of a million orders are from the East, averaging close to three units each, just as the Poisson mean of three suggests.
DT_alias <- DT
DT_alias[, flag := TRUE]
"flag" %in% names(DT)
Yes, TRUE. Because colon equals modifies in place, both names point at one table. It is the same reference semantics as environments and R6.
DT[, flag := NULL]
DT_copy <- copy(DT)
DT_copy[, flag := TRUE]
"flag" %in% names(DT)
This time the answer is FALSE. The copy has the flag column, but the original does not.
fwrite(DT, "sales_big.csv")
system.time(DT2 <- fread("sales_big.csv"))
dim(DT2)
round(file.size("sales_big.csv") / 1e6, 1)
fread loaded the whole file in a fraction of a second. DT2 has one million rows and six columns, including revenue, and the file is about twenty eight megabytes.
Pause and try it: data.table practice#
Total units per product in the West region, sorted high to low.
DT[region == "West",
.(units = sum(units)),
by = product][order(-units)]
We get five rows, one per product, each with roughly one hundred and fifty thousand units, sorted from the largest total down.
rm(DT, DT2, DT_alias, DT_copy, tb)
invisible(file.remove("sales_big.csv"))
Tidy evaluation: your own tidyverse functions#
dplyr uses column names as if they were variables.
group_mean_wrong <- function(df, group, value) {
df |>
group_by(group) |>
summarise(mean = mean(value))
}
group_mean_wrong(mpg, class, hwy)
We get an error: must group by variables found in the data, column group is not found.
This one is meant to fail, so you can see the error message.
group_mean <- function(df, group, value) {
df |>
group_by({{ group }}) |>
summarise(
n = n(),
mean = mean({{ value }}, na.rm = TRUE),
.groups = "drop"
)
}
group_mean(mpg, class, hwy)
It works. Seven car classes, with compact and subcompact cars leading on highway economy at about twenty eight miles per gallon, and pickups lowest at about seventeen.
Column names as strings: .data[[ ]]#
col_range <- function(df, var) {
df |>
summarise(
min = min(.data[[var]]),
max = max(.data[[var]])
)
}
c("hwy", "cty", "displ") |>
set_names() |>
map(\(v) col_range(mpg, v)) |>
list_rbind(names_to = "variable")
A tidy three row table: highway ranges from twelve to forty four, city from nine to thirty five, and engine displacement from one point six to seven litres.
across() and ... inside functions#
summarise_many <- function(df, group, cols) {
df |>
group_by({{ group }}) |>
summarise(
across({{ cols }}, \(x) round(mean(x), 1)),
.groups = "drop"
)
}
summarise_many(mpg, drv, c(hwy, cty, displ))
One row per drive type. Front wheel drive cars have the best economy, around twenty eight highway, and the smallest engines, about two point six litres.
top_counts <- function(df, ..., n = 5) {
df |>
count(..., sort = TRUE) |>
head(n)
}
top_counts(mpg, manufacturer)
top_counts(mpg, manufacturer, drv, n = 3)
Dodge is the most common manufacturer with thirty seven cars, then Toyota and Volkswagen. By manufacturer and drive type, the top three are front wheel drive Volkswagens, four wheel drive Dodges, and front wheel drive Toyotas.
Pause and try it: top rows by a column#
Write top_by(df, var, n = 3) using slice_max().
top_by <- function(df, var, n = 3) {
df |>
slice_max({{ var }}, n = n, with_ties = FALSE)
}
top_by(mpg, hwy) |> select(manufacturer, model, hwy)
The three most economical cars on the highway are Volkswagens: two at forty four miles per gallon and one at forty one.
Unit testing with testthat#
library(testthat)
testthat masks a few functions, such as matches from dplyr and tidyr, and some is functions from purrr. None of them matter for us today.
test_that("adder factory adds correctly", {
add_two <- adder_factory2(2)
expect_equal(add_two(3), 5)
expect_equal(add_two(-2), 0)
expect_equal(add_two(1:3), 3:5)
})
We see test passed, along with a small random celebration emoji. testthat likes to cheer you on.
test_that("temperature class behaves", {
expect_s3_class(temperature(20), "temperature")
expect_error(temperature(-500), "absolute zero")
expect_equal(length(t1[1:2]), 2)
expect_error(validate_price(-1), class = "negative_price")
})
Test passed again. All four expectations held.
test_that("floating point is exact", {
expect_identical(0.1 + 0.2, 0.3)
})
We see a failure report: zero point one plus zero point two is not identical to zero point three. The objects are equal, but not identical, so the test stops with test failed.
This one is meant to fail, so you can see the error message.
test_that("floating point is close enough", {
expect_equal(0.1 + 0.2, 0.3)
})
Test passed. Always compare decimals with a tolerance, never with exact equality.
Recursion and memoisation#
fact <- function(n) {
if (n <= 1) return(1)
n * fact(n - 1)
}
fact(5)
fact(20)
Five factorial is one hundred and twenty. Twenty factorial is about two point four times ten to the eighteen, printed in scientific notation.
fib <- function(n) {
if (n <= 2) return(1)
fib(n - 1) + fib(n - 2)
}
fib(10)
system.time(fib(25))
fib of ten is fifty five. The twenty fifth takes a noticeable fraction of a second, because it makes about one hundred and fifty thousand calls, recomputing the same values again and again.
make_memo_fib <- function() {
memo <- c(1, 1)
fib_m <- function(n) {
if (n <= length(memo) && !is.na(memo[n])) return(memo[n])
val <- fib_m(n - 1) + fib_m(n - 2)
memo[n] <<- val
val
}
fib_m
}
fib_fast <- make_memo_fib()
system.time(fib_fast(25))
fib_fast(80)
length(environment(fib_fast)$memo)
The memoised version takes essentially zero time, and the eightieth Fibonacci number is about two point three times ten to the sixteen. The memory now holds eighty values.
Project: a reusable sales analytics toolkit#
S3 class + validated constructor + methods + tests + tidy eval.
set.seed(99)
n_sales <- 300
list_price <- c(Laptop = 900, Phone = 600, Tablet = 350)
raw_sales <- tibble(
date = as.Date("2026-01-01") + sample(0:179, n_sales, TRUE),
region = sample(c("North", "South", "East", "West"),
n_sales, replace = TRUE),
product = sample(names(list_price), n_sales, replace = TRUE),
units = sample(1:10, n_sales, replace = TRUE),
price = round(list_price[product] * runif(n_sales, 0.9, 1.1))
)
glimpse(raw_sales)
glimpse shows three hundred rows and five columns: dates, region and product as text, then units and prices.
Step 1: constructor, validator and helper#
new_sales <- function(df, currency) {
stopifnot(is.data.frame(df), is.character(currency))
structure(list(data = df, currency = currency),
class = "sales_data")
}
validate_sales <- function(x) {
needed <- c("date", "region", "product", "units", "price")
absent <- setdiff(needed, names(x$data))
if (length(absent) > 0) {
stop("missing columns: ", paste(absent, collapse = ", "))
}
if (any(x$data$units < 0)) stop("units cannot be negative")
x
}
sales_data <- function(df, currency = "USD") {
obj <- validate_sales(new_sales(as_tibble(df), currency))
obj$data <- mutate(obj$data, revenue = units * price)
obj
}
sales <- sales_data(raw_sales)
class(sales)
The class is sales data, our very own S3 class.
sales_data(select(raw_sales, date, region))
The validator stops with a precise message: missing columns product, units, price.
This one is meant to fail, so you can see the error message.
Step 2: print and summary methods#
print.sales_data <- function(x, ...) {
d <- x$data
cat("<sales_data>", nrow(d), "transactions\n")
cat("Period: ", format(min(d$date)), "to",
format(max(d$date)), "\n")
regions <- paste(sort(unique(d$region)), collapse = ", ")
cat("Regions:", regions, "\n")
total <- format(round(sum(d$revenue)), big.mark = ",")
cat("Revenue:", x$currency, total, "\n")
invisible(x)
}
sales
Our object now introduces itself: three hundred transactions, a period from early January to late June, four regions, and total revenue of a little over one million dollars.
summarise_sales <- function(sales, ...) {
sales$data |>
group_by(...) |>
summarise(
orders = n(),
units = sum(units),
revenue = round(sum(revenue)),
.groups = "drop"
) |>
arrange(desc(revenue))
}
summarise_sales(sales, product)
Laptops earn by far the most, about five hundred and twenty two thousand dollars, then phones and tablets. Interestingly, tablets sold the most units, five hundred and seventy four, but at a much lower price.
summary.sales_data <- function(object, by = region, ...) {
summarise_sales(object, {{ by }})
}
summary(sales)
summary(sales, by = product)
By region, South leads with about two hundred and eighty two thousand, and West is lowest at about two hundred and seven thousand. By product, we get the same table as before.
Step 3: a plot method#
plot.sales_data <- function(x, ...) {
monthly <- x$data |>
mutate(month = floor_date(date, "month")) |>
group_by(month, product) |>
summarise(revenue = sum(revenue), .groups = "drop")
p <- ggplot(monthly, aes(month, revenue, colour = product)) +
geom_line(linewidth = 1) +
geom_point(size = 2) +
scale_y_continuous(labels = scales::comma) +
labs(title = "Monthly revenue by product",
x = NULL, y = paste("Revenue,", x$currency)) +
theme_minimal(base_size = 14)
print(p)
invisible(p)
}
plot(sales)
The Plots pane shows three lines across six months. Laptops sit well above the others every month, peaking in February at about one hundred and twenty thousand. Phones stay in the middle, and in June tablets almost catch up with them.
Step 4: tested helpers#
growth_rate <- function(old, new) {
stopifnot(is.numeric(old), is.numeric(new))
if (any(old == 0, na.rm = TRUE)) stop("old values must be non-zero")
(new - old) / old
}
test_that("growth_rate works", {
expect_equal(growth_rate(100, 110), 0.1)
expect_equal(growth_rate(c(50, 200), c(25, 300)), c(-0.5, 0.5))
expect_error(growth_rate(0, 5), "non-zero")
})
Test passed. Our helper does exactly what we expect.
test_that("sales_data is built and validated", {
expect_s3_class(sales, "sales_data")
expect_true("revenue" %in% names(sales$data))
expect_equal(nrow(summary(sales)), 4)
expect_error(sales_data(raw_sales[, 1:3]), "missing columns")
})
Test passed. If we ever change the toolkit and break something, these tests will tell us immediately.
monthly_growth <- function(sales) {
sales$data |>
mutate(month = floor_date(date, "month")) |>
summarise(revenue = sum(revenue), .by = month) |>
arrange(month) |>
mutate(growth = round(100 * growth_rate(dplyr::lag(revenue),
revenue), 1))
}
monthly_growth(sales)
Six months of revenue with growth in percent. January is NA, since there is no earlier month. February jumped forty four percent, then April fell nineteen percent and May almost thirteen.
batches <- list(good = raw_sales, broken = raw_sales[, 1:2])
built <- map(batches, safely(sales_data))
map_lgl(built, \(r) is.null(r$error))
built$broken$error$message
The good batch is TRUE and the broken batch is FALSE. Its error message tells us the missing columns: product, units and price.
Recap: what you can do now#
Functions, closures, environments, purrr, conditions, S3/S4/R6,
copy-on-modify, fast code, data.table, tidy eval, tests, recursion.
What next: Shiny, Quarto, package development, tidymodels.
Found this useful?
All lessons, notebooks and datasets here are free. If they helped you, a coffee keeps new lessons coming.



