Non-linear Regression & GAMs

Week 4. Relationships that aren’t straight lines. One trick behind almost every curve: make new columns from x, give each a coefficient, and add them up. Quadratics and their optimum, polynomials and why they flap at the edges, splines, sin and cos columns for cycles, then GAMs, where a penalty lets the data choose how wiggly the curve is. Finally, models whose parameters mean something biological, fitted with nls().

Problem Set 4: Non-linear Models and GAMs — due Monday, October 26, 1:10 PM. Submit on Canvas.

Answer key: posted here at 3 PM on Monday, October 26.

Data: seedling_temp.csv (300 wheat seedlings grown at 8 to 42 °C, two varieties: shoot mass and leaf damage; the running example in the lectures, simulated), soil_season.csv (soil respiration on 8 collars every two weeks for two years, simulated), field_gradient.csv (a 200-plot rice trial on an uneven field, simulated), maize_nitrogen.csv (a maize nitrogen-rate trial, simulated), and for Problem Set 4: climate.csv, hemispheres.csv and leaf_light.csv (light-response curves of sun and shade leaves, simulated)

This week — Non-linear regression

One deck across both lectures, in six sections. First lecture: Bend the line (is it curved? the residual plot; logs and GLM links are already curves; the quadratic and its vertex as an estimated optimum; higher polynomials), Splines (pieces joined at knots; the basis as bumps, weights and their sum; how many knots; sin and cos columns for cycles; loess), and GAMs (the wiggliness penalty, λ, edf, REML, bias and variance, k as a ceiling, a first gam()). Second lecture: Reading a GAM (the summary line by line, partial-effect plots, gam.check() and choosing k, reporting a curve), the GAM toolkit (binomial GAMs, one curve per group with by =, cyclic smooths, random effects, smooth surfaces over a field, te()), and Mechanistic models (nls(): a linear-plateau optimum N rate, starting values, confidence intervals for parameters; describe a shape vs estimate a parameter).

Slides (PDF) — one deck for the week (the live-poll links are clickable)
Live coding in class — the short scripts we open in RStudio and run together, line by line: Week4_live_1_curves.R (lines, residuals, quadratics, polynomials, splines, sin and cos columns, a first GAM) and Week4_live_2_gam.R (reading a GAM, the k check, binomial and by = GAMs, the field, a plateau model with nls()).
The week’s code, with output — one page for the whole week, in the same order as the slides, with the code behind every figure and number in the lecture and the output under each block. Every block is labelled Core (the analysis you would run on your own data), Handy (useful, not essential), or Demo (simulations and by-hand sums for the lecture; folded). Start with the Core blocks for the problem set.
Week4_Nonlinear.R — the same code as a plain script, each block tagged [CORE], [HANDY], or [DEMO]. It reads the data straight from the course website, so there is no working directory to set. The notebook version (.qmd) is linked from the code page.
Interactive: splines and the penalty — move the number of basis functions and λ and watch the bumps, their sum and the edf change; then a global polynomial against a local spline, and an ordinary vs a cyclic smooth of the soil-respiration season. Runs in the browser.

Fri Oct 23 — Hands-on: non-linear models in R, step by step

Practice: fitting a curve, step by step — from an empty script to a fitted GAM, an nls() growth curve and a figure ready for a paper, on sunflower_growth.csv (sunflower growth over 12 weeks, two varieties, simulated). Each step says what to type, what you should see, and how to read the common errors. Works as a self-paced lab too.

Learning objectives

Check whether a relationship is curved from a residual plot, and explain why a large p-value for a straight-line slope does not mean “no relationship”
Explain how lm() fits a curve by adding columns built from x (powers, spline bumps, sines and cosines), and find the optimum of a quadratic
Explain why high-degree polynomials misbehave at the edges, and how splines avoid it
Fit a GAM with gam(y ~ s(x), method = "REML"), read every line of its summary, and plot it with a 95% confidence band on the scale of y
Explain what λ, edf and k do, and check k with gam.check()
Use the GAM toolkit: family = binomial, by = smooths with their factor, bs = "cc", bs = "re", s(x, y) and te()
Fit a mechanistic model with nls() from sensible starting values, and report its parameters with confidence intervals

Also useful

Packages this week: ggplot2, mgcv (for gam()), splines (for bs() and ns(); comes with R), gratia (derivatives of a fitted smooth), patchwork
Textbook: ISLR (2nd edition) chapter 7, Moving Beyond Linearity: sections 7.1 (polynomial regression), 7.3 (basis functions), 7.4 (regression splines), 7.5 (smoothing splines), 7.6 (local regression), and 7.7 (generalized additive models)

--- title: "Non-linear Regression & GAMs" --- ::: {.week-header} **Week 4.** Relationships that aren't straight lines. One trick behind almost every curve: make new columns from x, give each a coefficient, and add them up. Quadratics and their optimum, polynomials and why they flap at the edges, splines, sin and cos columns for cycles, then GAMs, where a penalty lets the data choose how wiggly the curve is. Finally, models whose parameters mean something biological, fitted with `nls()`. ::: **Problem Set 4: [Non-linear Models and GAMs](../Week%204/ps-nonlinear.html)** — due **Monday, October 26, 1:10 PM**. Submit on Canvas. ```{r} #| echo: false #| output: asis source("../tools/keys.R"); key_line(4) ``` **Data:** [seedling_temp.csv](../data/seedling_temp.csv) (300 wheat seedlings grown at 8 to 42 °C, two varieties: shoot mass and leaf damage; the running example in the lectures, simulated), [soil_season.csv](../data/soil_season.csv) (soil respiration on 8 collars every two weeks for two years, simulated), [field_gradient.csv](../data/field_gradient.csv) (a 200-plot rice trial on an uneven field, simulated), [maize_nitrogen.csv](../data/maize_nitrogen.csv) (a maize nitrogen-rate trial, simulated), and for Problem Set 4: [climate.csv](../data/climate.csv), [hemispheres.csv](../data/hemispheres.csv) and [leaf_light.csv](../data/leaf_light.csv) (light-response curves of sun and shade leaves, simulated) --- ## This week — Non-linear regression One deck across both lectures, in six sections. First lecture: **Bend the line** (is it curved? the residual plot; logs and GLM links are already curves; the quadratic and its vertex as an estimated optimum; higher polynomials), **Splines** (pieces joined at knots; the basis as bumps, weights and their sum; how many knots; sin and cos columns for cycles; loess), and **GAMs** (the wiggliness penalty, λ, edf, REML, bias and variance, k as a ceiling, a first `gam()`). Second lecture: **Reading a GAM** (the summary line by line, partial-effect plots, `gam.check()` and choosing k, reporting a curve), the **GAM toolkit** (binomial GAMs, one curve per group with `by =`, cyclic smooths, random effects, smooth surfaces over a field, `te()`), and **Mechanistic models** (`nls()`: a linear-plateau optimum N rate, starting values, confidence intervals for parameters; describe a shape vs estimate a parameter). ::: {.file-list} - [Slides (PDF)](../Week%204/Week4_Nonlinear_slides.pdf) — one deck for the week (the live-poll links are clickable) - **Live coding in class** — the short scripts we open in RStudio and run together, line by line: [Week4_live_1_curves.R](../Week%204/Week4_live_1_curves.R) (lines, residuals, quadratics, polynomials, splines, sin and cos columns, a first GAM) and [Week4_live_2_gam.R](../Week%204/Week4_live_2_gam.R) (reading a GAM, the k check, binomial and `by =` GAMs, the field, a plateau model with `nls()`). - [The week's code, with output](../Week%204/Week4_Nonlinear.html) — one page for the whole week, in the same order as the slides, with the code behind every figure and number in the lecture and the output under each block. Every block is labelled **Core** (the analysis you would run on your own data), **Handy** (useful, not essential), or **Demo** (simulations and by-hand sums for the lecture; folded). Start with the Core blocks for the problem set. - [Week4_Nonlinear.R](../Week%204/Week4_Nonlinear.R) — the same code as a plain script, each block tagged `[CORE]`, `[HANDY]`, or `[DEMO]`. It reads the data straight from the course website, so there is no working directory to set. The notebook version (`.qmd`) is linked from the code page. - [Interactive: splines and the penalty](../Week%204/Week4_interactive.html) — move the number of basis functions and λ and watch the bumps, their sum and the edf change; then a global polynomial against a local spline, and an ordinary vs a cyclic smooth of the soil-respiration season. Runs in the browser. ::: ## Fri Oct 23 — Hands-on: non-linear models in R, step by step ::: {.file-list} - [Practice: fitting a curve, step by step](../Week%204/Week4_friday_practice.html) — from an empty script to a fitted GAM, an `nls()` growth curve and a figure ready for a paper, on [sunflower_growth.csv](../data/sunflower_growth.csv) (sunflower growth over 12 weeks, two varieties, simulated). Each step says what to type, what you should see, and how to read the common errors. Works as a self-paced lab too. ::: --- ## Learning objectives - Check whether a relationship is curved from a residual plot, and explain why a large p-value for a straight-line slope does not mean "no relationship" - Explain how `lm()` fits a curve by adding columns built from x (powers, spline bumps, sines and cosines), and find the optimum of a quadratic - Explain why high-degree polynomials misbehave at the edges, and how splines avoid it - Fit a GAM with `gam(y ~ s(x), method = "REML")`, read every line of its summary, and plot it with a 95% confidence band on the scale of y - Explain what λ, edf and k do, and check k with `gam.check()` - Use the GAM toolkit: `family = binomial`, `by =` smooths with their factor, `bs = "cc"`, `bs = "re"`, `s(x, y)` and `te()` - Fit a mechanistic model with `nls()` from sensible starting values, and report its parameters with confidence intervals ## Also useful - Packages this week: `ggplot2`, `mgcv` (for `gam()`), `splines` (for `bs()` and `ns()`; comes with R), `gratia` (derivatives of a fitted smooth), `patchwork` - Textbook: ISLR (2nd edition) chapter 7, Moving Beyond Linearity: sections 7.1 (polynomial regression), 7.3 (basis functions), 7.4 (regression splines), 7.5 (smoothing splines), 7.6 (local regression), and 7.7 (generalized additive models)