D1 Activity: The Code Audit

What do you actually understand?

Can be used as: an in-class activity (20–30 min), a take-home exercise, or a recurring homework requirement. No dataset needed — all examples use built-in R data.

The Idea

Understanding code and being able to recognize code are different things. When you copy a script from the internet, or accept output from an AI tool, the code may run correctly — and you may have no idea why.

This activity makes that gap visible.

Your Annotation System

Go through a block of code line by line. For every non-trivial line, add a comment with one of three marks:

Mark	Meaning
`# ✓`	I could write this from scratch without looking anything up. I could explain it to someone else.
`# ?`	I recognize what this does but couldn’t produce it independently. I’d need to look up the syntax.
`# ✗`	I don’t know what this does or why it’s here.

The ✓ bar is intentionally high. “I’ve seen this before” is not ✓. “I understand what every argument does and why this line is here” is ✓.

Part 1 — Audit This Code

Below is a block of R code, written as if by an AI assistant responding to the prompt:

“I have plant height data for three treatment groups (control, drought, heat stress) with 10 plants each. Run an appropriate statistical test and make a figure.”

Read it carefully. Annotate every non-trivial line. Then answer the questions at the bottom.

set.seed(42)
n <- 10

heights <- data.frame(
  treatment = rep(c("control", "drought", "heat"), each = n),
  height    = c(rnorm(n, mean = 22, sd = 3),
                rnorm(n, mean = 14, sd = 3),
                rnorm(n, mean = 17, sd = 3))
)

# Descriptive stats
tapply(heights$height, heights$treatment, function(x) c(mean = mean(x), sd = sd(x)))

# Check normality
by(heights$height, heights$treatment, shapiro.test)

# Levene's test for homogeneity of variance
library(car)
leveneTest(height ~ treatment, data = heights)

# One-way ANOVA
fit <- aov(height ~ treatment, data = heights)
summary(fit)

# Effect size: eta-squared
ss   <- summary(fit)[[1]][["Sum Sq"]]
eta2 <- ss[1] / sum(ss)
cat("eta^2 =", round(eta2, 3), "\n")

# Post-hoc comparisons
TukeyHSD(fit)

# Figure
library(ggplot2)
ggplot(heights, aes(x = treatment, y = height, fill = treatment)) +
  geom_boxplot(alpha = 0.6, show.legend = FALSE) +
  geom_jitter(width = 0.1, alpha = 0.5) +
  stat_summary(fun = mean, geom = "point", shape = 18, size = 4, color = "black") +
  labs(title = "Plant height by treatment",
       x = "Treatment", y = "Height (cm)") +
  theme_classic()

Questions

After annotating:

Count your marks. What proportion of lines are ✓ vs ? vs ✗? Be honest.
Find the line that calculates eta-squared:
```
ss   <- summary(fit)[[1]][["Sum Sq"]]
eta2 <- ss[1] / sum(ss)
```
What is summary(fit)[[1]]? Why [[1]]? What does [["Sum Sq"]] return? Could you have written this line without seeing it first?
The code runs leveneTest() but never uses the result to make a decision. What should happen next? What would change about the analysis if Levene’s test were significant?
by(heights$height, heights$treatment, shapiro.test) — what is by() doing here? How is it different from tapply()? What does the output tell you?
The code uses aov() and then TukeyHSD(). Under what conditions is this combination appropriate? What assumptions are being made?
Is there anything this code does that you think is wrong or questionable? Anything you’d do differently?

Part 2 — Audit Your Own AI Code

Now generate your own code using an AI tool (ChatGPT, Claude, Copilot, etc.).

Prompt to use: > “I have a dataset with continuous measurements from two experimental groups. Write R code to determine whether the groups differ significantly, calculate an appropriate effect size, and make a publication-quality figure.”

Copy the output. Annotate it with ✓ / ? / ✗.

Then:

How many ✗ lines did you get? What are they?
For each ✗ line: look it up, understand it, and change the annotation.
Did the AI make any statistical choices you wouldn’t have made? Explain.
Would you feel confident submitting this analysis in a paper? Why or why not?

Part 3 — The Ownership Test (optional, pairs)

Swap your annotated code with a partner. Your partner asks you to explain any line you marked ✓.

If you can explain it clearly — what it does, what the arguments mean, what would happen if you changed them — the ✓ stands.

If you can’t, change it to ?.

The point: The ✓ category should represent code you genuinely own. This is the standard that matters in a thesis defense, a code review with a collaborator, or a revision request from a reviewer who asks “how exactly was this analysis done?”

The Course Policy Connection

A note on AI use in this course

You may use AI tools to help write code in this course. There is no blanket prohibition.

The standard is this: you are responsible for every line you submit. If your code runs a test you can’t explain, makes an assumption you weren’t aware of, or uses a function you couldn’t describe — that’s a problem, regardless of where the code came from.

The code audit is practice for the standard that applies in your actual research. No one will accept “the AI wrote it” as an explanation in your dissertation defense.

Reflection (written, submit on Canvas)

After completing Parts 1 and 2, write a short paragraph (3–5 sentences) answering:

What did you discover about what you actually understand vs. what you recognize?
Did anything in the AI-generated code surprise you, concern you, or teach you something?
What’s one thing you’re going to look up or practice as a result of this exercise?

There is no grade for correctness here — only for honest engagement.

--- title: "D1 Activity: The Code Audit" subtitle: "What do you actually understand?" --- ::: {.week-header} **Can be used as:** an in-class activity (20–30 min), a take-home exercise, or a recurring homework requirement. No dataset needed — all examples use built-in R data. ::: --- ## The Idea Understanding code and being able to recognize code are different things. When you copy a script from the internet, or accept output from an AI tool, the code may run correctly — and you may have no idea why. This activity makes that gap visible. --- ## Your Annotation System Go through a block of code line by line. For every non-trivial line, add a comment with one of three marks: | Mark | Meaning | |------|---------| | `# ✓` | I could write this from scratch without looking anything up. I could explain it to someone else. | | `# ?` | I recognize what this does but couldn't produce it independently. I'd need to look up the syntax. | | `# ✗` | I don't know what this does or why it's here. | **The ✓ bar is intentionally high.** "I've seen this before" is not ✓. "I understand what every argument does and why this line is here" is ✓. --- ## Part 1 — Audit This Code Below is a block of R code, written as if by an AI assistant responding to the prompt: > *"I have plant height data for three treatment groups (control, drought, heat stress) with 10 plants each. Run an appropriate statistical test and make a figure."* Read it carefully. Annotate every non-trivial line. Then answer the questions at the bottom. ```r set.seed(42) n <- 10 heights <- data.frame( treatment = rep(c("control", "drought", "heat"), each = n), height = c(rnorm(n, mean = 22, sd = 3), rnorm(n, mean = 14, sd = 3), rnorm(n, mean = 17, sd = 3)) ) # Descriptive stats tapply(heights$height, heights$treatment, function(x) c(mean = mean(x), sd = sd(x))) # Check normality by(heights$height, heights$treatment, shapiro.test) # Levene's test for homogeneity of variance library(car) leveneTest(height ~ treatment, data = heights) # One-way ANOVA fit <- aov(height ~ treatment, data = heights) summary(fit) # Effect size: eta-squared ss <- summary(fit)[[1]][["Sum Sq"]] eta2 <- ss[1] / sum(ss) cat("eta^2 =", round(eta2, 3), "\n") # Post-hoc comparisons TukeyHSD(fit) # Figure library(ggplot2) ggplot(heights, aes(x = treatment, y = height, fill = treatment)) + geom_boxplot(alpha = 0.6, show.legend = FALSE) + geom_jitter(width = 0.1, alpha = 0.5) + stat_summary(fun = mean, geom = "point", shape = 18, size = 4, color = "black") + labs(title = "Plant height by treatment", x = "Treatment", y = "Height (cm)") + theme_classic() ``` ### Questions **After annotating:** 1. Count your marks. What proportion of lines are ✓ vs ? vs ✗? Be honest. 2. Find the line that calculates eta-squared: ```r ss <- summary(fit)[[1]][["Sum Sq"]] eta2 <- ss[1] / sum(ss) ``` What is `summary(fit)[[1]]`? Why `[[1]]`? What does `[["Sum Sq"]]` return? Could you have written this line without seeing it first? 3. The code runs `leveneTest()` but never uses the result to make a decision. What should happen next? What would change about the analysis if Levene's test were significant? 4. `by(heights$height, heights$treatment, shapiro.test)` — what is `by()` doing here? How is it different from `tapply()`? What does the output tell you? 5. The code uses `aov()` and then `TukeyHSD()`. Under what conditions is this combination appropriate? What assumptions are being made? 6. Is there anything this code does that you think is wrong or questionable? Anything you'd do differently? --- ## Part 2 — Audit Your Own AI Code Now generate your own code using an AI tool (ChatGPT, Claude, Copilot, etc.). **Prompt to use:** > *"I have a dataset with continuous measurements from two experimental groups. Write R code to determine whether the groups differ significantly, calculate an appropriate effect size, and make a publication-quality figure."* Copy the output. Annotate it with ✓ / ? / ✗. Then: 1. How many ✗ lines did you get? What are they? 2. For each ✗ line: look it up, understand it, and change the annotation. 3. Did the AI make any statistical choices you wouldn't have made? Explain. 4. Would you feel confident submitting this analysis in a paper? Why or why not? --- ## Part 3 — The Ownership Test (optional, pairs) Swap your annotated code with a partner. Your partner asks you to explain any line you marked ✓. If you can explain it clearly — what it does, what the arguments mean, what would happen if you changed them — the ✓ stands. If you can't, change it to ?. **The point:** The ✓ category should represent code you genuinely own. This is the standard that matters in a thesis defense, a code review with a collaborator, or a revision request from a reviewer who asks "how exactly was this analysis done?" --- ## The Course Policy Connection ::: {.callout-note} ## A note on AI use in this course You may use AI tools to help write code in this course. There is no blanket prohibition. The standard is this: **you are responsible for every line you submit.** If your code runs a test you can't explain, makes an assumption you weren't aware of, or uses a function you couldn't describe — that's a problem, regardless of where the code came from. The code audit is practice for the standard that applies in your actual research. No one will accept "the AI wrote it" as an explanation in your dissertation defense. ::: --- ## Reflection (written, submit on Canvas) After completing Parts 1 and 2, write a short paragraph (3–5 sentences) answering: - What did you discover about what you actually understand vs. what you recognize? - Did anything in the AI-generated code surprise you, concern you, or teach you something? - What's one thing you're going to look up or practice as a result of this exercise? There is no grade for correctness here — only for honest engagement.