TDD (test-driven development). Tag: verified.
Definition: A development practice that writes each test before the code that passes it and lets the tests steer the design. Lecture 11.2 treats it as a design process and not a testing technique.
Scope and terms
The other scope sections have pages of their own. Limits is on Analysis and validity. Study stage and Course context are on Experimental design.
Glossary
The terms and short labels this page uses.
Glossary, 35 rows and 3 columns
Method and slug. Tag: verified.
Definition: A method is one TDD methodology written as a short numbered procedure in runs/full-1/tdd-methodologies.md. Its slug is its short lowercase name, which the Branch column shows after method/.
Kalah. Tag: verified.
Definition: A board game for two players. Each player has a row of houses holding seeds and a store. A move sows the seeds of one house round the board one at a time. The A02 task is a Java console program that plays Kalah in four variants of board size and sowing direction.
Family. Tag: verified.
Definition: One of four groups of methods in the Summary table of runs/full-1/tdd-methodologies.md, by what each changes in classic TDD. Baseline holds Classic greedy TDD alone. Structure first maps the classes before or after the code is written. Step size changes the size or order of the steps or lets early refactors accept a worse design. Search and validation tries several starts or changes how the tests check the code. The figure legends colour and shape each mark by family.
Cycle. Tag: verified.
Definition: One red-green pass after Beck's steps in Lecture 11.2. Add a test, see it fail, make it pass, then refactor. One agent call runs at most 15 cycles.
Iteration and loop. Tag: verified.
Definition: A loop is a run of method steps that repeats. Each pass is one iteration. A loop runs at most 15 iterations in full-1 and 2 in the pilots.
Clone and start clone. Tag: verified.
Definition: A clone is a fresh copy of the starter repository where one method's agents work. Multi-start TDD builds 3 start clones before its final clone, so 11 methods use 14 clones.
Arm. Tag: verified.
Definition: One configuration of a run. In tcp-1 each arm follows one order of the same test list.
Functional mark /15. Tag: verified.
Definition: The functionality mark of the A02 rubric computed from the hidden tests. The Columns section gives its formula.
STA and STC. Tag: verified.
Definition: Spec folder names for the sowing direction. STA means anticlockwise and STC clockwise sowing. Pairs such as 6-4 and 7-5 give houses and seeds.
A01 and A02. Tag: verified.
Definition: The first two assignments of the course. The A02 spec and rubric define the task here and the A02 reference row is a solution to it. The A01 rubric used the course PMD ruleset.
AI and AI coding agent. Tag: verified.
Definition: AI stands for artificial intelligence. An AI coding agent is a language model that edits code and runs commands unattended.
PMD and strict PMD compliance. Tag: verified.
Definition: PMD is a static analysis tool for Java. Strict PMD compliance passes when the course ruleset kalah_pmd.xml finds no violation of any priority in src/.
CPU. Tag: verified.
Definition: Central processing unit. Each bench run uses one processor core.
Mutation score and K/N. Tag: verified.
Definition: Mutation testing makes small changes to the code, called mutants, and counts those the tests catch. K/N is killed plus timed-out mutants out of the N that compiled.
Adjusted mutation score. Tag: verified.
Definition: The mutation score after dropping each surviving mutant that fails none of the hidden tests the unmutated clone passes. Such a mutant need not pass all 76 hidden tests. It may be equivalent or expose a gap in the hidden suite.
Weighted tokens. Tag: verified.
Definition: Model tokens weighted by relative price. Input counts 1, cache creation 1.25, cache read 0.05 and output 5 per token. The weights state no dollar cost.
Alterability and change case. Tag: verified.
Definition: Alterability is the cost of a change compared with its expected effort and likelihood. A change case is one likely change with its likelihood and impact. docs/SPEC.md names 2 change cases. No change case was carried out and the rating leaves alterability out, so it is not measured.
Blind review and blind rating. Tag: verified.
Definition: A review agent scored each method's code and tests once from a prompt that left out the method's name, and the blind rating supersedes those scores. In the blind rating three raters score each item from 1 to 5 on each cqp.html principle. An item is a rebuilt clone whose history names no method. The finals are the 11 final clones. The calibration items add the A02 reference, the starter stub and 7 mutants of functions-first, each damaging one or two named principles.
Audit flag. Tag: verified.
Definition: A command or commit by an agent that may show access to the hidden tests. Each high flag gets a disposition.
Test list. Tag: verified.
Definition: The list of tests a TDD developer plans to write before writing them.
Test case prioritisation. Tag: verified.
Definition: Ordering a test suite so that the tests most likely to reveal faults run first, after Rothermel et al. 2001. tcp-1 carries its criteria over to ordering a TDD test list.
Kendall distance. Tag: verified.
Definition: The share of test pairs that two orders rank differently. It is 0 for the same order and 1 for the reverse.
Its formula and inputs are on Measures.
APFD-I. Tag: verified.
Definition: The area under a clone's hidden-pass curve over its steps, normalised to 0 to 1. Higher means hidden tests start to pass earlier. harness/report/curve.py defines it.
Its formula and inputs are on Measures.
APFDc-H. Tag: verified.
Definition: The area under the same hidden-pass curve over cumulative weighted tokens divided by the cost to completion. harness/report/curve.py defines it.
Cost to completion. Tag: verified.
Definition: Weighted tokens of the lead cost plus the final clone's step costs, measure and review agents left out. A method's steps are the steps.csv rows of its graded points. An arm's steps are its loop calls, each costed by the costs.json agent its batch.json tcp.calls label names. A stopped call without a sha counts though it grades no point. harness/report/curve.py defines it.
M and M_io. Tag: verified.
Definition: M is a trapezoidal checkpoint-area estimate of weighted-token cost per hidden test: the cost to completion times 1 minus APFDc-H. M_io recomputes the curve with output tokens excluded. It is a sensitivity check and does not establish comparability across CLI builds. harness/report/curve.py defines both.
Its formula and inputs are on Measures.
Survey and survey build. Tag: verified.
Definition: The survey is run full-1, which built each of the 11 methods once. A survey build is that one build of a method.
Replicate and representative build. Tag: verified.
Definition: A replicate is one of the 5 builds of a method in run rep-1 or rep-2. The representative build is the middle replicate by adjusted mutation score, then by lower cost to completion.
Planned contrast. Tag: verified.
Definition: A pair of methods and an outcome fixed before any build of its run. rep-1 holds two, Functions first, classes later against Noun-verb map and Anchor Engineering against Classic greedy TDD, each on adjusted mutation score by an exact Mann-Whitney U test with Holm correction over the two. rep-2 holds three, Held-out TDD, Coarse-to-fine TDD and Most informative next test each against Classic greedy TDD, on M by the same test with Holm correction over the three.
Batch. Tag: verified.
Definition: The builds launched together in one run. full-1, rep-1 and rep-2 are three batches, and the costs of full-1 do not compare with those of rep-1 or rep-2. The Limits explain why.
CQP and CQP accepted. Tag: verified.
Definition: CQP sums a final's rating medians over the four counted cqp.html principles fixed before rep-1, Simple Constructs, Minimal Duplication, Modular Structure and Problem Alignment, from 4 to 20. CQP accepted sums the accepted ones, from 3 to 15. The CQP sensitivity check tests each planned contrast on CQP accepted.
Its formula and inputs are on Measures.
Relative spread and consistency score C. Tag: verified.
Definition: A method's relative spread divides its mean absolute deviation from its own median by the pooled mean absolute deviation across eligible builds off the metric's bound. The pooled reference weights builds, so unequal replicate counts contribute unequally. C is the mean of its relative spreads on unadjusted mutation score, classes and cost to completion. Lower means more consistent. A metric with at least 60 percent of a method's builds at its bound reads at bound and leaves C and the other methods' mean. A metric with fewer than two methods off its bound reads not comparable.
Awards and E. Tag: verified.
Definition: The Headline's awards rank the survey methods, lower first, with ties sharing a place. E sums a final's distinct PMD findings, adjusted surviving mutants, hidden failures and unseen failures. Alignment needs the blind CQP rating to agree with PMD on every principle and PMD to report no issue. Best overall is the plain sum of each method's ranks on M among 15/15 methods, E and alignment. Cheapest, the lowest cost to completion, and most consistent design, the lowest C pooled over every replicate of rep-1 and rep-2, take bronze badges only and never count toward best overall. A medal only styles a place and carries no score. harness/report/awards.py defines them.
Its formula and inputs are on Measures.
Short forms
The summary and design pages use these short forms. Each term links to its full entry above.
- Functional mark
- Marks out of 15 from the hidden course tests, scored as the A02 rubric does.
- E
- Errors: distinct PMD findings plus adjusted surviving mutants, those the hidden tests kill, plus failed hidden and unseen tests. Lower is better.
- Cost to completion
- Weighted tokens the agents spent to finish the build.
- M
- The trapezoidal checkpoint-area estimate of weighted-token cost per hidden test. Lower is better.
- M_io
- A sensitivity check that recomputes M without output-token costs.
- Alignment
- The blind code-quality rating agrees with PMD on every principle and PMD finds no issue.
- C
- Consistency score over repeat builds of one design. Lower means steadier results. Exploratory.
- PMD
- A static analysis tool for Java. A finding is one rule broken in one place.
- Weighted tokens
- Model tokens weighted by price: input 1, cache write 1.25, cache read 0.05, output 5.
- APFDc-H
- How early the hidden tests pass: the area under the hidden pass curve over weighted tokens spent, divided by the cost to completion.
- CQP
- Code Quality Principles, the course's design principles. A CQP score sums the blind rating's medians over four of them, 4 to 20.
- Adjusted mutation
- Surviving mutants missed by the hidden suite are excluded. They may be equivalent or expose gaps in that suite. Higher is better.
- Planned contrast
- A comparison of two designs fixed before the builds ran, tested over their repeat builds.
- U
- The exact two-sided Mann-Whitney statistic of a planned contrast: how many pairs of builds, one from each design, have this design's value above the other's, a tie counting half.
- Holm
- A correction that raises each contrast's p value, the chance of a gap this large with no real difference, so testing several contrasts at once keeps the false alarm rate at its usual level.