Each consistency score pools every replicate of runs rep-1 and rep-2 per method, as the consistency table states.
C combines unadjusted mutation, classes and cost to completion. It pools raw costs across CLI builds 2.1.308 and 2.1.309 without correcting that change. The Classic greedy TDD drift test gave M p 0.310. The global spread test did not detect a difference (p 0.464).
Consistency across runs
Figure 8.1
Consistency score per method
Informs Q22.
Exploratory. Consistency scores C: Functions first, classes later 1.14, Noun-verb map 1.65, Anchor Engineering 1.06, Classic greedy TDD 0.87, Held-out TDD 0.96, Coarse-to-fine TDD 1.06, Most informative next test 0.38. The spread of C across methods gives permutation p 0.464.
Exploratory, since only the replicated methods enter it. Each shape is one metric's relative spread, a method's mean absolute deviation from its own median divided by the pooled mean absolute deviation across eligible builds, and the tick is C, their mean. Lower C means less observed spread on this composite. A metric with at least 60 percent of builds at its bound reads at bound and leaves C. A metric with fewer than two methods off its bound reads not comparable and is not drawn. Tag: recomputed.
Table view, 7 rows and 8 columns
| Method | Adjusted mutation | Unadjusted mutation | Classes | Cost to completion |
|---|---|---|---|---|
| Functions first, classes later | 1.37 | 1.98 | 1.14 | 0.30 |
| Noun-verb map | 1.61 | 2.18 | 1.43 | 1.34 |
| Anchor Engineering | at bound | 0.21 | 1.43 | 1.53 |
| Classic greedy TDD | 0.33 | 0.75 | 0.86 | 1.00 |
| Held-out TDD | 1.36 | 1.73 | 0.57 | 0.59 |
| Coarse-to-fine TDD | at bound | 0.25 | 1.71 | 1.23 |
| Most informative next test | at bound | 0.14 | 0.00 | 1.01 |
| Method | Strict PMD violations | CQP accepted | Consistency score C |
|---|---|---|---|
| Functions first, classes later | at bound | 1.68 | 1.14 |
| Noun-verb map | at bound | 0.00 | 1.65 |
| Anchor Engineering | at bound | 0.84 | 1.06 |
| Classic greedy TDD | 0.97 | 1.47 | 0.87 |
| Held-out TDD | 1.12 | 0.84 | 0.96 |
| Coarse-to-fine TDD | 1.22 | 1.26 | 1.06 |
| Most informative next test | 0.71 | 0.42 | 0.38 |