Alterability. Item: Change Case 1: sow seeds anti-clockwise. Score /5: 5.
Reason: Sowing direction is handled only by mirroring the board in BoardView.lines, which is one if/else. Board and the game loop do not depend on direction at all.
Kalah TDD study · Part 15 of 29 · Designs
Pick tests the simplest ranked transformation can pass.
Source run full-1 at tdd-experiment c3dd988
Step size
What it changes: Move order. Cheapest transformation first.
No award in this survey.
One method of the study, from runs full-1. The survey build is the method's single build in run full-1. Claims carry a tag. Recomputed means redone from source data. Verified means checked against a primary source. Unchecked means written or judged by an agent and not checked.
The procedure supplied to the agents. Indented lines run under the line above them.
Family: Step size. Idea: Pick tests the simplest ranked transformation can pass. What it changes in the search: Move order. Cheapest transformation first.
once:1. Put Martin's ranked list with one-line definitions in the assistant prompt.repeat per feature until none is open:2. Pick the next test that the highest-ranked transformation can pass.3. Pass it with that one transformation. Name it in the commit message.4. Revert and try a simpler test if it needs while, function or assignment.5. Accept the lower-ranked move and log it if no simpler test exists.6. Refactor on green. Refactors change structure and never behaviour.7. Repeat until the test list is empty.
Once means one agent runs the steps. Repeat per unit means one agent per unit until no unit is open. Throughout means the rule joins every prompt. For up to n starts means separate attempts with the best kept. The harness does means the harness runs the step without an agent. If names the event that starts a stage, and with adds a rule borrowed from another method. Steps are quoted from args.json with their numbers. Tag: verified.
Figure 15.1
Next: Steps 2 to 7.
Loop: repeat per feature, back to Steps 2 to 7.
2 stages from the run facts, each checked against its block of the steps above. Details and notes are in the stage table. Tag: recomputed.
Next: Steps 2 to 7.
Loop: repeat per feature, back to Steps 2 to 7.
Each stage as the run facts list it. Runs as and Note are quoted verbatim and Steps repeats the steps above. Tag: verified.
| Stage | Runs as | Steps | Details | Note |
|---|---|---|---|---|
| Once: step 1 | once: | 1. Put Martin's ranked list with one-line definitions in the assistant prompt. | none | Martin's list from highest to lowest rank is {}->nil, constant, constant+, scalar, statements, if, array, container, recursion, while, function and variable->assignment. Write it with one-line definitions to TPP.md and commit it. Later stages read TPP.md as the prompt list. |
| Steps 2 to 7 | repeat per feature until none is open: | 2. Pick the next test that the highest-ranked transformation can pass. 3. Pass it with that one transformation. Name it in the commit message. 4. Revert and try a simpler test if it needs while, function or assignment. 5. Accept the lower-ranked move and log it if no simpler test exists. 6. Refactor on green. Refactors change structure and never behaviour. 7. Repeat until the test list is empty. | unit feature | Read TPP.md first. An impasse is a bar that stays red for 3 red-green cycles. Log each impasse in the LOG.md actions. |
One build in run full-1.
One survey build. It passed every hidden test with 3 classes, 7 PMD violations and adjusted mutation 0.879. It had passed 0.05 of the hidden tests at 500000 weighted tokens, and later commits rewrote 34 percent of the src lines it added. Tag: recomputed.
The report's class diagram for this design is identical to Figure 15.2 below, so it is shown once. Tag: recomputed.
| Functional mark /15 | 15/15 |
|---|---|
| Hidden pass /76 | 76/76 |
| Unseen pass /69 | 69/69 |
| Strict PMD compliance | fail (7 violations) |
| Mutation score | 94/114, 0 not run, adjusted 94/107 (13 of 20 survivors fail hidden tests) |
| Classes | 3 |
| Simple Constructs /5 | 4 |
| Minimal Duplication /5 | 4 |
| Modular Structure /5 | 4 |
| Problem Alignment (tentative) /5 | 4 |
| Weighted tokens | 1,291,180 |
| Bench median ms | 205 |
Survey tiles place the survey build among the single survey builds. One build per method describes that build and does not rank the methods. Contrast tiles are medians over the replicates. Tag: recomputed.
Best, median and worst order a method's replicates by adjusted mutation, highest first, then lower cost to completion. The median build is the representative. Tag: recomputed.
| Build | tpp |
|---|---|
| Adjusted mutation | 0.879 |
| Classes | 3 |
| Cost to completion | 1,232,581 |
| Strict PMD violations | 7 |
Figure 15.2
3 classes and 2 links. Tag: recomputed.
Same family comes from the methods doc. Nearest and farthest average the rank differences over adjusted mutation, classes and cost to completion of the survey builds, over the metrics both methods have. Anchor Engineering has no survey build, so the replicate medians of adjusted mutation and classes stand in. Cost to completion leaves out such a method, since costs of a different batch do not compare. Tag: recomputed.
Each clone's own report, as in its REPORT.md.
Branch method/tpp. Claims carry a tag. Recomputed means redone from source data. Verified means checked against a primary source. Unchecked means written or judged by an agent and not checked.
| Functional mark /15 | 15/15 |
|---|---|
| Hidden pass /76 | 76/76 |
| Unseen pass /69 | 69/69 |
| Acceptance pass /6 | 6/6 |
| Strict PMD compliance | fail (7 violations) |
Recomputed from the grader's grade.tsv and summary.json and from metrics.json. The functional mark follows A02-RUBRIC.md: 2 for the original set with no failure, 3 per other set less one per failure, and 4 when every test passes. A02 is the course assignment whose spec and rubric the experiment uses. STA means anticlockwise and STC clockwise sowing. 6-4 and 7-5 are houses and seeds. A02-RUBRIC.md names the original set "Clockwise 6 Houses 4 Seeds". This page names each set by the spec folder its tests play, so the original set reads STA 6-4. Strict PMD compliance passes when PMD with the starter's kalah_pmd.xml finds no violation in src/ and otherwise fails with the count of distinct violations. The lint sets no minimum priority, so rules of every priority count. It covers src/ only, the scope of the starter's ./run.sh lint. A02-RUBRIC.md awards no marks for PMD, so the functional mark leaves it out.
| Rubric set | Tests | Failures | Marks |
|---|---|---|---|
| Original set, STA 6-4 | testST64SKHHSH_ | 0 | 2/2 |
| STC 6-4 | testSTC64SKHHSH_ | 0 | 3/3 |
| STA 7-5 | testST75SKHHSH_ | 0 | 3/3 |
| STC 7-5 | testSTC75SKHHSH_ | 0 | 3/3 |
| All requirements | all tests | 0 | 4/4 |
| Metric | Value | Tag |
|---|---|---|
| Total lines | 528 (src 193, own tests 335) | recomputed |
| Lines of code | 418 (src 152, own tests 266) | recomputed |
| Classes | 3 | recomputed |
| Methods | 21 | recomputed |
| Fields | 2 | recomputed |
| Bench median ms | 205 | recomputed |
| Bench volume | 100614 inputs, 40083576 output chars | recomputed |
Own tests leave out the starter's two Example tests and their ExampleSpec helper. Classes count top-level, nested and anonymous classes in src/.
| Metric | Value | Tag |
|---|---|---|
| Mutation score | 94/114, 0 not run, adjusted 94/107 (13 of 20 survivors fail hidden tests) | recomputed |
| PMD lines | 7 (7 violations) | recomputed |
| Dependency edges | 2 | recomputed |
| Review scores (superseded) | Alterability 4, Code quality principles 4.1, Test quality 4 | unchecked |
| Tokens | 11,999,055 (input 542, cache creation 486,444, cache read 11,490,457, output 21,612) | recomputed |
| Weighted tokens | 1,291,180 | recomputed |
| Wall time | 0:33:03 | recomputed |
| Agent time | 0:32:07 | recomputed |
| Agent calls | 10 | recomputed |
| Audit flags | 0 high, 0 low | recomputed |
PMD lines count violations once per file and line. Agent time sums each agent's first to last timestamp, so agents that ran side by side add up. Weighted tokens count input 1, cache creation 1.25, cache read 0.05 and output 5 per token. These are claude-opus-5-5 prices relative to an input token and state no dollar cost. Costs, times and agent calls cover this clone only. The comparison row covers every clone of the method as its count basis states.
A blind reviewer agent scored src/ and test/ from 1, worst, to 5, best, so the scores are unchecked. Reasons are the reviewer's own words, verbatim. This review is superseded by the blind rating of three raters in rating.json, whose medians fill the comparison's principle columns.
Alterability. Item: Change Case 1: sow seeds anti-clockwise. Score /5: 5.
Reason: Sowing direction is handled only by mirroring the board in BoardView.lines, which is one if/else. Board and the game loop do not depend on direction at all.
Alterability. Item: Change Case 2: 7 houses with 5 seeds. Score /5: 3.
Reason: The house and seed counts go into the Board constructor in one place, and Board and cells scale with them. border and divider hardcode literal strings for 6 and 7 houses, so any other size would need new literals.
Code quality principles. Item: 1. Clear Presentation. Score /5: 4.
Reason: Formatting is tidy and the files are short and well ordered. The long BoardView.lines call in printBoard and the index arithmetic in sow are a little dense to read.
Code quality principles. Item: 2. Explanatory Language. Score /5: 4.
Reason: Doc comments explain intent, for example the mover's-lap pit indexing. Names like n, last and pit, and literals such as 2*n+1 and the 0/1 player ids, are only partly explained.
Code quality principles. Item: 3. Consistent Code. Score /5: 4.
Reason: Players are 0 and 1 throughout, and the static view helpers follow one style. Houses are numbered from 1 in the API but from 0 inside it, and some helpers are package-private for tests while their siblings are private.
Code quality principles. Item: 4. Used Content. Score /5: 5.
Reason: Every element is used. The starter placeholder was removed and nothing is dead.
Code quality principles. Item: 5. Simple Constructs. Score /5: 4.
Reason: The control flow is simple and the nesting is shallow. sow packs sowing, capture and the extra-move check into one method that uses modular arithmetic.
Code quality principles. Item: 6. Minimal Duplication. Score /5: 3.
Reason: border and divider each repeat near-identical literal strings for 6 and 7 houses where a loop could build them. play also repeats the Game over plus printBoard sequence on both exit paths.
Code quality principles. Item: 7. Modular Structure. Score /5: 5.
Reason: Board holds the rules, BoardView handles rendering and Kalah handles I/O and turns. Their dependencies are minimal and clean.
Code quality principles. Item: 8. Problem Alignment. Score /5: 4.
Reason: Board, sow, store, score and hasMoves match the domain language. Modelling direction as a pure rendering mirror is apt, but the code never states that this is why it works.
Test quality. Item: Test quality. Score /5: 4.
Reason: Many small, clearly named unit tests pin sowing, wrapping, skipping the opponent's store, captures, scoring and rendering, and spec tests cover each scenario end to end. The spec scenarios exercise only two of the four variants, anticlockwise 6-4 and clockwise 7-5.
0 high and 0 low flags from agent tool calls and the method's commits. Only audit.json lists the low flags.
Method agents wrote the LOG.md cells. Agent-written text is shown verbatim and unchecked. Words such as recomputed inside it are the agent's own and are not tags of this page. Its Measure lines describe measurement during the run. Harness scores were measured separately after the run. Agent text may quote git commit times in the host's local zone. Harness times are UTC. Agent, tokens and seconds are recomputed from transcripts by row time. A dash in Agent means no agent run spans the row.
2026-10-15T23:53:23Z. Step: s1. Unit: TPP ranked list. Tests added: 0. Cycles: 0. Sha: d5642a9. Agent: tpp:s1. Tokens: 118327. Seconds: 16.195.
Actions: Wrote Martin's 12 transformations with one-line definitions to TPP.md and committed it. No code or tests touched so no checks run.
2026-10-15T23:58:32Z. Step: L2-7 iter 1. Unit: Board rendering for all four variants. Tests added: 15. Cycles: 15. Sha: 8f21b36. Agent: tpp:L2-7#1. Tokens: 3250312. Seconds: 310.637.
Actions: Wrote UNITS.md with 8 units. Built BoardView (house, pad, border, divider, row, lines) and wired play() to print the starting board, one TPP transformation per commit. Lower-ranked moves accepted with no simpler test: if->while for row joining, if->while for building board cells and expression->function plus loop in play(). Border and divider stay as 6/7 if-branches because a loop would change behaviour for other sizes. No impasse. Hit the 15-cycle limit exactly. Full ./run.sh test: BoardViewTest 13 and KalahBoardTest 2 pass. The 6 example tests still error as expected until later units land.
2026-10-16T00:00:21Z. Step: L2-7 iter 2. Unit: Prompt and quit with the Game over board. Tests added: 2. Cycles: 1. Sha: e85537d. Agent: tpp:L2-7#2. Tokens: 1040657. Seconds: 107.514.
Actions: Replaced KalahBoardTest with KalahQuitTest since play() now prompts and the board-only specs no longer describe its behaviour. One red-green cycle on the anticlockwise 6-4 quit spec, passed with statement->statements (prompt read, Game over, board again). The clockwise 7-5 quit spec was then updated to the new behaviour. Refactored on green by extracting printBoard. No lower-ranked move and no impasse. Conflict logged: the Write and Edit tools strip the prompt's trailing space, so the 6-4 spec was copied from the identical examples quit.txt and the 7-5 spec was assembled with head and cat from the example's first six lines plus a scratch tail. Full ./run.sh test: BoardViewTest 13, KalahQuitTest 2 and ExampleAnticlockwiseTest.testQuit pass. The other 5 example tests still error as expected.
2026-10-16T00:11:44Z. Step: L2-7 iter 3. Unit: Sowing a move within the player's own houses in either direction. Tests added: 7. Cycles: 7. Sha: 5b10280. Agent: tpp:L2-7#3. Tokens: 1321298. Seconds: 680.431.
Actions: Added a Board model (houses per player, sow) and wired play() to alternate moves, one TPP transformation per commit. Lower-ranked moves accepted with no simpler test: expression->function for the house fill, scalar->array with variable->assignment to empty the chosen house, if->while to spread several seeds, and if->while with variable->assignment and an extracted prompt function to give P2 a move. Sowing runs toward higher house numbers in both directions so direction only affects BoardView. No refactor needed on green and no impasse. Conflict logged: the two new specs were written with printf in Bash because the Write tool strips the prompt's trailing space. Full ./run.sh test: BoardTest 5, KalahSowTest 2, BoardViewTest 13, KalahQuitTest 2 and ExampleAnticlockwiseTest.testQuit pass. The other 5 example tests still fail as expected until the store, wrap, capture and game end units land.
2026-10-16T00:14:30Z. Step: L2-7 iter 4. Unit: Last seed in own store gives another move. Tests added: 6. Cycles: 5. Sha: 0ef6b38. Agent: tpp:L2-7#4. Tokens: 1268106. Seconds: 169.752.
Actions: Added stores to Board and made sow report another move, then wired play() to show stores and keep the mover, one TPP transformation per commit: {}->nil for an empty store, unconditional->if with scalar->array to sow into the store, nil->constant then constant->scalar for the another-move result. Lower-ranked move accepted with no simpler test: unconditional->if with constant->scalar in play(), since a store seed cannot show without another move until wrapping lands. A clockwise 7-5 spec for P2's another move passed on arrival and was kept as a guard with no cycle counted. No refactor needed on green and no impasse. Conflict logged: the two new specs were written with printf in Bash because the Write tool strips the prompt's trailing space. Full ./run.sh test: BoardTest 9, KalahStoreTest 2, KalahSowTest 2, BoardViewTest 13, KalahQuitTest 2, ExampleAnticlockwiseTest.testQuit and one ExampleClockwiseTest pass. The other 4 example tests still fail as expected until the wrap, capture and game end units land.
2026-10-16T00:17:47Z. Step: L2-7 iter 5. Unit: Sowing wraps onto opponent houses and skips the opponent store. Tests added: 6. Cycles: 4. Sha: aa6582a. Agent: tpp:L2-7#5. Tokens: 1596744. Seconds: 193.322.
Actions: Extended Board.sow past the mover's store, one TPP transformation per commit: unconditional->if to wrap onto opponent houses and unconditional->if to skip the opponent store back onto own houses. Lower-ranked moves accepted with no simpler test: variable->assignment with expression->function for a modulo lap position so sowing can lap the board more than once, and expression->function reusing that position so a lap ending in the store gives another move. Refactored on green by extracting pit(). Two play() wrap specs for anticlockwise 6-4 and clockwise 7-5 passed on arrival once a store-placement error in the hand-written clockwise spec was corrected, and were kept as guards with no cycle counted. No impasse. Conflict logged: the two new specs were written by a printf script in .scratch because the Write tool strips the prompt's trailing space. Full ./run.sh test: BoardTest 13, BoardViewTest 13, KalahQuitTest 2, KalahSowTest 2, KalahStoreTest 2 and KalahWrapTest 2 pass. ExampleAnticlockwiseTest and ExampleClockwiseTest each still have 2 errors as expected until the capture, empty house and game end units land.
2026-10-16T00:20:21Z. Step: L2-7 iter 6. Unit: Capture when the last seed lands in an empty own house. Tests added: 7. Cycles: 3. Sha: 1c1eec4. Agent: tpp:L2-7#6. Tokens: 940241. Seconds: 149.972.
Actions: Added capture to Board.sow, one TPP transformation per commit: unconditional->if to add the landing seed and the opposite house to the store, then unconditional->if narrowing the condition so an empty opposite house gives no capture. The first test was split before going green so the store gain came alone and needed no assignment. Lower-ranked move accepted with no simpler test: statement->statements with variable->assignment to empty the landing and opposite houses. Refactored on green by extracting the opposite house index. Guards that passed on arrival with no cycle counted: no capture in an empty opponent house, a capture into the chosen house after a full lap, and two play() capture specs for anticlockwise 6-4 and clockwise 7-5. No impasse. Conflict logged: the two new specs were written by a printf script in .scratch because the Write tool strips the prompt's trailing space, and the UNITS.md tick used sed in place of Edit. Full ./run.sh test: BoardTest 18, BoardViewTest 13, KalahCaptureTest 2, KalahQuitTest 2, KalahSowTest 2, KalahStoreTest 2 and KalahWrapTest 2 pass. ExampleAnticlockwiseTest still has 1 error and ExampleClockwiseTest 2 errors as expected until the empty house and game end units land.
2026-10-16T00:21:39Z. Step: L2-7 iter 7. Unit: Choosing an empty house asks for another move. Tests added: 2. Cycles: 1. Sha: 62dd06d. Agent: tpp:L2-7#7. Tokens: 509320. Seconds: 71.18.
Actions: Added two play() specs for anticlockwise 6-4 and clockwise 7-5 where P1 picks its emptied house 1, seen red on line 22. One red-green cycle passed with unconditional->if in play() that prints House is empty. Move again. and the board, then prompts the same player. Refactored on green by printing the board once after either branch. No lower-ranked move and no impasse. Conflict logged: the two new specs were written by a printf script in .scratch because the Write tool strips the prompt's trailing space. Full ./run.sh test: BoardTest 18, BoardViewTest 13, KalahCaptureTest 2, KalahEmptyHouseTest 2, KalahQuitTest 2, KalahSowTest 2, KalahStoreTest 2 and KalahWrapTest 2 pass. ExampleClockwiseTest.testUseEmpty now passes. ExampleAnticlockwiseTest and ExampleClockwiseTest each still have 1 full game error as expected until the game end unit lands.
2026-10-16T00:24:44Z. Step: L2-7 iter 8. Unit: Game end with final scores and a winner or a tie. Tests added: 10. Cycles: 10. Sha: 245e63c. Agent: tpp:L2-7#8. Tokens: 1781453. Seconds: 187.804.
Actions: Added Board.score and Board.hasMoves and ended play() with Game over, the board, both scores and a result line when the player to move has no seeds, one TPP transformation per commit: {}->nil for a zero score, constant->scalar for the store and then a single house, nil->constant then constant->scalar for hasMoves, unconditional->if in play() driven red by ExampleAnticlockwiseTest.testFullGame2 with a constant Player 2 wins!, unconditional->if for A tie! driven red by ExampleClockwiseTest.testFullGameTie, and unconditional->if for Player 1 wins! in KalahResultTest. Lower-ranked moves accepted with no simpler test: expression->function summing every house for score and expression->function matching any seeded house for hasMoves. Refactored on green by extracting Kalah.result. Two KalahResultTest guards for the tie and the P2 win passed on arrival with no cycle counted. Quitting still prints no scores, as the quit examples show. No impasse. Full ./run.sh test: 11 classes pass with 0 failures, including all six example tests.
The method's measure agent answered its Measure line from LOG.md and git. Agent-written text is shown verbatim and unchecked. Words such as recomputed inside it are the agent's own and are not tags of this page. Its Measure lines describe measurement during the run. Harness scores were measured separately after the run. Agent text may quote git commit times in the host's local zone. Harness times are UTC.
# Measure
Sources are LOG.md, UNITS.md, TPP.md and the commit subjects in git log. GAP.md and ATTEMPTS.md do not exist. Credits and time are measured by the harness and are not reported here.
## Summary
46 green commits used a median transformation rank of 6 (`unconditional->if`) and a mean of 6.07. 12 of 46 (26%) took a lower-ranked move of rank 10 or below, each logged with no simpler test available. There was 1 backtrack to a simpler test and 0 impasses across all 8 features.
Rank numbers come from TPP.md, where 1 is `{}->nil` and 12 is `variable->assignment`. A commit that combined moves is scored by its lowest-ranked move, which is the highest number. Green commits are the behaviour commits that count as red-green cycles. Refactor, pinned-guard, tick and log commits are excluded. The 46 green commits match the 46 cycles in LOG.md.
## Transformation rank per green commit
| rank | transformation | green commits |
| --- | --- | --- |
| 1 | `{}->nil` | 2 |
| 2 | `nil->constant` | 8 |
| 3 | `constant->constant+` | 1 |
| 4 | `constant->scalar` | 6 |
| 5 | `statement->statements` | 2 |
| 6 | `unconditional->if` | 14 |
| 7 | `scalar->array` | 1 |
| 8 | `array->container` | 0 |
| 9 | `statement->recursion` | 0 |
| 10 | `if->while` | 3 |
| 11 | `expression->function` | 5 |
| 12 | `variable->assignment` | 4 |
## Per feature
| feature | green commits | ranks in commit order | lower-ranked (10 to 12) | backtracks | impasses |
| --- | --- | --- | --- | --- | --- |
| Board rendering for all four variants | 15 | 2 4 6 2 6 2 4 10 2 6 2 10 6 3 11 | 3 | 0 | 0 |
| Prompt and quit with the Game over board | 1 | 5 | 0 | 0 | 0 |
| Sowing within own houses | 7 | 2 11 12 5 10 6 12 | 4 | 0 | 0 |
| Last seed in own store gives another move | 5 | 1 7 2 4 6 | 0 | 0 | 0 |
| Sowing wraps and skips the opponent store | 4 | 6 6 12 11 | 2 | 0 | 0 |
| Capture into an empty own house | 3 | 6 12 6 | 1 | 1 | 0 |
| Choosing an empty house asks again | 1 | 6 | 0 | 0 | 0 |
| Game end with scores and a result | 10 | 1 4 4 11 2 4 11 6 6 6 | 2 | 0 | 0 |
| Total | 46 | median 6, mean 6.07 | 12 | 1 | 0 |
## Backtracks to a simpler test
1. Capture. The first capture test was split before going green so the store gain came alone at `unconditional->if` (5e9b9d1). Emptying the two houses followed as its own step (3715c0e). LOG.md records this. No revert commit exists in git log, so the backtrack happened before any commit.
## Impasses per feature
0 for every feature. Each LOG.md row states no impasse. Every lower-ranked move was accepted with the logged reason that no simpler test existed.
## Notes
- Ranks come from commit subjects and LOG.md. The diffs were not re-scored. Tag `unchecked` for the per-commit classification and `recomputed` for the counts and statistics.
- Passed-on-arrival guard tests add no green commit and are excluded from the rank counts.
See Methodology: measures and the Glossary for the terms and table columns this page uses.