Kalah TDD study · Part 27 of 29 · Data

Supporting data: method table

The method table, with each column linked to its measure.

Source run full-1 at tdd-experiment c3dd988

Contents

One row per method of run full-1. Each column heading links to its definition, and Columns on the measures page lists all 32 columns.

Method table

Methods, 11 rows and 32 columns
Run
MethodBranchRun order
Classic greedy TDDmethod/classicparallel
Noun-verb mapmethod/noun-verbparallel
Mapped-path TDDmethod/mapped-pathparallel
Functions first, classes latermethod/functions-firstparallel
Transformation Priority Premise TDDmethod/tppparallel
Coarse-to-fine TDDmethod/coarse-to-fineparallel
Annealed refactoringmethod/annealedparallel
Multi-start TDDmethod/multi-startparallel
Most informative next testmethod/mutation-guidedparallel
Property-based TDDmethod/property-basedparallel
Held-out TDDmethod/held-outparallel
Correctness
MethodFunctional mark /15Hidden pass /76Unseen pass /69Acceptance pass /6
Classic greedy TDD15/1576/7669/696/6
Noun-verb map15/1576/7669/696/6
Mapped-path TDD15/1576/7669/696/6
Functions first, classes later15/1576/7669/696/6
Transformation Priority Premise TDD15/1576/7669/696/6
Coarse-to-fine TDD15/1576/7669/696/6
Annealed refactoring15/1576/7669/696/6
Multi-start TDD15/1576/7669/696/6
Most informative next test15/1576/7669/696/6
Property-based TDD15/1576/7669/696/6
Held-out TDD15/1576/7669/696/6
Code
MethodStrict PMD complianceStatic analysis violationsMutation scoreLines of code
Classic greedy TDDfail (10 violations)10105/113, 0 not run, adjusted 105/105 (0 of 8 survivors fail hidden tests)206
Noun-verb mappass088/115, 0 not run, adjusted 88/109 (21 of 27 survivors fail hidden tests)400
Mapped-path TDDpass0107/113, 0 not run, adjusted 107/109 (2 of 6 survivors fail hidden tests)975
Functions first, classes laterpass0117/132, 0 not run, adjusted 117/122 (5 of 15 survivors fail hidden tests)538
Transformation Priority Premise TDDfail (7 violations)794/114, 0 not run, adjusted 94/107 (13 of 20 survivors fail hidden tests)418
Coarse-to-fine TDDfail (6 violations)6102/111, 0 not run, adjusted 102/102 (0 of 9 survivors fail hidden tests)539
Annealed refactoringfail (2 violations)2115/123, 0 not run, adjusted 115/117 (2 of 8 survivors fail hidden tests)181
Multi-start TDDfail (5 violations)5103/123, 0 not run, adjusted 103/118 (15 of 20 survivors fail hidden tests)384
Most informative next testfail (9 violations)9112/119, 0 not run, adjusted 112/112 (0 of 7 survivors fail hidden tests)513
Property-based TDDfail (5 violations)5116/123, 0 not run, adjusted 116/116 (0 of 7 survivors fail hidden tests)1237
Held-out TDDfail (4 violations)4112/149, 0 not run, adjusted 112/141 (29 of 37 survivors fail hidden tests)331
Structure
MethodClassesMethodsFields
Classic greedy TDD190
Noun-verb map146030
Mapped-path TDD166730
Functions first, classes later64514
Transformation Priority Premise TDD3212
Coarse-to-fine TDD3236
Annealed refactoring1163
Multi-start TDD3251
Most informative next test3196
Property-based TDD4225
Held-out TDD52613
Quality ratings
MethodClear Presentation /5Explanatory Language /5Consistent Code /5Used Content /5
Classic greedy TDD3345
Noun-verb map4444
Mapped-path TDD4444
Functions first, classes later4443
Transformation Priority Premise TDD3445
Coarse-to-fine TDD4445
Annealed refactoring5445
Multi-start TDD4445
Most informative next test3445
Property-based TDD3445
Held-out TDD4445
Quality ratings
MethodSimple Constructs /5Minimal Duplication /5Modular Structure /5Problem Alignment (tentative) /5
Classic greedy TDD3433
Noun-verb map4434
Mapped-path TDD4444
Functions first, classes later4444
Transformation Priority Premise TDD4444
Coarse-to-fine TDD4454
Annealed refactoring4433
Multi-start TDD4443
Most informative next test4343
Property-based TDD4344
Held-out TDD4454
Cost
MethodTokensWeighted tokensWall timeAgent calls
Classic greedy TDD5,369,214843,7340:22:3111
Noun-verb map4,798,738961,2060:23:399
Mapped-path TDD19,389,9752,126,2190:54:3414
Functions first, classes later9,108,6261,273,0660:28:519
Transformation Priority Premise TDD11,999,0551,291,1800:33:0310
Coarse-to-fine TDD5,705,380969,3660:21:567
Annealed refactoring7,926,085958,0930:26:328
Multi-start TDD8,187,2041,295,7990:21:1912
Most informative next test4,795,045760,6860:44:128
Property-based TDD7,851,4341,234,9540:29:4812
Held-out TDD4,966,338844,9590:21:5211
Curve and speed
MethodAPFDc-HCost to completionBench median ms
Classic greedy TDD0.326804,727605
Noun-verb map0.4065918,336562
Mapped-path TDD0.28312,058,511520
Functions first, classes later0.60651,195,424521
Transformation Priority Premise TDD0.2811,232,581205
Coarse-to-fine TDD0.4196926,685605
Annealed refactoring0.3125910,035526
Multi-start TDD0.01821,255,948553
Most informative next test0.3265708,760576
Property-based TDD0.29761,171,669553
Held-out TDD0.3522801,898606
Cost per hidden test
MethodMM_io
Classic greedy TDD542,369496,442
Noun-verb map545,018415,798
Mapped-path TDD1,475,6581,386,111
Functions first, classes later470,362441,270
Transformation Priority Premise TDD886,267829,734
Coarse-to-fine TDD537,832420,899
Annealed refactoring625,663557,219
Multi-start TDD1,233,1301,118,377
Most informative next test477,347448,873
Property-based TDD823,020703,501
Held-out TDD519,499443,244
MethodAudit flags
Classic greedy TDD0 high, 10 low
Noun-verb map1 high, 20 low, 1 of 1 high have a disposition
Mapped-path TDD0 high, 6 low
Functions first, classes later0 high, 6 low
Transformation Priority Premise TDD0 high, 0 low
Coarse-to-fine TDD0 high, 13 low
Annealed refactoring0 high, 13 low
Multi-start TDD0 high, 2 low
Most informative next test0 high, 0 low
Property-based TDD0 high, 8 low
Held-out TDD9 high, 4 low, 9 of 9 high have a disposition