8.7. Write a scenario that could be used to help design tests for the wilderness weather station system
Gary is a guy who works at a remote wilderness weather station. He is currently stationed at a wilderness base. He wakes up, gets dressed, and makes breakfast. After drinking a nice cup of coffee, he sits at his desk, but does not log in to the computer; it stays powered on. The system is running already, constantly scanning for weather anomalies. But as there are no alerts visible on his screen, Gary knows that there is nothing that demands his immediate attention. Everything will be automatically logged and recorded, and he will be able to scan through a nice summary of the conditions and latest predictions if and when anything comes up to report on. He sits back in his chair and relaxes, trusting the weather system to warn him if anything happens, and thinking about Saturdays with the family.
8.10. A common approach to system testing is to test the system until the testing budget is exhausted and then deliver the system to customers. Discuss the ethics of this approach for systems that are delivered to external customers.
The implications of a limited-budget test system are generally negative. It’s an unethical (or at best ethically ignorant) system, due to it’s complete disregard for the quality of testing, instead reducing it to a simple number. It is also in ineffective way to structure testing because:
- Costs have nothing to do with quality: they can be high if resources are wasted, and low if they are used sparingly. A developer who wants to test thoroughly might stretch their resources thin and stay within the limit while wasting time. Contrastingly, a developer who hates testing might outsource it to an expensive consultant; he doesn’t care about the testing, doesn’t want to do it, and is happy to burn the money to get the testing ‘done’ more conveniently, but get a lot less testing for the money.
- Costs have nothing to do with time: Say the team decides to split one developer off to finish the testing while they move on to the next project. His budget might cover the ‘man-hours’ necessary, but with one man, those tests might take far too long. The project will end ‘on budget’ but not ‘on time’.
- Budgets can be incorrectly distributed: As many will admit, testing is few developer’s favorite part of a project. What, then, if the budget manager hates tests? ‘Here’s $5 for testing, make it work’. That will surely lead to a thoroughly and properly tested result. Definitely.
- Order of operations: ‘deliver when the testing budget is exhausted’ implies that testing is the final stage of the project. And yes, once the project is done additional tests will need to be completed. But testing does not start when the project ends; testing done properly starts before coding on the project begins; testability is designed into the core project structure, and is present in the process throughout.
- Artificial boundaries: Testing is not a disparate element that can be wholly separated from development’s other costs. Just as elements of testing are part of the planning process, and start before any programming beings, testing should also be a continuous process to validate and verify the project as it proceeds. A consequence of the 7 +- 2 quality of humanity is that we’re likely to lose track of details once things have piled up and become too large; this means making tests while one still understands the tested material. To achieve this, many tests will have to be done while coding, and will thus blend with core programming ‘budgeting’. And in so doing, the budgets for ‘testing’ and ‘everything else’ become inextricably intertwined and time sheets start getting really complicated. These budgets were never meant to be separated, and doing so causes more problems than it solves.
However, this system will continue to exist, despite these issues, because:
- A budgetary constraint has to exist.
- As much as we might wish otherwise, money and time are the two primary constricting factors in every project. Complaining about it doesn’t change that fact.
- Having a clear goal and clear limitations is helpful for everyone.
- Do we like the customers? Maybe this system is in place because the long-term requirements of the project are immaterial; it’s all about the sale. This very well might be the case if your business is run by business majors. It might be unethical, but they’re okay with that, and you work for them, not the client. Unless you work directly for a client; in which case, they set the budget. Either way, if the person with the paychecks is going to budget your testing, your testing is budgeted. So ethics are forced to bend to the realities of the business world.