Testing, Systems Development:  

Systems Home | Introduction | Needs Assessment | Requirements Specification | Requirements Analysis | Preliminary Design | Detailed Design | Implementation | Testing

Systems Development: Requirements, Design, Implementation, Testing

Testing

Which of the following librettists got it right with regards to systems development?

Things never are as bad as they seem. –Johnny Mercer (1909-1976), U.S. songwriter. "Dream," Her Highness and the Bellboy, Michael H. Goldsen, Inc. (1944).
Things are seldom what they seem, Skim milk masquerades as cream. –Sir William Schwenck Gilbert (1836-1911), British librettist. Mrs. Cripps (Little Buttercup), in HMS Pinafore, Act 2 (1878), published in The Savoy Operas (1926).

Perhaps Americans are a tad optimistic compared to the more stoic Brits, but Gilbert, of Gilbert and Sullivan fame, got it right: when it's all said and done, systems tend to lag behind customer expectations. Significantly.

We talked about this earlier: in requirements specification, we mentioned that software systems are different from so many other human endeavors because they are intangible: software quality cannot be observed directly. Consequently we are forced to rely on testing to help us determine if a developed system meets its specified requirements–or not.

Testing is best performed as an independent, parallel effort from design and implementation. The test process can be developed by a separate team concurrently using the same base set of requirements. Many organizations do this, especially when the output product is a large scale system with serious consequences for failure.

Process Activities

Testing consists of documentation and execution activities. The major test documents are an optional test requirements document, a test plan, test procedures, and test steps. Like other development documents, these start with a general level of detail and grow into finer details as engineering continues.

Execution, or the performance of actual tests, comes in many flavors. There are private and public executions. Generally testing that is expected to fail is performed in private. But testing designed to prove to the customer that the system was developed as specified is performed in public with the customer witnessing the execution of documented test steps.

Exercise

We will write two of the four test documents, namely the plan and the steps. Some of the steps will be performed in public with a customer witness:

Test Procedure Example

Here is a sample test procedure document. The procedure describes the test environment and each test case. But the detail is not sufficient for a novice to execute the tests. Each sentence in a test case would involve five or six steps that the customer could witness and verify independently:

Study Guides

There is no study guide for this unit. There will be a unit quiz, however.

Lecture Slides

Here are copies of the lecture slides presented during class:

Readings

Each of the listed links will open a new browser window:

Concept Notes Links
Definitions Here are some basic articles about topics which are fundamental to this unit.

Software testing: A Wikipedia article.

Test plan: A Wikipedia article.

Test case: A Wikipedia article. In our model, the test plan contains several test cases. Each case is amplified into test steps.

Software Testing as a Process Here are two papers that talk about software testing in general. These papers define the what and why of testing.

Software Testing: Jiantao Pan, Carnegie Mellon University.

The Testing Team's Model: Brian Marick, Testing Foundations.

Test Strategies We mention several different test methods in the lecture. Here are two overall strategies that use these methods to accomplish different testing goals:

Shoestring Manual Testing: Rex Black, RCBS Consultants. Describes the differences between automated and manual testing. Argues for the use of manual testing.

What is Exploratory Testing? James Bach, StickyMinds.com. We talk a great deal about test steps or scripts. Here is an argument for planned but unscripted testing.

Test Economics Here are a few articles that deal with the cost of testing, or how much should an organization spend on testing compared to development.

Quality Cost Analysis: Benefits and Risks: Cem Kaner, Florida Institute of Technology. A formal method for calculating how much an organization should spend on testing.

Software Negligence and Testing Coverage: Cem Kaner, Florida Institute of Technology. Another formula for computing testing budgets, but this one focuses on legal implications in the USA.

It Depends: Deciding on the Correct Ratio of Developers to Testers, Johanna Rothman, Rothman Consulting Group, Inc. A proposal on computing a ratio between developers and testers for an organization. Cites Microsoft as using a 1:1 ratio.

Light Hearted Testing Our last two articles deal with testing in a more light-hearted fashion. The titles speak for themselves!

Incomplete Tests Are Worse Than None At All!: Edward Miller, Software Research, Inc.

Software Engineering Proverbs: Collected by Tom Van Vleck, a Multics operating system author.

Systems Home | Introduction | Needs Assessment | Requirements Specification | Requirements Analysis | Preliminary Design | Detailed Design | Implementation | Testing


Copyright ©2006-2009, Jason Paul Kazarian. All Rights Reserved. License for reproduction, derivative works, internal commercial, and external non-commercial use granted to GITC faculty, staff, and students.