Software Testing

Terms:

Software Testing -- The organized execution of a program with defined data sets following standard procedures to ensure it meets all of the stated requirements and does so without errors.

Validation -- The process of determining the degree to which software meets the stated requirements. Does it do what it is supposed to do? Did we build the right software?

Verification -- The process of determining the degree to which software functions correctly. Does it do what it is supposed to do correctly? Did we build the software right?

I.V. & V. -- Independent Verification and Validation. An area of specialization within computer science. Some companies specialize in performing the testing of software developed by other companies.

Walk through -- a verification method in which a team performs a manual simulation of the program or design.

Code Inspection -- A verification method in which one member of a team (not the author) reads the program or design line by line and the others point out errors.

Black Box Testing -- Testing a function based solely on the inputs and expected outputs without regard to what is actually happening inside the function.

White or Clear Box Testing -- Testing a function based on knowing the internal workings of the function. This testing ensures that all possible paths through the code in the function are tested.

Two Approaches to Software Testing

1. Bottom Up Testing
	In this approach you identify all functions in the program which do not call any other functions. Drivers are created which test these functions thoroughly. When this is completed all functions which call these low level functions are added and drivers created to call those functions. Continue working back up the levels until you reach the top level (usually in main()).
2. Top Down Testing
	In this approach you start with the top level function (usually main()) and devise functions stubs to mimic the action of lower level functions which are called by main. After thoroughly testing main with the stubs you substitute the real functions which main calls and create stubs for the next level of functions called by those which are called by main. Continue working down the levels until you reach the level where functions do not call any other functions.

Five Types of Software Testing

1. Statement Testing
	Testing each line or small group of lines of code as they are added to the source.
2. Unit Testing
	The objective is to to test all the possible behaviors of each individual function in a program.
3. Integration Testing
	After each function has been thoroughly tested with Unit Testing, tests are performed to determine how functions work together.
4. Acceptance Testing
	Before a software product is delivered to the customer it is tested to make sure all of the stated requirements have been met.
5. Regression Testing
	Testing done during the maintenance phase. Invariably bugs will be found in the software, after it has been delivered, these will have to be fixed. The software must then be tested to ensure that the bugs were fixed and that fixing the bugs did not introduce other errors into the software.

Designing a Software Test

	1. Determine what is the goal of the test.
	2. Determine the inputs that will demonstrate the goal of the test and the expected output from the test. Ensure that all possible cases are covered.
	3. Modify the code or create stubs and/or drivers as needed to run the test.
	4. Run the program with the test code and record the results.
	5. Compare the results to the expected results. If discrepancies are noted determine why they occurred, fix the problem, and retest.

Software Test in Reality
The information above relates primarily to the testing performed on a software system as guided by the Software Test Plan. This can be referred to as Formal Testing. There is, however, a lot of Informal Testing that takes place that, while it may not be documented in a test plan, it is understood by every member of a development team that this testing will also be conducted. Some software companies have in place a means of documenting all of the informal testing as it is conducted. This type of testing actually begins during the design phase and continues into the coding phase.

Testing the Design

Development teams hold round table discussions. One team member presents some ideas, a proposed algorithm, a code outline, etc. The team discusses the positive and negative aspects of the proposal. In this fashion the new ideas are given their first test. Any time the software development team meets to discuss some aspect of the system that one member is working on this is referred to as a Peer Review.
After the design document is completed there are meetings to review and discuss the validity and "workability" of the overall design.
Any proposed algorithms are reviewed by the development team before implementation. The meetings to perform these reviews are referred to as Code Reviews.

Testing the Code

Sometimes just before a portion of the code is added to the system the development team will meet with the person who wrote the code and examine the validity of the code. The team will step through the code one line at a time and verify that it looks acurate and will perform as desired. This is called a Code Walk-through
As the code is being written, each programmer, will perform informal unit testing on each segment of a function as it is implemented. This is testing apart from the formal unit testing outlined in the test plan.
Emphasis is constantly given to the importance of testing. Programmers learn to create informal tests that verify even the smallest segment of code in every function of every class in the system.