Test Strategies for WebApps
- The strategy for WebApp testing adopts the basic principles for all software
testing and applies a strategy that are used for object-oriented systems.
- The following steps summarize the approach :
- The content model for the WebApp is reviewed to uncover errors.
- The interface model is reviewed to ensure that all use cases can be
accommodated.
- The design model for the WebApp is reviewed to uncover navigation
errors.
- The user interface is tested to uncover errors in presentation and/or
navigation mechanics.
- Each functional component is unit tested.
- Navigation throughout the architecture is tested.
- The WebApp is implemented in a variety of different
environmental configurations and is tested for compatibility
with each configuration.
- Security tests are conducted.
- Performance tests are conducted.
- The WebApp is tested by a controlled and monitored
population of end users.
Test strategies for Object-Oriented Software
- Introduction: The objective of testing, stated simply, is to find the
greatest possible number of errors with a manageable amount of
effort applied over a realistic time span.
- Although this fundamental objective remains unchanged for objectoriented software.
- The nature of object-oriented software changes both testing
strategy and testing tactics (Plan).
Unit Testing in the OO Context
- When object-oriented software is considered, the concept of the
unit changes. Encapsulation drives the definition of classes and
objects.
- This means that each class and each instance of a class packages
attributes (data) and the operations that manipulate these data.
- An encapsulated class is usually the focus of unit testing.
- However, operations (methods) within the class are the smallest
testable units. Because a class can contain a number of different
operations, and a particular operation may exist as part of a number
of different classes.
- Class testing for OO software is the equivalent of unit testing
for conventional software.
- Unlike unit testing of conventional software, which tends to
focus on the algorithmic detail of a module and the data that
flow across the module interface,
- Class testing for OO software is driven by the operations
encapsulated by the class and the state behavior of the class.
Integration Testing in the OO Context
- Different strategies for integration testing of OO systems.
- Thread-based testing
- use-based testing
- Cluster testing
- The first, thread-based testing, integrates the set of classes
required to respond to one input or event for the system.
- Each thread is integrated and tested individually.
- Regression testing is applied to ensure that no side effects occur.
- The second integration approach, use-based testing, begins the
construction of the system by testing those classes (called
independent classes) that use very few (if any) server classes.
- After the independent classes are tested, the next layer of classes,
called dependent classes, that use the independent classes are
tested.
- This sequence of testing layers of dependent classes continues until
the entire system is constructed.
Cluster testing
- Cluster testing is one step in the integration testing of OO software.
- Here, a cluster of collaborating classes is exercised by designing test
cases that attempt to uncover errors in the collaborations.
Strategic Issues
- The best strategy will fail if a series of overriding issues are not addressed. Tom
Gilb argues that a software testing strategy will succeed when software
testers.
- Specify product requirements in a quantifiable manner long before testing
commences.
- Objective of testing is to find errors, a good testing strategy also assesses
other quality characteristics such as portability, maintainability, and
usability.
- State testing objectives explicitly.
- The specific objectives of testing should be stated in measurable terms.
- Understand the users of the software and develop a profile for each user
category.
- Develop a testing plan that emphasizes “rapid cycle testing.
- Build “robust” software that is designed to test itself.
- Use effective technical reviews as a filter prior to testing.
- Conduct technical reviews to assess the test strategy and test cases
themselves
- Develop a continuous improvement approach for the testing
process.
Short note on Software Testing Strategic
- The software process may be viewed as the spiral illustrated in
Figure.
- Initially, system engineering defines the role of software and
leads to software requirements analysis, where the information
domain, function, behavior, performance, constraints, and
validation criteria for software are established.
- Moving inward along the spiral, you come to design and finally
to coding.
Software Testing Strategic
- Unit testing begins at the vortex of the spiral and concentrates on
each unit (e.g., component, class, or WebApp content object) of the
software as implemented in source code.
- Integration testing, where the focus is on design and the construction
of the software architecture.
- Validation testing, where requirements established as part of
requirements modeling are validated against the software that has
been constructed.
- Finally, system testing, where the software and other system
elements are tested as a whole…
- Process from a
procedural point of
view :
- Testing within the
context of software
engineering is
actually a series of
four steps that are
implemented
sequentially.
- The steps are
shown in Figure :
Introduction - strategy for software testing
- Testing is the process of exercising a program with the specific intent
of finding errors prior to delivery to the end user.
- A strategy for software testing provides a road map that
describes the steps to be conducted as part of testing.
- Any testing strategy must incorporate
- Test planning,
- Test case design,
- Test execution,
- Resultant data collection and Evaluation.
What Testing Shows
A Strategy Approach to software testing
- Testing is a set of activities that can be planned in advance
and conducted systematically.
- For this reason a template for software testing—a set of steps
into which you can place specific test case design techniques
and testing methods—should be defined for the software
process.
- Generic characteristics of Strategy Approach to software testing.
- To perform effective testing, you should conduct effective technical reviews. By doing this, many errors will be eliminated before testing
commences.
- Testing begins at the component level and works “outward” toward the
integration of the entire computer-based system.
- Different testing techniques are appropriate for different software engineering
approaches and at different points in time.
- Testing is conducted by the developer of the software and (for large projects) an
independent test group.
- Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy.
Verification and Validation (V & V)
- Software testing is one element of a broader topic that is often
referred to as verification and validation (V&V).
- Verification refers to the set of tasks that ensure that software
correctly implements a specific function.
- Validation refers to a different set of tasks that ensure that the
software that has been built is traceable to customer requirements.
- Boehm states this another way:
- Verification: “Are we building the product right?”
- Validation: “Are we building the right product?”
Who Tests the Software?
Short note on Reviews Guidelines
- Review the product, not the producer.
- Set an agenda and maintain it.
- Limit debate and rebuttal.
- spending time debating the question, the issue should be recorded for further
discussion off-line.
- Enunciate (Identify) problem areas, but don't attempt to solve every
problem noted.
- Review is not a problem-solving session. The solution of a problem can often
be accomplished by the producer alone or with the help of only one other
individual. Problem solving should be postponed until after the review meeting.
- Take written notes.
- Limit the number of participants and insist upon advance preparation.
- Develop a checklist for each product that is likely to be reviewed.
- Allocate resources and schedule time for FTRs.
- Conduct meaningful training for all reviewers.
- Review your early reviews.
Short note on Formal Technical Reviews. (FTR)
- Formal technical review (FTR) is a software quality control activity performed by software engineers (and others).
- The objectives of an FTR are:
- (1) To uncover errors in function, logic, or implementation for any representation of the software;
- (2) To verify that the software under review meets its requirements;
- (3) To ensure that the software has been represented according to predefined standards
- (4) To achieve software that is developed in a uniform manner;
- (5) To make projects more manageable. In addition, the FTR serves as a training ground, enabling junior engineers to observe different approaches to software analysis, design, and implementation
- The FTR is actually a class of reviews that includes walkthroughs and inspections
The Review Meeting:
- Every review meeting should abide by the following constraints:
- Between three and five people (typically) should be involved in the review.
- Advance preparation should occur but should require no more than two hours of work for each person.
- The duration of the review meeting should be less than two hours. Given these constraints, it should be obvious that an FTR focuses on a specific (and small) part of the overall software.
- For example, rather than attempting to review an entire design, walkthroughs are conducted for each component or small group of components.
Review Summary Report
- What was reviewed?
- Who reviewed it?
- What were the findings and conclusions?
The Players of Review Meeting
- Producer—the individual who has developed the work product
- Informs the project leader that the work product is complete and that a review is required.
- Review leader—evaluates the product for readiness, generates copies of product materials, and distributes them to two or three reviewers for advance preparation.
- Reviewer(s)—expected to spend between one and two hours reviewing the product, making notes, and otherwise becoming familiar with the work.
- Recorder— a reviewer who records (in writing) all important issues raised during the review.
Short note on Informal Reviews
- Informal reviews include
- A simple desk check of a software engineering work product with a colleague,
- A casual meeting (involving more than two people) for the purpose of reviewing a work product,
- The review-oriented aspects of pair programming.
- A simple desk check or a casual meeting conducted with a colleague is a review.
- However, because there is no advance planning or preparation, no agenda or meeting structure, and no follow-up on the errors that are uncovered,
- the effectiveness of such reviews is considerably lower than more formal approaches. But a simple desk check can and does uncover errors that might otherwise propagate further into the software process.
Review Metrics & their use
- Introduction : Technical reviews are one of many actions that are
required as part of good software engineering practice.
- Each action requires dedicated human effort.
- The following review metrics can be collected for each review that
is conducted
- Preparation effort, Ep—the effort (in person-hours) required to
review a work product prior to the actual review meeting.
- Assessment effort, Ea— the effort (in person-hours) that is
expending during the actual review
- Rework effort, Er— the effort (in person-hours) that is dedicated to
the correction of those errors uncovered during the review
- Work product size, WPS—a measure of the size of the work product
that has been reviewed (e.g., the number of UML models, or the
number of document pages, or the number of lines of code)
- Minor errors found, Errminor—the number of errors found that can be
categorized as minor (requiring less than some pre-specified effort
to correct)
- Major errors found, Errmajor— the number of errors found that can
be categorized as major (requiring more than some pre-specified
effort to correct)
What Are Reviews?
- Introduction : Software reviews are a “filter” for the software process.
- Reviews are applied at various points during software engineering and serve to
uncover errors and defects that can then be removed.
- Software reviews “purify” software engineering work products, including
requirements and design models, code, and testing data.
- Technical reviews – TR (Peer Reviews) are the most effective mechanism
for finding mistakes early in the software process.
- Six Steps are employed (Planning-Preparation-Structuring meeting Noting
error-Making correction-Verifying correction)
What Do We Look For?
- Errors and defects
- Error — A quality problem found before the software is released
to end users
- Defect — A quality problem found only after the software has
been released to end-users
- The primary objective of technical reviews is to find errors during
the process so that they do not become defects after release of the
software.
- The obvious benefit of technical reviews is the early discovery of
errors so that they do not propagate to the next step in the
software process.
Defect Amplification (Extension / Increase)
- A defect amplification model can be used to illustrate the
generation and detection of errors during the design and code
generation actions of a software process.
