Debugging

 Debugging

• Debugging occurs as a consequence of successful testing.
• When a test case uncovers an error, debugging is the process that results in the removal of the error.
• Although debugging can and should be an orderly process.

Debugging: A Diagnostic Process

The Debugging Process

System Testing & its types

System Testing & its types

• System testing is actually a series of different tests whose primary purpose is to fully exercise the computer-based system.
• Although each test has a different purpose, all work to verify that system elements have been properly integrated and perform allocated functions.
• Types of system tests :
• Recovery Testing
• Security Testing
• Stress Testing
• Performance Testing
• Deployment Testing

High Order Testing

• Validation testing
• Focus is on software requirements
• System testing
• Focus is on system integration
• Alpha/Beta testing 
• Focus is on customer usage
• Recovery testing
• forces the software to fail in a variety of ways and verifies that recovery is properly performed
• Security testing
• verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration
• Stress testing
• executes a system in a manner that demands resources in abnormal quantity, frequency, or volume
• Performance Testing
• test the run-time performance of software within the context of an integrated system

In Brief;

• Recovery Testing
• Recovery testing is a system test that forces the software to fail in a variety of ways and verifies that recovery is properly performed.
• If recovery is automatic (performed by the system itself), reinitialization, checkpointing mechanisms, data recovery, and restart are evaluated for correctness.
• If recovery requires human intervention, the mean-time-torepair (MTTR) is evaluated to determine whether it is within acceptable limits
• Security Testing
• Security testing attempts to verify that protection mechanisms built into a system.
• Stress Testing
• It executes a system in a manner that demands resources in abnormal quantity, frequency, or volume.
• Stress tests are designed to face programs with abnormal situations. In essence, the tester who performs stress testing asks: "How high can we crank this up before it fails?"
• variation of stress testing is a technique called sensitivity testing.
  
• Performance Testing
• Test the run-time performance of software within the context of an integrated system.
• Performance tests are often coupled with stress testing.
• Performance testing occurs throughout all steps in the testing process. Even at the unit level, the performance of an individual module may be assessed as tests are conducted
• Deployment Testing
• In many cases, software must execute on a variety of platforms and under more than one operating system environment.
• Deployment testing, sometimes called configuration testing, exercises the software in each environment in which it is to operate.
• In addition, deployment testing examines all installation procedures and specialized installation software (e.g., “installers”) that will be used by customers, and all documentation that will be used to introduce the software to end users.

Validation Testing

Validation Testing

• Validation testing begins at the conclusion of integration testing,
• When individual components have been exercised, the software is completely assembled as a package, and interfacing errors have been uncovered and corrected.
• At the validation or system level, the distinction between conventional software, object-oriented software, and WebApps disappears.
• Testing focuses on user-visible actions and user-recognizable output from the system.
• In simple meaning, validation succeeds when software functions in a manner that can be reasonably expected by the customer.
• Validation-Test Criteria
• Software validation is achieved through a series of tests that demonstrate conformity with requirements.
• A test plan outlines the classes of tests to be conducted and A test procedure defines specific test cases that are designed to ensure that
• All functional requirements are satisfied,
• All behavioral characteristics are achieved,
• All content is accurate and properly presented,
• All performance requirements are attained,
• Documentation is correct, and usability and other requirements are met (e.g., transportability, compatibility, error recovery, maintainability).
• After each validation test case has been conducted, one of two possible conditions exist:
• The function or performance characteristic conforms to specification and is accepted.
• A deviation (Error) from the specification is uncovered and a deficiency list is created.
• Configuration Review :
• An important element of the validation processis a configuration review.
• The objective of the review is to ensure that all elements of the software configuration have been properly developed.
• The configuration review, sometimes called an audit…
• Acceptance Testing :
• When custom software is built for one customer, a series of acceptance tests are conducted to enable the customer to validate all requirements.
• Conducted by the end user rather than software engineers, an acceptance test can range from an informal “test drive” to a planned and systematically executed series of tests.
• In fact, acceptance testing can be conducted over a period of weeks or months.
• Alpha and Beta Testing :
• If software is developed as a product to be used by many customers, it is impractical to perform formal acceptance tests with each one. 
• Most software product builders use a process called alpha and beta testing to uncover errors that only the end user seems able to find.
• The alpha test is conducted at the developer’s site by a representative group of end users.
• The software is used in a natural setting with the developer “looking over the shoulder” of the users and recording errors and usage problems.
• Alpha tests are conducted in a controlled environment.
• The beta test is conducted at one or more end-user sites.
• Unlike alpha testing, the developer generally is not present. Therefore, the beta test is a “live” application of the software in an environment that cannot be controlled by the developer.
• The customer records all problems (real or imagined) that are encountered during beta testing and reports these to the developer at regular intervals.
• As a result of problems reported during beta tests, you make modifications and then prepare for release of the software product to the entire customer base.

Test Strategies for WebApps

Test Strategies for WebApps

• The strategy for WebApp testing adopts the basic principles for all software testing and applies a strategy that are used for object-oriented systems.
• The following steps summarize the approach :
• The content model for the WebApp is reviewed to uncover errors.
• The interface model is reviewed to ensure that all use cases can be accommodated.
• The design model for the WebApp is reviewed to uncover navigation errors.
• The user interface is tested to uncover errors in presentation and/or navigation mechanics.
• Each functional component is unit tested.
• Navigation throughout the architecture is tested.
• The WebApp is implemented in a variety of different environmental configurations and is tested for compatibility with each configuration.
• Security tests are conducted.
• Performance tests are conducted.
• The WebApp is tested by a controlled and monitored population of end users.

Test strategies for Object-Oriented Software

Test strategies for Object-Oriented Software

• Introduction: The objective of testing, stated simply, is to find the greatest possible number of errors with a manageable amount of effort applied over a realistic time span.
• Although this fundamental objective remains unchanged for objectoriented software.
• The nature of object-oriented software changes both testing strategy and testing tactics (Plan).

Unit Testing in the OO Context

• When object-oriented software is considered, the concept of the unit changes. Encapsulation drives the definition of classes and objects.
• This means that each class and each instance of a class packages attributes (data) and the operations that manipulate these data.
• An encapsulated class is usually the focus of unit testing.
• However, operations (methods) within the class are the smallest testable units. Because a class can contain a number of different operations, and a particular operation may exist as part of a number of different classes.
• Class testing for OO software is the equivalent of unit testing for conventional software.
• Unlike unit testing of conventional software, which tends to focus on the algorithmic detail of a module and the data that flow across the module interface,
• Class testing for OO software is driven by the operations encapsulated by the class and the state behavior of the class.

Integration Testing in the OO Context 

• Different strategies for integration testing of OO systems.
• Thread-based testing
• use-based testing 
• Cluster testing
• The first, thread-based testing, integrates the set of classes required to respond to one input or event for the system.
• Each thread is integrated and tested individually. 
• Regression testing is applied to ensure that no side effects occur.
• The second integration approach, use-based testing, begins the construction of the system by testing those classes (called independent classes) that use very few (if any) server classes.
• After the independent classes are tested, the next layer of classes, called dependent classes, that use the independent classes are tested.
• This sequence of testing layers of dependent classes continues until the entire system is constructed.

Cluster testing

• Cluster testing is one step in the integration testing of OO software. 
• Here, a cluster of collaborating classes is exercised by designing test cases that attempt to uncover errors in the collaborations. 

Strategic Issues

Strategic Issues

• The best strategy will fail if a series of overriding issues are not addressed. Tom Gilb argues that a software testing strategy will succeed when software testers.
• Specify product requirements in a quantifiable manner long before testing commences.
• Objective of testing is to find errors, a good testing strategy also assesses other quality characteristics such as portability, maintainability, and usability.
• State testing objectives explicitly.
• The specific objectives of testing should be stated in measurable terms.
• Understand the users of the software and develop a profile for each user category.
• Develop a testing plan that emphasizes “rapid cycle testing.
• Build “robust” software that is designed to test itself.
• Use effective technical reviews as a filter prior to testing.
• Conduct technical reviews to assess the test strategy and test cases themselves
• Develop a continuous improvement approach for the testing process.

Short note on Software Testing Strategic

Short note on Software Testing Strategic

• The software process may be viewed as the spiral illustrated in Figure.
• Initially, system engineering defines the role of software and leads to software requirements analysis, where the information domain, function, behavior, performance, constraints, and validation criteria for software are established.
• Moving inward along the spiral, you come to design and finally to coding.

Software Testing Strategic

• Unit testing begins at the vortex of the spiral and concentrates on each unit (e.g., component, class, or WebApp content object) of the software as implemented in source code.
• Integration testing, where the focus is on design and the construction of the software architecture.
• Validation testing, where requirements established as part of requirements modeling are validated against the software that has been constructed.
• Finally, system testing, where the software and other system elements are tested as a whole…
• Process from a procedural point of view :
• Testing within the context of software engineering is actually a series of four steps that are implemented sequentially.
• The steps are shown in Figure :

Introduction - strategy for software testing | Verification and Validation (V & V)

Introduction - strategy for software testing

• Testing is the process of exercising a program with the specific intent of finding errors prior to delivery to the end user.
• A strategy for software testing provides a road map that describes the steps to be conducted as part of testing.
• Any testing strategy must incorporate 
• Test planning,
• Test case design,
• Test execution,
• Resultant data collection and Evaluation.

What Testing Shows

A Strategy Approach to software testing

• Testing is a set of activities that can be planned in advance and conducted systematically.
• For this reason a template for software testing—a set of steps into which you can place specific test case design techniques and testing methods—should be defined for the software process.
• Generic characteristics of Strategy Approach to software testing.
• To perform effective testing, you should conduct effective technical reviews. By doing this, many errors will be eliminated before testing commences.
• Testing begins at the component level and works “outward” toward the integration of the entire computer-based system.
• Different testing techniques are appropriate for different software engineering approaches and at different points in time.
• Testing is conducted by the developer of the software and (for large projects) an independent test group.
• Testing and debugging are different activities, but debugging must be accommodated in any testing strategy.

Verification and Validation (V & V)

• Software testing is one element of a broader topic that is often referred to as verification and validation (V&V).
• Verification refers to the set of tasks that ensure that software correctly implements a specific function.
• Validation refers to a different set of tasks that ensure that the software that has been built is traceable to customer requirements.
• Boehm states this another way:
• Verification: “Are we building the product right?”
• Validation: “Are we building the right product?”

Who Tests the Software?