Software Testing - Metrics

---

---

Software testing is an integral part of software development life cycle (SDLC). Efforts are made to make the testing process more effective, robust etc so that the software quality is not compromised. To have more confidence in the software, various metrics are adopted to make testing more efficient.

What are Test Metrics?

The metrics are like a measurable yardstick of the software testing methodologies, approaches, quality, overall process etc. It enhances the quality and effectiveness of testing. It also aids in making better plans and decisions for the future course of testing by generating data and evidence. A metric indicates the degree to which a software, and its components, features etc has a specific numerical attribute.

Why are Test Metrics Important?

The test metrics are important because of the reasons listed below −

• They help to improve the software quality and performance, and identify the areas, scope of continuous improvements etc for the team.
• They help to improve the productivity of both developers and testers.
• Certain metrics namely, the defect density, defect arrival rate assist the testing team to get the patterns and trends of defect in the early stages of the SDLC.
• They identify the places where the testing efforts are needed, by doing so resource optimization can be done. The testing resources are deployed at critical stages, thus it improves the overall test strategy and plan.
• They determine the progress of the testing process, by keeping track of the total number of tests executed, completion percentage, if the testing is progressing as per the plan, etc.

Types of Test Metrics

The various types of test metrics are listed below −

Metrics for Projects

It is created to monitor the quality of the project and is used to estimate various parameters including the budget for the project.

Metrics for Product

It is created to measure the size, architecture, quality etc for the product and can be utilized by the developers to improve their development processes.

Metrics for Process

It is created to measure the processes followed in the project. It can be used to improve the testing processes in the SDLC.

It is recommended that the target users and reasons for having metrics are determined before creating them. Besides, they should be driven by the project requirements and the financial numbers involved with each of the statistics obtained from those metrics.

What are the Different Test Metrics?

There are different test metrics are listed below −

• Base Metric − It is created by analyzing the data gathered by executing the test cases throughout the SDLC. A project status report is created and shared with the project stakeholders. The base metrics are measured in terms of the total count of test cases and total executed out of them.
• Calculated Metric − The data from the base metric is used to generate the calculated metric.
• Defect Metric − It is created to examine the various quality parameters of the software namely the stability, performance, functionality, usability etc.
• Schedule Metric − It is created to obtain the time difference between the expected and actual test execution.
• Defect Severity − It helps the developer to determine how the defect can affect the functionality and quality of the software.
• Test Case Effectiveness − It is created to identify how effective the test cases are at identifying errors in the software.
• Defects Finding Rate − It is used to understand the trends of defects over a period of time.
• Defect Fixing Rate − It is used to determine the time required for a defect fix.
• Test Coverage − It is used to identify the number of requirements which have been covered.
• Defect Root Cause − It is used to determine root cause of defects.

Life Cycle of Test Metrics

The various phases of test metric life cycle are listed below −

• Analysis − Identify and describe the metrics to be used by the testing team.
• Communicate − All the team members and project stakeholders are made aware of the need of a test metric and train them on the data required to process the metric.
• Evaluate − Data obtained from metric are obtained and captured.
• Report − Share the test metric report to all the project stakeholders and gather feedback from them.

Test Metric Formula

% of Tests Executed = (Total Count of Tests Executed)/Total Count of Tests Written) * 100

Test Case Effectiveness = (Total Count of Defects Identified)/(Total Count of Tests Executed) * 100

% of Passed Tests = (Total Count of Tests Passed)/(Total Count of Tests Executed) * 100

% of Failed Tests = (Total Count of Tests Failed)/(Total Count of Tests Executed) * 100

% of Blocked Tests = (Total Count of Tests Blocked)/(Total Count of Tests Executed) * 100

% of Defected Resolved = (Total Count of Defects Resolved)/(Total Count of Defects) * 100

Rework Effort = (Total rework effort in a stage)/(Total effort in that stage) * 100

% of Accepted Defects = (Total Count of Valid Defects)/(Total Count of Defects) * 100

% of Deferred Defects = (Total Count of Deferred Defects)/(Total Count of Defects) * 100 

Test Metric Example

Based on the above data, let us calculate various test metrics −

% of Tests Executed =  250/500 * 100 = 50.

Test Case Effectiveness = 20/250 * 100 = 8.

% of Failed Tests = 40/250 * 100 = 16.

% of Blocked Tests = 10/250 * 100 = 4.

% of Defected Resolved = 15/20 * 100 = 75.

% of Accepted Defects = 18/20 * 100 = 90.

% of Deferred Defects = 2/20 * 100 = 10.

Conclusion

This concludes our comprehensive take on the tutorial on Software Testing Metrics. We’ve started with describing what are test metrics, why are test metrics important, what are the types of test metrics, what are the different test metrics, what is the life cycle of test metrics, test metric formula, and test metric examples.

This equips you with in-depth knowledge of Software Testing Metrics. It is wise to keep practicing what you’ve learned and exploring others relevant to Software Testing to deepen your understanding and expand your horizons.