Software Testing Metrics (Definition, Types, Example)

Software TestingAutomation TestingTesting Tools

Software Testing Metrics

In the software testing process, Software Testing Metrics are quantitative measures that are utilized to evaluate progress, quality, productivity, and general wellness. The major objective of software testing metrics is to enhance the efficiency and effectiveness of the software testing process, as well as to assist in making smart decisions for future testing by giving dependable data about the testing process.

The Metric quantifies the degree to which The system, system component, or process possesses a specific attribute. A weekly mileage of a car compared to its ideal mileage recommended by the manufacturer is an excellent example of metrics.

Software testing metrics - Increases the efficiency and efficacy of software testing.

Example of test metrics −

  • How many flaws are there in the module?

  • How many test cases does each person run?

  • What is the percentage of test coverage?

The quantitative indicator of the extent, capacity, dimension, amount, or size of some feature of a process or product is known as software testing metrics or software test measurement.

As an example of software test measurement, consider the following −

Total number of defects

Why Do We Need Test Metrics?

The most important responsibility of the Software Test Lead/Manager is the generation of Software Test Metrics.

Test Metrics are employed to −

  • Make a decision about the next phase of activities, such as estimating the cost and timeline for future projects.

  • Recognize the type of improvement required to ensure the project's success.

  • Make a decision on whether or not to modify the process or technology, for example.

Why are Test Metrics Important?

"We cannot improve what we do not measure," and Test Metrics enables us to do just that.

  • Make a decision on the next phase of activities.

  • Proof of the claim or prediction

  • Recognize the type of enhancement required.

  • Consider a decision, a process, or a technological change.

Manual test metrics

Manual test metrics are divided into two categories in software engineering.

  • Base Metrics

  • Calculated Metrics

Base Metrics

These are the raw data that the test engineer collects during the testing process.

Here are some examples of Base Metrics given below −

  • Number of test cases

  • Number of test cases completed

Calculated Metrics

These are attained by converting the data collected in Base Metrics into utilitarian information.

Here are a few examples of calculated metrics −

  • Test efficiency

  • Test coverage

Some of the key metrics depend on the project or business model.

  • Test case execution productivity metrics

  • Test case preparation productivity metrics

  • Defect metrics

  • Defects by priority

  • Defects by severity

  • Defect slippage ratio

In what ways do matrices help

  • Metrics are used to improve the quality of products and services, which aids in the achievement of customer happiness.

  • Different metrics assist teams in monitoring and controlling the efficiency of the process.

  • It identifies areas where the present process can be improved.

Lifecycle Metrics

  • Analyze −

    • Identify and define metrics

    • Establish parameters for assessing metrics

  • Communicate −

    • Explain the importance and necessity of metrics to stakeholders and the testing team

    • Educate the testing team on the data points that must be captured in order to process the metric

  • Evaluating −

    • Capturing the required data

    • Testing the validity of the data recorded

    • Calculating the value of the metrics

  • Reports −

    • Create a report with an effective conclusion.

    • Distribute reports to stakeholders, developers, and testing teams

    • Collect feedback for future improvements

8 Effective Software Testing Metrics

Here is a list of eight useful and effective software metrics that the team should use.

  • Test Case Productive Preparation Test Case productive Preparation = Total test steps / effort (hours) e.g., TPP = 60/8 = 7.5 8 test cases/hour

  • Test Execution Summary Summarize your reports using parameters such as

    • Test Case Passed

    • Failed Test Case

    • Execution of the Test Case

    • The test case was not carried out.

    • Blocked Test Case

  • Test Case Coverage  = [Executed Test cases / total no. of test cases] * 100

  • Defect Acceptance  Defect Acceptance= {[No. of valid flaws / total no. of defects] * 100}

  • Defect Rejection  Defect Rejection= [No. of invalid flaws / total no. of defects] * 100

  • Test Efficiency Test Efficiency = {[DT / (DT + DU)] * 100} DT is the Defect by Testing team & Development team DU is the Defect by customer

  • Effort Variance  Effort variance = {[(Actual effort – estimated effort) / estimated effort] * 100} You can discover from the above formula that −

    • If Effort Variance is positive, we took more time (effort) to complete the planned work.

    • A negative Effort Variance indicates that we took less time (effort) to complete the planned work.

    • If the Effort Variance is zero, you are on track with your estimated effort.

  • Schedule Variance  Schedule Variance = [(Actual no. of days – estimated no. of days) / estimated no. of days)] * 100 We can deduce from the preceding formula that −

    • When Schedule Variance is (+) positive, you are ahead of schedule.

    • A (-) negative Schedule Variance indicates that you are behind schedule.

    • If Schedule Variance is zero, this indicates that you are on track.

Calculation of test metrics

There are some specific steps that should be taken.

  • Find the key software testing processes to be measured 1st.

  • The tester uses the data as the foundation for defining the metrics in this step.

  • Determination of the information to be tracked, the frequency with which it will be tracked, and the person in charge of the task

  • Effective estimation, management, and interpretation of the defined metrics

  • Determine the areas that require improvement based on the interpretation of the defined metrics.

Calculation of Test Metric

Steps to test metricsExample
Determine the primary software testing processes that will be evaluated.Process for tracking progress is being tested.
The tester utilizes the data as a baseline to define the metrics in this step.The amount of test cases that will be run each day.
Identifying the information to be tracked, the frequency of tracking, and the person in chargeThe test manager will record the actual test execution every day at the conclusion of the day.
Calculation, management, and interpretation of defined metrics that are accuratePer day, the number of actual test cases executed
Based on the assessment of defined metrics, identify opportunities for improvement.When the Test Case execution falls short of the target, we must examine the cause and make recommendations for improvement.

Test Metric Example

We'll look at an example of a percentage test case executed to see how the test metrics are calculated.

The formula is used to calculate the percentage of test cases that have been executed.

$$\mathrm{\% \:of \:Test\: Cases\: Executed = \frac{No.\: of\: Executed\: Test\: Cases}{Total\: No.\: of \:Test \:Cases\: Written}\:\times \:100}$$

Similarly, you can compute for other parameters such as test cases that were not executed, test cases that were passed, test cases that failed, test cases that were blocked, and so on.

Updated on 23-Sep-2021 11:03:32