Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Risk-Based Testing – Approach, Matrix, Process & Examples
Risk-based Testing (RBT)
It is a sub-category of software testing based on the probability of risks. In this test, the software is assessed to identify risk. It includes assessing criticality of business, frequency of usage, possible areas with problems, etc. This type of testing emphasizes testing of software’s features and functions which are vulnerable to defects.
Risk is the occurrence of nay unwanted event that may have an effect (positive or negative) on the results of the project. Risks can either be events that have occurred previously or current events, or even something that could take place in future. These events affect the cost, business, technical and quality standards of a project.
There are two types of risk −
Positive risks − These are known as opportunities and enable business sustainability. e.g., investing in a new project, changing business procedures, etc.
Negative risks − These are known as threats and recommendations to reduce or remove them should be executed for project success.
When to perform Risk-based Testing
It can be done in −
Projects having appropriate time, resource, budget, etc.
Projects in which risk analysis can be done to find redundancies and vulnerabilities to SQL injection attacks.
Security testing in cloud computing.
Projects with high risk factors, e.g., lack of expertise with technologies involved in the project or lack of domain knowledge.
Iterative and step-by-step models.
Risk management process
Following are the steps of risk management process −
Risk identification − This can be achieved with risk workshops, checklists, brainstorming, interviewing, Delphi technique, cause & effect diagrams, observations from previous projects, root cause analysis, domain experts and subject matter experts.
Risk register − It is a spreadsheet that includes a list of risks identified, possible responses, and root causes. It tracks and monitors risks throughout the project. Risk response strategies are used to handle risks, both positive and negative. Risk breakdown structure is vital in risk planning. It helps identify risk prone areas and in effective evaluation as well as monitoring of risks, throughout the project. It helps enable enough time and resources for risk management. It also helps categorize sources from which risks to project may arise.
Risk Analysis − After the potential risks have been listed, they are analysed and filtered based on their significance. It includes quantitative and qualitative risk analysis. A technique of qualitative risk analysis is Risk Matrix, used to estimate the probability and effect of the risk.
Risk Response Planning − Based on this analysis, it can de determined whether or not the risks need a response. Some risks essentially need a response in project planning, while some may need it in project monitoring, and some may not even need a response at all. The owner of the risk identifies options to minimize the probability and effective of the assigned tasks.
Risk mitigation − This is a method to minimize the effects of the risks or possible threats. It is executed by removing the risks or minimizing them to a specified acceptable level.
Risk contingency − This is the probability of an unforeseen event, whose effect is is neither known nor can be estimated. Contingency plan is also referred to as action plan or backup plan for worst case scenarios. It helps decide steps to take in case an unexpected event materializes.
Risk monitoring and control − It is a process to track the identified risks, monitoring the residual risks, discovering new ones, updating the existing risks, analysing the change, executing the response plan and monitoring risk triggers. This can be possible through risk assessments, risk audits, variance and trend analysis, measuring the technical performance, updating status meetings and retroactive meetings.
Note that risk increases with technology variations, project size, project length or duration, number of sponsoring agencies, efforts and lack of appropriate skills.
Approach of Risk-based Testing
Analyse the needs.
Review documents, such as SRS, FRS, Usecases, etc. to detect and remove errors and ambiguities.
To avoid the introduction of any late change in the project, a risk-reduction technique known as requirements sign-off’s is adopted. After baselining the document, if any change has to be made, that would demand a change control process and subsequent approvals.
Evaluate the risks by determining the effect and likelihood that each requirement can have on the project considering defined criteria’s such as cost, schedule, resources, scope, technical performance safety, reliability, etc.
Determine the probability of failure and risk-prone areas. This is done using risk assessment matrix.
Maintain a risk register to record identified risks. Update, track and monitor the risks from time to time.
Risk profiling is performed at this stage to determine the capacity and tolerance levels of risks.
Prioritize the needs according to their rating.
Define risk-based testing process.
The highly critical and moderate risks are taken into account for mitigation planning, implementation, and progress monitoring, while low risks are on a watch list.
Assess the quality of data through risk data quality assessment.
Plan the test based on the rating.
Adopt a proper approach and test design technique for designing test cases in such a way that tests the highest risk items first. Such items can be tested using resource with expert domain knowledge.
Different techniques for test design can be adopted, such as decision-table technique for high-risk test items, while using equivalence partitioning for low-risk items.
Test cases also cover multiple functionalities and end-to-end business scenarios.
Develop test data, conditions and test bed.
Review test plans, test strategy, test report, and other documents created by testers. This in vital for identifying and reducing defects.
Do dry runs and check results’ quality.
Ensure traceability between risks, tests covering them, their results, and defects identified while testing.
At the system level, focus on what’s most important in the software. This can be done by looking at the visibility of functions, frequency of usage, and estimated cost of failure.
Evaluate exit criteria. By now, all high risk-prone areas have been tested, and only minor risks are left unattended.
Report the risk-based testing results.
Reassess previous and new risk events on the basis of Key Risk Indicators.
Update risk register.
Analyse and prevent defects to remove them.
Perform regression testing to authenticate the fixes on the basis of estimated risk analysis. High-risk prone area must be intensively covered.
If feasible, perform risk-based automation testing.
Calculate residual risks.
Use exit or completion criteria for different risk levels.
Assess risk profiling and client feedback.
Risk-based Testing Approach
Technical System Test − It is also known as environment test and integration test, that includes testing in development, testing, and the production environment.
Functional System Test − This tests all functionalities, features, programs and modules. Its objective is to evaluate whether the system meets the specified requirements.
Non-functional System Test − This tests the non-functional aspects, performance, load tests, stress-test, configuration, security, backup and recovery procedures and documentation.
Prioritization and Risk Assessment Matrix
Risk assessment matrix, also known as probability impact matrix, enables testers to have a quick view of the risks and the priority with which a risk is addressed.
Probability is the measure of prospects for an unexpected event’ occurrence. Exposure in time, proximity and repetition, is expressed in terms of percentage.
They are
Frequent (A) − Expected to occur several times in most scenarios. (91-100%)
Probable (B) − Likely to arise several times in most scenarios (61-90%)
Occasional (C) − May arise sometime (41-60%)
Remote(D) − Unlikely to arise/may arise sometime (11-40%)
Improbable(E) − May arise in rare scenarios (0-10%)
Eliminated(F) − Impossible (0%)
Severity is the measure of loss due to unexpected event. It is scored 1-4 and is categorized as −
Catastrophic = 1 − Harsh effects that reduces the productivity to zero. It can also cause the project to be terminated. It has a top priority in risk management.
Critical = 2 − Large effects, can cause great loss, and severely threatens the project.
Marginal = 3 − Short-term, yet reversible.
Negligible = 4 − Little loss, can be managed through routine procedures.
Priority has 4 categories − serious, high, medium and low.
Serious − Project must be terminated, and take immediate and effective steps to isolate the risk. The project is not resumed, unless the risk has been reduced to low or medium level
High − Take immediate steps to isolate, remove and substitute the risk, take effective risk controls. If not resolved immediately, define strict timelines to resolve them.
Medium − Take sensible and practical actions to minimize the risks.
Low − Do not usually pose any threat, can be ignored. However, make periodical reviews to make sure the controls are effective.
Results Reporting and Metrics
Test report preparation − Resorting test status includes efficiently communicating the results with stakeholders. It provides a clear understanding and comparison of test results and objectives.
Number of test cases planned and executed.
Number of test cases passed and failed
Number of identified defects, their severity and status.
Number of existing critical defects.
Environment failures.
Metrics preparation − A metric is a combination of multiple measures to compare software processes, projects and products.
Schedule and effort changes.
Productivity of test case production.
Test design coverage.
Risk identification efficiency
Risk mitigation efficiency
Test efficacy.
Test execution coverage.
Test execution productivity.
Defect leakage.
Defect identification efficacy.
Requirement stability index.
Quality cost
Analyse the risks involved in functional non-functional aspects based on defect status and number of test pass and fail status, their relationship with risks.
Determine key lead and lag indicators, prepare warning indicators.
Track and report lead and lag indicators by analysing data patterns, trends and dependencies.
Advantages of Risk-based Testing
Risk-based testing tests all the important functionalities of the software. It offers realtime clear understanding of the risks involved in the project.
Intensively emphasize risks of business project rather than functionality of the information system.
The testing can be done in a language that all stakeholders can understand.
It concentrates on optimal test delivery, productivity and cost reduction.
2K+ Views
