In my previous blog I wrote how embracing failure can be a viable strategy, depending on the current quality level. Before we can determine this quality level, we first need to understand what quality is and ask the question “why do we even want quality?”.
Many books have been written on the concept of quality and the Wikipedia page on quality provides the following summary:
“Quality is a perceptual, conditional, and somewhat subjective attribute and may be understood differently by different people. Consumers may focus on the specification quality of a product/service, or how it compares to competitors in the marketplace. Producers might measure the conformance quality, or degree to which the product/service was produced correctly. Support personnel may measure quality in the degree that a product is reliable, maintainable, or sustainable.“
Aligning these different perceptions seems to be a challenge, so how do we determine what amount of quality is required? How do we determine the bar for ‘this would be enough quality for now’?
When you start analyzing the various viewpoints, it becomes clear they share one common factor: they all aim at maintaining a certain risk at a certain level. You want to know how well your product performs to prevent customers from walking away. You want to have insights into your operational performance to prevent going bankrupt. You want to know process effectiveness to prevent fines.
Risk management can help with that. The concept of risk management is mentioned in various ISO norms (e.g., 9001, 14001, 27001, 22301), but they all use similar strategies to handle risks. In general, you can summarize them into four categories:
To lower risks, we introduce quality assurance. Quality assurance can therefore be considered as a risk management process and spending effort on quality is in fact a risk mitigation strategy. Understanding the risk management strategies enables us to design a quality strategy.
So, how can you determine what quality strategy to use when you start looking at the underlying risks?
Deciding what strategy to use depends on your risk appetite for a specific system or functionality and on how much certainty you already have. Risk profiles can also be applied on multiple levels. Some systems are more critical than others, but even within these systems there are likely components with different risk profiles.
Choosing a strategy that provides less assurance (i.e. allows for more risk) will result in more deviations not being identified as early as possible, but by properly looking at your risk profile that might not be a problem. Since you can only spend an hour or euro once, you must decide where it yields the best results.
An example of a high-level risk-based quality strategy is the following:
Figure 1: example of risk management to quality strategy mapping
When the risk profile of a system, or the certainty on it changes, the best quality strategy might also change. In high-risk situations with little certainty, you want to spend more effort in quality and decide to go for a highly-preventive approach such as Test-Driven Design. When you have more certainty about the solution, you can opt for less preventive strategies like classic test automation or even just monitoring your system for problems. By definition, a strategy is therefore not static, but should rather act as a guide.
Let’s take a fictive example about the evolution of a product, the risk profile at each stage, and the quality strategy that would fit that situation:
Figure 2: the risk profile journey
Managing these different aspects is not new within the field of quality assurance. Approaches like risk-based testing have been around for some time and can help quickly determine the most effective strategy for a certain component or system.
The shift-left movement requires that we start doing continuous risk management. Determining the accurate risk profile can be achieved with techniques like risk storming or threat modeling. An accurate risk profile allows for determining the most fitting quality strategy and adjusting your quality assurance process where needed. Using the optimal quality assurance process prevents ‘over-processing’ and ‘gold-plated engineering’ and makes sure your quality assurance process fits the risk profile.