Since the introduction of the ICH 09 Risk Management guidelines in November of 2005, the entire industry has been engrossed in risk assessments, risks management, risk analysis tool , ‘Risk- based’ decision making and a host of risk associated activities.

Companies have designed, and implemented their version of a Risk Management Program (RPM) and now rely on them to make quality related decisions daily. From the design phases of new projects to implementing CAPAs, risk management is now intertwined with nearly all GxP activities, and is as common a daily occurrence that one can barely remember a time when risk assessments were not done.

Now, ten years on, companies and regulators are asking: ‘What have we achieved?’, ‘Has Risk Management lived up to the promises of ten years ago?’ and ‘Are our patients truly better protected now that Risk Management is behind our decision making process?’ This article will set forth a methodology of evaluating the effectiveness and consistency of Risk Management Programs.

Background

The ICH Q09 is not a regulatory requirement. It is not mandatory. Companies have no obligation to adopt it, address it, or even acknowledge its existence. If you are in a position, ten years after its’ introduction, to believe this is true, good for you.

The rest of us in the industry know that Risk Management is firmly established within our industry, and it is here to stay. The advantage of having a guidance document such as the ICH Q09 is that everyone in the industry: engineers, scientist, regulators, senior management and everyone else can assess problematic issues from the same point of view, with the same terminology, and intent. Or so we hoped.

We should stop hoping, and begin to review: what has worked, and what has not. We do so not to discredit the merits of ICH Q09, nor to advocate against it. It is ‘how the industry has embraced, or, perhaps, misinterpreted it?’ that the industry must answer.

Related: Learn more about risk based equipment qualification

Step 1: The Quality Risk Management Program (QRPM) and Environment

In the ICH Q09 Guideline, the first two chapters address ‘Responsibilities’ and ‘Initiating a Quality Risk Management Process’ . When assessing your QRPM, it is simpler to consider these steps together.

In order to have an effective and robust Quality Risk Management Program (QRPM), an organization must ensure that the relevance of the RMP is evident and consistent at all levels within the organization. This step then is to evaluate whether the concepts of Risk Management is consistent from the senior level managers to the shop floor.

There are three areas in which to investigate risk environment: Processes and Procedures, Risk Management Oversight, and Culture.

Processes and Procedures

In assessing the effectiveness of your program, these policies and procedures should be evaluated to ensure that they meet the overriding principles of the ICH Q09 principles:

• Evaluate risk to quality bases upon science and protection of the patient.
• Is the effort of formality and documentation commensurate with the level of risks

Are your policies and procedures in place providing these intentions?

Although many companies start with these two overriding principles as the cornerstone of their Risk Management Program, over time revisions to procedures, and changes in personnel can divert from these original intentions. Many organizations have realized that revisions to standard operating procedures (SOPs) generally include added ‘checks and balances’, and very rarely include the elimination of steps.

So too, Risk Management procedures have evolved from a slim, streamlined, effective, and efficient process, to multiple burdensome, complex, and redundant processes.

It is the opinion of this author that QRPMs which ‘bolt on’ the risk assessment process to existing Quality systems are far more likely to suffer from these ailments. Many organizations have simply added Quality Risk Management Procedures to the list of other Quality Systems.

For example: Change Control Procedures that require an evaluation of and provide mitigating actions against risks are far more efficient than those Change Control Processes that simply point the requester to perform a separate Risk Assessment and mitigating actions. (Remember: Change Control IS a means of risk management).

Risk Management Oversight

Is there a clearly defined process owner, one person or entity that is responsible for the entire program? How does it report up to senior management and down to the shop floor?

In evaluating the effectiveness of QRPMs, there is a clear correlation between inconsistent, ineffective, and inefficient ones and those lacking in some level of reporting structure. A QRPM that is implemented without some type of governance is one which will be uncontrollable, and the outputs of such a program will be variable and inconsistent. Alternatively, a QRPM which monitors its performance will be able to ensure faster delivery of a high quality risk management process.

Oversight activities should include the key performance indicators (KPIs), cycle times, periodic reviews, and senior management responsibilities. These need to be timely, and widely advertised within the organization. Assessing the QRPM will require finding where and when these are communicated.

Culture

Is there the same view and level of importance demonstrated consistently throughout the organization? Checking to see where QRPMs are discussed, reported on, and participated in, is a good indication of whether there is or is not a unified culture of QRPM. Further, listing the sources of risk assessments will indicate areas utilizing QRPM and those which do not. This type of listing can show which departments, managers or project teams are actually involved in risk management. The lack of a uniform distribution of departments conducting risk assessments may be an indication that there areas within an organization that are not utilizing risk management.

Step 2: Risk Assessment

The Risk Assessment consists of three components:

1. Risk Identification: recognizing risk and defining the scope of the assessment
2. Risk Analysis: quantifying or qualifying the level of risks
3. Risk Evaluation: determining if the risk level is acceptable or if mitigation must occur.

For each of the components above, the QRPM must consider the three fundamental questions of risk:

1. What might go wrong?
2. What is the probability it will go wrong?
3. What are the consequences (severity)?

The accepted analysis of risk generally includes establishing a level of ‘severity, likelihood and detectability. Some QRPM procedures call for a qualitative assessment, calling the different levels ‘High’, ‘Medium’ and ‘Low’’. Other QRPMs require a quantitative ranking, such as a numerical scale 1 to 10.

Either way, precise and descriptive definitions of each level is necessary. When assessing this portion of the QRPM, one must determine if the tools that are being used within the QRPM are being applied with consistency and openness.

For example: one risk assessment ranked the likelihood to be ‘Low’ because there were downstream quality checks (stability and microbiologic testing). Another risk assessment on the same process ranked it ‘High’, as reliance on downstream testing should not be relied on to control risks. Nowhere in the QRPM was the use of downstream checks discussed or defined, thus it was open to interruption and yielded varying results

Assessing the QRPM requires comparing the analyses over a period of time and across different disciplines. For example: Are the risk assessments conducted in the QC laboratory evaluating the risk levels the same way the engineering department is? Is there a variance in the risk evaluations done during the summer months, before and after audits, and during the holiday season?

When assessing this portion of the QRPM, one must determine if the tools that are being used within the QRPM are being applied with consistency and openness. This task requires reading through several risk assessments from various authors, teams and sources. In some cases it will be clear to the reader that the author had begun the risk assessment with forgone conclusions already in mind. It can be evident in the description of the risk. For example, language such as ‘making a small change to the system’, ‘an insignificant amount of material’, ‘has been done elsewhere with no significant implications’ and the like can tip off the reader to risk assessments done disingenuously.

Also, the risk team itself can influence the outcome of a risk assessment. Participants who are adamant that there is no risk based on a ‘gut feel’ are dangerous to a scientifically sound risk assessment. Likewise, members who are risk adverse can alter the risk assessment in such a way that all risks are intolerable.

Step 3: Risk Controls

In this step, one must evaluate the decision making process that leads an organization to either accept the risks, or mitigate against them. Again, a comparison between points in time and between disciplines is warranted.

This step provides insight into what is the risk evaluation yielding. For example: A department of engineers performed eighty six risk assessments in the course of a year, well documented in hundreds of pages of Quality Risk Assessments as required by the governing QRPM. Yet, in all of the assessments and all of the effort to produce the documents, they only found two risks that required remediation, or ‘intolerable risks, as they were called.

Determining the cause of a risk assessment yielding such minuscule amount of risks requiring mitigating actions is imperative to improve the QRPM. It could be the team or author that wrote the assessment had predetermined the output of the assessment prior to performing it. It could also show weaknesses in the risk assessment tools, training, or a lack of the ‘culture’ discussed in step 1.

Step 4: Risk Communication

Risk communication is a step within the ICH Q09 guidelines that is the most difficult not only to put into practice but also to assess. It is not required for each and every risk assessment performed and thus is overlooked by many QRPMs.

When a risk assessment identifies risks that require mitigation, how is this shared with other stakeholders that maybe unaware that this risk exists. Within a production plant, there may be many other departments or business units that may be susceptible to the same risks. How are they to know about the ones identified in your assessment?

Are there other sister sites, contract manufacturers, suppliers etc.. that need to be inform? How are their mitigating actions tracked? Having a strong well defined governance (as discussed in step 1) can easily remedy this difficulty.

A robust QRPM will also have defined mechanisms to evaluate the whether or not the risks identified require notification to regulatory authorities, who is responsible for deciding this, and what is the procedure to report such a risk?

Related: Learn more about risk based equipment qualification

Step 5: Risk Review

Many companies produce an annual Risk Management Summary Report to fulfil this requirement. In it, an analysis of the sources of risk assessments, the number of risks and the associated levels, mitigation actions and an analysis if they worked, and areas for continuous improvements are all topics that are essential for a thorough review.

Assessing the QRPM not only needs to ensure such a review is taking place, but that is also effective and suggests definitive actions for continuous improvements Like any feedback to any process, Risk Reviews should prompt positive unambiguous measures to eliminate inefficiencies, and yes, risks. Finally, the assessment needs to determine that such actions did in fact produce the improvements they sought.

Although still optional, establishing a robust Quality Risk Management Program is crucial to remaining compliant and competitive. It is the language the industry and the regulators speak now and will continue to do so in the near future. But, establishing a QRPM is just the start. Assessing your QQRPM is imperative to make it effective and efficient. This can not happen by chance: it must as carefully thought through.