1. Mr. Stokes, what kind of unknown-unknown or unforeseeable risks can you envisage in project management of pharmaceutical companies?

The one thing that we can be sure about in a project is that there are going to be surprises. It’s basically a mathematical certainty. When you cannot even imagine all that could go wrong, it’s like the law of coincidences, it would be almost impossible that something might not occur. Most surprises will happen in first contact with the patient, as individuals. With patients, if something can go wrong, it will go wrong. For example, many medicines have names that are quite similar; they will be mixed up. People will misread the medical leaflet, misunderstand the informed consent form; investigators in a trial will misunderstand the protocol, or not have time to fill in the case report form; staff will get sick. Yes, amazingly, we even overlook the obvious. We may have an industry which employs many brilliant statisticians, but we are still vulnerable to many forms of cognitive bias. Here are just a few that I have observed in pharmaceutical companies: confirmation bias, base rate fallacy (ignoring the background context), clustering illusion, hindsight bias, attentional bias, anchoring effect, framing effect, illusory correlation and loss aversion.

The purpose of risk analysis is essentially to convert unknown-unknowns into known-unknowns by a structured process of identifying and articulating risks, qualifying and quantifying risks, defining and managing actions. The process of risk analysis begins with the planning of risk management. When planning risks, determine how opportunities will be managed as well as threats; without this you will never be as good as you had planned. Remember also to look for unknown-knowns. It sounds quite strange, but quite simply this is the information that others know, perhaps on a different project, service, company, competitor, country, or between patients and practitioners.
 

2. What risk do you find mostly underestimated ? Why ? Could you give us some examples ?

In general, as human beings we are poor at assessing probabilities, and easily inclined to miscalculate cumulative effects and inter-dependencies. For example, in the case of a medical condition there are common causes and interactions that are frequently misunderstood. To take another example, with communication people influence each other in complex ways. Communication is a process in any case, the end result is comprehension. People often confuse communication with providing information. A medical leaflet, an informed consent form, and even an investigator’s brochure are incomprehensible unless we have written in terms that people can comprehend and have verified that they have understood. This is spelt out in the regulations, but too often we do the minimum to save the pennies and it ends up costing us millions.

If patients drop out of a trial due to misunderstanding, it can invalidate the trial. There are common errors that can be avoided. As many as 50% of drug trials fail in Phase III because the drug cannot be proved effective and for two primary reasons: a novel mechanism of action, i.e. a new biological pathway, and a subjective definition of the endpoint, i.e. a patient opinion. Learning about effectiveness is an objective of Phase II. The remedy is to perform risk management in a way that enables appropriate classification of risks in Phase II, benefitting from more operational experience and more comprehensive decision making.
 

3. What techniques could you recommend to manage these risks ?

When assessing patient recruitment rates there are parameters over a given period. These can be adjusted according to the therapeutic area, nature of the syndrome, number of sites and investigators, other behavioural patterns. Companies should manage their own experience and statistics, recycle and adjust them for future trials. Each range estimate – best case, most likely, worst case – is associated with a standard probability curve, and simulation can reveal possible outcomes. This method termed Monte Carlo works particularly well with a project planning approach.

Decision trees are useful for assessing multiple options that depend upon both events and information. They enable decisions about choices and whether or not to invest in information in order to reduce uncertainties. Expected monetary value is the result of multiplying the expected outcomes by the possible probabilities to obtain an expected risk exposure. It should not be the only criterion to determine contingency, but it can help the decisions about contingency and risk actions.

The Haddon Matrix is a technique that could be used more often, because it reminds us to assess what could happen in the case of an accident or a crisis; before, during and after. It looks at personal aspects, agent or material attributes and environmental factors. Thus what is often neglected, which is prevention, reaction and correction is addressed with more prescience.
 

4. How should risks be managed in practice in drug development projects ?

Risk analysis is very data and information hungry: to convert unknown unknowns into known unknowns (uncertainties into risks) and unknown knowns (ignorance into issues). Questions should be asked in order to reach the roots of the problem. It is easy to say that the project might be late because we may not get enough resources. However, by continuing to ask questions about why this may be so we will eventually reveal something that can be managed. Risks should be articulated in a specific way. For example, “if we fail to promote the importance of our project to the executive steering committee, then the other projects may be judged as more important and we will lose our resources thus making the project late.” If… then…

Once risks have been identified, articulated and qualified, then we can define actions. There are actions to manage threats that take place before the project has started – avoiding, mitigating or transferring – and to manage opportunities – exploiting, enhancing and sharing – and actions that take place during the project when the event giving rise to the risk is occurring or has occurred – accepting the threat or opportunity.

The word ‘share’ is one I prefer to ‘transfer’ since no risk is completely transferred; there is inevitably a residual business risk. Business risks are those that we can manage better than any third party; they are the consequences of our competencies and competitive advantage. To a great extent business is a choice of the risks we feel best able to manage, instead of the ones that are outside are sphere of competency. It would be ridiculous to avoid every risk in our lives and expensive in terms of lost opportunities.

Very often one risk can cause another risk, or conceal another risk. Risks can escalate due to a change in their likelihood or impact. Some risks require many actions, and sometimes one action can address many risks. There is a potential multiple relation between risks and actions, and also there are many interdependent risks.

It is very important to understand the signals, or triggers that provide an indication that the risk event is occurring or is going to occur. Some risks give clear advance warning that they are going to occur, whereas others hit us unexpectedly. Risks that are difficult to detect can be placed on watch list.

A great amount of common ground exists between risk analysis and stakeholder analysis. Many risk actions are about communication. Therefore, understanding communication styles and conflict patterns is a great enable of effective risk management. It is good practice to assemble the team and to use some kind of team brainstorming method at least once during the project. The team brain is usually greater than the sum of its parts.
 

5. What is the role of product managers in the risk management of pharmaceutical projects ?

Product managers should encourage the team to focus on the big picture, the overall vision and to clearly understand the priorities. The purpose and justification for the project can create a sense of excitement and reinforce motivation. Product managers can enable the team to understand the portfolio strategy and the decisions to be made, because “there are no solutions, only trade-offs”.

A visionary product manager understands the science, the technology, the regulatory environment, and the market the customers, the patients. They can explain the end-to-end connection between the initial inspiration and its resolution. They can also remind people that “it’s not what you don’t know that gets you into trouble, but what you think you know that just isn’t so.” Entrepreneurs take sensible risks and avoid the ones that are not worth the trouble. Successful entrepreneurs are those that stop the wrong projects early enough. “Bad news early is good news”, because “we cannot manage secrets”.
 

6. How would you recommend to budget risk management in pharmaceutical companies?

Budget management is determined at portfolio level using portfolio analysis with the aim of building a balanced portfolio in terms of technological and scientific risk, and commercial and regulatory risk. It is neither a good idea to produce only blockbusters that may fail, nor incremental generic products that may not give enough payback. Projects are analyzed in terms of return on investment, while not forgetting that value is even harder to estimate than cost, and there are many vital factors that are not easily measurable.

Escalation thresholds, triggers and tolerances enable risks to be reported early enough and to the right organizational levels. The method of “expected monetary value” facilitates decisions about the appropriate levels of risk tolerance and risk aversion in terms of the exposure and potential payback.

The project should be empowered to manage the threats and opportunities that have the highest probabilities with a ‘project contingency’, whilst the organization manages high impact risks with a ‘management reserve’. The higher the level of risk tolerance, the higher can be the threshold before escalation. An escalation and decision making process is essential in order that management can rise to the challenge when serious adverse events occur. This is part of crisis planning ad once again it relies upon a high degree of preparation and communication capability to stakeholders.
 

7. What practical tip would you give to risk managers in pharma companies?

Risk managers should ensure that risk is given adequate attention and suitable priorities. For example, the most important risk items can be addressed first in project status meetings. When performing go/no-go gate reviews, the project can progress “knowing the risks”, which means knowing who owns and manages each risk; otherwise it is the project manager.

The managers of individual risks do not have to be part of the project, for example for a regulatory risk or a safety risk. However, the project manager and the risk manager must ensure that they are aware they are managing the risks. If there are any doubts about the likelihood or impact of a risk, then it is good practice to allow the person who is managing the risk to make the decision.

Those responsible for risk management should be sensitive to the difficulties of evaluating risks. They can heighten risk awareness by asking probing questions, and insisting upon data, tests, checks, inspections, observation and tangible scientific evidence.

Techniques that are useful for developing awareness are playing the role of devil’s advocate, using check-lists and reading them aloud to raise the levels of consciousness and organizing a pre-mortem to heighten sensitivity to risks. The principle of ‘respectful disobedience’ authorized team-members to “speak truth to power”.

Risk managers can encourage awareness of cognitive and decision-making bias. Projects do not behave as they are supposed to do, and we have a tendency to be over-optimistic and to discount the fact that other events will arise that will distract us from the project goals. This is called the ‘planning fallacy’. Typically it means that project budgets and schedules should be multiplied by a factor of 2.54. Project participants compensate by balancing the planning fallacy with contingency, and senior managers add their contingency, but only at the top level and their instinct is to confiscate the detailed contingency. Good practice is to be open and transparent about the contingency and to manage it at project level, thereby allowing project contingency to be managed by accountable professionals.
 

Thank you, Mr Stokes for your extensive answers!

To practice these and other techniques, subscribe to the course of Ian Stokes in risk management. If you are a risk manager, check also our course “Risk fournisseur”.

Keywords:

IAN Stokes

Expert en Management de Projets

Ian est impliqué dans la facilitation et l'amélioration des processus de gestion de projets, le pilotage de projets technologiques et de développement de produits, la gestion du changement, par l’adoption de méthodes agiles qui sont centrées sur le client et le travail transdisciplinaires dans le contexte de projets complexes. Il a une grande expérience internationale et interculturelle.