Prof. dr. Bruno Stricker: Projections from the past

During his presentation at the MEB Science Day, he takes the audience through the four decades he has been active in the field of pharmacovigilance. “The progression has been enormous, both in methodology as in available resources in the pharmacovigilance playing field”, according to Bruno Stricker, who started his career at the Dutch agency that investigated suspected adverse effects and was a predecessor of current day institutions like Lareb and MEB.  “Reporting adverse drug reactions was exclusively voluntary. We received about 2,000 up to 2,500 reports of suspected adverse drug reactions each year. A manageable amount of data.”

Reports were done pro-actively and came from a steady network of intelligent medical observers. Bruno: “A big advantage in those days was that a large percentage of those reports really was significant. Good quality data. Which was important, since  there was no way to quantify the reports. Are they incidental or part of a pattern? We had no way of knowing, other than using clinical judgement and common sense.” New methods were developed in the late eighties and the first half of the 1990’s. “For instance, we were the first to propose the use of the Reporting Odds Ratio with WHO reports, which is used for calculating the exposure odds of a particular event compared to the exposure odds of all other events in similar drugs. At that time, only the WHO database was big enough for statistically stable estimates. And it was around that time that the first ideas of linking information databases arose. The PHARMO initiative which we took with professor Ron Hering is an example of that: linking hospital records, pharmacy information, and observational cohort studies to investigate and quantify signals of adverse drug reactions.”

Digitalisation

New European legislation on pharmacovigilance, taking effect in 1995, really added to the momentum. “That brought a lot of changes. Both expected and unforeseen changes”, Bruno adds. “We anticipated that voluntary reporting by healthcare professionals would remain very important for signal detection. That was and still is a fact. And we also saw how digitalisation in the nineties took off and started to really aid us in making more and more healthcare databases available. Larger sets of data consequently led to more signals and techniques to test out hypotheses.”

The fact that marketing authorisation holders from that time on have had the obligation to report ADRs, brought an increase in reports and in the administrative burden that comes with that. “The number of reports grew, explosively. And with that, the need for proper management and better and more sophisticated analytics. Because with the amount of data, the overall quality did not go up at the same speed. Detecting and identifying relevant signals became more of a challenge.”

From dangers to scares

With these expected effects, Bruno also witnessed a less expected change. “An unforeseen effect that doesn’t really have anything to do with science, but is more societal”, he explains. “We live in a risk-averse society. Our risk tolerance has been decreasing by the decade. What we accepted as a fact of life in 1980 is now considered as a frightening danger. Combine that with the fact of ever growing numbers of ADRs and a growth in databases and systems with data and knowledge. That combination may lead to a decreased perception of safety. It seems like a paradox: the better view we have of danger, the more we fear it. For every deleted real danger we get back a scare, so to speak.”

In his presentation, Bruno refers to ‘regulations turn danger into scare’: he mentions the code red or code orange warnings for certain weather circumstances that are issued in the Netherlands nowadays. “In 1980, nobody stayed at home for snow or a heavy storm, but now some people almost panic when they issue ‘code red’. And if you let your young child ride a bicycle without a helmet, you are more and more considered to be a bad parent. Risk aversion is increasing everywhere. Globalization and mass media contribute to that as well. We focus so much on health issues that we forget that we should appreciate quality of life more than health alone: our health is a means to an end, and not a goal by itself. After all, in the end we are dead anyhow. Life is a lost battle for everybody, so strive to enjoy life as long as you have it. We should keep that in mind when communicating on adverse events and avoid to make our population neurotic and focus too much on dangers and health alone.

Increase of available data

More data, specifically in the field of ADRs, leads to more and better conclusions and more ways of interpreting signals, right? “Yes and no: more data doesn’t always bring us closer to the truth”, Bruno stresses. “The increase of available data is a huge step forward. But it’s all in the handling of that data and the way you analyse it and extract information. In current day pharmacovigilance, there is a European collaborative infrastructure in place for pharmacovigilance, with well-trained epidemiologists and numerous resources of data. Pooling all that data helps us to study efficacy and effectiveness of medicines, but also enables us to study rare but serious ADRs, or the heterogeneity of drug exposure.”

But there’s also a different perspective. “Sure, the data are there anyhow, so why not try to get everything out of it? Well, there is also a set of potential limitations. Huge databases, for example, also mean that the confidence interval gets more and more narrow. So, more precision but not necessarily more validity. And generating data based on data-mining might lead to false positives, and the results may be prone to confounding.”

Proper interpretation

Data interpretation and signal detection become more and more important with the growing amount of data. Bruno: “Besides multivariable analysis, propensity scores are gaining traction in scientific publications the last years. Propensity scores can be useful, for example when analysing multiple clinical endpoints, or when confounding factors are present that are hard to categorize. However, I think multivariable statistical techniques are always preferable if possible. In my opinion, propensity scores can be hard to fathom and are easily misinterpreted. The amount of factors that are taken into account can get very large and the relevance of the individual factors for the chosen clinical endpoints might differ enormously.”

“Always keep in mind what the real risk factors are, and don’t go for a score that’s based on what’s available”, Bruno argues. “Many administrative, but marginally relevant determinants may be present, but really relevant confounders, like body-mass index, smoking habits et cetera, might not be available. Propensity scores should only be used in factors which can’t be directly adjusted for.” Another important aspect Bruno stresses, is never to forget the original raw data. “In large databases of information, queries might lead to outcomes that aren’t always realistic. When in doubt, always look at the original medical records. You might come across much more plausible causes or explanations than any statistical score might tell you.”

Large data pools are a luxury, but at the same time, the luxury problem of big data is the huge sample size pooled databases have. Bruno: “The narrower the confidence interval CI gets, the sooner a signal seems significant. Validity of data is key there. One of the biggest fallacies in epidemiology is that in a large dataset problems will smooth out themselves. That is wrong: what is invalid data, remains invalid data.”

In conclusion

“Methods, regulations and resources in the area of pharmacovigilance have progressed enormously over the past several decades. Our abilities to detect signals and to analyse hypotheses have also improved, but we should be very cautious. Caution in how big data is analysed and interpreted, and how we communicate this information in society. The more information we have available, the bigger dangers seem to become in perception of people. Second to that, the interpretation of data is strongly dependent of the validity of the (pooled) data. That’s where our challenges are the coming years: lots of data, pooled data in a complex environment, will ask for very careful and accurate analysis and communication.”