skip to Main Content

Opportunities and Challenges with Real World Data: A Conversation with Duke Margolis

The 21st Century Cures Act now allows the Food and Drug Administration to approve medical trials based on real-world evidence and real-world data. The potential for many new life-saving drugs and devices to reach the market quickly is exciting, but this blue ocean frontier still needs policy to guide safety and efficacy standards, leading many to ask how and when to leverage real-world evidence/real-world data (RWE/RWD).

Earlier this year, Golden Source Consultants partnered with TAG (the Technology Association of Georgia) to present a “A Conversation with Duke Margolis: Real-World Data—Opportunities and Challenges.” The event featured a keynote by Dr. Gregory Daniel of Duke University’s Margolis Center for Health Policy as well as a Q&A with Daniels, Tim Eggena of Diasyst, and Dan O’Connor of CioxHealth. The event was moderated by Julie Krommenhoek of CioxHealth.

We have included both a recording of the event and a full transcript below.


KEYNOTE SPEAKER: Dr. Gregory Daniel

Gregory W. Daniel, PhD, MPH, RPh, directs the Duke-Margolis office in Washington, D.C. He leads the Center’s pharmaceutical and medical device policy portfolio, which includes policy and data strategies for improving development and access to innovative pharmaceutical and medical device technologies. His expertise spans post-market evidence development to support increased value, improving regulatory science and drug development tools, optimizing biomedical innovation, and supporting drug and device payment reform.

Dr. Daniel is also a Senior Advisor to the Reagan-Udall Foundation for the FDA and Adjunct Associate Professor in the Division of Pharmaceutical Outcomes and Policy at the UNC Eshelman School of Pharmacy. Previously, he was Managing Director for Evidence Development & Biomedical Innovation in the Center for Health Policy and Fellow in Economic Studies at the Brookings Institution and Vice President, Government and Academic Research at HealthCore (subsidiary of Anthem, Inc). Dr. Daniel’s research expertise includes utilizing electronic health data in designing research in health outcomes and pharmacoeconomics, comparative effectiveness, and drug safety and pharmacoepidemiology.

Thank you everyone, for coming on a Friday afternoon. I wasn’t sure if everybody would come, but this is very engaged group, so it is an important topic. I’m going focus a little bit on the opportunities with real-world evidence from a policy perspective.

The Duke-Margolis Center is a brand-new center at Duke University. It’s a center for health policy. We’re about three-and-a-half years old. We have a big office in Washington D.C. I direct that one that has about 30 staff members up in D.C. working on a whole host of policy issues. We also have a big office on campus at Duke University, where we have 40 core affiliated faculty members across Duke Law, the Fuqua School of Business, Duke Medicine, DCRI, the Policy School at Duke.

Our center reports into Sally Kornbluth, who’s the Provost of the university, which means that we have a sister relationship to all of the other schools and departments at Duke. And that actually allows us to collaborate much more effectively on any policy issue that affects healthcare. We have our policy experts in D.C., and we can engage them with the faculty at Duke and do really good practical evidence-based policy. In the D.C. office and in the Durham office, our center focuses on two big areas of health policy. I run the biomedical innovation part, which includes any policy issue related to drugs and devices, that’s my sort of area. And then we also do a lot with physician, health system, payment reform, value-based care, health policies that affect the delivery of care. And on the third piece of our focus areas is that we’re also trying to do a better job of educating students. Duke has not had a health policy program, degree program, and so we’re building that within our health policy center as well.

So now to turn to real-world evidence. I know it’s a wide-ranging group here today. So, what I thought I’d do is start out a little bit with how drugs and devices are developed. And I’m going start showing what I call a “cone of evidence.” So, from the beginning of human testing to FDA approval, we have phase 1, 2, 3 clinical trials. It takes about 10 to 12 years on average to get a drug or device through the clinical trial process and evidence gets developed. And we all know the phase 3 clinical trials are the pivotal trials for regulatory approval. Thousands of patients, potentially, over a number of years, three or four or five years, to develop the evidence that FDA needs for approval. And so, the cone of evidence is pretty narrow, but it gets developed. Once the drug is on the market, then there are FDA required post-marketing studies. There are often safety studies or confirmation studies.

For a product that might be approved under accelerated approval, the FDA would require that company to still do post-market safety or effectiveness studies. And so that cone of evidence begins to develop. It gets a little wider and we get more evidence about how those products work in the patient population, but those studies are still, according to the FDA indication, according to the labeling, the people who are in those trials are generally, they’re sort of the healthy people with the disease. You don’t see a lot of patients with multiple comorbidities, extremes of age, where the real-world people that have these diseases and use these products aren’t necessarily represented in the clinical trials, but we still develop a cone of evidence. But what actually happens, once the product is approved, either medical device or drug, is that the utilization of the product far out-passes or surpasses the evidence that we have for. So, when a brand new drug is on the market and a patient comes into physician’s office and asks “Is this drug for me?” a lot of times it’s like, “Well, it’s for this disease. So, let’s try it with you. We think it might work.”

But the reality is, that patients is often not necessarily reflected in the clinical trial. So, we get a lot of off-label uses of the products and we get variants in populations. And we see that when you actually take a look at the real-world setting, really the patients who are using these products, they have multiple comorbidities, there are extremes of age, products are being used differently than they were in the clinical trials. And so we have a lot of evidence potential, but that evidence historically and usually is not necessarily reflected in the regulatory approval of that particular product. So, when a new patient says “I have a disease and so I’m going to talk to my doctor about what therapy he’s recommending” … or you have a family member who’s starting on a new therapy, what does everybody do? You go on to Google and Google will tell you information about that product. And 99% of the information that Google tells you is the approved labeling.

You hear about the studies, you hear about this drug, and the challenge is, oftentimes, it’s not relevant to the patient who’s going to take it.

My favorite example is in pulmonary dysfunction, either asthma or COPD. When in clinical trials, the primary endpoint is FEV1, Forced Expiratory Volume in one minute. And so, I think of a patient, it might be a family member who suffers from COPD, and they go into the doctor’s office and it hasn’t been controlled. And the doctor’s like, “Oh, we’ve got this brand-new drug. It’s going be perfect. It improves your FEV1.” And the patient says, “Yes, I’ve been waiting for that drug to improve my FEV1. I think about my FEV1 every night.”

But really, the patient says, “You know what would be good for me? Instead of going to the ER three or four times a month, because of asthma exacerbated complications or COPD attributed complications, I would love it if this drug could help me get to the ER twice a month or once a month. That would be a huge difference for me instead of four or five times a month.” And the physician says, “Maybe, we don’t really know. That wasn’t studied in the clinical trials, we have no idea. But let’s just assume if your FV1 is better that you’re not going to go to the ER.” So, it’s a logical leap.

The question is: is the information in clinical trials? Yes, it’s really important to FDA in making those approval decisions, but oftentimes it’s not relevant to the patient making a decision, or in the physician-patient interaction, or even Medicare or commercial payers making coverage decisions. So, payers started making these coverage decisions, and I think if you’re a developer, you know that the evidence that’s developed for FDA approval is not necessarily all the evidence that you need for that product.

So that brings in real-world data and real-world evidence. Those two things are different. Real-world data are the raw materials, like the data from claims data, electronic health record data, patient registries, and even newer technologies that we’re going hear about from some of our panelists. This includes mobile apps or wearable devices that actually collect information directly from patients. Those typically are not in clinical trials. So, real-world data are just almost everywhere.

Any time a patient sees a physician, goes to the hospital, takes a prescription, monitors their blood glucose levels, that happens across the healthcare system. We don’t do a really good job of leveraging that data and turning it into actionable evidence. So real-world evidence quite simply is taking the real-world data, the raw materials, and doing really good research studies to turn that data into evidence. And so while everybody thinks of real-world data as sort of claims data and electronic health record data, increasingly there are new technologies that are doing a much better job of getting data directly from patients, so that we can better understand how they’re experiencing the healthcare system and how these products are working for their particular disease.

Traditional clinical trials are always going to be the gold standard for drug and device development, but they’re challenging. They don’t necessarily get to all of the questions that the healthcare system has. They’re very time intensive, they’re very resource intensive, and they don’t necessarily leverage all of the information that we have from patients. So, should FDA use real-world evidence more? Well, there are a lot of questions about that.

The benefit, in my mind, is wouldn’t it be great if we learned from real-world evidence and that information that we learned can actually get into FDA-approved labeling? When it’s in the labeling, that means that the company can sort of talk about it. That means that when you Google that drug, you can actually get information about what might be more relevant to patients. And then really the only way to do that is to enable the FDA to sort of consider these broader ranges of data and this evidence to potentially get into labeling. So, it turns out that Congress thought that was a good idea, too. The 21st Century Cures Act was passed in 2016. It was the last piece of legislation under the Obama administration, and was probably the last piece of legislation that got overwhelming bipartisan support.

The FDA is an act of Congress and only exists because of Congress, and it could only do what Congress says that it can do. And so, the 21st Century Cures had a provision that said, “Okay, the FDA can now or must now develop a program that would begin evaluating the uses of real-world evidence in its regulatory decisions.” Since then, in December of 2018, the FDA put forth its real-world evidence framework for the program, and sort of how it views real-world evidence, and how it might develop guidances, and things like that. It’s the first official thing from the FDA about real-world evidence over the next several years. The FDA will be developing guidances for exactly how it plans to look at real-world data, how it plans to leverage that evidence into trials, and into regulatory decisions. And so, for the kinds of things that we’re talking about, it’s not going to be brand new drugs on the market approved based on real-world evidence, because the drug has to be on the market to generate the real-world evidence.

So, we’re talking about the kinds of decisions that FDA makes about new drug on the market, and then later on we learn more. So new indications, labeling changes, new populations for that particular product, that’s the opportunity for real-world evidence. And even post-market safety assessments and post-market confirmation studies as well. So, the FDA sort of put forth its framework, and the framework is right here, it’s three things.

One is, how will FDA look at real-world evidence? Well, one of the biggest things is data quality. How do we know the data are of good quality? The reviewers at FDA have decades of experience looking at clinical trial data. They’re pretty comfortable with it. They know what to look for, they know where the pitfalls are, they really know how to look at that data. But if you think about real-world data, where is that coming from? Who really knows? It’s coming from everywhere. How do you look at that data and know if it’s high quality? Some of it may be super high quality, some of it may be really low quality, but how do you really know? And one of the things that the FDA is really concerned about is if we start getting sponsors submitting evidence packages that include all kinds of data from all kinds of sources, and they say, “Trust us, it’s good quality.” The agency really needs to have methods and tools and techniques to look at this data and understand, is it high quality? Is it fit for use? So that’s a huge focus area for FDA, right now. They don’t have all the answers and they’re looking to the stakeholder community, the researchers, policy groups like Duke-Margolis and others as to how they should think about measuring data quality.

The second thing is research design. Real-world evidence can include randomized studies, but more often includes prospective observational studies, retrospective observational studies, and the question is, how can those studies get to the regulatory threshold of, according to Congress, substantial evidence? That’s how FDA approves. They have to approve based on substantial evidence. In statute, it means studies, plural, that are adequate and well-controlled. Then in regulation the FDA defines what adequate and well-controlled is. If you look at it, it kind of looks like traditional clinical trials. So, the big question is, how could these real-world evidence studies actually get to what FDA has to get it to: substantial evidence. A lot of questions about study design, propensity score matching, all of these sort of techniques, artificial intelligence, machine learning, those techniques. Can you actually develop causal inferences from these kinds of data?

The Duke-Margolis Center been working a lot in this area, on real-world data, real-world evidence, modernizing M Health, turning AI into solutions for the healthcare system, and we’ve put a lot of white papers and publications out there on these topics. The FDA has tapped our organization to help them with coming to terms with how to think about real-world evidence for regulatory uses. So, we do a lot of work in partnership with the FDA, a lot of public events, a lot of research, and a lot of bringing together of the experts in the field to answer, how we should measure data quality? How should we design these studies? How do we know that in a particular clinical situation that these studies can actually get to substantial evidence?

So here’s our answer, from a couple of years ago, and we keep iterating it, and this still holds true. How do you know that the FDA can use real-world evidence? Well, it depends, that’s what everybody said. It depends. On what? We think it depends on four things. It depends on the regulatory question. If there’s a drug on the market and a sponsor comes to them and says, “Okay, we know it’s being used in asthma, but we think it could actually help in diabetes.” It’s a totally new population. The question is, could real-world evidence support an entirely new population evaluating safety and efficacy?

Another kind of regulatory decision would be same population, same use. We just want to put the reduction in asthma-related hospitalizations as an additional end point. That’s a different kind of regulatory question. Here, the opportunity for real-world evidence really depends on, what’s the specific regulatory question? What’s the clinical context? Is there a clinical equipoise? Do we have a reason to believe that the people who are only using this product are much sicker than the ones who might be using your comparative product? And observational designs, that can be a concern, because you don’t know if the effects on the drug are because of differences in population or if it’s truly because of the drug, so you have to think about, in that clinical context, whether or not real-world evidence could be useful. Big areas are in the middle. So those two things taken together, regulatory context and clinical context, give rise to the questions, what kind of data do you need? Is it relevant data? And is it of high quality, depending on how you measure it?

One of the big projects that we’re doing right now with FDA, as well as in collaboration with a number of companies and academic groups, is coming up with standardized ways to measure data quality and to measure fit for use characteristics of the data. The second thing is, okay, you have to have the right data and it has to be good quality. Nobody really knows what good enough is, but you have to come up with something, is the method. So, is this a question that you could only answer with a randomized design? Believe it or not, you could do randomization and still do real-world evidence, where it’s essentially taking the trial to clinical practice. Pragmatic trials, where you decrease the restrictions on the patients who are in the study to relax the inclusion-exclusion criteria. You randomize them on day one, and then on day two forward it becomes an observational study, where you just step back and let patients do what they normally do in the healthcare system, versus in a traditional trial where you say, “Okay, you have to come back every week and you have to get all of these measures and you have to document for us that you’ve taken the drug, and all of these, the exact way.”

The reality is you miss dosage. You take them at different times. You have different kinds of patients. And so, a pragmatic trial would take the element of randomization and then allow normal, usual care to happen. All of those considerations taken together … can we really be confident that the real-world evidence that’s being developed is good? Parallel to all of this, drugs are expensive, okay? And this, in fact, is a dated slide, because this is like two years ago, when the media was like, “Oh no, a million-dollar gene therapy treatment is coming out.” It should now say two million, a two million drug out there. And if you’re paying attention to the political debates, drugs are expensive. Everybody thinks so. And so, the big question in policy around this is like, “How do we deal with drug prices?” And so, it’s a huge issue.

The Trump administration last year put out American Patients First, which is the blueprint to how to deal with drug prices. It turns out there are a lot of good ideas that were in the blueprint, improving competition, so relaxing some of the regulatory requirements to allow generic companies and bio-similar companies to get into the space to help bring down prices. Another thing was decreasing patient out-of-pocket costs. A lot of manufacturer rebates go to payers, so manufacturers jack up the prices. They give big rebates to the payers. This is what the media tells you. And then patients have to pay a percentage of the original price, not of the net rebated price.

So, there have been things coming out of the administration that sort of take away the ability for companies to do that and that require them to give those rebates to the patients at the point of sale, so that’s sort of working its way through. And then another way is to think about this is in terms of value-based payment arrangements. Why pay for a drug based on the list price? Wouldn’t it just make sense that if the drug works then you pay that price. If it doesn’t work, you get your money back or you don’t pay as much? It’s a concept in healthcare that’s sort of generating a lot of interest. And so we’ve been looking at this sort of question of, “What are value-based payment arrangements?” And so, quite honestly, we have this broad definition.

There are basically two ways to do this. One is that you develop a value-based price, which would be using value assessment frameworks, or ICER, where for new drugs on the market and they say, “Based on clinical trial results, based on how much it costs to treat this condition, this is what we think a value-based price is.” Usually it’s a third-party group that is not related to the manufacturer or the payers that determines this value-based price. And so that’s one way to do it. The issue there is that becomes the price, and then that’s what you pay no matter what, even if in your population it doesn’t work, you still pay that value-based price.

The other way to do this is called an outcomes-based contract. Where it’s a contract between the manufacturer and the payer that says, “Okay, if we actually observe in our patients who are using this product that we’re not getting to the clinical outcomes that you’re promising, we get money back. Or we don’t have to pay as much.” And for some conditions and some technologies that might make a lot of sense. For gene therapies, for example, $2 million for a gene therapy, and you’re going to the payer and saying, “You know, well, this is going to be a lifetime of benefits. So, it’s worth this much.” And the payer, if you’re savvy, is saying, “Well, okay. But your clinical trial was only three years long, or five years long. So how do I know it’s a lifetime of benefits, okay?” And, so, there’s a lot of uncertainty. It could be good for our patients, but we don’t know if it’s a value for our population. So, one way to think about it is, if you’re the company, “Okay, got it. You’re a little uncertain. Well, how about if we give you 80% of your money back if it doesn’t work, at year one, two, three, four and five?”

Turns out you can’t do that because we have regulations called Medicaid Best Price. And what that means is, let’s say the payer paid $2 million for that gene therapy, and at year two it stopped working, so the company gives 80% back—every state Medicaid program in the country would say, “Ah, that’s our new price for everyone.” Okay? So that makes sense for volume-based care, but in value-based payment arrangements what that has done, it has resulted in companies not willing to provide these deep discounts. So, you’ll see discounts less than 20% because if it’s any more in statute, if it’s any more than 23.1% of the list price, that triggers Medicaid Best Price, and that becomes the new price that all state Medicaid programs pay. These are challenges when you start thinking about value-based care.

So at the Duke-Margolis Center, we formed a new consortium that includes companies, payers, patient groups, to try to figure out, “Well, this is a good idea, but how can we do these better?”

Some of these legal regulatory barriers, like the Federal Anti-Kickback Statute, sort of prevents a lot of this stuff. The Medicaid Best Price issue is there. The other thing is that if we’re going to pay based on outcomes, we want to be able to measure the data. And we did a survey of companies and payers, and one of the biggest barriers to doing meaningful value-based payment arrangements is they’re saying, “We just don’t have the data.” Payers have claims data, but if you’re like, “We’re going to pay you more … and if the quality of life of our patients isn’t improving, we get money back.” Great idea, but we don’t collect quality of life data, so how will we ever know it? There are so many types of patient-centered meaningful outcomes that it would be great to adjudicate payment against. “Let’s pay for these things that are better for our patients.” But we don’t collect that data. And so, the opportunity to leverage real-world data and evidence, yes, it’s good for regulatory uses and we get more information labeling, like I said. What we’re really hoping would happen, is that as companies start doing that more, we start generating more real-world evidence that might support regulatory decisions.

That evidence could also support payment and coverage decisions. We’re increasingly smarter every day with new technologies, these devices and apps, that now it’s actually easy … instead of diabetes, you maybe measure quality of life in these smartphone apps. And voila, we have lots of great data that then we can link payment to. That’s where we’re moving, that’s where we need to be moving, but the real-world evidence, real-world data is sort of lagging behind. One of the challenges will be on the regulatory side, even if we start using these data for coverage and payment, how do we know if it’s good, how do we know the best uses of that data? So, a lot more understanding in the opportunity for these data, are they good quality data, what can they be used for? There’s so much opportunity to learn from those data, so we still need to take a good look at quality and issues like that, to make sure that we’re using these data in the right way.

The benefits toward moving to value-based payments, it’s not just bringing down drug prices. I don’t know if it’ll bring down drug prices. I think a combination of value-based pricing that I talked about and the outcomes-based contracts together could actually bring prices down, but the prices and the payment should be linked to value. So, if they’re fundamentally improving the way patients experience their disease, we should pay more. We shouldn’t pay more if it’s not helping the patients. We can get there with value-based payment arrangements.

There are other reasons to do it … on this payer uncertainty, a lot of times, payers might be like, “Well, we’re just not going to cover it until years and years down the road when we have enough evidence to say, ‘Okay, it is valuable for our patient population.’” One way to do it might be, “Okay, we’ll cover it now, let’s collect data and evidence along the way.” And that can help de-risk that decision from the payer group and can support better real-world evidence calculations. So, if you’re an innovative company with a new technology, and you’re partnering with a software company and you go to the payer and say, “Hey, we can not only deliver our product, but we can help you with data collection. And we can help you sort of make sure that we’re improving the way we’re collecting data, because it’s important to measure our technology.” And so, these kinds of arrangements could actually result in more support for better real-world evidence and data collection, even though right now, they’re limited because we don’t have good real-world data.

Okay, I know I’m way over my time, so thanks that’s it, and we will go to Q&A. Thank you.


Julie Krommenhoek: Thank you, Dr. Daniel. I think we’ll introduce the panel and open it up for questions.

Tim Eggena is joining us today. Tim is the Chief Strategy Officer with Diasyst. It is a health IT venture that offers remote patient monitoring and pharmacotherapy solutions specific to the management of type 2 diabetes. We’ll be able to talk to Tim about how applications and IoT might affect how our real-world data is collected and analyzed.

Today we also have Dan O’Connor, the Chief Strategy Officer of CioxHealth, head of M&A and corporate strategy. Dan and I worked together on the health technology side where CioxHealth is really dedicated to improving health outcomes in the U.S. We have an interesting way of transforming how clinical data is used and where it resides, and we use that to curate and utilize it to produce really interesting outcomes.

CioxHealth is also the catalyst for real-world evidence starting with real-world data. Dr. Daniels made a great distinction, explaining that data is really the potential energy behind the kinetic of getting to the evidence, which really does turn into approvals expansions and where the FDA is really looking for the market and looking to us to make some really interesting inroads and actionable insights.

I’ll kick us off now with our first question. Would each of you briefly offer us how your organization fits into the puzzle piece of real-world data and real-world evidence in the healthcare ecosystem? I’ll start with Dan.

Dan O’Connor: Yes. So, CioxHealth is probably the largest health IT company you’ve never of. It started as an organization that helped providers deal with all of the requests for clinical information in their facilities. They come from everywhere. Attorneys, payers, patients, health insurance, life insurance, and clinical researchers. Ciox was essentially a roll-up of a lot of these companies, and we’ve invested about $1.2 billion in building technology that can standardize the way those requests are taken in and the way data is then shipped out, so at least providers have an understanding of what’s being asked for and what’s going out.

When I came on board a few years ago, one of my goals was to figure out how we can help the life sciences industry be one of those requesters and help them get good real-world data out of providers in a compliant, lawful way. And so, we’ve spent the last year-and-a-half, Julia and I and a couple of other folks, designing a business to enable that in a way that satisfies the needs of clinical researchers and in a way that satisfies the expectations of a hospital, with the data that they create about the patients. We’re very excited about that. We’re in the second inning of that ball game, but we’ve got a lot of investment behind it, we’ve got some great clients who’ve anchored around us and said we believe in what you’re doing, we believe in how you’re doing it, and we think we’re going to be a real catalyst to serve up real-world data for the researchers who are going to do great things with it and turn it into evidence.

Tim Eggena: I don’t know how I’m going be able to speak without a white board, but I’ll do my best. [Laughter]. As Julie said, Diasyst is a remote patient monitoring platform for type 2 diabetes and medication management. So, we started this center here with the patient and they’re capturing the glucose levels, their sugars via our smartphone app, and we’re tracking the sugars and their medications. So, our solution then, when sugars are out of whack, is to advise the clinician when and how to adjust meds. So ultimately, I’d describe our solution as, Dr. Dan, if you can look over the shoulder every time they did a finger stick and then you can adjust meds, wouldn’t that improve outcomes? Of course it would. You provided them a ton of resources to do that. So, that’s essentially what we do. With Diasyst at the core, now let’s talk about the ecosystem.

Not only do you have one medication, you may have multiple meds, there are diabetics on many meds, up to six, seven… Our algorithms and platform support all 106 medications on the market today, and combinations of medications. So, you’ve got meds, you also have devices, you’ve got glucometers. There’s actually a state-of-the-art glucometer with a chip in it, so now they don’t have to manually key in the sugar entry into our smartphone app, it automatically goes to the cloud and comes into our product. If you just begin to think about those three dynamics alone—meds, devices, glucometers—and then talk about real-world evidence and real-world data, I can only begin to imagine what we’re going to be able to figure out. I know that we will have empirical evidence with the drugs, with the devices, and with other factors, but what’s it going to show us is still yet to be seen.

We’re brand new to market, just a few thousand patients on the system, but looking at what’s being presented here today, my techie propeller head starts going off thinking about, “What other points should we capture?” If I partner with a payer or I partner with a pharma or device company, what would be important to them to capture with the smartphone device. It could be geographic location, it could be, “What did you eat this morning?”, “Are you happy today?” Dr. Daniel’s presentation really begins to let me think about what other data points could be valuable for both data and evidence-based type decisions.

I have one more thing … anything that Dr. Daniel writes lands on my desk immediately. I’ve got an RS feed that I read, and I go “That makes a lot of sense, we should be thinking about this.” For example, we’re working with the largest pharmaceutical companies in the world. The challenge that we run into is that everybody looks at the capability of and promise of a solution like Ciox— get clinical information out that’s de-identified, structured, and analyzable, and it’s everything, it’s not just what’s in a CCD or a summary document, it’s what’s in a pathology report—and they look at that think, “this is regulatory approval right here, just waiting for me.” And you know what comes in the door is a proposal to turn it into a commercial for Enbrel. So it’s a non-regulatory use of that, so if you think about, watch the nightly news tonight and you’ll see an eczema commercial and at the beginning, you’re thinking, “That could really help me,” and at the end you’re going, “My hair could fall out.” All of those data points that they describe generally come from the review of real-world data in a post-market study, in an analysis that someone’s done, and they’ve dug through all that data. That’s good for patients, that’s good for providers to know those things, but where it ends up is in a commercial.

Dan O’Connor: What we really want at Ciox in a big way when we’re working with our pharma providers—and we’ll be working with the FDA and in places like Duke—is to help companies like the Pfizers out there who have exposure in the paper about Enbrel. Just recently, maybe it was the Washington Post that said, Pfizer had seen a 64% reduction in Alzheimer’s prevalence with people who are on Enbrel, and they did that using claims data. And Pfizer said, “Well, we did the work and we realized that there’s not actually a causal effect there, so we didn’t want spend the money on clinical research.”

Dr. Gregory Daniels: Couldn’t measure a causal effect?

Dan O’Connor: Or couldn’t measure a causal effect. Well said. So, the question for Pfizer was, “Should I spend $500 million chasing this down as an Alzheimer drug and get in the Alzheimer’s game, or should I not?” And they said, “Well, it doesn’t make sense to do that, I don’t have the evidence to do that.” Was there an in-between? Is there a way that they could’ve taken claims data and clinical data and done some really hard work to figure out, if, at a much lower cost, if there are some causal relationships that warrant expensive studies, and I think the answer is yes. I think we’re developing one of those answers, but it wasn’t on the market six months ago, or 12 months ago when Pfizer was looking at this. So that’s the promise I see is the ability to use this in a regulatory setting to advance clinical therapies faster and cheaper, but responsibly. And to your point, Dr. Daniel, about, “I trust it’s good data because you said so,” we think that you need a third party in between setting the boundaries on how it comes out, how it should be standardized, how it should look. And you can’t be the ball player and the referee. So that’s why we think that there’s something there and we’re spending lot of time on it right now.

Julie Krommenhoek: Good point.

Dr. Gregory Daniels: Can I just, sort of react to that?

Julie Krommenhoek: Yes.

Dr. Gregory Daniels: What’s the estimate, like, 1% or 2% of a patient’s time is actually spent in a clinic, or in the physician’s office? 99, 98% of their time is enjoying life and being outside, and being in this room, and doing other things that you do. Our healthcare system from a policy perspective and a regulatory perspective has been, “Oh, but we can only look at the data that were collected during that 1% of the time when the patient actually goes into the office and then the physician does something or writes something down.” And what we really need to be doing is figuring out how to better measure this data … patients’ experiences when they’re not in the clinic. But the issue is, while they’re being more innovative and we’re getting more technologies out there, the policy is not as innovative or modern. And the FDA has done a wonderful job in doing what it can to modernize with things like 21st Century Cures, and it’s starting to look at, realistically, how this data can be useful.

How can we start using that data to actually inform our decisions instead of just closing our eyes to that data and only looking at the data that are collected during a clinical trial. That just doesn’t make sense. And I’m really excited that finally we got Congress to do something about it. And the FDA has a new commissioner now. This started under Rob Califf, it continued under Scott Gottlieb, and now this is an issue for the Agency that isn’t dependent upon who the commissioner is. Each commissioner, both under a Democratic and under a Republican administration, has put this issue as top priority to figure out how the Agency can actually learn from these kinds of data.

But it’s hard, it’s difficult. You get a bunch of scientists in the room, and you try to ask them, “Let’s come to consensus today on the best way to measure data quality.” And you leave with more confusion than you had when you came in, but we’re getting there, and there are ways to come to a consensus on what really matters.

Julie Krommenhoek: So to dovetail on that, a couple of the questions that we had that came in earlier … if you were to crystal ball this and kind of do the secret magic wand … what kind of policies do you think would change how real-world data could be expedited, and maybe help us with those odds that I spoke of earlier. What do you think would be ideal?

Dan O’Connor: I think that the focus would almost certainly need to be on data interoperability. And so right now while it’s going to be great to get information from these smartphone apps and these wearable devices, you still want to link it back to the clinical record and you want to marry it with the kinds of medications and the things that are happening in the hospital and in the clinic so you get a total view, and even include patient-generated health data, a total view to make those decisions more well-informed. And the issue right now is that it’s just so costly and hard and time-consuming to integrate electronic medical record data with claims data with these patient-generated data sources, so that you can get a full picture. And for us to modernize and get to where we’re talking about like a learning healthcare system, it’s that data interoperability that we need to solve. I think the other issue is: is it good data, can we get to standard ways of measuring it. I think in 10 years we are going solve those things. The next biggest thing is going to be this interoperability.

Tim Eggena: I agree, but it’s going to be very complex. Can we extend it into these other datasets and patient data points and even beyond that? It’d be very difficult, but you’ve got to start, right?

Julie Krommenhoek: So Tim, are you integrating today with your device into the MR systems?

Tim Eggena: Diasyst is integrated with a couple of glucometers. So, we’re integrated at that level. We are in the process of integrating into the system.

Julie Krommenhoek: So magic wand, that would be happening instantly.

Tim Eggena: That’s right, Plus, again, when I get to think about … what about the other data points, like geography, where was that captured? Is there a correlation between the outcomes and the person’s medication? Where are they? And there’s so many different data that once we begin to capture it and analyze it, it’s going tell a story.

Dan O’Conner: I disagree with both of these guys. I’ve got a story about interoperability. When I came into this industry in the summer of 2017, I spent a lot of time in other industries and I got acquainted with health information technology and clinical information in 2017. So, I went to the IMS Conference in Las Vegas … and I walked into that show for the first time and I said “Oh, interoperability, it’s solved.” We landed on the moon, right? Everybody in that building was describing how important their solution in clinical interoperability was, and how great it was, and how the problem was gone now, right? It absolutely blew my mind when I learned from clients that that’s not exactly how it works.

There’s two problems with data in this country. There’s probably more … I’m one of those people who says, “They’re two types … ” There’s two problems with data as I see it: it’s getting access to clinical data that’s created by providers about patients. The easy stuff is the sociology of the visit, right? It’s claims data. Who was in the room? What did we land on? Diagnosis and procedure? How much was spent? It’s really to lubricate the financial system, healthcare system, make sure everybody gets paid for the resources that they use. The rest of it is just a document dump.

And for some providers it’s better because they get really involved and make sure that workflow was useful for the clinical side, and for others it’s just a reimbursement system. But either way it’s hard to access if you’re not the provider, it’s hard to get out and for a lot of good reasons, HIPAA, privacy, etcetera. And once it’s out, it’s hard to analyze, because most of it is unstructured and it’s unstructured in very different ways. So essentially, what we say is, “Okay, stop crying about it. Figure out ways to get it out from the highest tech to the lowest tech. Send somebody in with a backpack scanner and figure out how to structure it. If that means OCR and NOP-ing and then applying machine learning to it, great. If that means then putting somebody on there to type it into rows and columns, do that too.

So, for us, the use case is, who’s using it at the end and what do they need it for? Right? If it’s a diabetes patient, they want know if a certain biomarker is exhibited about that diabetes patient, they want to know what their levels are and they want know what meds were adjusted and when. That’s what they want to know in order to do better in diabetes treatment. They don’t want to know how many millions of patients you have in your database, right? So, I think … I think if I wave the magic wand, people get myopically focused on the problems we’re trying to solve with bringing treatments faster and cheaper to patients, rather than adopting regulation-based technologies for the sake of standardization. Because it just takes a lot longer, and it’s not based on the end use case. And you see it in pockets and it’s exciting. But that would be my goal for, for this.

Julie Krommenhoek: One thing that I will add, if anyone’s worked on both the U.S. side and then clinical trials in European market, or for me it was the Australian market, it’s very different. When I was clinically trained, we had multiple EMR systems within one system. I might have been at one hospital, and had four to five different EMR systems, depending on what rotation I was on. So, to Dan’s point and to back to Tim and Greg, there are so many different ways of being able to capture your data and it really depends on the system that you’re working with. The story is really difficult to create when you already have a system that has all these little pieces and nooks and crannies to be able to combine that data.

I’ll give you an example of working in Australia on more of a socialized medicine system. When I entered data for one patient that came in for chronic dialysis, that data went into one system, I didn’t have other options. So, therefore, the innovation wasn’t quite there because I might have had different ideas, I might have wanted to add ad hoc or write more streamlined tech notes, but I couldn’t do that because it didn’t fit into their system.

So their data is really good because you can go to that one patient to really build the story, but on the other side you lose some of the innovation. So, it’s pretty interesting to have a tech play, a policy play, and then a patient play, on how you get that data and still try and bring all this together and make it work.

One of the closing questions we have is specific to the FDA Cures Act. The title today, of course, was opportunities and challenges, and I think we talked about both in really great length. So what is next? How will we use that data in a way that’s going really benefit the public?

Dr. Gregory Daniels: So for the Cures Act is … what’s most immediately next is that companies are not necessarily embracing the use of real-world evidence right now, because they’re like, “Why would we do that study, if it’s never going be accepted by the FDA? It’s such a waste of time.” But the FDA is saying, “Well, show us that you can do a real-world evidence study that is going to be good enough.” And if you’re like, “No, you tell us what we could do and then we’ll … ” and so on. So it’s almost like this chicken and the egg … so, companies and really waiting for gospel from the Agency, which is called guidance. [laughter]

We had a debate the other day about binding guidance, the FDA uses that term, binding guidance, it’s the opposite of what that means, it’s actually like, “This is guidance, you could do it any way you want, but this is what we recommend that you do.” And so, companies are waiting for that to come out from the Agency. The good thing is, from the FDA framework in 2018, they are committed to seven different guidances—this is huge, seven different guidances from the FDA over the next two years—that will put forth how they view data quality, what they think the sponsors should be measuring, how they should be using it in studies so that the companies can be more confident that if we do it this way, we have more guidance from the FDA that they’re going to accept it and they’re going to look at it.

Dan O’Connor: Which is why I read everything that hits my desk.

Tim Eggena: I’ll just simply take his comments and say the Agency … they can be responsive, so if you got a new shiny thing, whether it’s capturing data or not, and we know it’s going to improve outcomes and it’s preventing companies from moving forward and taking a product through the cycle into market, maybe there’s an advocacy group like Duke-Margolis to say to the FDA, red flag, you’re holding up innovation. Maybe mandate some type of turnaround times in terms of, “You’ve got to have guidance within 30 days.”

Dr. Gregory Daniels: Yeah, I think we’re pretty pragmatic about it. I’ve got a line in my budget that says “FDA regulatory collaboration.” Spend money, spend time, put executive manpower behind the collaborative nature of the work, because we can sit back and complain that the regulators are going too slow or they’re not listening or their turnaround time’s too slow, or we can spend money, put things forward ,and have them get rejected. I think that at least our approach has been to work with the people who have been very public that they want to advance the ball on this, right? Our goal is aligned with their goal, so when they’re putting a budget against it, we should put a budget against it because we’re figuring this out together. So that’s just been our approach and our policy on it, and I think it’s going to yield benefit, it’s going to help us both go faster.

Dan O’Connor: The only thing I would add is that the FDA has been really focusing on this and they’ve been doing a ton of public events … basically, they recognize that the experts within the Agency, they’re not the experts necessarily in real-world evidence, they’re asking the community. So, the Duke-Margolis Center’s doing stuff, the National Academies of Sciences has been hosting a number of meetings, companies are doing things, anybody who has thoughts can send them to the Agency. They have a lot of opportunities, and when they do guidance, they look across all possible inputs that they can get. And any time stakeholder groups or researchers have thoughts on things going to these public events, and responding to the federal register request from the Agency on input, it’s dependent on the innovators, the academic groups, the patient groups to respond to those and provide all the inputs that the Agency needs. We try to do that in some of our convening, but it does take broader groups to provide that input.

Tim Eggena: Just one quick response to what he said, and that is, you’ve got a nice line item budget to be able to throw money at it …

Dan O’Connor: Oh, you startups don’t?

Tim Eggena: Startups don’t. So, if you’ve got an entrepreneur down in Georgia Tech, and he’s got this great idea that he goes to try to raise capital … and there’s guidance here that says, “That’s not going fly” … gonna stifle innovation.”

Julie Krommenhoek: If you do have a question, if you could either raise your hand or head to the middle section.

Mary Beth Marchione: Hi, yes. My name is Mary Beth Marchione. I’m with Porter Keadle Moore. And my question is related to helping startups and the FDA. Is there any guidance around maybe getting an attestation once those guidances are formed, if these companies are conforming to them?

Dr. Gregory Daniels: No. [laughter]

Mary Beth Marchione: Can we accelerate the process?

Dr. Gregory Daniels: No. In terms of the attestation from the companies that they follow the guidance, is that what you’re…

Mary Beth Marchione: Yeah. Sort of how other industries have to follow SOP 2 or SOP 1.

Dr. Gregory Daniels: Standard protocols?

Mary Beth Marchione: Yeah. That way the FDA can evaluate if somebody’s gone through the guidance and meet that guidance.

Dr. Gregory Daniels: Yeah. So, there are standard protocols for how you’re collecting clinical data to support efficacy claims and things like that, and you have to follow those kinds of things. But in terms of guidances, specifically, those are in fact just guidances and they’re not legally binding. You don’t have to follow them, but it’s the Agency saying, “If you do it this way, this is what we would determine to be acceptable so that… ” The attestation, necessarily, doesn’t work there. But once you get into standards, then if it’s standards for the way that data are collected, that would have that there.

Dan O’Connor: So, the way data are collected is vitally important. As a third party in between, we’re designing a standard way of doing that that we think is market-based. So, we’ll have researchers look at it and validate this is useful and interesting to them. And then also, when we work with regulators we’re going to say, “This is kind of how we’ve done it. And we believe that this is how the market wants to see it. Do you have significant problems with the way that this is collected? Is it fair?” So, we will be the arbiter of that.

But what will we be doing for the regulatory use cases I described earlier? We haven’t done any yet, but it’s on our plan. We’ll be getting attestations that we, as Ciox, are, on behalf of the sponsor, CFR 21 Part 11 certified. Not certified as from where, but we attest that the data coming out of the EMR and going into our rows and columns, and ending up in a SAS or NR … Whatever program that they’re running for the analytics, and then into a report, there is traceability for that data all the way back to its source. And we will get attestations for various studies that we do there. We set up our processes to be that, but until you go on through a study, that’s probably what we would require. It’s just a cost of doing business.

Jugal Malpani: Hi. Jugal Malpani. Ex-IBM project manager. I’d like your perspective on what Tim Cook said—I’m an Applebot user, too—that their biggest contribution is going to be in healthcare. What are your thoughts on what they’re doing and where it’s going?

Tim Eggena: I’m able to deliver our smartphone app to our diabetics, and just having a platform like this is incredible. I bet if you go out into the App Store today and just type in ‘medical app’, it’s got thousands. So just being able to put this in a patient’s hand and transmit readings through the device, or whatever, is almost … limitless. And it’s frankly amazing.

Dan O’Connor: Yeah. I think Apple’s biggest contribution to healthcare … well, healthcare might be the biggest contribution to Apple’s bottom line. At the end of the day, we talk about devices and eyeballs, but in terms of privacy, I don’t think the platform’s designed for that. But I think the biggest contribution is going to be the sheer volume of people who have those devices in their pockets and can wield that leverage, Apple’s leverage, in certain ways. So, whether that’s using applications that sit on the Apple device, or whether that’s bringing 8 million, 10 million, 25 million users at a time with the consent from their phone to do certain things with their data.

Dan O’Connor: CMS and ONC just released new rules related to clinical interoperability, right? Payers have to keep and store all of your data. They have to have all of your data, which they don’t yet. But they have to have all your data, keep it in storage, so when you leave United because you switch jobs … and go to Aetna, right, then your data can go over. So, the payers are going to have to do that. That’s going be hard, it’s going to be a big lift. And when providers are asked for your data, they have to be able to give it quickly. And it’s got to be the whole thing. So, both of them are looking at that and going, “That’s really hard.” And we generally require the patient to authorize that movement in some way. I think when you look at something like Apple that just has massive scale and reach, and the ability to grab that patient’s permission, I think that’s going to be their biggest contribution.

Jugal Malpani: Thank you.

Julie Krommenhoek: I’m going to close by saying, first of all, thank you for our speakers today in our panel. I appreciate everyone that’s here today. Also, special thanks to Jordan Lofton with Golden Source Consultants who put this together for us. And thank you to Jessica Weiss, Patty Green, and Michelle Madison. Thank you all for spending part of your day with us and have a great Friday.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back To Top