Chair: Rep. Merwyn R. (Mitch) Greenlick, Ph.D., is a Democratic representative in the Oregon House of Representatives. He spent 30 years as Director of Research for Kaiser Permanente and 10 years as Chair of the Department of Public Health and Preventive Medicine at the Oregon Health and Science University's School of Medicine.
Panelists: Bruce Goldberg, M.D., is the Administrator of the Office of Oregon Health Policy and Research. Previously, Goldberg was the Medical Director for CareOregon, a Portland-based non-profit organization that serves Oregon Health Plan recipients. He was also an Associate Professor in the Departments of Family Medicine and Public Health and Preventive Medicine at Oregon Health Sciences University School of Medicine.
Rep. Phil Lopes is a freshman Democratic representative in the Arizona House of Representatives and the ranking Democrat on the State's health committee. He was formerly the Senior Manager of the Arizona Department of Health Services and Executive Director of the Health Systems Agency of Southeastern Arizona.
James Tallon is President of the United Hospital Fund of New York, a philanthropic and health services research organization. He served in the New York State Assembly for 19 years; he was Majority Leader from 1987 to 1993, and Chair of the State's health committee from 1979 to 1987.
Mitch Greenlick: 'Be dear on who the novice is ...'
I am a recovering health services researcher. I ,an for public office partly because I grew tired of the legislature not implementing health policy that should have followed from the research. It has been very interesting for me to see policy from the inside, and I would like to share four tips for success in effectively translating health services research into state health policy.
1. Start developing the answer years before being asked the question. Imagine getting a call from someone who is planning to have a hearing on a health policy topic the following week. He wants to know if you can research this subject between now and next Tuesday. The answer to that question can only be yes. Ideally, a researcher would figure out 5 years before that time that that particular question was going to be asked. To be policy sensitive in health services research, you must think ahead about what the critical questions will be. For somebody like me who has been in the field for 40 years, it is pretty easy because the questions tend to be the same ones that were asked 40 years ago. The problem is that federal agencies do not understand that. They typically only support questions that were asked yesterday. So it is very important for researchers to think carefully, as they seek federal funding, about how to develop proposals that are based on questions that were asked yesterday but that will also be germane to questions that will be asked 5 years from now.
One of the first federal grants I received was to implement home health and skilled nursing services for the under-65 population of Kaiser Permanente in 1966. The grant came after we had done a study showing that posthospital needs estimates of Medicare were underestimated by a factor of three.
The Public Health Service amazingly gave us the funds because they believed somebody was going to ask them at some point what would happen if Medicare benefits were extended to the whole population; they wanted to have an answer in terms of posthospital needs. We delivered the study in 1972--a year late--and, three weeks later, the agency director testified before Congress on the Nixon Health Insurance Plan and addressed the question of what the under 65 population's posthospital expenditures would be. Fortunately, several of us had anticipated how the research would be used and were able to provide answers at the time that they were needed.
2. Remember the difference between being an advocate and a policy expert. It is appropriate for everybody as a citizen to advocate for policies that make a difference to them. But it becomes a problem when researchers wrap the cloak of science around positions for which they are advocating. That destroys the ability of legislators and their staff to trust what the experts are saying, because everyone knows to take an advocate's words with a grain of salt. The bottom line is to always tell the truth. Bruce Goldberg, who is on this panel, has built the most trust of any researcher I have known in the state of Oregon, and it is because he is willing to tell the truth, even when it means the legislature may move forward on policies he does not want to see implemented.
3. Remember the difference between a point estimate and a confidence interval. As researchers we are not always good at giving unqualified answers when policymakers ask for information. My suggestion is to try to give them answers that are clear, straightforward, and reasonably valid. Often, what matters is having a solid estimate, not whether the number is exactly precise and accurate.
I was on the commission that started the Oregon Health Plan in 1989. In the very first meeting, somebody asked, 'How many uninsured are there in Oregon?' I raised my hand and told them, '400,000,' which was based on a study we conducted recently. I was then asked how many of those uninsured are below 100 percent of the poverty level, to which I replied, '200,000.'
That was the end of the discussion. The commission would not have gotten anywhere had I said, 'Well, it is something like 400,000, but we only interviewed 400 people and we would have liked to have a larger sample size but we did not have the money, and the confidence interval around that 400,000 number is somewhere between 300,000 and 550,000.' If I did what we usually do as researchers, we would have spent the next 6 months arguing about how many uninsured are in the state rather than trying to find solutions to the problem.
Now, were there really 400,000 uninsured? Well, of course not. But it was the best point estimate we had, and it really did not matter if there were 350,000 or 540,000; the policy answers were not going to change.
4. Do not show off Researchers are accustomed to showing off. They come to meetings and give presentations and strut their stuff for their colleagues; they want to boast about how erudite they are. Well, do not do that. In addition, do not assume, as researchers sometimes do, that legislators are less intelligent than you. The fact is that legislators are not stupid. Of course, there are stupid legislators, but there are also stupid health services researchers.
It is critical to be clear on who the novice is and who the expert is in any given situation. In fact, when researchers are involved in the health policy formulation process, they are the novices, and the legislature or the long-time legislative staffs are the experts.
One of the joys for me of being a 40-year veteran health services researcher is that, for probably the last 10 years, I was the person in the room who knew the most about any topic. Now, in the legislature, all of a sudden I became a rookie. I have learned that it takes time to understand the real policy formulation process.
Phil Lopes: 'What evidence and whose evidence?' ...
I suspect that, if I took a survey of my 60 colleagues in the Arizona House of Representatives, no one would disagree that we should use evidence-based research to make our decisions. The question is: what evidence and whose evidence?
There are some issues for which we simply have no good empirical research. For example, we were going to raise the fines for driving under the influence (DUI) in the State. Proponents of the increase argued that higher fines would deter people from drinking and driving. I looked for the research on which they based that statement. As it turned out, there was no empirical evidence that the amount of a DUI fine affects whether someone drives under the influence.
I found someone at Arizona State University who did a literature search, which concluded that there was no evidence. I also called the people who conducted the research and asked them what their gut feeling was. None of them thought that higher fines would act as a deterrent. By that time, 3 months had passed, and I missed the opportunity to use research (or the lack of it) to influence the decision. But I am not sure I could have made a great argument against the fine even if I had, because it seemed to be a reasonable way for the legislature to increase state revenue.
On the other hand, some issues that we address have a lot of empirical research behind them. In that case, the question becomes, how do you make sense out of it so that you can talk about the preponderance of the evidence? My experience has been that, no matter what your position, you can find some piece of research out there that justifies it. For that reason, it is important to have some person or mechanism to sift through all the data, determine what is valid, and establish what the aggregate findings are, even though there may be some exceptions.
An example of this was when the Arizona legislature looked at the effects of preschool on the later academic performance of students. People came forth and testified that the preponderance of the evidence pointed to a positive correlation between preschool and later academic performance. However, there were a couple of studies that did not conclude that, and those studies were cited again and again by opponents of preschool.
The reason for that is that policy decisions are not based solely on evidence. To quote John McDonough's book, Experiencing Politics: A Legislator's Stories of Government and Health Care, what takes place in the state legislature is really values masquerading as data. That has been abundantly true in my experience. Thus, in the preschool example, if my value is, 'Kids ought to be home with their parents,' which is a big argument, then I can always use a select few studies and ignore the preponderance.
I appreciated that Mitch drew a distinction between advocates and policy experts. In Arizona, which is a term-limited state, we not only rely on staff; we have to rely on lobbyists and advocates. Many times lobbyists will answer questions quickly--but it is important to examine carefully the information they provide.
During a health committee meeting on Medicaid--we have Medicaid managed care in Arizona--someone stated that, because Medicaid did not reimburse hospitals adequately, commercial health plans were making those funds up in their commercial agreements with hospitals. I asked what the evidence was for that statement. Nobody at the health care committee had it, but later a lobbyist told me that the evidence was contained in a Medicaid agency report done about a year ago.
I asked him to get me the report--which he did. But after looking through it, I could not find any relevant data about the issue at hand. Eventually, I called the author of the report, who told me that it contained no such information and that he did not know of any evidence for the claim elsewhere either. He put a word out to his colleagues seeking the information. That was 2 months ago, and I still do not have the answer.
Bruce Goldberg: 'Health services researchers are the gears ...'
The most pressing issue facing health services researchers is how do we translate what we do into policy. I have been head of Oregon's Office of Health Policy for a year now and want to reflect on my experience.
When I took the job I thought my role was as a health services researcher and as a policymaker. What I have learned is that my job is as a translator: it is my responsibility to be certain that the 91 bosses that I have--90 legislators and I governor--are well informed. It has been an interesting process. Having lived in the gap between health services research and policy, I have experienced a number of surprises. The first surprise is that I have seen more of my colleagues in 15 minutes at this Annual Research Meeting of the AcademyHealth than I have during a year at the State House--and to me that says it all.
Health services researchers need to be more involved with legislators and with policymakers. I do not see my former colleagues at the state offices; I do not see them at the Capitol. I do not run into them in legislators' offices. When I am waiting to get into a legislator's office, the three people ahead of me are lobbyists and advocates. They are never health services researchers.
I have gotten calls for two things from health services researchers: data and money. I have gotten two calls--only two--from health services researchers asking me what is important to the state and what they could do to help inform state health policy.
Another surprise that I quickly learned is that research is actually a dirty word to many policymakers. Analysis, evaluation, and policy work are all acceptable terms but research is not; it is actually a nonstarter. Health services researchers may be offended by that, but it is the really: We have to put what we do into terms that are accessible and understandable to everyone. I think it would be tragic if, as researchers, we were to fall on our swords about language and let the policy get away from us.
I learned that all policy is parochial. That was interesting to me having come from a very heady world of trying to inform national health policy. The legislators in my state are not interested in national issues or in what Arizona or Colorado or New York are doing; they want information that will give them a better understanding of Oregon.
They also want information to be available to them when they need it. The reason that I think legislators often get information from lobbyists is because they are there. I cannot tell you the number of times that I have seen lobbyists scramble to answer a policymaker's question within 24-48 hours of having been asked. When the same questions are asked of researchers, they hedge. They say, 'Well, this is what we will have to do, and it is very complicated, and we will need to get funding; we need to be certain.' It is a nonstarter.
If we want to inform policymakers, we need to give them information that is timely. That does not mean that the data will be inaccurate. We do not need to sell out our craft. It means being able to take our knowledge and information and apply them to the issues that policymakers want. The goal is not to show policymakers how smart we are or that we know the answers to the six questions they have not asked us; it is to answer the question that they have asked.
The most important thing that I have learned is how critical it is to communicate clearly in order to translate health services research into policy. This is an area where we could all improve. Legislators and policymakers do not read journal articles. They do not even read the abstracts. They do not read them because the information is not understandable or relevant to them.
If we want our research to be relevant, we need to think about other communication formats. We cannot just rely on journals. Journals are a wonderful way for us to communicate with ourselves, and indeed we must continue to do that. However, we should also develop tools to communicate with others. Every journal article should be accompanied by a one-page policy piece that states the implications of the research in language that is understandable to lay people.
I understand that funding is a big issue, and that researchers operate under intense funding pressures. Indeed, I think there is a huge disconnect between funding and policy that we need to address.
Here is a story that illustrates the problem: We had an important piece of legislation passed that provided a subsidy for rural physicians. A young research fellow asked if we would want it evaluated, and I told him that that would be fantastic. We did not have a lot of dollars to fund it but we sat down and talked about how we could quickly and effectively evaluate the legislation for between $5,000 and $10,000. I never heard back.
Two months later the same individual told me that he wanted to use the evaluation as the foundation for a career development award. He was developing a 5-year proposal around evaluating the legislation. I realize that this young person needed career development and that is important. But by the time he gets that K-08 award and does the work, that piece of legislation is likely to have been changed twice and thus his evaluation may no longer be relevant. The opportunity to quickly and effectively evaluate the legislation would have passed.
What can we do to address this disconnect? Perhaps we could provide more hands-on training that is done in conjunction with policymakers. I think we should look seriously at the issue of research translation and how we can best teach that. We need to develop better state funding sources for policy research and move away from national dollars. Lastly, I think we must look at how we can create research collaboratives like what we have done in our state--where we have put together a state health policy office and a health services research community to create a group that has expertise at identifying important policy issues, carrying out evaluative research, and developing an effective means of translating that research into language that policymakers can understand. Health services researchers are the gears for doing that work.
James Tallon: 'Researchers are from Mars; policymakers are from Venus....'
Health services researchers and policymakers are Mars and Venus; they are oil and water; they are different worlds. I have spent the better part of my career--which runs out to about 35 years--trying to bridge these two incredibly important and fundamental fields.
Both research and policy are broad fields that exist in many contexts. On the policy side of this discussion, we have members of the legislature who vary enormously in terms of their tenure, seniority, interest, and focus. They work with staffs. Some of those staffs have fiscal responsibilities; some of them are more programmatic-oriented. In some states there are programmatic agencies that take on analytic responsibilities.
Those legislators work within a broader context of state government, which includes governors, executive agencies, executive staffs, budget divisions, and the like. They also work within a context of interest groups, of media attention, and of course of a broader public who are, in a final analysis for legislators, their constituents.
On the research side, there is clearly in our world of health services research a research-policy continuum. We see that clearly in AcademyHealth. Some of our members are so close to policy that they move in and out of research and policy roles. Others are strictly academic researchers. There are also people who play operational roles in the public and private sectors.
Within this broad continuum, researchers deal with a range of topics, and only a subset of them are of direct interest to state legislatures. But researchers can make their work relevant to state health policy if they are willing to focus on how to operate in that world. Most of us think of our research as our findings, our observations, our analysis. Let me take a step back and remind researchers that they also do two other things--they frame the question and they create the context in which the question is analyzed.
The portion of a research article that produces the findings may not be the most important part, at least for those at the working end of state government. The capacity to flame a question is often of much greater value to them.
For legislators, there is also an immediacy in their need for information. That underscores the importance in this business of being able to forecast decision points. I did not know a year ago, for example, that legislators would be clamoring to figure out how to buy drugs in Canada. I did not know 6 months ago that they would focus on the causes and potential remedies for obesity.
Legislators want to talk with researchers in terms that are specific to the context within which they work. And so, an analysis of spending is to them a budget question. An analysis of service utilization patterns is a programmatic issue. They will always place a great deal of emphasis on how to solve problems.
In a recent discussion I had with a senior staff member in Albany, I laid out for him what I thought was a brilliant analysis of the health care issues facing the state government in New York. His response was, 'Okay, what do we do?' I had not thought about that.
Researchers can help policymakers to navigate the complexity of the health care system by answering the what-if questions about their proposals. This is not a matter of presenting a whole series of findings on a system that a legislator is thinking about creating. It is about developing models and helping them understand the dynamics of those models, and how the choices that they make might affect various parties.
What can health services researchers do to bridge the gap between research and state health policy? First, I want to recommend that health services research do what it does well: Tell a broad story to a broad audience.
The Institute of Medicine did this well in the past couple of years by bringing together the analyses of so many researchers to zero in on the uninsured and debunk some of the myths about them--such as the myth that most of them are unemployed.
Similarly, Beth McGlynn and her colleagues at RAND have done an excellent job of communicating through their research the need to improve the quality of our nation's health care system. I had a conversation a couple weeks ago in which someone characterized one of the RAND findings this way: 'You have as good a chance of getting the right health care in this country as Shaquille O'Neil does of making a foul shot.' When you have a message that resonates with the public and the media, they help do the translation for the legislature.
It is also important for researchers not to fear the short-term. I recognize that many of us feel uncomfortable with that methodologically, but there are ways to get good information quickly, and sometimes it is necessary to do so. I have learned a great deal from focus groups, for example.
I worked with a focus group in Brooklyn to evaluate how immigrant communities feel about health insurance or direct health care services. We were quite surprised when the first question to come out of the focus groups was: What is health insurance? They did not understand the concept because they came from countries with health systems that were dramatically different than ours.
Finally, researchers need to be a little careful about their own self-importance. There is always the pressure to be recognized by colleagues and funders, and indeed that is important. But ultimately, researchers' success may be tied most strongly to the success of decision-makers; they are the ones who move our research findings into policy solutions.
Prepared for publication by:
Christina E. Folz, former Director of Communications, AcademyHealth
