The tangled web of scientific responsibility

When I was doing my PhD, I spent a great deal of time thinking about the reasons why I was pursuing a particular line of research. Between personal existential crises and a demanding thesis committee, a PhD candidate must always be able to explain why a specific scientific question is worth her time.

To me, the reason for my work was clear: the climate system is the biggest influence on life as we know it; we want to predict how the climate might change but our predictions are uncertain, and in some places more than others; my dissertation will help improve predictions over Southeast Asia by enabling climate models to make more realistic clouds.

Straightforward, right? Well, it was to me. It was pretty straightforward to my thesis committee, too.

But I found it much harder to explain to others – my family, the stranger seated next to me on a plane, my dentist and ob/gyn – why this work was important. It was very difficult to justify spending taxpayer dollars for me to spend 4 years fixing a dozen lines of code buried among tens of thousands of other lines of code hidden inside a model that, in the minds of many people, is ambiguous anyway.

To me, this communication problem is merely the tip of a very large frozen thing. I’ll come back to it in a minute. First, I’d like to bring two other issues into the mix: research funding and scientific integrity.

Two high-profile articles were published recently that strongly criticize the methods of scientific research: ‘How science goes wrong’, on The Economist, and ‘A dig through old files reminds me why I’m so critical of science’, by John Horgan, writing on his Scientific American blog. The thesis of both these articles is that the practice of scientific research is insufficiently rigorous, and that scientists are not living up to the public’s expectation that they will self-police for poor practice. The end result, according to these articles, is that a lot of junk science is being published as the real thing.

Concurrent to these criticisms, certain members of the US House of Representatives, led by Republican Lamar Smith, have put forward a draft bill that would call on the National Science Foundation (NSF) – the largest funder of science and engineering research – to justify every grant it bestows as being in the “national interest”. This bill is similar to one put forward earlier this year that was widely panned.

(Two responses to the original bill can be read here and here. You can read about the newest draft of this bill, which provides more detail on what exactly constitutes ‘national interest’, in this Nature article.)

These recent developments, in the press and on the Hill, are important because they’re suggestive of intent: to question how science is conducted, what science is worthy of pursuit, and how we make those decisions. I think that the underlying question here is one of value and responsibility – who decides what science is of value, how value is defined and conferred, and how a scientist is held accountable for adhering to the prescribed measure of value? To whom, ultimately, is a scientist responsible? And how does that accountability play out in practice?

“Bec, I’m confused,” I hear you say. “If you’re worried about funding and scientific integrity, then why did you start out by talking about science communication?”

It’s because these issues are fundamentally interconnected. You can’t separate questions about the scientific process and sources of funding from associated questions about science communication, the tenure process, and the mechanisms that transform basic research into a commercialized public product. It’s a tangled web of responsibility.

To begin, consider the life of a brand new, tenure-track scientist – someone working at a university as an Assistant Professor, who hopes one day to reach the esteemed position of Tenured Professor and no longer be forced to move nomadically around the country in search of a stable job.

Much of an untenured professor’s life revolves around the quest for funding. It’s particularly important in the science and engineering fields, where a professor is responsible for securing the money necessary to pay graduate students a living wage (though the definition of ‘living’ is debatable), buy equipment, do experimental work, pay for publications and attend conferences. Without grant funding, a young professor simply can’t do research.

As if the stress of keeping students housed and fed weren’t enough, there are unwritten but implicitly understood rules that vary by university and by department: bring in $X of grant funding, publish Y papers in certain prestigious journals, and graduate Z number of students in the next few years or you can kiss your chances for tenure goodbye. Oh, and you’ll also have to keep teaching classes and contributing to university ‘service’ (sitting on committees about college strategy and the like). You thought you were busy completing a PhD? Try working towards tenure.

At the end of the pre-tenure period (which varies between 4 and 7 years in most places), the tenured members of a department will review the young professor’s file to check if the unspoken requirements have been met. A package of material outlining the professor’s accomplishments will be sent out to her peers – researchers who work in the same area but have not worked directly with the young professor, and who can ‘objectively’ provide an impression of her tenure case.

Part of that impression will be the degree of rigor in a professor’s research, the robustness of her methods and results and the ability to think about big questions in her field. But part of that impression will also be how many papers she has published, how often they have been cited, and how many people know her name and her work. Those subjective human measures of value may or may not be the same thing as the rigor test for her work.

These requirements create a huge amount of pressure on an untenured professor to do two things: attract money, primarily through grant funding, and generate publicity. The pressures of the tenure process represent the first tier of responsibility in the life of a scientist – to herself and her students, her own small research world.

I’m sure you can guess where this pressure leads. Or, rather, might lead.

The tenure process does not mean that all researchers are on a quest to become scientific celebrities.  Most are motivated by innate curiosity and a desire to solve big problems of societal import, which they try to squeeze into the constraints of the tenure process. Many scientists still do the best work they can, and hope that it will be enough in the end.

But some scientists will feel pressured into over-hyping their work, or publishing immature research, or leveraging connections to increase the chances of funding and publication. This is particularly true for those branches of science in which being first means everything, getting ‘scooped’ by another research lab is tantamount to funding death, and a sexy new hot topic guarantees funding success.

I think that these issues contribute towards the lack of rigor that science critics abhor, and rightfully so. The gold standard for scientific research shouldn’t be its position in a publishing race; it should be the rigor, repeatability and robustness of the finished work (the three R’s of science). I believe that many (hopefully, most) scientists adhere to this standard. But it’s important to remember that scientists are people too, and just as fallible as you and me.

I don’t know what the solution is to the tenure pressure issue. Would the lack of tenure pressure make scientists less defensive about their work and more willing to publicise their mistakes and failures? Maybe, maybe not. No one wants egg on their face. But it might make scientists feel less pressured into overstating positive results and pushing out work that’s premature. How would tenure cases be decided without some metric for success? How would you set a success metric without using funding dollars acquired or number of papers published? I don’t know. But I do know that this issue plays a part in undermining scientific integrity and it needs revision.

A second thread of the tangled web of scientific responsibility is the (often giant-sized) leap from basic research to usable public product. Thinking about the draft bill that would require NSF funding to be in the national interest, how would someone judge whether a proposed research question is going to be of value?

Basic research, on the face of it, often doesn’t appear to have a tangible and useful outcome. Sometimes not even the researcher herself knows where a question might lead, only that the question seems intriguing and scientifically important. Quantum mechanics is the quintessential example of how basic research can have unforeseen benefits.

Quantum mechanics started out as an obscure branch of theoretical physics, attempting to understand why certain physical principles appeared to break down at the smallest of scales – inside the atom. It was initially a purely intellectual exercise, a diving down the rabbit hole of curiosity, and nothing more.

But the field of quantum mechanics is responsible for much of the technology we take for granted and depend on today. Among other things, you can thank quantum mechanics for the transistors in your cell phone and computer, which enable modern telecommunication networks, the operation of energy and water systems, and the phenomenon that is Grumpy Cat.

How could someone have predicted at the beginning that pursuing ideas about quantum mechanics might be in the national interest? If that funding measure had been applied decades ago, would we still have ended up where we are today?

There are many other unexpected practical applications that have arisen from basic research, e.g. pretty much everything in NASA’s Spinoff. This article by Katherine Rogers over at Harvard’s policylab presents some of the unforeseen benefits of research that might fail to be funded under a strict definition of ‘national interest’.

The other side of this coin is when basic research has an intended outcome but a lack of follow-through.

Why doesn’t more of the research done in universities end up in the public sphere? One reason is that most universities are not equipped to help transition basic research outcomes into tangible products for commercial or social enterprise. Frequently that role falls to private industry, for which a research outcome must be of immediate financial value in order to justify the R&D needed to bring a new idea into the usable public space.

What happens if there is no perceived immediate financial gain from a new piece of research? Most likely, that research won’t travel outside the firewall of a subscription-based academic journal.

Note that here I’m not just referencing basic research that helps other academics to do more basic research. I’m also talking about breakthroughs that carry social but not financial benefit, such as new understanding of a disease too rare to warrant the commercial manufacture of new treatments. In these cases, there might be completely understandable public resentment at a perceived lack of scientific integrity and follow-through, even though the scientists themselves are effectively impotent.

I would argue that, in these cases, it is not the scientist’s responsibility to ensure that her research is transformed into something of public value. But she does have an obligation to agitate for the funding needed to make it happen and to support her colleagues working for the same goals. I would call this the second tier of responsibility in the life of a scientist – the responsibility to her work and to her peers in the field.

Of course, policy decisions about science are usually not made by the people who do science, the people who understand science or the potential for its application in wider society. Policies (and funding decisions) are sometimes made by people who don’t have any understanding of science beyond how it is publicly perceived.

Now, this doesn’t have to be a bad thing. A perception isn’t necessarily inaccurate just because it’s made by a non-specialist. But it does mean that scientists carry a responsibility to ensure that the people making and influencing policy are well-informed.

Which brings me to the third tier of responsibility in the life of a scientist – communication to non-scientists. This might be the hardest tier for scientists to navigate on their own.

Two weekends ago, I attended the annual Science Writers conference, a joint effort between the National Association of Science Writers and the Council for the Advancement of Science Writing. For me, one of the most eye-opening sessions of this conference was a panel about ethics in the world of freelance science writing.

As someone who was previously engaged in scientific research but is now writing about the work of other scientists, the ethics panel raised more questions than it answered. The main source of my confusion was a suggestion by the editors of two very prominent magazines that scientists cannot talk about their field of expertise without bias, simply due to their training. The same panel also brought up questions about the conflict between wanting to communicate science (i.e. as a promoter of science) and wanting to write about science as a journalist (i.e. with perceived neutrality and lack of bias).

Herein lies the root of my confusion: the media frequently calls upon scientists to participate in the communication process, but seemingly doesn’t want scientists to be the ones doing the talking. What role, then, should scientists play in talking about research? If a scientist can’t talk about her science without bias, in what way is her communication valuable? Even if a scientist could be objective, would the perception of bias be insurmountable?

I do understand the journalist’s fear that a scientist may be given to proselytizing. When does the desire to better communicate science for the purpose of an informed citizenry cross over into publicity for an individual scientist’s work? If there are not always two sides to a story, in the traditional sense of the newsroom, how do standards of journalistic integrity play into the reporting of science?

I’m still trying to figure out the answers to these questions, and I’m sure that others with scientific training are equally confused. I sincerely believe that scientists have a responsibility to talk about their work and not wall it up in the academic tower where only scientific peers can see it. But here is where this tier of responsibility bleeds outside the scientific walls: equally, I think that those with training in journalism and communication have an obligation to teach scientists how to do it better.

It’s important for scientists to communicate about the potential for what their research might become when others can’t see it. It’s also important for non-scientists to remember that not all basic research has an obvious beneficial outcome, and that public value can arise from unexpected places.

It’s important for journalists and the public to keep scientists accountable for how research funding is spent, to ensure that the science being publicized is rigorous, repeatable and worthy of publication. But it’s also important for non-scientists to agitate for the support needed to bring scientific research out to the public and keep that research going.

Above all, it’s important to remember that the scientific process has extremely lofty goals but very human practitioners, and everyone has a responsibility to help scientists reach as high as possible.

Gary Marcus recently published an article in The New Yorker in response to the public criticisms of the scientific process. I’ll sign off with this excerpt from his essay (and strongly encourage you to read the whole thing):

It is absolutely correct for onlookers to call for increased skepticism and clearer thinking in science writing. I’ve sometimes heard it said, with a certain amount of condescension, that this or that field of science “needs its popularizers.” But what science really needs is greater enthusiasm for those people who are willing to invest the time to try to sort the truth from hype and bring that to the public. Academic science does far too little to encourage such voices.

At the same time, it is facile to dismiss science itself. The most careful scientists, and the best science journalists, realize that all science is provisional. There will always be things that we haven’t figured out yet, and even some that we get wrong. But science is not just about conclusions, which are occasionally incorrect. It’s about a methodology for investigation, which includes, at its core, a relentless drive towards questioning that which came before. You can both love science and question it.

Yours in the pursuit of spaceships,

Bec