The slippery slope of novelty

Coming up with a novel idea in science is actually very difficult. The many years during which smart people have, thought, researched, and written about ecology and evolution means that there aren’t necessarily many easy openings remaining. If you are lucky (or unlucky) enough to know someone with an encyclopedic knowledge of the literature, it becomes quickly apparent that only rarely has an idea not been suggested anywhere in the history of the discipline. Mostly science results from careful steps, not novel leaps and bounds. The irony is that to publish in a top journal, a researcher must convince the editor and reviewers that they are making a novel contribution.

There are several ways of thinking about the role of the novelty criterion - first, the effect it has had on research and publishing, but also more fundamentally, how difficult it is to even define scientific novelty in practice. Almost every new student spends considerable effort attempting to come up with a completely "novel" idea, but a strict definition of novelty – research that is completely different than anything published in the field in the past - is nearly impossible. Science is incrementally built on a foundation of existing knowledge, so new research mostly differs from past research in terms of scale and extent. Let's say that extent characterizes how different an idea must be from a previous one to be novel. Is neutral theory different enough from island biogeography (another, earlier, explanation for diversity which doesn’t rely on species-differences) to be considered novel? Most people would suggest that it is distinct enough as to be novel, but clearly it is not unrelated to works that came before it. What about biodiversity and ecosystem functioning? Is the fact that its results are converging with expectations from niche theory (ecological diversity yields greater productivity, etc) take away from its original, apparent novelty? 

Then there is the question of scale, which considers the relation of an new idea to those found in other disciplines or at previous points in time. For example, when applying ideas that originate in other disciplines, the similarity of the application or the relatedness of the other discipline alters our conclusions about its novelty. Applying fractals to ecology might be considered more novel than introducing particular statistical methods, for example. Priority (were you first?) is probably the first thing considered when evaluating scientific novelty. But ideas are so rarely unconnected to the work that came before them, so then we evaluate novelty as a matter of degree. The most common value judgment seems to be that re-inventing an obscure concept first describe many years ago is more novel than re-inventing an obscure concept that was recently described.

In practice, then, the working definition of novelty may be that something like ‘an idea or finding doesn't exist the average body of knowledge in the field’. The problem with this is that not everyone has an average body of knowledge – some will be aware of every obscure paper written 50 years ago, and for them nothing is novel. Others have a lesser knowledge or more generous judgement of novelty and for them, many things seems important and new. A great deal of inconsistency in the judgement of papers for a journal with a novelty criterion results simply from the inconsistent judgement of novelty. This is one of the points that Goran Arnqvist makes in his critique of the use of novelty as a criterion for publishing (also, best paper title in recent memory). Novelty is a slippery slope. It forces papers to be “sold” and so overvalues flashy and/or controversial conclusions and undervalues careful replication and modest advances. And worse, it ignores the truth about science, which is that science is built on tiny steps founded in the existing knowledge from hundreds of labs and thousands of papers. And that we've never really come up with a consistent way to evaluate novelty.

(Thanks Steve Walker for the bringing up the original idea)

Holding fast to a good(?) idea

One of my favourite lists on the internet is tucked away in the credits for the PHYLIP software. PHYLIP was authored by Joe Felsenstein, a professor at the University of Washington and expert on methods for phylogenetic inference. PHYLIP is a free package of programs for inferring phylogenies, and probably the first and oldest widely-distributed phylogenetic program. Programs like PHYLIP made phylogenetic approaches easily accessible to ecologists and evolutionary biologists. Apparently it took years to get from the idea for PHYLIP to funding, and Felsenstein memorializes this with his “No thanks to” list (below). The list includes reviewers and panels from the US Dept of Energy, NSF, and NIH that turned down his proposals and made comments like "The work has the potential to define the field for many years to come.... All agreed that the proposal is somewhat vague. There was also some concern that the proposed work is too ambitious.”

(Click to enlarge)

There are obvious responses to this list, mostly relating to the short-sightedness of funding agencies, meaningless requirements for ‘broader impacts’, the fact that proposals might be improved through the process of multiple failed applications, and of course the benefit of being long-established and respected when posting such lists on your website. But what I always wonder about is how long do you hold on to an idea, a proposal, or a manuscript that it is repeatedly rejected, before you give up on it? 

This question is interesting to me for a couple of reasons. Firstly, because personality is so intertwined with confidence about an idea’s success. We all know people who would argue that all their ideas are Nature-worthy, criticism be damned. Other people need to be convinced of the merit of their own ideas. Obviously past success probably helps with judgment – having experience in identifying good ideas builds confidence in your ability to do so again. But what is the line between self-confidence and self-delusion? Secondly, it is a reminder that lots of good ideas and good papers were rejected many times. In any case, I am curious whether people tend give up on an idea simply because they became discouraged at the prospects of getting it published, or because they lost faith in the idea, or a combination of both.