Further studies of the decline effect find decline of the decline effect

“The Truth Wears Off: Is something wrong with the scientific method?”

The Decline Effect explored in an article by Jonah Lehrer in the New Yorker refers to a temporal decline in the size of an observed effect: for example, the therapeutic value of antidepressants appears to have declined threefold since the original trials. Based on the cases presented, this effect is not limited to medical and psychological studies. One example in evolutionary biology is the relationship between physical symmetry and female choice: initial studies consistently found strong selection for symmetry in mates by females, but as time passed, the evidence grew increasingly smaller.

This may be a result of selective reporting – scientists focus on results that are novel and interesting, even if they are in fact simply statistical outliers, or worse, the result of unconscious human bias. This sentiment is troubling; humans – scientists or not– are proficient pattern finders, but our subconscious (or conscious) beliefs influence what we search for. Lehrer argues that replication – the process of carrying out additional, comparable but independent studies – isn’t an effective part of the scientific method. After all, if study results are biased, and replications don’t agree, how can we know what to trust?

I don’t disagree with most of the article’s points: that scientists can produce biased results, PhD not withstanding, that more effort and time should be invested in data collection and experimental methodology, that the focus on 5% statistical significance is problematic. For one, it’s not clear from the article how prevalent the decline effect is. However, I wonder whether Lehrer, similar to the scientists he’s reporting on, has selected specific, interesting data points, while ignoring the general trend of the research. In 2001, Jennions and Moller published evidence of a small negative trend in effect size over time for 200+ studies, however, they suggest this is due to a bias toward high statistical significance, which requires either large effect sizes (the early studies published), or small effect sizes in combination with large sample sizes (a scenario which takes more time).

Even if the decline effect is rampant, does it represent a failure of replicability? Lehrer states that replication is flawed because “it appears that nature often gives us different answers”. As ecologists though, we know that nature doesn’t give different answers, we ask it different questions (or the same question in different contexts). Ecology is complex and context-dependent, and replication is about investigating the general role of a mechanism that may have been studied only in a specific system, organism, or process. Additional studies will likely produce slightly or greatly different results, and optimally a comprehensive understanding of the effect results. The real danger is that scientists, the media, and journals over-emphasize the significance of initial, novel results, which haven’t (and may never be) replicated.

Is there something wrong with the scientific method (which is curiously never defined in the article)? The decline effect hardly seems like evidence that we’re all wasting our time as scientists – for one, the fact that “unfashionable” results are still publishable suggests that replicability is doing what it’s supposed to, that is, correct for unusual outcomes and produce something close to the average effect size. True, scientists are not infallible, but the strength of the scientific process today is that it doesn’t operate on the individual level: it relies on a scientific community made of peers, reviewers, editors, and co-authors, and hopefully this encourages greater accuracy in our conclusions.

Carinval #32 and still going strong

Want to know what people are talking about? The 32nd Carnival of Evolution is online, hosted by Denim and Tweed. Check it out, pass it along.

Trend in ecology, 2010

Sciences are always in a state of ebb and flow (sorry), and topics of study fall in or out of fashion in response to paradigms shifts, methodological advances, and to support necessary ecological applications.
For the sake of curiosity, I've compiled the top keywords from ecology publications in 2010. Obviously there are many covariates, but it should come as no surprise that the top words were "biodiversity" (667 times), "climate change" (293), and "conservation" (274); other popular keywords were "evolution" (277), "population (ecology)" (273), and the rather vague "patterns" (196).

(click image for larger view)

The evolutionary story of ecosystem function

Twenty years of research has repeatedly shown that communities with greater diversity result higher functioning -namely greater production of biomass. One of the major mechanisms producing this relationship is that different species use differing resources, such that their complementary use of resources uses the total resource pool more thoroughly, thus converting more resources into biomass. Resource preference is the product of evolution and how organisms have adapted to using various resources can influence the strength of the diversity-function.

In a recent paper in Nature, Dominique Gravel and colleagues test how the evolution of specialization versus general resource use affect the strength of the diversity-function relationship. They use bacteria strains that have undergone evolution on diverse resources (generalist) versus on a singular resource (specialist). The resources in their case are different carbon substrates.

Assemblages of generalists were able to use many available resources and generally had greater productivity than specialist assemblages. Generalists also show an increasing relationship between diversity and productivity, because no generalist used all resources and they still showed some preferences. Combining multiple such generalists meant that more of the total resource pool was consumed. Specialists also resulted in the positive relationship, but a much steeper one. Because specialist use many fewer carbon substrates, additional specialists meant that new resources were tapped into. Thus increasing specialist diversity resulted in more new resources being consumed than with the generalist species.

While these results are logical, they are important for two reasons. First is that the strength of the relationship between diversity and function is mechanistically determined by the resource use efficiency of individual strains, and how many of the total substrates they can use. The mechanisms producing different relationships in previous experiments were hypothesized after the results analyzed, as opposed to being predicted. Second, recent work has shown that evolutionary history seems to be a better explanation of community function than the number of species. These results show how the history of evolution can have important consequences for function.

Gravel, D., Bell, T., Barbera, C., Bouvier, T., Pommier, T., Venail, P., & Mouquet, N. (2010). Experimental niche evolution alters the strength of the diversity–productivity relationship Nature, 469 (7328), 89-92 DOI: 10.1038/nature09592

Who is a scientist, I am a scientist: the bees of Blackawton

In discussions of the larger societal implications of scientific findings, the question of who is a scientist is frequently asked. I've talked with with creationists who invoke the authority of someone who has a PhD in a scientific discipline and happens to share their belief of supernatural origins, as a scientific authority. Does the fact that I have a PhD in ecology and evolutionary biology make me scientist or is being scientist something more?

This is an important question. It goes to the core of whose authority we believe for public discussion of such issues as climate change, evolution, risks of vaccines, and so on. Regardless of how we define 'scientist', a scientist participates in science by publishing peer-reviewed research articles in scientific publications. This notion of who is a scientist has been enjoyably stretched by the publication of a paper in Biology Letters by a group of elementary school children from Blackawton, UK. In consultation with a academic scientist and under the supervision of teachers, 25 8-10 year olds devised and carried out an experiment on bee visual perception and behavior, and wrote up their results into a publishable manuscript.

The students trained bees by offering them nectar rewards in different color containers. They then allowed these trained bees to forage in multicolored arenas and they conclusively show that the bees unambiguously select the colored containers they were trained on. Bess learn and adapt their behavior based on previous experience.

Publishing a paper by a group of children may sound like a gimmick, but the study is very interesting. The commentary from the journal says it best: "The children's findings show that bees are able to alter their foraging behaviour based on previously learned colours and pattern cues in a complex scene consisting of a (local) pattern within a larger (global) pattern . As there has been little testing of bees learning colour patterns at small and large scales, the results can add considerably to our understanding of insect behaviour."

The paper is extremely enjoyable to read and will have you chuckling to yourself. Sincerity pours from the words and I was left wondering if I could have reasoned so well at that age. The children develop hypotheses using information available to them, such as watching Dave Letterman's 'Stupid Dog Tricks'. Reading this article made me realize why I love being scientist. The students note that "This experiment is important, because, as far as we know, no one in history (including adults) has done this experiment before" and because they were given the opportunity to carryout this study they "also discovered that science is cool and fun because you get to do stuff that no one has ever done before". Too true. I could not have said it better myself.

Being scientist can mean a lot of things, it can mean knowledge (which the Latin origin, Scientia means), it can mean training and acquired skills, but at its core, being a scientist means conducting research, testing hypotheses and writing publications that are deemed acceptable by other scientists. Therefore the children of Blackawton are scientists, I am a scientist.

Blackawton, P., Airzee, S., Allen, A., Baker, S., Berrow, A., Blair, C., Churchill, M., Coles, J., Cumming, R., Fraquelli, L., Hackford, C., Hinton Mellor, A., Hutchcroft, M., Ireland, B., Jewsbury, D., Littlejohns, A., Littlejohns, G., Lotto, M., McKeown, J., O'Toole, A., Richards, H., Robbins-Davey, L., Roblyn, S., Rodwell-Lynn, H., Schenck, D., Springer, J., Wishy, A., Rodwell-Lynn, T., Strudwick, D., & Lotto, R. (2010). Blackawton bees Biology Letters DOI: 10.1098/rsbl.2010.1056

Tweet

The carnival is on!

Check out the 31st edition of the Carnival of Evolution, hosted by The Dispersal of Darwin. Check it out, a fun read for sure!

Science 2.0 - science comes of age on the Internet

by Marc Cadotte, Nicholas Mirotchnick and Caroline Tucker

The Internet is not just for lolcats and porn anymore, scientists have begun using it in constructive ways. The past few weeks’ controversy about the ability (or lack thereof) of bacteria to incorporate arsenic exemplifies how the relationship between science and the Internet is changing. If you’ve missed the debate over the recent Science paper, researchers funded by NASA’s exobiology/evolutionary biology program published experimental results suggesting that a Halomonas species could incorporate arsenic into its DNA in the absence of available phosphorus. This paper received extensive attention in the mainstream media, but also vocal criticism, which was expressed primarily through postings and comments on scientific blogs. Until recently, for scientific communication the Internet has functioned primarily as an electronic source of published journal articles. Earlier attempts to take advantage of the Internet’s potential (immediacy, accessibility, and ability to connect individuals, organizations, and ideas) in scientific discourse have been mixed (e.g. Nature Precedings versus PLoS ONE). The use of blogs as a forum for scientific debate suggests that this is changing: posters tended to be active scientists and the comments were similarly knowledgeable. In contrast to this online approach, the authors of the Science paper stated that they would only respond to peer-reviewed critiques and would not engage in discussions on the blogosphere.

The story of the arsenic-utilizing bacteria highlights an emergent tension in the transition to internet-based scientific discourse. Traditional communication in science has been primarily unidirectional, from the authors of a study to the readership of a journal. Any discourse transpired on the pages of a journal, regulated by editorial and peer review. This gatekeeping meant that this discourse was technically sound and kept personal grudges and tangential discussions to a minimum. This also meant, however, that only a few voices were heard, the discussion was slow (occurring over months) and only happened for one back and forth (journals will not devote precious page space to on-going discussions and debates).

This method of discourse is changing. Journals have experimented with online discussion or commenting features on their websites. Methods in Ecology and Evolution, for example, has a correspondence page with discussion threads for each paper they publish, and PloS ONE allows for comments to be posted to every paper they publish. While, in concept, these are positive developments for scientific communication, commenting features are seldom, if ever, used. The main obstacle to their success is that they are only available on the publishers’ websites, but scientists access articles in many different ways, from database searches to library links. Few scientists actually go to individual journal websites to access papers. This is not to say that there are not discussions about scientific papers occurring online. As highlighted by the arsenic bacterial episode, blogs are an important avenue for discussing and disseminating new ideas in science. Blogs may not, however, actually foster conversations very well. One person or a few people usually run them and there is little discussion among blogs (a comment on a blog post at blog X will not be part of the discussion of the same story at blog Y). Rather, the greatest potential to foster discourse is through virtual networks where people are linked together either through friendships or professional self-identification (e.g., as fisheries biologists), with Google Reader being a particularly powerful communication tool.

It’s exciting to think about what the future of science will look like, given the changes that we’ve already started to see. The major upside of new channels of communication is that they give us the potential to quickly reach thousands of readers, instead of the handful that usually read any given journal article. They also let us communicate in both directions, and in real time. The pitfall, of course, is that they’re free-for-alls; anyone can blog about science.

But here’s what’s unexpected: these free-for-alls have been amazingly reliable at filtering out the bad and promoting the good. Inaccuracies are pulled from Wikipedia faster than anyone had predicted, the social news site Reddit is “astonishingly” altruistic, with users eliminating offensive or erroneous comments from the site and promoting other users’ questions and problems, and the reputations of blogs are shattered if their content becomes unreliable. Social networking has revolutionized the way we consume news, with sites like Facebook and Twitter launching the best articles into viral webspace. The open-access world has evolved self-regulating mechanisms that work surprisingly well so far and if these media are to continue to grow, we will have to ensure that these mechanisms remain built-in.

Seems like an easy task, right? Apparently not. For some reason, academics are slow and conservative when it comes to adopting new media. A letter to Nature two weeks ago scolded scientists for not contributing their share to Wikipedia pages. Various facebooks for academics, like Mendeley and ResearchGATE have emerged, but last week, another Nature article complained that researchers aren’t jumping on the bandwagon. These sites are potential collaborative goldmines, but we seem to be incapable mastering what tweens can do with two thumbs.

It’s not so hard to imagine a world where anyone with a broadband connection can contribute creative ideas to science, the good ideas get automatically filtered to the top and the information is all free to anyone. In this world, children count ants (or bees!) in their backyards and upload their data to global networks. Revolutionary discoveries are published instantly on blogs and thousands of scientists get to decide if they’re valid. Every gene ever sequenced and every tree height ever measured can be readily downloaded in an Excel (or OpenOffice) spreadsheet. In this world, the report on our little arsenophilic friends might never have been published in Science, because instead of being reviewed by two referees, the thousands of readers on the blogosphere would have filtered it out, if was in fact porous.

Academics should be the first, not the last, to adopt new communication tools. We are no longer limited by the postal service, email or PDFs; the web has gone 2.0 and we should follow suit. So go forth, young researchers, and blog, edit and share. And then go tweet about it all so your eight year-old kid knows how hip you are.

Biodiversity and ecosystem functioning – without fungi?

Different subfields of ecology have a propensity to remain remarkably isolated – researchers in aquatic systems independently develop hypotheses that already exist in some form in other systems, and vice versa. Population ecology and community ecology, despite their obvious relevance to each other, are rarely integrated. There is a tendency – resulting from limits on our time, experience, and possibly imagination – to stay within whatever box we’ve defined for ourselves.

Historically, it seems that biodiversity and ecosystem functioning has lost sight of the progress made in classical ecology in understanding the mechanisms behind species coexistence (and all the functional implications that follow). Studies of ecosystem functioning often vaguely reference concepts such as “niche partitioning”, which would hardly be explicit enough for most papers on coexistence. Fortunately, there are periodically attempts to unifying ecological knowledge.

One of the most important contributions to understanding coexistence is Chesson’s (2000) framework of equalizing and stabilizing effects. Unlike previous approaches to species interactions, which tended to reference these vaguely-defined “niche differences”, Chesson proposed that species interactions depended on both fitness differences (differences in absolute growth rates after niche differences are controlled for) and niche differences (ecological differences between species which cause intraspecific competition to exceed interspecific competition). He also suggested rigorous methods to quantify these concepts. This framework has been applied both to the obvious questions of species coexistence and diversity maintenance, as well as predator-prey relationships (2008) and the phylogenetic structure of communities (2010).

In a recent paper, Ian Carroll et al. apply this framework to the search for the mechanisms behind biodiversity and ecosystem functioning. They point out that the questions in studies of ecosystem functioning are directly analogous to Chesson’s concepts – selection effects result from fitness or competitive differences between species, while complementarity relates to the partitioning of resources, or niche differences between species. The added benefit is that Chesson has provided clear definitions for these concepts.

While this may not be world-altering, it’s encouraging. Anytime different areas of ecology intersect, both benefit. Of course there are difficulties – no doubt the question of how to measure niche differences and fitness differences will be contentious (as attempts to translate ecological theory into ecological methodology often are) - but the possibility that a few general ecological concepts explain diverse observations is worth pursuing.

Biodiversity and ecosystem functioning - only with fungi

Once again scientists have come to an age-old conclusion: fungus is behind all of life’s great mysteries. It's responsible for curing strep throat, delicious veggie burgers, that unique musk emanating from your gym bag, the colour-morphing walls at last night’s party and now, biodiversity and ecosystem functioning.

The world of biodiversity and ecosystem functioning (BEF), like many other high-profile disciplines of science, has often been bogged down by controversy. In such situations, we often spend a disproportionate amount of time focusing on the controversy instead of actually advancing the science itself (sound familiar?)

There have been several posts about BEF on this blog in the last few months, but briefly and oversimplified, here's how it works. Ecosystem functions are things like productivity, nutrient cycling and decomposition. Ecosystems that contain many species produce higher levels of these functions than monocultures do. The controversy here surrounds the cause of this phenomenon. In the 1990s, researchers originally disagreed over whether the relationship they observed was due to complementarity (different species partitioning resources) or selection effects (the higher chance of a really productive species being included in a community with many species). The question was largely settled a few years ago; selection effects do exist, but most of the relationship is driven by complementarity. Nonetheless, many biologists who are only tangentially familiar with this area of research are unaware of the consensus and continue to believe that the issue remains unresolved. Some still dismiss the whole field of BEF because of selection effects. I guess people just love a controversy.

The result of all this is that the ecologists studying these relationships have had to spend an undue amount of time parsing their results into selection and complementarity and discussing the two phenomena. They have even come to refer to these as the “mechanisms” behind BEF. And this is where we start to have a problem. Selection and complementarity are not mechanisms - they are symptoms of mechanisms. They do not tell us what is actually causing the positive effect that biodiversity has on ecosystem functioning, only what the shape of the relationship is. In fact, very few studies have actually looked for true mechanisms that explain the effects that we have repeatedly observed.

But this week I read a new paper in Ecology Letters that actually did find a mechanism, and it wasn’t one that we expected. John Maron and his coauthors at the universities of Montana and British Columbia found that belowground fungi were causing plant productivity to increase with diversity.

In an impressively complete experiment, Maron et al. put together a classic BEF setup of many plots containing varying levels of plant diversity and then measured plant biomass. But this time they added a twist; they applied fungicide to the soil in some of these plots. The result was that in the absence of fungi, the common BEF relationship disappeared. The low diversity plots became much more productive, while productivity at high diversity only increased slightly. The authors explained their results by the fact that fungi can be both species-specific and density-dependent, so as plant diversity increases, the fungi’s negative impact on plant productivity diminishes. And for good measure, they of course also ruled out a significant selection effect in their results.

So what does this all mean? Well for one thing, it means that we now have at least one good mechanistic explanation for that black box that we’ve been calling “complementarity” for years. But perhaps more importantly, it means that the link between biodiversity and ecosystem functioning is now more real than ever. If plant species go extinct, the remaining ones will be more susceptible to fungal pathogens and productivity will decline. So let’s try to not let that happen, k?

Maron, J. L., Marler, M., Klironomos, J. N. and Cleveland, C. C. , Soil fungal pathogens and the relationship between plant diversity and productivity. Ecology Letters, DOI: 10.1111/j.1461-0248.2010.01547.x

The effects of forest fragmentation after 30 years

Large-scale alteration of nature landscapes has had profound implications for biological diversity. The single biggest contributor to the current extinction crisis is the wholesale destruction of habitats. As habitats are destroyed, formerly contiguous landscapes become fragmented into smaller patches. But what exactly the effects of fragmentation are, independent of habitat destruction, is not always so clear (e.g., Simberloff 2000. What do we really know about fragmentation? Texas Journal of Science 52: S5-S22).

The biological dynamics of forest fragments project (BDFFP) in the Amazon, was started in 1979 and created 11 tropical forest patches ranging from 1 to 100 ha in size. The dynamics of these fragments have been consistently monitored and compared to plots in intact forest. This experiment represents the world's largest, longest-running fragmentation experiment and has told us more about fragmentation then any other study system. In a recent publication by William Laurance and many colleagues involved in this project, they summarize 30 years of data and show how fragmentation affects ecological patterns and processes.
Fragments turn out to be very dynamic and defined by change, compared to interior plots. They have higher tree mortality and are much more susceptible to weather events such as storms or droughts. The effects are especially pronounced at the edges of these fragments. The edge community face high mortality but also have higher tree density. Faunal communities in fragments and especially near edges are depauperate.

One interesting aspect highlighted by this 30 years of research is that the edge effects are strongly influenced by what is happening around the fragments. The fragment edge effects are sensitive to the composition of the inter-patch matrix, giving managers the opportunity to influence fragment diversity and health by managing the matrix in ways that support fragments. Because of over 30 years of perseverance of the researchers involved, this experiment give scientists, managers and policy-makers information to help manage an increasingly fragmented world and to find ways to reduce to negative impacts of habitat destruction.

Laurance, W., Camargo, J., Luizão, R., Laurance, S., Pimm, S., Bruna, E., Stouffer, P., Bruce Williamson, G., Benítez-Malvido, J., & Vasconcelos, H. (2010). The fate of Amazonian forest fragments: A 32-year investigation Biological Conservation DOI: 10.1016/j.biocon.2010.09.021