Is ecology really failing at theory?

Sam Scheiner. 2013. The ecological literature, an idea free distribution. Ecology Letters. Early View. DOI: 10.1111/ele.12196 

“Ecology is awash with theory, but everywhere the literature is bereft”. That is Sam Scheiner's provocative start to his editorial about what he sees as a major threat to modern ecology. The crux of his argument is simple – theory is incredibly important, it allows us to understand, to predict, to apply, to generalize. Ecology began as a study rooted in system-specific knowledge or natural history in the early 1900s, and developed into a theory-dominated field in the 1960s, when many great theoreticians came to the forefront of ecology. But today, he fears that theory is dwindling in importance in ecology. To test this, he provides a small survey of ecological and evolutionary journals for comparison (Ecology Letters, Oikos, Ecology, AmNat, Evolution, Journal of Evolutionary Biology), recording papers from each journal as either containing no theory, being ‘theory motivated’, or containing theory (either tests of, development of, or reviews of theory). The results showed that papers in ecological journals on average include theory only 60% of the time, compared to 80% for evolutionary papers. Worse, ecological papers seem to be more likely to develop theory than to test it. Scheiner’s editorial (as the title makes clear) is an indictment of this shortcoming of modern ecology.

Plots made based on data table in Scheiner 2013. Results combined for all evolution and all ecology papers.
The proportion of papers in each category - all categories starting with
 "Theory" refer to theory-containing papers.

Plots made based on data table in Scheiner 2013. Results for papers from individual journals.
The proportion of papers of each type - all categories starting with
 "Theory" refer to theory-containing papers.

This is not the kind of conclusion that I find myself arguing against too often. And I mostly agree with Scheiner: theory is the basis of good science, and ecology has suffered from a lack of theoretical motivation for work, or pseudo-theoretical motivation (e.g. productivity-diversity, intermediate diversity patterns that may lack an explanatory mechanism). But I think the methods and interpretation, and perhaps some lack of recognition of differences between ecological and evolutionary research make the conclusions a little difficult to embrace fully. There are three reasons for this – first, is this brief literature review a good measure of how and why we use theory as ecologists? Second, do counts of papers with or without theory really scale into impact or harm? And third, is it fair to directly compare ecological and evolutionary literature, or are there differences in the scope, motivations, and approaches of these fields?

If we are being truly scientific, this might be a good time to point out that The 95% confidence intervals for the percentage of ecology papers with theory overlap with the confidence intervals for the percentage of evolutionary papers with theory suggesting the difference that is the crux of the paper is not significant. [Thanks to a commenter for pointing out this difference is likely significant]. While significant at the 5% level, the amount of overlap is enough that whether this difference is meaningful is less clear. (I would accept an argument that this is due to small sample sizes though). The results also show that choice of journal makes a big difference in terms of the kinds of paper found within – Ecology Letters and AmNat had more theoretical papers or theory motivated papers, while Oikos had more tests of theory and Ecology had more case studies. This sort of unspoken division of labour between journals means that the amount of theory varies greatly. And most ecologists recognize this - if I write a theory paper, it will be undoubtedly targeted to a journal that commonly publishes theory papers. So to more fully represent ecology, a wider variety of journals and more papers would be helpful. Still, Scheiner's counterargument would likely be that even non-theory papers (case studies, etc) should include more theory.

It may be that the proportion of papers that include theory is not a good measure of theory’s importance or inclusion in ecology in general. For example, Scheiner states, “All observations presuppose a theoretical context...the simple act of counting individuals and assessing species diversity relies on the concepts of ‘individual’ and ‘species,’ both of which are complex ideas”. While absolutely true, does this suggest that any paper with a survey of species’ distributions needs to reference theory related to species’ concepts? What is the difference between acknowledging theory used via a citation and more involved discussion of theory? In neither of these cases is the paper “bereft” of theory, but it is not clear from the methods how this difference was dealt with. As well, I think that ecological literature contains far more papers about applied topics, natural history, and system-specific reports than evolutionary biology. Applied literature is an important output of ecology, and as Scheiner states, builds undeniably on years of theoretical background. But on the other hand, a paper about the efficacy of existing reserves in protecting diversity using gap analysis is both important and may not have a clear role for a theoretical section (but will no doubt cite some theoretical and methodological studies). Does this make it somehow of less value to ecology than a paper explicitly testing theory? In addition, case reports and data *are* a necessary part of the theoretical process, since they provide the raw observations on which to build or refine theory. In many ways, Scheiner's editorial is a continuation of the ongoing tension between theory and empiricism that ecology has always faced.

The point I did agree strongly with is that ecology is prone to missing the step between theory development and data collection, i.e. theory testing. Far too few papers test existing theories before the theoreticians have moved on to some new theory. The balance between data collection, theory development, and theory testing is probably more important than the absolute number of papers devoted to one or the other.

Scheiner’s conclusion, though, is eloquent and easy to support, no matter how you feel about his other conclusions: “My challenge to you is to examine the ecological literature with a critical eye towards theory engagement, especially if you are a grant or manuscript reviewer. Be sure to be explicit about the theoretical underpinnings of your study in your next paper…Strengthening the ecological literature by engaging with theory depends on you.”

Science shutdown

As the United States enters the 10th day of its government shutdown, the impacts on science and scientific researchers are becoming very apparent. Ignoring the political mess, the fact is that the shutdown is having devastating effects for science, ranging from small to very large scales.

Some of the largest obvious effects relate to the cessation of highly necessary services. For example, the majority of employees the Centers for Disease Control and Prevention were furloughed. Days later experts on foodborne illness were called back to work, because, surprise, they are essential in cases of E. coli outbreaks. Similarly, nearly all of the Environmental Protection Agency’s employees were furloughed, which means that air and water quality monitoring is furloughed too. According to one employee: "No one is going to be out inspecting water discharges, or wet lands. Nobody is going to be out inspecting waste water treatment plants, drinking water treatment plants, or landfills – nothing". Most of the employees whose work relates to climate science research and related environmental controls at the EPA, NASA, and the National Oceanic and Atmospheric Administration (NOAA) are also no longer at work.

Non-government scientists of every form are also being affected in multiple ways. The overwhelming issue is that the National Science Foundation (NSF) and National Institute of Health (NIH), have been all but closed. (For a great illustration, flip through the messages currently on US gov't science webpages, here). Technically, these furloughed employees are prohibited from replying to phone calls or answering emails from their work email. From one furlough letter: “Due to legal requirements, working in any way during a period of furlough is grounds for disciplinary action, up to and including termination of employment”. If your supervisor or collaborator is a federal scientist, this is a very direct impact of the furlough. But more generally and most importantly, these agencies (particularly the NSF for EEB academics) is the primary funding source for many scientists and institutes, and that means that until the shutdown is over, there will be no new payments or grants awarded.

(From nsf.gov/outage.html, regarding new awards)

"Proposal Preparation & Submission
No new funding opportunities (program descriptions, announcements or solicitations) will be issued.
FastLane proposal preparation and submission will be unavailable."
(existing awards)

"Performance of Work
Awardees may continue performance under their NSF awards during the shutdown, to the extent funds are available, and the term of the grant or cooperative agreement has not expired. Any expenses must be allowable and in accordance with the Office of Management and Budget (OMB) cost circulars. During the shutdown, NSF cannot authorize costs exceeding available award amounts or obligate additional funds to cover such costs.
Payments
No payments will be made during the shutdown."

If your grant is up for review or you are applying for a new grant (it was DDIGs season), then you are out of luck for the moment (FastLane, necessary for submitting grants, is not available), and there undoubtedly will be delays in processing once the shutdown does end.


Even if you have money to spend, limitations may arise from the fact that permits for research activities can’t be filled, and National Parks, National Wildlife Reserves, National Forests are all closed, so planned field activities may be limited. For example, Long-term Ecological Reseach projects (LTERs), which by definition require continuity in sampling, some projects are now inaccessible. Researchers in charge of long-term studies of rodent populations in Sevilleta National Wildlife Refuge, New Mexico, found the gates locked and the combinations changed, making monitoring stations impossible to reach. If you were planning a field season in Antarctica, you are especially out of luck, since the field season has been cancelled altogether. If you planned to use the collections at a national museum (e.g. the Smithsonian), or use government websites and federally funded data repositories, you can’t. I’ve received several mass emails from researchers attempting to locate other sources of necessary data, since their government source is gone. Even if you aren’t a researcher in the US, you might notice the effect of the shutdown because the high profile open-access journal from the Public Library of Science, PLOS ONE, is also at a standstill. And if the keynote speaker at your conference is a US government employee, you’ll notice their absence, since they are not allowed to attend conferences. 

Given the global nature of science, these effects are not isolated to only just a single country in the world – scientists everywhere share US websites and institutions, museum collections, collaborate with US-located researchers, government or otherwise, do fieldwork in US national parks, or rely on the continuity of data from long-term research projects. The impacts of this shutdown are broad and deep, and the most important question is how long will it take for science and scientists to recover?


*If anyone has examples of how this shutdown is affecting their research, please leave a comment!

Why greater diversity – even of parasites – might decrease infections

Pieter T. J. Johnson, Daniel L. Preston, Jason T. Hoverman, and Bryan E. LaFonte. 2013. Host and parasite diversity jointly control disease risk in complex communities. PNAS. Early view.

(Host competence - the tendency of host species to become infected and maintain infection.)

There is often a disconnect between the reality that communities and ecosystems are diverse assemblages with numerous, often complicated and variable interactions, and ecological research, which (perhaps necessarily) focuses on interactions between at most two or three species at a time. Disease ecology similarly has often considered interactions between particular host/parasite species pairs. Some researchers have considered the diversity of host species as an important factor in explaining disease transmission and mortality, but the reality is that parasites also interact, and most hosts harbour multiple parasites. Studying disease dynamics in the context of multi-parasite, multi-host interactions is increasingly recognized as key to understanding disease transmission and severity in communities.

With this in mind, a new paper from Pieter Johnson and colleagues attempts to combine research into the effects of both parasite and host diversity on disease. Two possible hypotheses predict the effect of diversity on disease transmission: the ‘dilution effect’ suggests that the presence of multiple hosts should decrease transmission risk, if the result of additional species is a decline in community competence. It is also hypothesized, somewhat contradictorily, that increased host diversity should support a greater variety of parasites, and parasite life cycles. Both these hypotheses take a host-centric view: understanding how changes in host diversity alter disease risk also requires that we understand how changes in parasite diversity affect disease transmission.

Mutations caused by Ribeiroia infection.
From:http://www.nature.com/scitable/knowledge/library/ecological-consequences-of-parasitism-13255694

The authors looked at the contribution of host and parasite diversity to parasite transmission success using field data and laboratory experiments. First, they looked at existing data on infections of the pathogenic trematode Ribeiroia ondatrae, in amphibian species. Observations showed a positive correlation between larval trematode diversity (parasites) and the richness of free-living species (hosts). Of course the two diversities might be correlated for many unrelated reasons, like site isolation, evolutionary history, or habitat productivity. But a closer analysis showed what appeared to be an interaction between Ribeiroia infection in Pseudacris regilla (Pacific tree frog, the most common amphibian in the survey) and the total number of amphibian species at a site (figure below). Infection by Ribeiroia was highest when there was low amphibian richness and low parasite richness. It dropped significantly lower when amphibian richness was high and/or parasite richness was high.

From Johnson et al. 2013 PNAS. Results of field observations.

In addition to these observations, the authors manipulated both parasite and host species richness, first in small laboratory microcosms and then in larger and more realistic outdoor mesocosms. Results from the laboratory microcosms showed that increases in both amphibian richness (one vs. three species) and parasite richness (one vs. five species) reduced the average number of Ribeiroia in Pseudacris regilla as well as the total infection rate in the amphibian community. The mesocosms had similar results, with both host and parasite diversity negatively influencing Ribeiroia infection. In support of the generality of these results, effect sizes were comparable between the two experiments. These effects were also quite large: for example, in the mesocosm high-host, high-parasite richness treatment there were 52.4% fewer Ribeiroia per P. regilla and 38.2% fewer Ribeiroia overall compared to the low-host, low-parasite richness. Clearly multi-species interactions are crucial for understanding infection by Ribeiroia.

From Johnson et al. 2013 PNAS. Results of the microcosm and mesocosm experiments,
 showing the effects of host and parasite diversity on transmission.

The results make it clear that if you want to understand disease transmission in communities, both host and parasite diversity should be considered. To some extent, both of the initial hypotheses were supported – host and parasite diversity were correlated in the wild, but (in agreement with the dilution effect) infection rates declined as host diversity increased. One factor missing from these hypotheses is the dynamics of the parasite community: in the paper, the authors found models of transmission that included both host and parasite richness were superior. Further, past and future studies that consider only host richness may be inadvertently accounting for the effects of parasite richness on transmission as well, if those two host and parasite diversities are correlated.

There are a number of possibilities for why both host and parasite communities alter parasite transmission success. If host diversity changes the susceptibility of the community to infection (i.e. as diversity increases, the number of low competence/susceptible species increases) then low-competence hosts could act as sinks for parasite infections. Increases in parasite diversity could result in inter-parasite competition and interactions via host immunity.

One future step will be to move beyond simple measurements of species richness to understanding how species identity or characteristics are tied to the putative mechanisms. For example, how do communities of host species vary from low to high diversity sites? Do sites in fact tend to assemble with increasing numbers of low competence host species? The implications are also of interest to other types of studies of community ecology – after all, host-parasite interactions are not very different from predator-prey interactions, and similarly, despite knowing that interactions are complex and involve multiple species, we tend to focus on two or three species examples.

The demise of peer review?

Scientists are often inundated with numerous requests to submit their papers to 'open access' journals with no track record, no editor-in-chief, and dubious sounding promises. Why this explosion in dubious journals? -simple, there is money to be made. Many of these journals are scams meant to maximize profits. As it turns out, they don't even bother with peer review. In a recent, fascinating story in Science, John Bohannon sent fatally flawed bogus articles to hundreds of open access journals, with more than half being accepted with little or no peer review. This article is a must read for anyone concerned about science publishing.

Struggling (and sometimes cheating) to catch up

Scientific research is maturing in a number of developing nations, which are trying to join North American and European nations as recognized centres of research. As recent stories show, the pressure to fulfill this vision--and to publish in English-language, international journals--has led to some large-scale schemes to commit academic fraud, in addition to cases of run-of-the-mill academic dishonesty.

In China, a widely-discussed incident involved criminals with a sideline in the production of fake journal articles, and even fake versions of existing medical journals in which authors could buy slots for their articles. China has been criticized for widespread academic problems for some time, for example, 2010 the New York Times published a report suggesting academic fraud (plagiarism, falsification or fabrication) was rampant in China and would hold the country back in its goal to become an important scientific contributor. In the other recent incident, four Brazilian medical journals were caught “citation stacking”, where each journal cited the other three excessively, thus avoiding notice for obvious journal self-citation, while still increasing their journal’s impact factor. These four journals were among 14 having their impact factors suspended for a year, with other possible incidences that were flagged but could not be proven involved Italian, a Chinese, and a Swiss journal.

There are some important facts that might provide context to these outbreaks of cheating. Both Brazil and China are nations where to be a successful scientist in the national system, you need to prove that you are capable of success on the world stage. This is probably a tall order in countries where scientific research has not traditionally had an international profile and most researchers do not speak English as their first language. In particular it leads to focus on values which are comparable across the globe, such as journal impact factors, as measures of success. In China, there is great pressure to publish in journals included on the Science Citation Index (SCI), a list of leading international journals. When researcher, department, and university success is quantified with impact factors and SCI publications, it becomes a numbers game, a GDP of research. Further, bonuses for publications in high caliber journals can double a poorly-paid researcher’s salary: a 2004 survey found that for nearly half of Chinese researchers, performance based pay was 50+ percent of their income. In Brazil, the government similarly emphasizes publications in Western journals as evidence of researcher quality.

It’s easy to dismiss these problems as specific to China or Brazil, and there are some aspects of the issue that are naturally country-specific. On the other hand, if you peruse Ivan Oransky’s Retraction Watch website, you’ll notice that academic dishonesty leading to article retraction is hardly restricted to researchers from developing countries. At the moment, the leading four countries in retractions due to fraud are the US, Germany, Japan, and then China, suggesting that Western science isn’t free from guilt. But in developing nations the conditions are ripe to produce fraud. Nationalistic ambition is funnelled into pressure on national scientists to succeed on the international stage; disproportionate focus on metrics of international success; high monetary rewards to otherwise poorly paid individuals for achieving these measures of success; combined with the reality that it is particularly difficult for researchers who were educated in a less competitive scientific system and who may lack English language skills, to publish in top journals. The benefits of success for these researchers are large, but the obstacles preventing their success are also huge. Combine that with a measure of success (impact factor, h-index) that is open to being gamed, and essentially relies on honesty and shared scientific principles, and it is not surprising that system fails.

Medical research was at the heart of both of these scandals, probably because the stakes (money, prestige) are high. Fortunately (or not) for ecology and evolutionary biology, the financial incentives for fraud are rather smaller, and thus organized academic fraud is probably less common. But the ingredients that seem to lead to these issues – national pressures to succeed on the world stage and difficulty in obtaining such success; combined with reliance on susceptible metrics  – would threaten any field of science. And issues of language and culture are so rarely considered by English-language science (eg.), that it can be difficult for scientists from smaller countries to integrate into global academia. There are really two ways for the scientific community to respond to these issues of fraud and dishonesty – either treat these nations as second-class scientific citizens and assume their research may be unreliable, or else be available and willing to play an active role in their development. There are a number of ways the latter could happen. For example, some reputable national journals invite submissions from established international researchers to improve the visibility of their journals. In some nations (Portugal, Iceland, Czech Republic, etc), international scientists review funding proposals, so that an unbiased and external voice on the quality of work is provided. Indeed, the most hopeful fact is that top students from many developing nations attend graduate school in Europe and North America, and then return home with the knowledge and connections they gained. Obviously this is not a total solution, but we need to recognize fraud as problem affecting and interacting with all of academia, rather than solely an issue of a few problem nations.

Can common tradeoffs predict your supervisor’s functional type?

With Lanna Jin

If you’re a graduate student, the most important question (even more important than who you’re writing your paper with) is who is running your lab. New graduate students everywhere are settling in and getting to know their supervisors are little better. Supervisors come in all types, and hypothesizing a priori what their style is can be difficult. Fortunately, a couple of common tradeoffs underlie most functional styles...

---

1)

The invisible man/woman: The ultimate laissez-faire approach. You have the freedom to choose the ideas and projects that interest you, and the responsibility to make them work. Freedom can be replaced by frustrations when you need a signature, some support, or a manuscript commented on.

El generalissimo: "Here's my idea, now go do it." These labs are usually run in a top-down manner. Day to day operations are fairly hands off though, giving you room to work through those problems on your own. El generalissimo will reward their supporters well for good work.

The coach: The coach provides you the best of both worlds: enough rope to explore your ideas, but not so much to hang yourself. They are available for troubleshooting and brainstorming, but you ultimately have responsibility for your project. But if you rely on them to much, it's going to be hard to function without them.

The micromanager: These supervisors expect regular presence, frequent meetings, records of progress, and milestones to be met promptly. If you like working from home, leaving early or starting late, or need lots of freedom to be most productive it could be a poor fit. For students who thrive on structure and prefer set goals though, this might be an ideal environment.

---

2)

The skeleton: This supervisor has established themselves in their career and been active for some time, but now other interests consume them. The scientific meat that made their name seems to be gone, and all that's left is the skeleton of their earlier career. They are happy to chat with you about the many things they are interested in, but supervising your science doesn't seem to be a priority. They often see you as a person who has with non-academic interests and responsibilities, which can be a nice feeling.

The superwoman/superman: This career superstar has made their name, possibly quite early, and they are passionate about their science. This can make for an exciting and successful lab experience, as new ideas and opportunities are always on the horizon. But since they have so many demands on their time, sometimes their capes (and they) are feeling a bit ragged.

The silverback: Labs of influential individuals can be an amazing opportunity for a graduate student. Silverback experiences might be quite variable, depending on how involved they are in day to day lab activities, their travel schedules, and the size of the lab. When they are available, they have a lot to teach a student about making a successful academic career.

Bad idea: This corner of the tradeoff (low interest in science, poor establishment in the career) probably doesn't exist in tenure-track faculty. If you do manage to find such a person, run away.

The unknown: A motivated but still unestablished supervisor is a blank slate. Their early career state means that they might have time and energy to devote to you and be especially motivated to see you and the lab succeed. On the other hand, they may not yet know how to manage people and their supervisor style could morph into anything - the coach, the micromanager, el generalissimo. A bit of a gamble.