Another round in Diamond vs. Simberloff: revisiting the checkerboard pattern debate

Edward F. Connor, Michael D. Collins, and Daniel Simberloff. 2013. "The Chequered History of Checkerboard Distributions." Ecology. http://dx.doi.org/10.1890/12-1471.1.

One of the most vociferous recent debates in community ecology started in the 1970s between Jared Diamond and Dan Simberloff (and colleagues) regarding whether 'checkerboard patterns' of bird distributions provided evidence for interspecific competition. This was an early and particularly heated example of the pattern versus process debate that continues in various forms today. Diamond (1975) proposed that the distribution of birds in the Bismark Archipelago, and particularly the fact that some pairs of bird species did not co-occur on the same islands (producing a checkerboard pattern), was evidence that competition between species limited their distributions. The issue with using this checkerboard pattern as evidence of competition, which Connor and Simberloff (1979) subsequently pointed out, was that a null model was necessary to determine whether it was actually different from random patterns of apparent non-independence between species pairs. Further, other mechanisms (different habitat requirements, speciation, dispersal limitations) could also produce non-independence between species pairs. The original debate may have died down, but the methodology for null models of communities suggested by Connor and Simberloff has greatly influenced modern ecological methods, and continues to be debated and modified to this day.

The original null model of bird distributions in the Bismark Archipelago involved a binary community matrix (rows represent islands, columns represent species) with 0s and 1s representing species presences or absences. Hence, all the 1s in a row represent the species present on the island. The original null model approach involved randomly shuffling the 0s and 1s, maintaining island richness (row sums) and species range sizes (column sums). The authors of a new paper in Ecology admit that the original null models didn’t accurately capture what Diamond meant by a "checkerboard pattern". This is interesting in part because two of the authors (E.F. Connor and Dan Simberloff) lead the debate against Diamond and introduced the binary matrix approach for generating null expectations. So there is a little bit of a ‘mea culpa’ here. The authors note that earlier null models captured patterns of non-overlap between species' distributions but didn’t differentiate between non-overlap between species with overlapping ranges compared to non-overlap between species which simply occurred on sets of geographically distant islands (referred to here as 'regional allopatry'). The original binary matrix approach didn’t consider spatial proximity of species ranges.

With this fact in mind, the authors re-analyzed checkerboard patterns in the Bismark Archipelago, but in such a way as to control for regional allopatry. True checkerboarding was defined as: “a congeneric or within-guild pair with exclusive distribution, co-occurrence in at least one island group, and geographic ranges that overlap more or significantly more than expected under an hypothesis of pairwise independence”. This definition appears closer to Jared Diamond's original definition and so a null model that captures this is probably a better test of the original hypothesis. The authors looked at the overlap of convex hulls defining species’ ranges and when randomizing the binary matrix, added the further restriction that species could occur only within the island groups where they were actually found (instead of being randomly shuffled through any island, as before).

Even with these clarified and more precise null models, the results remain consistent. True checkerboarding appears to rarely occur compared to chance. Of course, this doesn't mean that competition is not important, but “Rather, in echoing what we said many years ago, one can only conclude that, if they do compete, competition does not strongly affect their patterns of distribution among islands.” More generally, the endurance of this particular debate says a lot about the longstanding tension in ecology over the value and wealth of information captured by ecological patterns, and the limitations and caveats that come with such data. There is also a subtle message about the limitations of null models: they are often treated as a magic wand for dealing with observed patterns, but null models are limited by our own understanding (or ignorance) of the processes at play and our interpretation of their meaning. 

The slippery slope of novelty

Coming up with a novel idea in science is actually very difficult. The many years during which smart people have, thought, researched, and written about ecology and evolution means that there aren’t necessarily many easy openings remaining. If you are lucky (or unlucky) enough to know someone with an encyclopedic knowledge of the literature, it becomes quickly apparent that only rarely has an idea not been suggested anywhere in the history of the discipline. Mostly science results from careful steps, not novel leaps and bounds. The irony is that to publish in a top journal, a researcher must convince the editor and reviewers that they are making a novel contribution.

There are several ways of thinking about the role of the novelty criterion - first, the effect it has had on research and publishing, but also more fundamentally, how difficult it is to even define scientific novelty in practice. Almost every new student spends considerable effort attempting to come up with a completely "novel" idea, but a strict definition of novelty – research that is completely different than anything published in the field in the past - is nearly impossible. Science is incrementally built on a foundation of existing knowledge, so new research mostly differs from past research in terms of scale and extent. Let's say that extent characterizes how different an idea must be from a previous one to be novel. Is neutral theory different enough from island biogeography (another, earlier, explanation for diversity which doesn’t rely on species-differences) to be considered novel? Most people would suggest that it is distinct enough as to be novel, but clearly it is not unrelated to works that came before it. What about biodiversity and ecosystem functioning? Is the fact that its results are converging with expectations from niche theory (ecological diversity yields greater productivity, etc) take away from its original, apparent novelty? 

Then there is the question of scale, which considers the relation of an new idea to those found in other disciplines or at previous points in time. For example, when applying ideas that originate in other disciplines, the similarity of the application or the relatedness of the other discipline alters our conclusions about its novelty. Applying fractals to ecology might be considered more novel than introducing particular statistical methods, for example. Priority (were you first?) is probably the first thing considered when evaluating scientific novelty. But ideas are so rarely unconnected to the work that came before them, so then we evaluate novelty as a matter of degree. The most common value judgment seems to be that re-inventing an obscure concept first describe many years ago is more novel than re-inventing an obscure concept that was recently described.

In practice, then, the working definition of novelty may be that something like ‘an idea or finding doesn't exist the average body of knowledge in the field’. The problem with this is that not everyone has an average body of knowledge – some will be aware of every obscure paper written 50 years ago, and for them nothing is novel. Others have a lesser knowledge or more generous judgement of novelty and for them, many things seems important and new. A great deal of inconsistency in the judgement of papers for a journal with a novelty criterion results simply from the inconsistent judgement of novelty. This is one of the points that Goran Arnqvist makes in his critique of the use of novelty as a criterion for publishing (also, best paper title in recent memory). Novelty is a slippery slope. It forces papers to be “sold” and so overvalues flashy and/or controversial conclusions and undervalues careful replication and modest advances. And worse, it ignores the truth about science, which is that science is built on tiny steps founded in the existing knowledge from hundreds of labs and thousands of papers. And that we've never really come up with a consistent way to evaluate novelty.

(Thanks Steve Walker for the bringing up the original idea)

Speaking the language: is jargon always bad?

You hear mostly about the evils of jargon in science. Undeniably jargon is a huge barrier between scientific ideas and discoveries and non-scientists. Translating a complex, nuanced result into a sound bite or recommendation suitable for consumption by policymakers or the public can be the most difficult aspect of a project (something Alan Alda, as part of his Center for Communicating Science, is attempting to assist scientists with). But sometimes the implication in general seems to be that scientific jargon is always undesirable. Is jargon really always a bad thing?

Even between scientists, you hear criticism about the amount of jargon in talks and papers. I have heard several times that community ecology is a frequent offender when it comes to over-reliance on jargon (defn: “words or expressions that are used by a particular profession or group and are difficult for others to understand”). It is fun to come up with a list of jargon frequently seen in  community ecology, because examples are endless: microcosm, mesocosm, niche, extinction debt, stochastic, trophic cascades, paradigm shift, priority effects, alternate stable states, or any phrase ending in ‘dynamics’ (i.e. eco-evolutionary, neutral, deterministic). Special annoyance from me at the usage of multidisciplinary, trans-disciplinary, and inter-disciplinary to all express the exact same thing. I don’t think, despite this list, that jargon is necessarily problematic.

If the meaning implied by the word or phrase is more than the sum of its parts it is probably jargon. Ideally, jargon is a shared, accurate shorthand for communicating with colleagues. A paper published without any jargon at all would be much longer and not necessarily clearer. Instead of saying, “we used protist microcosms”, it would have to say, “we used a community of protist species meant to encapsulate in miniature the characteristic features of a larger community”. (And arguably ecology is still relatively understandable for a newcomer, compared to disciplines like cell and systems biology, where an abstract might seem impenetrable: “Here, we report that, during mouse somatic cell reprogramming, pluripotency can be induced with lineage specifiers that are pluripotency rivals to suppress ESC identity, most of which are not enriched in ESCs.”)

Jargon is useful as a unifying tool: if everyone is using the same nicely defined label for a phenomenon, it is easier to generalize, contrast and compare across research. Jargon is many pieces of information captured in a single phrase: for example, using the term 'ecophylogenetics' may imply not only the application of phylogenetic methods and evolutionary biology to community ecology, but also the accompanying subtext about methodology, criticism, and research history. At its best, jargon can actually stimulate and unify research activities – you could argue that introducing a new term (‘neutral dynamics’) for an old idea stimulated research into the effects of stochasticity and dispersal limitation on community structure.

That’s the best case scenario for jargon. There are also consequences to developing a meaning-laden dialect unique to a subdiscipline. It is very difficult to enter a subdiscipline or move between subdisciplines if you don’t speak the language. New students often find papers difficult to penetrate because of the heavy reliance on jargon-y descriptions: obtaining new knowledge requires you already have a foundation of knowledge. Moving between subdisciplines is hard too – a word in one area may have completely different meaning in another. In a paper on conservation and reserve selection, complementarity might refer to the selection of regions with dissimilar species or habitats. In a biodiversity and ecosystem functioning paper, a not-very distant discipline, complementarity might refer to functional or niche differences among co-occurring species. Giving a talk to anyone but the most specialist audience is hampered by concerns about how much jargon is acceptable or understandable.

Jargon also leads to confusion. When using jargon, you can rely on understood meaning to delimit the boundaries of your meaning, but you may never specify anything beyond those boundaries. Everyone has heard a 30-second spiel so entirely made of jargon that you never develop a clear idea of what the person does. The other issue is that jargon can quickly become inaccurate, so laden with various meanings as to be not useful. The phrase ‘priority effect’, for example, has had so many particular mechanisms associated with it that it can be uninformative on its own. And I think most ecologists are well aware that jargon can be inaccurate, but it’s a difficult trap to get out of. The word “community”, essential to studying community ecology, is so broadly and inconsistently defined as to be meaningless. Multiple people have pointed this out (1, 2, 3) and even suggested solutions or precise definitions, but without lasting impact. One of the questions in my PhD defense was “how did I define an ecological community and why?”, because there is still no universal answer. How do we rescue words from becoming meaningless?

Something interesting, that you rarely see expressed about jargon is that linguists tells us that language is knowledge: how we understand something is not independent of the language we use to describe it. The particular language we think in shapes and limits what we think about: perhaps if you have many ways of finely delineating a concept you will think about it as a complex and subtle idea (the 100-words-for-snow idea). On the other hand, what if you have to rely on vague catch-alls to describe an idea? For example, a phrase like ‘temporal heterogeneity’ incorporates many types of differences that occur through time: is that why most researchers continue to think about differences through time in a vague, imprecise manner? Hard to say. It is hard to imagine where community ecology would be without jargon, and even harder to figure out how to fix all the issues jargon creates. 

Some ways you might not expect your research to be used

Most ecologists recognize that ecological knowledge is a tool, with useful applications to conservation and management, recreation, and ecosystem services and goods. Many of us have even written or said something suggesting uses for our work, no matter how likely. But few ecologists expect their research to be cited for military applications or support for the superiority of capitalism.

For example, a recent editorial in the New York Times detailed how conservation of biodiversity became part of American Cold War strategies. In those days, the American military was considering the role for ‘environmental warfare’, and the research of Charles Elton, who wrote of the dangers of simplifying landscapes by reducing biodiversity resonated. Strategists advocated maintaining biodiversity in food supplies and stockpiles (wisdom which transcends the military motivation). Ecological research into invasive species has also informed the US military in modern times. For example, the report "Invasive Threats to the American Homeland" considers the possibility of introduced species being used as terrorist weapons. Such introduced species might be crop parasites or vectors for human diseases, theoretically wreaking economic, structural, and human costs. 

Sometimes attempts to adapt research to other uses fall rather short of the mark. Evolutionary biology is not unfamiliar with this: for example, the misapplication of evolution to social Darwinism and some of the ideas touted in evolutionary psychology misrepresent evolutionary theory. This can happen in ecology too. A recent PNAS paper presented the result that evolutionary diversity increases ecosystem productivity. One writer in the Washington Post blogging community presented this finding as evidence that capitalist concepts like division of labour are found even in nature. It seems difficult to accept the link the writer attempts to make (the title is rather over the top as well: “Darwin’s free market wisdom: division of labor starts in the genes”). The writer states that nature wouldn’t exhibit a relationship between diversity and higher productivity if it wasn’t optimal, so “[t]he same findings would also appear to suggest that species, like humans, are not all created equal and some are more adept at certain tasks than others.” Therefore, apparently, capitalism is superior to communism. 

This kind of thing makes me think that Darwin was lucky that he did not live to see his words and ideas so frequently misquoted and misapplied (although he certainly suffered this during his own lifetime). This is the danger of sending an idea or result into the world: you no longer fully control how it is used and understood. A successful idea is one that, for better or worse, has an independent life. 

(There are probably many misapplications or unusual uses of ecology and evolution that I haven't thought of. If you think of other examples, feel free to mention them in the comments.)

Has academic advancement changed your point of view?

From http://researchinprogress.tumblr.com/

We regret to inform you that you paper has not been accepted

as a graduate student:

as a postdoc:

 as a professor:

We are pleased to inform you that your paper has been accepted

as a graduate student:

as a postdoc:

as a professor:

Evidence for the evolutionary diversity-productivity relationship at several scales

Cadotte, M. W. 2013. Experimental evidence that evolutionarily diverse assemblages result in higher productivity. Proceedings of the National Academy of Sciences.

John J. Stachowicz, Stephanie J. Kamel, A. Randall Hughes, and Richard K. Grosberg. Genetic Relatedness Influences Plant Biomass Accumulation in Eelgrass (Zostera marina). The American Naturalist, Vol. 181, No. 5 (May 2013), pp. 715-724

Ecology is increasingly recognizing the value of non-species based measures of diversity in relation to ecosystem services, community diversity and invasibility, and conservation activities. One result is that we are seeing increasingly strong and interesting experimental evidence for the importance of genetic diversity in understanding populations, species, and communities are structured. Two recent papers are good examples of how our understanding is progressing.

For example, we are now at the point where research has clearly demonstrated the relationship between ecosystem functioning and evolutionary history, and now well-designed experiments can begin to explore the mechanisms that underlie the ecosystem functioning-evolutionary diversity link. The oft-demonstrated correlation between evolutionary diversity and productivity is explained based on the assumption that ecological similarity and evolutionary relatedness are connected. Diverse communities are often thought to have lower niche overlap (i.e. higher complemenarity), but these experiments often rely on highly distinct species (such as a grass and a N-fixer), which could over-emphasize the importance of this relationship. In Cadotte (2013), independent manipulations of phylogenetic diversity and species richness allow the author to explore separately the role of complementarity and selection effects (the increased likelihood that a highly productive species will be present as species richness increases).

The experiment involved old field plots, planted with between 1 and 4 species chosen from a pool of 17 possible species; evolutionary diversity (high, medium, or low) and species richness are manipulated to include all possible combinations. The study found found a much stronger relationship between phylogenetic diversity (PD) and biomass production then between species richness and biomass production, but this isn't especially novel. What is interesting is that it could also identify how selection effects and complementarity were driving this response. High levels of complementarity were associated with high levels of PD: polyculture plots with high complementarity values were much more likely to show transgressive overyielding. Plots with close relatives had a negative or negligible complementarity effect (negative suggesting competitive or other inhibitory interactions). There was also evidence for a selection effect, which was best captured by an abundance-weighted measure of evolutionary diversity (IAC), which measured the abundance of closely related species in a plot. Together, PD and IAC explain 60% of the variation in biomass production.

From Cadotte (2013).

The second study asks the exact same question – what is the relationship between biomass production and genetic diversity - but within populations. Stachowicz et al. (2013) looked at genetic relatedness among individuals in monocultures of the eelgrass Zostera marina and its relationship to productivity. Variation within a species has many of the same implications as variation within a community – high intraspecific variation might increase complementarity and diverse assemblages might also contain more productive genotypes leading to a selection effect. On the other hand, it is possible that closely related, locally adapted genotypes might be most productive despite their low genotypic variation. 

Similar to most community-level experiments, Stachowicz et al. found that looking at past experimental data suggested there was a strong relationship between genetic relatedness and biomass/density in eelgrass beds. Taxa (i.e. the number of genotypes) tended to be a poorer predictor of productivity. However, the relationship was in the opposite direction usually seen – increasing relatedness predicted higher biomass. This is difficult to explain, since it goes against the expected direction of complementarity or selection effects. Possibly cooperative/facilitative relationships are important in eelgrass monocultures. Data obtained from field surveys (rather than experimental data) suggested an alternative: possibly these studies didn’t cover a large enough range of relatedness. This field data covered a much larger range of relatedness values, and showed a unimodal relationship (below), indicating that the productivity-relatedness relationship had an optimum, where highly related or highly diverse assemblages were less productive. Although further work needs to be done, this is an intriguing possibility.

From Stachowicz et al. (2013). Grey dots represent range of relatedness values from experimental data only, compared to range covered by field survey.

At some scales, ecologists are now refining what we know about popular research questions, while at others we are just scratching the surface. Stachowicz et al. suggest that as we scale up or down our expectations should differ -  “the slope and direction of the relationship between genetic differentiation and ecological functioning might depend on the genetic scale under consideration”.

(Disclaimer - obviously Marc Cadotte was my PhD supervisor until very recently. But I think it's a nice paper, regardless, and worth a post :) )

The end of the impact factor

Recently, both the American Society for Cell Biology (ASCB) and the journal Science both publicly proclaimed that the journal impact factor (IF) was bad for science. The ASCB statement argues that IFs limit meaningful assessment of scientific impact for both published articles and especially other scientific products. The Science statement goes further, and claims that assessments based on IFs lead researchers to alter research trajectories and try to game the system rather than focussing on the important questions that need answering.


Impact factors: tale of the tail
The impact factor was created by Thomson Reuters and is simply the number of citations a journal has received in the the previous two years, divided by the number of articles published over that time span. Thus it is a snapshot of a particular type of 'impact'. There are technical problems with this metric -for example, that citations accumulate at different rates across different subdisciplines. More importantly, and what all publishers and editors know, is that IFs generally rise and fall with the extreme tail of the distribution of the number of citations. For a smaller journal, it just takes one heavily cited paper to make the IF jump up. For example if a journal publishes one paper that accumulates 300 citations and it published just 300 articles over the two years, then its IF can jump up by 1, which can alter the optics. In ecology and evolution, IFs greater than 5 are usually are viewed as top journals.

Regardless of these issues, the main concern expressed by ACSB and Science is that a journal-level metric should not be used to assess an individual researcher's impact. Should a researcher publishing in a high IF journal be rewarded (promotion, raise, grant funded, etc.) if their paper is never cited? What about their colleague who publishes in the lower IF journal, but accrues a high number of citations?

Given that rewards are, in part, based on the journals we publish in, researchers try to game the system by writing articles for certain journals and journals try to attract papers that will accrue citations quickly. Journals with increasing IFs usually see large increases in the number of submissions, as researchers are desperate to have high IF papers on their CVs. Some researchers send papers to journals in the order of their IFs without regard for the actual fit of the paper to the journal. This results in an overloaded peer-review system.

Rise of the altmetric
Alternative metrics (altmetrics) movement means to replace journal and article assessment from one based on journal citation metrics to a composite of measures that include page views, downloads, citations, discussions on social media and blogs, and mainstream media stories. Altmetrics attempts to capture a more holistic picture of the impact of an article. Below is a screenshot from a PLoS ONE paper, showing an example of altmetrics:

By making such information available, the impact of an individual article is not the journal IF anymore, but rather how the article actually performs. Altmetrics are particularly important for subdisciplines where maximal impact is beyond the ivory towers of academia. For example, the journal I am an Editor for, the Journal of Applied Ecology, tries to reach out to practitioners, managers and policy makers. If an article is taken up by these groups, they do not return citations, but they do share and discuss these papers. Accounting for this type of impact has been an important issue for us. In fact, even though our IF may be equivalent to other, non-applied journals, our articles are viewed and downloaded at a much higher rate.

The future
Soon, how articles and journals are assessed for impact will be very different. Organizations such as Altmetric have developed new scoring systems that take into account all the different types of impact. Further, publishers have been experimenting with altmetrics and future online articles will be intimately linked to how they are being used (e.g., seeing tweets when viewing the article).

Once the culture shifts to one that bases assessment on individual article performance, where you publish should become less important, and journals can feel free to focus on an identity that is based on content and not citations. National systems that currently hire, fund and promote faculty based on the journals they publish in, need to carefully rethink their assessment schemes.

May 21st, 2013 Addendum:

You can sign the declaration against Impact Factors by clicking on the logo below:

Holding fast to a good(?) idea

One of my favourite lists on the internet is tucked away in the credits for the PHYLIP software. PHYLIP was authored by Joe Felsenstein, a professor at the University of Washington and expert on methods for phylogenetic inference. PHYLIP is a free package of programs for inferring phylogenies, and probably the first and oldest widely-distributed phylogenetic program. Programs like PHYLIP made phylogenetic approaches easily accessible to ecologists and evolutionary biologists. Apparently it took years to get from the idea for PHYLIP to funding, and Felsenstein memorializes this with his “No thanks to” list (below). The list includes reviewers and panels from the US Dept of Energy, NSF, and NIH that turned down his proposals and made comments like "The work has the potential to define the field for many years to come.... All agreed that the proposal is somewhat vague. There was also some concern that the proposed work is too ambitious.”

(Click to enlarge)

There are obvious responses to this list, mostly relating to the short-sightedness of funding agencies, meaningless requirements for ‘broader impacts’, the fact that proposals might be improved through the process of multiple failed applications, and of course the benefit of being long-established and respected when posting such lists on your website. But what I always wonder about is how long do you hold on to an idea, a proposal, or a manuscript that it is repeatedly rejected, before you give up on it? 

This question is interesting to me for a couple of reasons. Firstly, because personality is so intertwined with confidence about an idea’s success. We all know people who would argue that all their ideas are Nature-worthy, criticism be damned. Other people need to be convinced of the merit of their own ideas. Obviously past success probably helps with judgment – having experience in identifying good ideas builds confidence in your ability to do so again. But what is the line between self-confidence and self-delusion? Secondly, it is a reminder that lots of good ideas and good papers were rejected many times. In any case, I am curious whether people tend give up on an idea simply because they became discouraged at the prospects of getting it published, or because they lost faith in the idea, or a combination of both. 

Love the lab you’re with or find the lab you love? Being happy in grad school.

Every grad student is unhappy at some point; existential angst is basically required, hence the success of PhD comics. A surprisingly common reason for grad school unhappiness results when students feel they have diverged from the path they want to be on - that they are somehow in the wrong lab, learning the wrong thing, or working with the wrong person. That they dislike their research. Some people might argue that few people start in their dream job, but grad school is more like an apprenticeship than a 9-5 job: a place to obtain skills and experiences rather than a source of income.

Every unhappy student is different in their own way, but there are a few predictable causes. The path between undergrad and grad student is highly stochastic. Most undergraduates make choices about grad school while under-informed about their options and unclear on where their interests truly lay. Choosing a lab for a PhD or Masters is a huge commitment for an undergrad who has had comparatively limited interactions with ecological research. Academic labs tend to be so specialized that even if an undergrad has had the opportunity and motivation to interact with a number of labs as a student, they have experienced only a tiny fraction of the areas available for study. Students in schools with general biology programs, rather than specialized EEB departments, may be more limited again in the ecological experiences they can have. I can’t help but think that few undergrads are really equipped to make a definite, informed decision about what they want to spend the next 5 years (or more) of their lives doing. Even if they are, a successful graduate student should grow as a researcher and their interests will naturally expand or shift. Expanding interests and changing foci are part of a successful graduate experience, but what initially felt like a good fit may suddenly feel less comfortable. Students may also end up in uncomfortable fits simply because their choices for grad school were limited by geographical constraints or the availability of funded positions, causing them to compromise on their interests.

My own experience moving from undergrad to PhD student was pretty much in line with this. I knew I wanted to go to grad school and I felt reasonably prepared – I had good grades, three years of research experience in a lab, I researched and contacted a few potential supervisors – but I hadn’t specialized in ecology and didn’t exactly know what my interests were. It took more than a year as a PhD student, reading deep into the literature and taking classes, to realize what I really was interested in was completely different than what I was supposed to be doing. This was accompanied by a period of unhappiness and confusion – I had apparently gotten what I wanted (grad school, funding, etc), but it wasn’t what I wanted after all. No one prepared me for this possibility. Eventually, but with some hassle, I changed labs and was lucky to have the opportunity to get the skills I was really wanted.

I don’t think this outcome is anyone’s “fault”. I think most departments and many supervisors are sympathetic to these sort of graduate student issues. Formal advising of undergraduates at the department level, in addition to the usual informal advising that grad students and advisors provide, should be focused on guiding students in determining what they want from grad school (or if they really want grad school at all!), how to identify areas of interest and programs/supervisors that would suit their interests. In particular, they need to be empowered in how to contact potential supervisors and how to discuss the supervisor’s expectations and approach, what changing interests mean on a laboratory and department level, and what resources are available for student who may wish to obtain particular skills not available in the lab. Supervisors benefit too when their students are informed and more likely to be happy and engaged.

The lab rotation system, which some departments have, also seems like a good way to expose students to their options (although I have no personal experience with it). In addition, when grad student funding comes through the department, rather than from individual supervisors, students can change labs with less difficulty. Some supervisors have very relaxed approaches to grad student projects, allowing students to explore their interests well outside of the lab’s particular approach. But other supervisors (or funding sources) are very much organized around a particular project, making it difficult for students to do anything but the project they were hired to work on.

So what is a student who realizes they want to be working with a different system, approach, sub-discipline, or supervisor, supposed to do? How much does unhappiness really matter in the long run? This depends a lot on what a student wants to get out of grad school and what they need to achieve it. One thing students need to do is elucidate what they hope to achieve as a grad student. Though a student may ultimately be unsatisfied with some aspects of their position, they may be able to gain the experiences they want from grad school regardless. There are many tangible and intangible skills students learn in grad school. Students may decide that they want to obtain particular quantitative skills (statistics, ArcGIS, coding and modeling experience, etc, etc) that they want for the job market; if these aren’t available, change may be necessary. On the other hand, even if a student is less interested in the particular system they are working in, it may be possible to obtain experimental and technical skills that are transferrable elsewhere. If students wish to remain in academia, but realize they are interested in a different subdiscipline than where they work, one consideration is whether it will be easier to make the shift now compared when finding a postdoc and attempting to convince a potential employer that their knowledge is transferrable. This is a difficult question – having read a large number of Ecolog post-doctoral position ads, it seems that the request for system-specific experience occurs in about 50% of ads. The need to have a particular skill set (say, Python and R, or experimental design) tends to be mentioned in every ad. So if you want to go from a protist-microcosm PhD to a postdoc in kangaroo ecology, it seems difficult to predict how well your experimental design skills will trump your lack of understanding of Australian ecosystems.

Of course, there is no one-size-fits-all answer about what to do. Sometimes, unhappiness will pass, sometimes it won’t. Students need to be proactive above all. The truth is that sometimes it is better to be willing to drop out, to change labs, or take other drastic action. Students commonly fall victim to the sunk-cost fallacy, the idea that they’ve spent 2 years on this degree, so they might as well not “waste” it. Sometimes it is worth sticking it out, but there should be no stigma in deciding not to.

Testing the utility of trait databases

Cordlandwehr, Verena, Meredith, Rebecca L., Ozinga, Wim A., Bekker, Renée M., van Groenendael, Jan M., Bakker, Jan P. 2013. Do plant traits retrieved from a database accurately predict on-site measurements? Journal of Ecology. 101:1365-2745.

We are increasingly moving towards data-sharing and the development of online databases in ecology. Any scientist today can access trait data for thousands of species, global range maps, gene sequences, population time series, or fossil measurements. Regardless of arguments for or against, the fact that massive amounts of ecological data are widely available is changing how research is done.

For example, global trait databases (TRY is probably best known) allow researchers to explore trait-based measures in communities, habitats, or ecosystems without requiring that the researchers have actually measured the traits of interest in the field. And while few researchers would suggest that this is superior to making the measurements in situ, the reality is that there are many situations where trait data might be required without the researcher being able to make them. In these cases, online databases are like a one-stop shop for data. But despite the increasing frequency of citations for trait databases, until now there has been little attempt to quantify how well database values act as proxies for observed trait values. How much should we be relying on these databases?

There are many well-recorded reasons why an average trait value might differ from an individual value: intraspecific differences result from plasticity, genotype differences, and age or stage differences, all of which may vary meaningfully between habitats. How much this variation actually matters to trait-based questions is still up for debate, but clearly affects the value of such databases.  To look at this question, Cordlandwehr et al. (2013) examined how average trait values calculated with values from a North-west European trait database (LEDA) corresponded with average trait values calculated using in situ measurements. Average trait values were calculated across several spatial scales and habitat types. The authors looked plant communities growing in 70 2m x 2m plots in the Netherlands, divided between wet meadow and salt marsh habitats. In each community, they measure three very common plant traits: canopy height (CH), leaf dry matter content (LDMC), and specific leaf area (SLA).

In situ measurements were made such that the trait value for a given plot was the median value of all individuals measured; for each habitat it was the the median value of all individuals measured in the habitat. The authors calculated the average trait values (weighted by species abundance) across all species for each community (2m x 2m plot) and each habitat (wet meadow vs. salt marsh). They then compared the community or habitat average as calculated using the in situ values and the regional database values. 

From Cordlandwehr et al. 2013. Habitat-level traits at site scale plotted against habitat-level traits calculated using trait values retrieved from a database. 

The authors found the correspondence between average trait values measured using in situ or database values varied with the scale of aggregation, the type of trait and the particular habitat. For example, leaf dry matter content varied very little but SLA was variable. The mesic habitat (wet meadow) was easier to predict from database values than the salt marsh habitat, probably because salt marshes are stressful environments likely to impose a strong environmental filter on individuals, so that trait values are biased. While true that rank differences in species trait values tended to be maintained regardless of the source of data, intraspecific variation was high enough to lead to over- or under-prediction when database values were relied on. Most importantly, spatial scale mattered a lot. In general, database values at the habitat-scale were reasonable predictors of observed traits. However, the authors strongly cautioned against scaling such database values to the community level or indeed using averaged values of any type at that scale: “From the poor correspondence of community-level traits with respect to within-community trait variability, we conclude that neither average trait values of species measured at the site scale nor those retrieved from a database can be used to study processes operating at the plot scale, such as niche partitioning and competitive exclusion. For these questions, it is strongly recommended to rigorously sample individual plants at the plot scale to calculate functional traits per species and community.” 

There are two conclusions I take from this. First, that the correlation between sampling effort and payoff is still (as usual) high. It may be easier to get traits from a database, but it is not usually better. The second is that studies like this allow us to find a middle ground between unquestioning acceptance or automatic criticism of trait databases: they help scientists develop a nuanced view that acknowledges both strengths and weaknesses. And that's a valuable contribution for a study to make.