Monday, March 17, 2014

How are we defining prediction in ecology?

There is an ongoing debate about the role of wolves in altering ecosystem dynamics in Yellowstone, which has stimulated a number of recent papers, and apparently inspired an editorial in Nature. Entitled “An Elegant Chaos”, the editorial reads a bit like an apology for ecology’s failure at prediction, suggesting that we should embrace ecology’s lack of universal laws and recognize that “Ecological complexity, which may seem like an impenetrable thicket of nuance, is also the source of much of our pleasure in nature”.

Most of the time, I also fall squarely into the pessimistic “ecological complexity limits predictability” camp. And concerns about prediction in ecology are widespread and understandable. But there is also something frustrating about the way we so often approach ecological prediction. Statements such as “It would be useful to have broad patterns and commonalities in ecology” feel incomplete. Is it that we really lack “broad patterns and commonalities in ecology”, or has ecology adopted a rather precise and self-excoriating definition for “prediction”? 

We are fixated on achieving particular forms of prediction (either robust universal relationships, or else precise and specific numerical outputs), and perhaps we are failing at achieving these. But on the other hand, ecology is relatively successful in understanding and predicting qualitative relationships, especially at large spatial and temporal scales. At the broadest scales, ecologists can predict the relationships between species numbers and area, between precipitation, temperature and habitat type, between habitat types and the traits of species found within, between productivity and the general number of trophic levels supported. Not only do we ignore this foundation of large-scale predictable relationships, but we ignore the fact that prediction is full of tradeoffs. As a paper with the excellent title, “The good, the bad, and the ugly of predictive science” states, any predictive model is still limited by tradeoffs between: “robustness-to-uncertainty, fidelity-to-data, and confidence-in-prediction…. [H]igh-fidelity models cannot…be made robust to uncertainty and lack-of-knowledge. Similarly, equally robust models do not provide consistent predictions, hence reducing confidence-in-prediction. The conclusion of the theoretical investigation is that, in assessing the predictive accuracy of numerical models, one should never focus on a single aspect.” Different types of predictions have different limitations. But sometimes it seems that ecologists want to make predictions in the purest, trade-off free sense - robustness-to-uncertainty, fidelity-to-data, and confidence-in-prediction - all at once. 

In relation to this, ecological processes tend to be easier to represent in a probabilistic fashion, something that we seem rather uncomfortable with. Ecology is predictive in the way medicine is predictive – we understand the important cause and effect relationships, many of the interactions that can occur, and we can even estimate the likelihood of particular outcomes (of smoking causing lung cancer, of warming climate decreasing diversity), but predicting how a human body or ecosystem will change is always inexact. The complexity of multiple independent species, populations, genes, traits, all interacting with similarly changing abiotic conditions makes precise quantitative predictions at small scales of space or time pretty intractable. So maybe that shouldn’t be our bar for success. The analogous problem for an evolutionary biologist would be to predict not only a change in population genetic structure but also the resulting phenotypes, accounting for epigenetics and plasticity too. I think that would be considered unrealistic, so why is that where we place the bar for ecology? 

In part the bar for prediction is set so high because the demand for ecological knowledge, given habitat destruction, climate change, extinction, and a myriad of other changes, is so great. But in attempting to fulfill that need, it may be worth acknowledging that predictions in ecology occur on a hierarchy from those relationships at the broadest scale that we can be most certain about, moving down to the finest scale of interactions and traits and genes where we may be less certain. If we see events as occurring with different probabilities, and our knowledge of those probability distributions declining the farther down that hierarchy we travel, then our predictive ability will decline as well. New and additional research adds to the missing or poor relationships, but at the finest scales, prediction may always be limited.


D.T. de Kerckhove said...

This post reminds me of driving one of the chief engineers who put the RADARSAT-2 satellite in orbit across a small lake to work on some fisheries hydroacoustics together. I was curious how they predicted the failure rate of the satellite once it was launched. He said it was easy. Each little component including the bolts had been testing many times for their individual failure rate, and so it took relatively simple math to conclude the overall failure rate of the satellite. I think we are often too hard on ecology to have each of those components of a question rigorously worked out. But I don't think its impossible to do. Fisheries models are great examples of a steady improvement in predictive power. With a great respect for the technical aspects of RADARSAT-2, I think the big public ecological questions are more complicated, with more components and receiving less funding. I am confident over time, that we will increasing get better handles on answering them.

JH said...

Hi Caroline, thanks for the chance to talk about prediction because I think it’s a conversation ecologist’s should be having. That said, I disagree with the notion that ecologists are too concerned about prediction. My sense is exactly the opposite – at least, in practice – we have not appreciated the fundamental need for prediction if we are going to demonstrate that we have some understanding of how the natural world works. And then potentially contribute to solving some of the problems you mention.
The first key point is that while ecological complexity may limit predictability it does not limit prediction. . My key concern isn’t that we can’t make very good predictions (I actually don’t know if that’s true although I suspect it is), it’s that we rarely do it so I don’t know what we can predict.
And the assertion that we can make good predictions at large spatial and temporal scales just seems to me to be almost without evidence. What we actually do is make predictions that are so imprecise that they are difficult to dispute. Species- area is a very good example – we can predict that the greater the ‘patch’ size the more species we will find. What is broad about this prediction is the range of observed values that would be consistent with it. And it’s a prediction that could be made by almost anybody who understands the idea of random distribution – you certainly wouldn’t need to be an ecologist. And sticking with the species-area relationship – this may be the closest thing we have to a law but when was the last time you saw somebody use a quantitative species-area relationship to predict the number of species in a patch that wasn’t used to build the model (I’m sure it’s happened but it’s rare)? Several other predictions that you assert we can make I don’t have your same level of confidence – mostly because I don’t think I’ve ever seen people make the predictions. For example, I am not at all convinced we can make good predictions of number of trophic levels based on productivity but mostly I’m not sure I’ve seen people try (except in experimental microbial communities). That said, I don’t know this literature very well so I may have missed something.
And ecology should be comfortable with predictions being probabilistic because in the vast majority of disciplines (and possibly all) predictions are always probabilistic. In some fields the probability of a prediction being true may be very high – approaching 1 but it seems to me that predictions are, by and large, by nature probabilistic and so if we’re not comfortable with that, you’re right., we need to get comfortable with it.
In addition, I agree that deciding how to evaluate the predictive ability of models may be a tricky question in some cases and one I barely have any feel for. But, the fact that it will be hard doesn’t change at all the need for doing it.
I believe that ecologists have escaped the demand for assessing predictive ability exactly because people don’t see our findings as critical to their lives. We have a very good understanding of human anatomy because people’s lives depend upon it. We have a very good understanding of aeronautical physics because planes need to get up in the air and stay there until we want them to come down. Where are we making explicit predictions and having to confront our uncertainty ? Climate change because it has such enormous economic and human health consequences. Fisheries because there are large economic consequences (although, Derrick, I’m not as convinced as you that fisheries scientists make very precise and accurate predictions – but, at least, they try). Agricultural productivity because there are economic consequences.
And so after more than 100 years of ecological science what do we really know? Well, they only way to demonstrate what we understand about the natural world is with predictions and we rarely do that. I don’t know what we know…because we don’t predict. Best, Jeff Houlahan

Hans Castorp said...

The species area relationship is much more than saying that more species will occur in a larger are. We know that in first approximation and for not too large a range of areas it is a power law and that the exponent is more or less the same for the same taxonomical group (for plants it is quite consistently 0.23). What Jeff says is that we cannot explain WHY it is a power law and WHY the expontent is that value - but in physics you cannot explain why the force of gravity is proportional to the inverse of the square of distance (why not the inverse of the cube of the distance?) and why the gravitational constant is about 9.8. I feel that the problem is not that we want to imitate engineering. but that we are unfamiliar with the method of exact sciences like physics and chemistry. Our model is not physics, chemistry or engineering, but anatomy, cell physiology, and other descriptive sciences. We want to explain thigns in terms of simple highly reductionistc processes and not in terms of complex abstract things like cynetic energy or chemical potential. Ecologists trained in mathematics or physics like Volterra, McArthur, Tilman and Chesson never complained that ecology was unpredictiv, but instead produced general theories that are often difficult to use in applications - but the same is true of physics, in fact engineering, that is applied physics, is a completely separate field, adn engineering relies, with respect to physics, on simplifications and approximations.

Caroline Tucker said...

Thanks for all the comments! You have all made some great points here. I think, and what I hope got across in the post, that there isn't really a black and white answer to the question of whether "ecology is predictive (or could easily be more predictive)", rather there is some merit to both sides of the argument.