Crusade against sloppy mathematics

Crusade against sloppy mathematics

A London-based Canadian mathematician, Douglas Keenan, has made it his mission to lead the battle against the sloppy or malicious use of mathematics. You might think that there isn't much scope for different opinions in mathematics, but when you're dealing with the interpretation of data, it's entirely possible for divergences of views to arise. The wrong interpretation can be made at times, and even consciously made. In particular, in the field of climate studies, opposing points of view are often backed up by scientific research that is based on the mathematical analysis of data. Because mathematics gives such pieces of work a stamp of credibility, politicians often rely on them. It is therefore all the more important for them to be carried out with care.

After studying mathematics at the University of Waterloo, Keenan worked on Wall Street for a few years, but in 1995 devoted himself completely to the forensic study of mathematics. Since then he has been leading—completely independently—a real crusade against shady mathematical machinations. The targets for his often vigorously worded attacks are numerous, and range from the misuse of statistical methods in determining the origin of volcanic ashes to the questionable use of tree-rings in evaluating the date of a shipwreck.

Three years ago, the scientific journal Nature published a study that used the ripening process of Pinot Noir grapes as an indicator for the warmth of the climate. The official start of the harvest in August is determined by the ripeness of the grapes, which in turn is determined by the temperature of the summer that has just ended. Since the dates for the beginning of the harvest in Burgundy have been recorded in city archives since 1370, they could conceivably be used as indicators for the way temperatures have developed over the past six centuries. A French research team came up with a model based on this data. The model showed that the summer of 2003 was the hottest in 600 years. The conclusion was clear: Burgundy is warming up.

The work aroused Keenan's suspicion, and he wanted to test its mathematical foundations. In order to do this however, he needed the raw data—but the authors were not prepared to hand it out. It was only after two requests to Nature that they finally handed their documents over. Keenan immediately made a find. The authors had smoothed the data for their study, confused standard errors with standard deviations, used incorrect parameters, and confused daily temperatures with average temperatures. Once all these sources of error are taken into account, the year 2003 does indeed display high temperatures, but not unexpectedly high ones. It's no surprise that the Nature editors hadn't noticed anything, since the data was never put at their disposal, and they never asked for it either. Had they done so, they would easily have seen through the authors' game. The mere fact that the grape harvest model gave a temperature for 2003 that was 2.4 degrees Celsius above the temperature actually measured by Météo France should have made the editors suspicious.

Keenan's most recent targets are two pieces of work that examine the influence of urbanization on climate change between 1954 and 1983. In order to be able to compare measurements made over different periods, it is absolutely crucial that the location of the station where the measurements are carried out not change throughout the observation period. For example, because a city generates warmth, a measuring station that is moved from the center of the city to its periphery would record lower measurements. On the other hand, the measurements would be more likely to rise if a measuring station was moved from a position upwind from the city to a position downwind. Even small changes of location, like for example from a field to the asphalt road next to it, lead to deviations. Keenan was above all doubtful about the measurements made in China. He didn't believe that during Mao's Cultural Revolution, when scientists were thought very little of, a scientific study would have been carried out with much care.

When he asked which stations had been used to make the measurements, Keenan once again found himself running into a brick wall. “Why should I make the data available to you, when your aim is to try and find something wrong with it?”, asked one of the authors. But the professor had not reckoned with Keenan's obstinacy. Since the professor was working at a university in England, he was subject to the Freedom of Information Act, which obliges employees of public institutions to release data. He was thus forced to hand over the list of the Chinese measuring stations to Keenan. And lo and behold: out of 35 measuring stations, 25 had been subjected to a change of location, sometimes even several changes, which often covered dozens of kilometers. For a further 49 measuring stations, documentation did not even exist.

g.szpiro@nzz.ch