Sunday, December 06, 2009

Garbage in, Garbage out

Even the BBC is coming to realize that the information revealed by the Climategate leaks raises serious questions about the data inputting problems used by the community of global warming alarmists at East Anglia University. What is clear is that there were major problems writing the code and that even those who aren't skeptical of human-caused climate change are deeply skeptical about the computer programming that went in to generating the data that was used to reach CRU's conclusions.

Link via Newsbusters

The report highlights just the criticisms that global warming skeptics have themselves been highlighting: the lack of clean data and explanations of adaptations made in the data. As the expert points out, this isn't the sort of coding you'd expect to see in source code from private business.
"The programming language actually has a problem," Graham-Cumming said. "And they put in some code to deal with that error. Unfortunately, in doing so they produced another error. And the upshot of this is the error occurs - the underlying error, they will skip over data that they're trying to plot without any warning to the end user. So in some sense there is data that is being lost."
And the reporter asks the key question: would you feel comfortable making decisions worth billions, even trillions of dollars, based on this code? And the answer is clear. Absolutely not.

That is why the British Meteorological Office is beginning a review of all its data and is asking all the nations involved for permission to release the raw data.
THE British Meteorological Office is to launch a review of its temperature data and has asked 188 nations - including Australia - for permission to release raw weather data in the wake of the so-called ''Climate-gate'' email scandal.

The investigation of temperature and global weather information by the Meteorological Office is significant because its database is one of three main sources of the temperature analyses that the United Nations climate change science body relied on for its assessment that global warming posed a major threat to world safety and wellbeing.

The decision comes in the wake of the theft - and publication on the internet - of thousands of emails and text files from the University of East Anglia's Climate Research Unit (CRU).

The emails, many of them written by its director, Professor Phil Jones, appeared to suggest that there were attempts to manipulate temperature data and to stymie the public release of information on raw data. The university has announced an investigation and Professor Jones, who denies the claims as ''rubbish'', has stood down while the inquiry takes place.

The Age reported on Saturday that a 247-page text file by one of the university's most senior computer programmers has also revealed frustration and anxiety about the integrity of the raw data provided from weather stations around the world and that Australian data came in for particular criticism.

The programmer found the Australian weather data to be riddled with entry errors, duplication and inaccuracies and described as a ''bloody mess'' attempts to homogenise information and entries.

In his text file, the programmer documents his attempts to make sense of the reams of raw historical data that was used by the CRU. He railed that the global information he has to work with has ''no uniform integrity''.
As AJ Strata points out, in the real world where decisions have to be made on the basis of what scientists and programmers produce, there is no room for programs written with poor codes that have "no uniform integrity." The results would be too catastrophic. Strata delineates a difference between what he derisively calls PhD code and NASA code.
There is a well known problem when research scientists have to interact with the real world: they hate to be bothered with rules, regulations, laws, etc. At NASA the lowest quality code you will find is in the science processing chain. It is just not held to the same quality standards as the operational code the runs all the flight and ground HW.

For example: you can’t have PhD quality code launching massive rockets over this countries large cities running up the eastern coast. You can’t have PhD level code controlling large antennas, since you don’t want them be destroyed as you move these massive machines or fry someone by turning them on when people are working on them. You don’t want PhD quality code landing the Space Shuttle. It’s that simple and no NASA PhD will argue that point.

The truth is the science teams don’t get enough funds to do it right, but that is only half the problem. The other half is the scientists like to write crappy code only they can use – creates a lot of job security. For much of science this is a livable and reasonable arrangement. Let the PhD’s dabble in exploring the unknown, and leave the designing, operating and safety of large complex systems (which can kill lots of people if things go wrong) to lesser people – like engineers.

When the global warming canard migrated from niche research into trillions of dollars of policy changes effecting every human being on the planet, the PhD level of quality control should have been ejected immediately. With the fate of humanity at risk, it is not too much to ask for professional quality code, analysis, and a true peer review process. Not that silly science journal review process by the good ol’ PhDs network, a real review like we do when we launch people into space or build a rail system or a new airplane.

How delicious that all these politicians are going to be meeting in Copenhagen to discuss climate change and what sort of statement they can issue and promises they can make that will sound good but not totally tank their economies just at the moment when the shoddiness of the research underlying the whole climate change industry has been brought to light. Let's have that analysis by the British Meteorological Office looking at all their data gathering and at the coding done in the computer programs. Until they can vouch for the integrity of both their data and their coding, let's hold off on vast economic changes that will affect the world's economy for decades. As George Will writes,
China, nimble at the politics of pretending that is characteristic of climate-change theater, promises only to reduce its "carbon intensity" -- carbon emissions per unit of production. So China's emissions will rise.

Barack Obama, understanding the histrionics required in climate-change debates, promises that U.S. emissions in 2050 will be 83 percent below 2005 levels. If so, 2050 emissions will equal those in 1910, when there were 92 million Americans. But there will be 420 million Americans in 2050, so Obama's promise means that per capita emissions then will be about what they were in 1875. That. Will. Not. Happen.
Let the politicians make their fancy speeches, which is what they're good for, but no country in its right mind should make the kind of economic decisions that Al Gore and his allies have been advocating for over a decade until they can prove that the whole scare isn't based on garbage data and garbage programming.

The global change true believers have long been outraged at the temerity of anyone challenging their apocalyptic visions. They likened critics to Holocaust deniers as if asking questions about research that tries to reconstruct past temperatures for a millennium while also projecting temperatures for a century into the future is equivalent to questioning something that is a historic fact in many people's own lifetime. George Will rightly compares the climate change activists to true believers in a religion who wish to excommunicate those who don't ascribe to the right doctrine.
The travesty is the intellectual arrogance of the authors of climate-change models partially based on the problematic practice of reconstructing long-term prior climate changes. On such models we are supposed to wager trillions of dollars -- and substantially diminished freedom.

Some climate scientists compound their delusions of intellectual adequacy with messiah complexes. They seem to suppose themselves a small clerisy entrusted with the most urgent truth ever discovered. On it, and hence on them, the planet's fate depends. So some of them consider it virtuous to embroider facts, exaggerate certitudes, suppress inconvenient data, and manipulate the peer-review process to suppress scholarly dissent and, above all, to declare that the debate is over.

Consider the sociology of science, the push and pull of interests, incentives, appetites and passions. Governments' attempts to manipulate Earth's temperature now comprise one of the world's largest industries. Tens of billions of dollars are being dispensed, as by the U.S. Energy Department, which has suddenly become, in effect, a huge venture capital operation, speculating in green technologies. Political, commercial, academic and journalistic prestige and advancement can be contingent on not disrupting the (postulated) consensus that is propelling the gigantic and fabulously lucrative industry of combating global warming.

Copenhagen is the culmination of the post-Kyoto maneuvering by people determined to fix the world's climate by breaking the world's -- especially America's -- population to the saddle of ever-more-minute supervision by governments. But Copenhagen also is prologue for the 2010 climate change summit in Mexico City, which will be planet Earth's last chance, until the next one.
The problems revealed in that CRU source code underline the differences between true science and religion. In science, honest scientists embrace the scientific method and accept that other scientists want to use their data to try to replicate their results. Willis Eschenbach's post at Watts Up With That reveals how the researchers at CRU were scrambling and stonewalling to avoid doing exactly that.
Science works by one person making a claim, and backing it up with the data and methods that they used to make the claim. Other scientists then attack the claim by (among other things) trying to replicate the first scientist’s work. If they can’t replicate it, it doesn’t stand. So blocking the FOIA allowed Phil Jones to claim that his temperature record (HadCRUT3) was valid science.

This is not just trivial gamesmanship, this is central to the very idea of scientific inquiry. This is an attack on the heart of science, by keeping people who disagree with you from ever checking your work and seeing if your math is correct.
If CRU wasn't interested in true science, then the conclusion is clear - no decision based on their research should be made by any policy-maker. The irony is that President Obama and his supporters all trumpeted how they would be making their decisions based on science, but now they're all averting their eyes to what Climategate has made obvious - the science behind the United Nations' claims about global warming was done by a bunch of true believers who felt they had dispensation to ignore the rules of the scientific method simply because their motives were so pure.