Created: 21 Nov 2010 10:53
Updated: 21 Nov 2010 20:01

Despite still being in its planning stage, the BBC's Weather Test project has highlighted the challenges faced by forecasters in meeting the demands of the audiences that they seek to enlighten.

The project aims to determine "Who can we trust to forecast the weather". But reportedly, the project steering group, made up of representatives from the Royal Meteorological Society, the Royal Astronomical Society and the Royal Statistical Society is bogged down over whether it can assess probabilistic forecasts or deterministic forecasts or both.

Meanwhile, some members of the public told an open meeting that they favour "usefulness", ie practical advice, over "accurate" forecasts.

BBC News' environment reporter Roger Harrabin embarked upon the quest to compare the accuracy of several forecasters using different forecasting techniques after the UK Met Office's forecast for a 'barbecue summer' proved "spectacularly wrong". He soon found there were no comparative statistics on the accuracy of forecasts, so enlisted experts from the three royal societies. 

To date the steering group has met twice and the public meeting "was one of the most constructive" Harrabin said he'd ever attended. Nonetheless, he acknowledges "the possibility of failure of the project overall still exists". 

The steering group has been working to define which weather forecasts it could test; daily, monthly, seasonal and/or regional, and how that could be done in practice. For instance, the steering group proposed testing daily forecasts that use data from the weather station at Heathrow Airport. But some attendees of the public meeting objected because that site has "too much concrete" so the forecasts would not be accurate.

The debate over probabilistic and deterministic forecasts is particularly contentious. The steering group wants to test probabilistic forecasts, and has devised a testing regime using notional "tokens" for different aspects; temperature, rainfall etc, of the forecast that allows some hedging of bets over days when the weather is less or more difficult to predict.

But the BBC provides only deterministic forecasts because it "does not believe the public is ready for percentage likelihoods on getting drenched". In the US, broadcasters offer probabilistic forecasts routinely. But the BBC appears concerned that, in the UK, statistical literacy is an issue.

As the project progresses, the BBC is keen to get more people involved, and has posted a form online to contact the Weather Test project team.
The site is

Also of interest with regard to weather forecasting is Frank Singleton's website.  Frank has observed the weather from above as a senior Met Office and RAF forecaster,  meteorological researcher, and from below as a yachtsman having sailed over 40,000 miles with his wife, Jennifer, in their HR 34.  He sits on the MCA Marine Safety Information Group by invitation of the CYCC and the RYA.