In many ways Radio 4's report 'The State of Welfare', broadcast on 27 November to mark the 70th anniversary of the Beveridge report, showed the BBC at its best. There was something symbolically appropriate about Radio 4 disrupting its morning schedule, a revered national institution in its own right, to devote three whole hours to the social security system that Beveridge founded. The programme was genuinely informative and balanced, drawing on a wide range of views from experts, historians, commentators, politicians and members of the public, including people who rely on benefits.

It may therefore seem churlish to single out one detail of the programme for criticism, but that's what I'm going to do.

At a couple of points in the programme, participants cast doubt on the estimates of benefit fraud produced by the Department for Work and Pensions. The journalist James Batholomew asserted that the figures only captured /detected/ fraud, suggesting that the real level of fraud is unknown; Frank Field also seemed to query these statistics. The BBC is not responsible for views expressed by guests, but at one stage the presenter Julian Worricker also stated that our knowledge of the level of benefit fraud was based on 'anecdote'.

This is simply incorrect. The figures produced by DWP are among the most statistically robust estimates of fraud produced by any government department. You don't need to take my word for this: the following quotation comes from a report by the Centre for Counter Fraud Studies at the University of Portsmouth which is highly critical of the approach to fraud estimation taken by most other government departments: 'DWP provided estimates of losses to fraud involving the benefits which they administer. They estimate for 2008-2009 that £1.1 billion (or 0.8% of total expenditure) was lost to fraud. This is a low figure, but one which, for the most part, is based on statistically valid and accurate loss measurement exercises – something which the DWP should be congratulated for undertaking. The low figure may reflect the length of time that DWP have been tackling this problem.'

Given Mr Bartholomew's assertion that the DWP figures only show cases of detected fraud it is worth noting the kind of practice with which the Centre for Counter Fraud studies is /contrasting/ DWP's approach: 'In some cases organisations simply added up the value of the losses found in cases of detected fraud and assumed that this figure represented the total losses incurred. This conveniently ignored the reality that for the figure to be accurate there would need to be a 100% successful detection rate....' Far from doing this, DWP selects a large random sample of benefit claims every year, carries out benefit review interviews with claimants and uses standard statistical techniques to gross the sample data up to overall estimates, controlling for sample bias and missing information. The sample for the 2011/12 estimates was some 36,300. Anecdote?

Why is this important? The Radio 4 programme returned to the issue of benefit fraud several times- quite rightly given its salience in public attitudes and debate. But in going along with the claim that fraud represents an unknown quantity, the BBC was not only mistaken, it was missing an opportunity to challenge a widespread public misunderstanding which distorts debate on social security.

For our recent report on benefit stigma for the charity Turn2Us Ben Baumberg, Kate Bell and I commissioned a survey from Mori in which we asked people to estimate the level of benefit fraud. Previous surveys, and the survey commissioned by Radio 4 for its programme, have asked similar questions, but we tried to formulate our survey questions to be as unambiguous as possible so that answers would reflect beliefs about fraud rather than less specific negative attitudes towards claimants.Thus while the British Social Attitudes Survey in 2007 asked ‘Out of every 100 people receiving [sickness or disability/unemployment] benefits, how many do you think are falsely claiming the benefits?’; our survey question was: ‘The government releases figures on the amount of “benefit fraud” – where some people deliberately deceive the government, as they would not be entitled to benefits if they told the truth. Out of every 100 people claiming out-of-work benefits how many, if any, would you say, commit fraud in this way?’

Unsurprisingly, the average estimate of fraud in our survey is somewhat lower than that for people 'falsely claiming' in British Social Attitudes: 25% for all out of work benefits compared to BSAS results of 30% for disability benefits and 35% for unemployment benefits. But even 25% is an extraordinary value: DWP estimates that the percentage of claims which are overpaid due to fraud ranges from 2.3% (working age housing benefit) to 4.2% (lone parents on income support). For all income support and Jobseeker's Allowance claims taken together, the estimate is 3.4%. (See table 5 on page 28 of the report here for our and the BSAS results but note that the official estimates of fraud we cite there are based on expenditure rather than caseload, which gives slightly lower figures- caseload figures weren't available at time of drafting).

We also looked at newspaper reporting of social security: some 30% of all articles on working age benefits published in national titles between 1995 and 2011 made reference to benefit fraud. We found a striking relationship between people's estimates of fraud and the amount of coverage of fraud in the newspaper they read. This relationship remains statistically significant after controlling for other factors associated with beliefs about fraud. Our interpretation is that members of the public, including claimants, supplement the limited information on benefit fraud accessible through direct experience with information from the media- and that information is strongly skewed towards fraud.

That the UK public overestimates benefit fraud by an order of magnitude is surely an important aspect of current debate on social security. This finding turns on having something robust to compare public opinion with, and fortunately that is precisely what we have. We don't have to believe that DWP has identified /all/ the fraud in the system: no estimate is perfect and the fraud detection rate for the sample is unlikely to be anything like 100%. But DWP's estimates would have to be out by a factor of seven to match the public's estimate, and that is utterly implausible. We can be very confident that benefit fraud, which exerts such a strong hold over public opinion on social security, is /much/ lower than people generally think. It's unfortunate that the BBC didn't set this straight in a programme which was otherwise well-researched and balanced.