Imagine a 24-hour carpark which serves two mutually exclusive categories of customer, both of whom have obsessively regular patterns of usage. Those in the first category always stay for exactly one hour, while those in the second category always stay for exactly four hours. Each of the two groups account for exactly half the demand for parking spaces throughout the day: whenever a new car enters the carpark there’s a 50:50 chance of it falling into either category. To simplify things, lets assume that cars arrive at and leave the carpark on the hour.

During the first hour half of the cars in the carpark belong to one hour customers and half to four hour customers. At the end of the first hour the first cohort of one hour customers take their cars away while the cars of the four hour customers stay put. The incoming cohort is divided 50:50 between one-hour and four-hour customers, so the share of spaces taken up by four hour customers rises. Thus, during the second hour the split is two thirds/one third in favour of the four-hour customers. In the third hour, the four-hour customers account for three quarters of the occupied spaces and in the fourth hour for four fifths. This is the equilibrium position: at the end of the fourth hour the first cohort of four-hour customers take their cars away and from now on the hourly outflows of both categories balance the inflows and the ratio stays at 80:20 in favour of four hour customers.

So what proportion of the cars using the carpark over this period belong to four hour customers? We already know the answer, because we’ve set things up so that exactly half of the cars using the carpark belong to four hour customers. But if we were to take a snapshot of usage at any point other than in the first hour, the proportion of four-hour customers would always be more than 50%. A series of random observations over the 24 hours would converge towards a value of 74%. Thus if we tried to infer the pattern of demand for parking spaces from a snapshot, or even a sequence of snapshots, we would end up with massive error.

This simple model could be just as well instantiated by other types of behaviour: the carpark over the course of a day could be a hotel over the course of a year, or an out-of-work benefit where clients tend to have different durations on benefit, or just about anything where the amount of time people remain in a particular state varies. It can easily be expanded to allow for any number of different patterns of usage, with customers staying for any intervals we choose. Nor is it strictly necessary that individuals enter the state in question with a pre-existing disposition to a particular length of stay: external factors might come into play. It makes no difference if longer stay customers originally intended to stay for a shorter period but got stranded somewhere because of adverse weather conditions. The point remains that observations at points in time will understimate the proportion of shorter-stay car-park customers, hotel guests and out-of-work benefit claimants, whatever the causes of differences in stays, and will overstate -in some cases, vastly- the proportion of long-stay customers, guests, or claimants. And the error will always be in the same direction: short stay customers, guests or claimants will never be overestimated.

Politicians consistently push a view of the benefit system which rests on exactly the same arithmetical error as would be involved in trying to gauge the demand for long-term parking places from a point-in-time observation. It is certainly the case that long-term benefit claims account for a high proportion of live claims right now. But figures on the percentage of claims which are long term now are artefacts arising from the arithmetic of differential lengths of benefit claim.Taken in isolation, they have absolutely no significance when it comes to assessing welfare policies, because any number of possible combinations of claim durations could underlie these point in time estimates. Anyone who thinks that the fact that X% of claimants of a particular benefit have been claiming for N years tells us anything about work incentives or welfare dependency is, in effect, missing all the one hour customers moving in and out of the system. Conclusions on the incentive or dependency effects of a particular benefit could only be justified by evidence based on all the flows of claimants over an extended timeframe. They rarely are.

A topical example comes from some data released in a response to a parliamentary question by Stephen Timms http://www.publications.parliament.uk/pa/cm201212/cmhansrd/cm120206/text... this week. The government's household benefit cap is justified at least in part as a means of tackling long-term benefit dependency arising from excessive payments. Let's ignore the question of whether the payments are excessive and ask: what percentage of the very small number of people who receive these high levels of benefit are long-term claimants? The impact assessment for the cap stated that 'a majority' had been claiming for two years or more: the full data was released in the PQ response and indeed shows that 55% had been claiming for two years or more. That is extraordinarily close to the percentage of all out-of-work benefit claimants who have been claiming for two years or more, whatever their level of benefit (53%, from Nomis, May 2011 WPLS data) so there's no basis for any conclusions about the effect of the level of payment on claim duration. Very long term claims (>=5 years) actually constitute a smaller share of the benefit cap claimants than of all out-of-work claimants.

But as we've seen, these figures overstate the percentage of all claimants who have longer durations on benefit in any case. In the absence of flows data we can't estimate the duration distribution for all the households that receive the cap level (£26k pro rata, including housing benefit) over an extended period (say, a year). But we can make some inferences.We can be very confident that the majority over the course of a year will have claim durations of less than two years, because the majority with these durations in the point-in-time estimate is already very slender, and once we start counting the shorter duration claims properly, it will vanish very quickly. (To be precise, the majority will have had claim durations of less than two years by the end of the time-period we are looking at, measuring duration from the start of the claim of course, not the start of the time period.) And it is quite possible that the majority will have durations of less than one year, because an awful lot of people make a short-term benefit claim in the course of a year. There are a lot of one-hour customers.

We can push the inferences a bit further. We know that very few households at any point in time receive £26k pro rata, or anything like it, in benefits (government estimates 67,000 by the time the cap comes in.) But over a period of time, the number who receive this level pro rata will be a lot larger than this, while still constituting a tiny minority of claimants over the same period. Most of these households will claim for relatively short durations, because they are unemployed or temporarily incapacitated, and then move back into work. In many cases, they will never receive £26k (quite apart from the fact that they won't even see the housing benefit element) because their claim won't run for a year. Many of them will have been in receipt of substantial in-work benefits while employed. For these households, the effect of the cap will be to exacerbate the impact of unemployment.or incapacity on income: they will see a bigger fall in their After Housing Costs income on losing work than households with fewer children or lower housing costs. But the cap won't prevent them falling into long term 'benefit dependency' because like the great majority of people who make a benefit claim, they weren't going to in the first place.

That's a lot of inferences from a small amount of data and a toy model of the benefit system. Some of the inferences may turn out to be wrong. Maybe people who receive very high benefit payments do act differently to other claimants, and the government just hasn't got round to publishing the evidence.The evidence it has provided (and it had to be extracted via a parliamentary question) doesn't show this, but a more detailed analysis controlling for individual characteristics and local labour market conditions might change the picture. I doubt any such analysis has been carried out though, as the cap is not a serious piece of welfare reform. The point here is not to waste time on arguing against the cap but to illustrate the importance of looking at benefit caseloads over time, counting all the short-stay customers as well as the long-stay ones who dominate the point-in-time snapshots, public perception and political debate. Once we do that, things that seemed obvious start to look anything but. It's obvious that if people can collect £26k in out-of-work benefits that they'll be less inclined to return to work. But what about all the people who claim this amount pro rata and do return to work? Once you start counting them, the blindingly obvious becomes just another theory, and one that isn't supported by the evidence. It's obvious the car-park is mainly serving four hour customers, because that's what you find when you do a count. Your mistake was counting the wrong thing. Once we start counting the right things, the world becomes a much more interesting place. Even the world of labour market analysis and welfare reform....