Large families, MPs and Hume’s test

[Testing some assertions about those families with 'eight, nine or ten kids' we hear so much about these days. See the attached PDF for table, references and additional pedantry. Caveat lector: the figures in this article for large families by constituency are NOT estimates, they are intended to provide probabilistic anchors for judging the scale of the phenomena in question (with numbers this small, local figures would risk disclosing details of identifiable individuals, which is one reason they don't exist). Data is from Work and Pensions Longitudinal Study via Nomis and DWP Freedom of Information response 2012-3222 http://statistics.dwp.gov.uk/asd/PQ_FOI/foi/2012/foi_3222_2012.pdf ]

‘This is what my constituents tell me’ is one of the rhetorical boltholes MP’s rush for when their assertions are called into question- arguing, in effect, that ‘It’s the people I represent who are saying this so I don’t need to provide any evidence to back it up ’.

There was a good example of this on BBC1’s Sunday Politics last weekend, when the Conservative MP for Shipley Philip Davies and Kate Bell from the Child Poverty Action Group discussed the latest proposals to cut child benefit for out-of-work families with more than two children. Kate stated that there was no evidence that people on benefits were choosing to have more children, and the presenter challenged Mr Davies to provide the evidence. First the MP tried appealing to stories in the popular press (which even its greatest fans would hesitate to describe as an authoritative source on these matters): then he played his ’get-out-of-jail –free’ card.

Andrew Neil: ‘Where’s the evidence?’ Phil Davies ‘We do see this happening, I mean we read reports in the papers of families with eight, nine ten kids who are all on benefits expecting to be housed in bigger and bigger houses. My constituents come to me in my surgery and talk about these people living next door to them, living down the same street as them that they object to, and it’s building up huge resentments among many working class voters who actually are going out, doing the right thing making sensible responsible decisions ...’(and so on).

Now it’s pretty clear that somebody has got it wrong here: whether Mr Davies’ constituents or Mr Davies himself, it’s impossible to say. There is no reason to doubt that some of Mr Davies’s constituents complain about families with ‘eight, nine or ten kids’ living on benefits. But he says that these constituents are complaining about families of this size /living on the same street as them/, and that’s a different matter, because there is no reason to believe that there is any significant number of such families in his constituency.

This is because there are very few families on benefits with eight or more children- about 1,600 in all of Great Britain - and there are no obvious grounds for expecting there to be a high concentration of these statistically rare families in Shipley. Based on the number of families with children claiming benefits in Shipley and the national frequency of families of this size among claimant households with children the expected number of claimant families with eight children in Shipley is one – that’s right,one,1, n divided by n, 3 minus 2, unity- and the expected number of claimant families with more than eight children is zero. (To be spuriously precise, it is 0.46, but as a number less than one doesn’t make a great deal of sense in this context, I’m calling it zero.)

Of course these are simply expectations based on limited data, and random variation alone could well lead to the number of large families on benefits being substantially higher (if not lower) than expected in Mr Davies’s constituency. That might explain the anomaly in a single case. But Mr Davies does not seem to saying that he has an unusual cluster of these families (or unusually ill-informed constituents) : he clearly believes that this is a general pattern, and he is far from being the only MP to appeal to constituents’ concerns in this area. The Prime Minister himself has asked ‘Are constituents working hard to give benefits so people can live in homes that they can only dream of?’ http://www.guardian.co.uk/politics/2010/oct/27/government-committed-to-h... Indeed one of the main arguments offered for policies such as the benefit cap has been that they address concerns spontaneously raised with MP’s by constituents on the basis of their own personal experience.

What is the likelihood that large numbers of constituents have well-founded concerns about large families on benefits in their area? Put another way, the average adult population for a UK parliamentary constituency is about 80,000: how does this compare to the likely number of large families on benefits in each constituency? Using the same data as above we can calculate a prior probability as to whether the number of benefit families with a given number of children in a constituency exceeds a certain level- more than one, more than 100 and so on. Think of this as an aid to judging, in the event that a constituent raises this as an issue, how likely it is that their complaint has a basis in fact rather than hearsay; or if an MP offers constituents’ complaints as evidence of how serious the problem is, how much credence should be placed in that evidence.

Based on these calculations, there is a very high probability that any constituency has more than ten families on benefits with five children but there is a very low probability that it contains more than 80 (0.03, or a 97% chance against). There is about an even chance a constituency will have more than ten families on benefits with six children, but a low chance it will have more than 20 (0.13, or a 67% chance against). For families with seven children, there is a 74% chance against there being more than five (yes, 5) in the constituency. For larger families again, the probability of more than five is vanishingly small: this is because once we reach eight children there simply aren’t enough such families to allow more than one in most constituencies, while once we reach nine children there are far fewer families than constituencies, so any constituency is more likely to have no such families than to have even one.

David Hume gave a rule for how we should judge the credibility of an assertion which on the face of it has a low probability of being true: we should believe it only if we believe the probability that the person making the assertion might be incorrect is even lower than the probability of the statement. I don’t think the claim that Mr Davies’ or any other MP's constituents are worked up about claimant families with ‘eight, nine, ten kids’ living on their street comes anywhere near passing Hume’s test.

It’s not unlikely that some constituents have raised complaints about large families on benefits with Mr Davies and other MPs. What I don’t believe is that MPs are hearing large numbers of complaints /grounded in personal experience/, as they like to claim. Rather, constituents are expressing grievances based on what they read in the popular press and what they hear from politicians like Mr Davies. When an MP claims to be just repeating what his constituents are saying, a version of Hume’s test should be brought to bear: which is more improbable- that the MP has some solid evidence to support his or her assertions which for some reason he or she has decided not to present, or that there is in fact no such evidence and the MP is just trying to play the ‘get-out-of-jail free’ card?