Facing Up to Reality

A More Realistic Census Methodology

New Zealand census data is already out of date.  The census was held last year; the results and data will not now be published until sometime after April next  year.  You may ask why census data is so important?  One of the main reasons is that it is regarded as a long deep vein of pure gold.  And not without good reason.  It is one of the most influential institutions affecting and governing how the government spends tax payers' money.  

The adage is that "bad data" results in wasteful, inappropriate spending. Fair enough.  We get the point.  But now we are being told the data from the most recent census (2017) is already unreliable and it hasn't even been published yet.

Work on the first release of 2018 Census data is taking longer than expected due to the complex nature of the task, Stats NZ said today.  Stats NZ had previously reported that they had full or partial information for around 90 percent of individuals and indicated further analysis would be required that would delay the first release.

“We now know this is going to take us a bit longer, but by April 2019 we will be able to announce when census data will be available,” Government Statistician Liz MacPherson said.  “I would like to thank the public for their patience. I know how important census data is to so many people and I apologise for the inconvenience caused by the further delay. It is precisely because the information is so important that we need to take the additional time.”  [Stats NZ]

Apparently statistical data on 90% of the population is not complete enough.
  It is regarded as unreliable.  A question is begged at this point.  Why was the most recent census unreliable?  Why was data collected on only 90 percent of the population?  Well, Stats NZ decided that it would move to a predominantly electronic (via the Net) data capture, rather than the old method of manually delivering census papers at the front door, filling them in by hand, then having them collected by census workers knocking on every door or makeshift dwelling in the country.

However anyone who has ever worked on the recording and collection of census data would tell you that there was always a proportion of the population that avoided being counted, for whatever reason.  Moreover, the more complex society became, on the one hand, and the more ghettos developed, the more unreliable census data has become.

We have personally experienced the problem.  Attempting to get reliable census data from "no go" streets was impossible ten years or so ago.  Every second house contained adults (and likely children) who were permanently inebriated or befuddled with drugs.  If someone came to the door they would stare vacantly, and may or may not take the census papers.  Collecting them a couple of week later filled in and completed was a pipe dream.

And then there were the "badlands" areas pretty much controlled by gangs.  We personally witnessed one collector bitten by dogs (there were three on the property).  Eventually a council dog handler and the police were called.  The handler later opined that such dogs are bred for one purpose only--a savage defence of the "owner".  All the dogs were eventually put down.  But (unsurprisingly) no census papers were ever received from that dwelling.

The first electronic census only got 90 percent of the population, we are told.  We believe that is good going.  We believe the census model needs to change so that estimates for difficult areas ought to be acceptable.   The Government wastes billions of dollars now.  It matters not much at all if a few more dollars go the way of the dodo, does it?  What's a bit of waste among friends?

It seems as though Stats NZ has reached the point of reasonableness.  It has conceded defeat on the goal of 99 percent participation.  Now it has acknowledged that 90% participation is a jolly good result.  It will now augment and adjust and massage the baseline result with appropriate "other data".

Stats NZ is developing and extending a range of statistical methods to adjust for limitations in the data collected and to understand the quality of the results produced. This includes using other government data about real people (such as births, tax, health, and education records) to compensate for the information it is missing.

“We know we have good coverage of the population from the wide range of government data we hold, and we can use that data to build a more complete picture,” Ms MacPherson said.  However, Stats NZ's research shows that government data has strengths and weaknesses. There are some gaps and inconsistencies in what other data sources can tell us, for example, who lives together in a household, and Māori descent and iwi information.

“We are exploring how to address this issue and we need more time to evaluate the approaches we are developing. Where there are gaps that may be more difficult to fill, we are also considering other ways to find the information users need with complementary strategies.”

That seems to us to be the most appropriate way forward.   Total survey coverage was always a pipe-dream.  As society becomes more complex, total survey coverage has become more and more imaginary.

To put it bluntly, ghettos do not deliver sound, accurate door-knock census data.