What can we learn from the 2018 Census debacle?

By Brian Easton.

Analysis - The publication of the report reviewing the 2018 Population Census and the resignation of the Government Statistician who presided over the disaster is the beginning, not the end, of a discussion on the role of statistics and the state sector in New Zealand.

A senior member of the State Services Commission once told me that they thought the SSC got their appointments of chief executives to government agencies about right 60 percent of the time. Presumably it was a high standard, but little thought has been given to dealing with CEs who fail after the appointment.

Before the 1988 State Sector Act, the SSC was also involved in the appointment of the second tier of management. When a CE drifted into dementia, the SSC was able to appoint a strong number two to cover for him. Today, with the CE making all the appointments, there are well-known cases of incompetent CEs appointing a second level of similar inadequacy.

What one does about a seriously failing CE is more fraught. The Government Statistician has admitted her failure and resigned. But less than a year earlier her term was renewed by the SSC. The problems documented by the Report of the Independent Review of New Zealand's 2018 Census were already largely known at that time albeit not in as much detail.

Perhaps as instructive is that the Government Statistician's term has been extended to the end of December despite her resignation and the documented failings. Many were astonished. The big issue is to recover public trust in Statistics New Zealand. At the very best this extension delays the recovery four months, but more likely it will compound the public mistrust.

Probably the SSC had no choice. The implication of the census review report was that there was a failure by the entire executive leadership team so that no member of it was suitable for temporary promotion. The usual action would have been to appoint an SSC deputy commissioner, but the job is so technical that none could have got up to speed in a short time.

So the release of the first tranche of the synthetic census data (that is the collected census data augmented by other data sources) will be fronted by a Government Statistician who has been found to have failed and who has admitted her culpability.

What trust will the public give to that? What it needs to understand is that whatever the administrative flaws, SNZ has a fine professional team of qualified statisticians who will deliver quality statistics very aware of, and documenting, the problems in the data. Their standards are underpinned by the international statistics profession, with whom there is constant interchange.

I have looked at the background material already released on the construction of the synthetic census data. As a professional social statistician I judge it very satisfactory. I am tempted to write 'impressive', but as the 2018 Census review report comments there has been a tendency to over-promise and underperform: 'e find a level of optimism in the reporting to ministers that was not always consistent with the level of issues being managed by the programme.' (A nice example of that optimism is that not long ago we were promised the census data release this September. Now we are advised that there will be only a partial release with another tranche in December, and the final release sometime next year.)

So before one gives a final tick to the augmented census results, let us wait to see them robustly tested by independent, professional statisticians. I could not help thinking that some ministers never learn. The Minister of Statistics admits to having been misled earlier, but is once more confidently affirming how good the data will be without the cautious caveat 'my advice is that'.

Despite this reservation, I repeat my affirmation that I am impressed by the quality of the work by the SNZ's team of professional statisticians working on the census. Over eight decades, Government Statisticians with professional statistics backgrounds have fostered the development of their staff. It is these professionals who will do the remediation of the 2018 Census failures, despite the rhetoric of the Minister and the State Services Commissioner which focuses on the executive leadership team.

Hardly anyone has noticed the telling recommendation in the 2018 Census Review report that the Chief Methodologist - an ungainly title for SNZ's senior professional statistician - should be added to the Executive Leadership Team. Under the previous Government Statistician he had been a Deputy Government Statistician but had been demoted to the third level. That is right - in the current Statistics New Zealand there are no professional statisticians in the top two tiers of management.

This is characteristic of generic managers with their typical preference for distancing professionals from management. The SSC was unwise to appoint a generic manager to such a skilled job; I have wondered whether the advisory committee which assisted the State Service Commissioner to make the appointment of the current Government Statistician had any professional statistician on it or whether it, too, was stacked with generic managers. If the latter, it is most unfortunate that the individual appointed to one of the most technical jobs in the state services was a person who, as the 2018 Census review clearly demonstrates, did not have the requisite skills to do it. That is well demonstrated by the need to extend the tenure of the current Government Statistician until we get the next one who, hopefully will, like the previous nine, be a professional statistician who is alert to the issues which the current Executive Leadership Team fumbled. It is extraordinary that it embarked on the major task of introducing a new approach to collecting census records with so little expertise or experience. Such instances are not unique.

As far as I can assess, the integrity of the nation's data base has not been compromised and, one hopes, may even eventually be enhanced. But the public's trust in it has been badly damaged and will not be quickly remedied. The State Services Commissioner promised that trust will return. If he expects that it will return quickly he is over-promising. My guess is that with proper nursing it will take ten years for SNZ to recover full public trust; roughly the period it took the Canadian equivalent to recover from a similar failure. There is going to be considerable unrest in the short term - as ongoing, and some not unjustified, criticism from the National Opposition and Māori illustrates. Public perceptions of the 2023 Census** will echo current concerns; providing it is run successfully, but the trust should settle down for the 2028 census. Sadly, in the interim, concerns about the census will undermine public trust in the entire SNZ data base, even though they are not justified.

There are two big takeaways from this debacle. The first is that it would be unwise to see this as an isolated failure in a state bureaucracy that is otherwise functioning well. Rather, it is a prominent example of a systemic failure (although there are successes, of course) which I do not see being addressed in the review of the State Sector Act.

Second, it is a timely reminder of the importance of a quality data base. Goethe said that he did not know whether statistics ruled, but they did tell us whether we were well ruled.

* This article first appeared on Pundit.

** I favoured bringing the next census forward to 2021. I accept the judgement of the review team that given the weaknesses in the current SNZ executive leadership a quality one could not be delivered at that date. My judgement was based upon the fact that the previous leadership team delivered a good-quality one in 2013 with only a two-year lead time.