Reviewers suggest delaying next census ahead of big decision

Independent reviewers of the census have suggested pushing out the next iteration to 2030 or 2031, as the national statistics agency grapples with a potential new approach.

The statutory review of the 2023 census, published last week, noted the census was on track to exceed its target coverage rate of 98 percent. However, the general population response rate was only 88.3 percent, below the 90 percent target. Response targets for Māori, Pacific peoples and those aged 15 to 29 years will also be missed.

(What isn’t clear is the proportion of people who only partially completed their forms.)

New Zealand is the first among major English-speaking countries to combine a traditional census, through field operations and digital collection, with administrative data from other records, such as those for tax, health, and driver licences.

This past week, Stats NZ chief executive Mark Sowden told RNZ that census operations would “probably disappear over time”, and data collected through forms would become less important.

But reviewers Murray Jack and Geoff Bowlby said moving from a traditional census to administrative data would involve a large and complex work programme, and great care and caution should be taken.

(Jack, the former Deloitte NZ chairman, reviewed the disastrous 2018 census, while Bowlby is a director of General Statistics Canada).

“While administrative data availability has grown considerably and techniques are improving, the move to an administrative census should only occur if it can produce data which are of similar quality or better [original emphasis], relative to the traditional or hybrid methods.

“Safe and ethical methods or practices for the intake, use, and storage of large-scale records from various programmes and sources need to be in place.

“Also, the statistical agency needs to have social licence to access, combine, use, and store these data – in other words, statistical agencies must have the trust of the public and be able to sustain that trust, to conduct an administrative census.”

(The review noted there was a significant increase in the proportion of individuals, found in administrative data, to be “placed” in dwellings.)

A decision to move to an administrative census would make it difficult to put in place contingencies should it fail, the review said. Another difficulty is Stats NZ’s “significant organisational transformation”. (Thirty-nine job losses were announced this past December, with further changes possible.)

Under an administrative data-first census, some attributes, for “small domain estimates”, may have reduced data quality, and there might be difficulties interpreting data aggregated over several years.

Considerable research has already gone into the shape of the future census. A decision is expected to be made “mid-2024”, after which there would be broader public consultation.

Transparent consultation is a pre-condition, and collaboration with Māori is critical, the reviewers said.

“It is not clear that the timeframes for decision-making will allow sufficient time for meaningful consultation,” the report said. “Stats NZ may want to consider the advantages of timing its next census to be in-line with other major nations, whose census days are on years that end with 0 or 1.”

Census information is used to set electoral boundaries, to determine where to build new schools and hospitals, the requirements for new infrastructure investments, as well as in private and public sector research.

What did the public get for an extra $188m?

The 2023 census is expected to cost $316.3 million, up from $128m in 2018.

One of the review report’s major findings was response rates for Māori and Pasifika fell short of expectations, despite increased focus and investment.

As mentioned, the general population response rate is, right now, below the 90 percent target. Response targets were also 90 percent for Māori, Pacific peoples, Asian, and 15 to 29 year olds, but they were, respectively, 76.7 percent, 79.9 percent, 91.5 percent, and 84.8 percent.

Māori response rates have been below targets in successive censuses.

After the 2018 census, independent reviewers and an external data quality panel recommended deepening and strengthening the relationship with Māori, through “governance arrangements, early engagement and co-design”.

Reviewers Jack and Bowlby said there was a strong “relationship focus” at senior levels of Stats NZ, but within the census programme “the relationship defaults to a transactional rather than a partnership one”.

“2023 census, whilst a significant step up from prior censuses, did not achieve the level of partnership and co-design expected by Māori,” the latest review said. “This will likely require increased capability within Stats NZ and refreshed governance arrangements with Māori.”

An iwi-led collections and data analytics initiative, Te Mana Whakatipu, ran in Ōhua (in Te Taitokerau), Toitū Tairāwhiti, and Te Whānau-ā-Apanui. However, key decisions about field collections had been made by the time iwi communities became involved.

Interim response rates for those of Māori descent improved significantly:

• Toitū Tairāwhiti, from 70.6 percent to 80.1 percent;
• Te Whānau-ā-Apanui, from 63.5 percent to 79 percent;
• Ōhua, from 69.1 percent to 69.6 percent.

But those results came at a cost of $40 per head of population, as opposed to the nationwide cost of $13, in part because of high digital uptake elsewhere. It wasn’t affordable to expand the pilot, as designed, nationwide, the review said.

“If a future census is to build on the good work done in 2023 and the pilot experience, then engagement with Māori on the collection-approach process design and support needs must start now.”

One successful initiative, taken when it was realised Māori returns were lagging expectations, was the ad hoc involvement of the Whānau Ora Commissioning Agency, which collected responses from 10,000 Auckland households which hadn’t responded.

“Ultimately, they were successful in getting returns from 29,426 individuals.”

“While overall the main population counts will be of very high quality, in some parts of the country where both census form response and available administrative data were lower, quality will be affected.”

Report of the Statutory Review of New Zealand’s 2023 Census

The 2023 census was delayed, in some areas, by the Northland and Auckland floods and Cyclone Gabrielle. But it was much-improved from the previous one, in 2018, with strengthened management and oversight, and increased numbers of field staff (to about 3600) supported by sufficient paper forms.

The delays demonstrated good quality data can be achieved weeks after census day.

In 2022, the dress rehearsal was affected by Covid-19, severe weather, and a rise in anti-government sentiment. Despite that, it revealed issues with “mark-in” – the system allowing field staff to know who had already completed forms – and the register of addresses, known as the operational frame.

The register wasn’t as accurate as anticipated, which frustrated field staff, the review said. “This affected field operations and added confusion and effort in the iwi-led collections and the additional collection efforts in Auckland.”

There were other significant problems.

A reminder went to all dwellings about census day (March 7), confusing people who had already responded. The confusion was compounded by a new, secure internet access code being provided.

After census day, reminders were sent at the time as the “non-response follow up” field operation. It would be better, the reviewers said, to let the reminders have their intended effect before collectors visited.

A technology issue disrupted field operations. Late changes to the so-called application programming interfaces (API) meant it was slow to transfer large volumes of data between census systems, leaving collectors blind as to who had already responded online.

The fix was too slow, the review said, and wasn’t in place before field operations started.

“The result was a large number of visits to dwellings that had already responded, an effort that would have been better placed in pursuing non-responding dwellings.”

These unnecessary visits also meant “trust in Stats NZ was impacted”, because householders were left wondering if their information was received or managed properly. A manual work-around was found to ensure reminder mailouts weren’t also unnecessarily sent.

Reviewers said an all-hands-on-deck and 24-hour work period should be applied for such an urgent, “high severity” incident.

Another major technological issue was the “last-minute” development of the census processing system. (This delay also had a knock-on effect for a backlog in coding. The reviewers suggested a greater automation of coding in future.)

Larger and more complex changes than anticipated were required to the 2018 processing system, which was “poorly documented” and difficult to adapt. While the system was produced in “record time” – about six months – material progress on resolving issues only occurred in late 2022.

“There is a sense that there was a much stronger emphasis by the Census Programme on communications, marketing, engagement, and collection relative to processing and delivery.”

Recommendations by reviewers included increased oversight of “post-collection steps”, such as processing, and ensuring the building and testing of processing and data delivery systems were done earlier.

But before it looks ahead to the next census, Stats NZ has to tackle existing problems.

The review said the census will produce a “varying level of quality of the data”.

“While overall the main population counts will be of very high quality, in some parts of the country where both census form response and available administrative data were lower, quality will be affected.

“Also, for variables that cannot be found in administrative data, historical data, or statistical imputation, the question-level coverage will be lower than the overall national average, and therefore quality is likely to be impacted.”

The first census data will be published on May 29, with releases continuing until August next year.