In an ironic boost to its “digital-first” principle, next year’s census appears to be grappling with a shortfall of pre-printed paper forms. David Williams reports.
New Zealand’s census has been hit by a paper form shortfall, according to internal documents released to Newsroom under the Official Information Act.
The documents state NZ Post won’t be able to print the required number of paper pack forms to be left by field staff before census day on March 6. Some officials worry it will discourage early responses in some places, especially in bilingual areas, and possibly dilute the overall quality of the data collected. Significant changes have been made to census field operations.
It’s a further headache ahead of the country’s most important collection of authoritative data, after problems emerged with crucial backroom IT systems.
Of course, a paper shortage might help the census achieve its 70 percent internet response target. But it might also mean fewer people will respond before being visited by field staff, making the follow-up phase longer and more costly, with higher demand for paper forms at the tail-end of the exercise. That leads to risks of under-staffing or over-spending.
The 34th census, to be held on March 6, will be largely conducted online, with 80 percent of households receiving an internet code instead of a paper form. Using digital technology can save money, with fewer field staff being employed, and speed up processing of data. It also makes internet security crucial for the $121 million project, so there won’t be a repeat of the debacle across the Tasman, when Australia’s census website was taken down for two days to avoid cyber attacks.
The paper shortage problem emerged in an October 4 “change request” written by census field operations boss Alan Bailey, part of hundreds of pages of documents released to Newsroom.
Bailey’s memo – signed off by Stats NZ’s deputy chief executive and deputy government statistician, Teresa Dickinson – says: “NZ Post are not able to print the number of paper form packs that were required to support previously agreed ‘enable and visit’ phase paper pack distribution methodologies.” The “enable” phase runs from February 19 to March 5, during which every New Zealand household will be provided with internet access codes or paper forms. During the “visit” phase, between March 16 to April 16, all households that haven’t responded will be visited.
The paper form deficit means some areas that should have received a letter and a paper form will now get a letter containing an internet access code. It’s not clear how many forms short the census will be – and NZ Post has declined to comment.
In an emailed statement to Newsroom, Bailey tries to change the narrative – a narrative created by a document he wrote. He says the original request for proposal with suppliers estimated the number of forms required “because the model was still in development”.
“Following our testing and modelling, we have revised our numbers for printing paper forms. We’re confident that NZ Post are able to print the number of paper forms we need to make sure that anyone who requests a paper form next March will be able to take part in the census this way.”
Bailey’s October 4 memo says non-responding households in so-called “mainstream list-leave” areas will now receive two reminder letters in the first 10 days after census day, when previously they wouldn’t have been sent any.
In “bilingual list-leave” areas, non-responding households will get one reminder letter – when they were not meant to get any – and that’ll be followed up by a field officer visit by March 16, or soon after. Under-counting of Maori and Pacific people was identified as a problem with the 2013 census. As a result, Maori are a “targeted response group” in the coming census.
Performance issues with bilingual approach
Bailey’s memo contains comments from many Stats NZ business units. The field operations division says targeted visits by field staff in bilingual areas might not be as effective without bilingual paper forms being available. “While this could encourage more internet response in these areas, it could also run the risk of dis-engaging respondents and de-motivating the field force.”
Similar concerns are expressed by the content and customer relations business unit, considering a “real or perceived history of issues in performance in relation to our bilingual approach”.
In 2014, the Salvation Army said the census should abandon trying to count South Auckland’s true population, as people avoided filling out forms because of illegal immigration, their shame at poverty and a lack of internet access. Census figures are used by local and central government for planning and allocating money, so any under-count, particularly in poor areas, could leave some in need of help at risk of slipping through the cracks.
In another section of Bailey’s memo, the statistical methods and population statistics teams worry that lower census response rates could hurt “data quality in general”. The census aims to account for at least 98 percent of the resident population.
The testing and operations coordination team says in the memo the paper shortfall creates a “new catastrophic risk” of field staff running out of English paper forms during home visits. It will be important to track the whereabouts of paper forms “so we can react ahead of time to get the paper where it needs to be”.
Other concerns raised include: the need for increased advertising and marketing, greater iwi engagement and “assisted completion”, a reassessment of printing and post budgets and more calls to the census contact centre. On the plus-side, paper responses might be spread over a wider period of time making form processing easier.
Earlier this month, the New York Times reported there were growing concerns about the United States’ 2020 census, which will also be conducted largely online. The $US15.6 billion exercise has been buffeted by years of under-funding and cost overruns on the digital transition.
The last census there, in 2010, cost $US13 billion, or more than $42 per person. New Zealand’s 2018 census, meanwhile, is expected to cost between $24 and $25, based on the full $121 million budget, including capital expenditure and two related projects.
Newsroom asked Stats NZ for information to write an explanatory story about its troubled Salesforce software system, including what it’s used for, how much it costs and how it was road-tested before it was added to the census project. Senior communications and marketing manager Richard Stokes replied: “We would prefer to leave that at this stage.”
