SPNO Statement on the National Household Survey

The voluntary National Household Survey (NHS) has produced lower quality data than the mandatory long-form Census it has replaced. The Social Planning Network of Ontario (SPNO) only uses NHS data when higher quality data is not available.

Statistics Canada has said their evaluations of NHS data “support the general reliability of the data at the national, provincial and territorial levels” but they have not extended this confidence to using NHS data at the community level. They have noted that “risk of error in NHS estimates increases for lower levels of geography and smaller population.” Due to the increase in error at the local level compared to previous Census data, Statistics Canada has released very few NHS data tables for smaller geographies like municipalities and neighbourhoods.

Statistics Canada has also cautioned against doing historical comparisons between NHS and Census data because the data was collected differently.

Good quality data about demographics, social and economic conditions at the local level is essential for social planning and research, including the design and delivery of public services. The cancellation of the mandatory long form Census has resulted in a significant loss of reliable data that will greatly reduce the ability of residents to understand their own communities. This critical data gap will also affect the quality and cost-effectiveness of the services residents use for years to come. The Social Planning Network of Ontario with its partners at the local, provincial and national levels will continue to bring attention to this major data gap and to call for the return of a full mandatory Census.

January 2014

PDF version of statement

Data day will not be celebrated in Hamilton

This is a guest post by Sara Mayo, Social Planner with Social Planning & Research Council of Hamilton. This is on the eve of the first release of data from the National Household Survey, which replaced the long-form census in Canada. The original post can be found here.


SPRC reserves judgement on NHS data until questions are answered about data quality

By Sara Mayo, Social Planner

On May 8, Statistics Canada will be releasing the first data from the 2011 National Household Survey (NHS). Major new data releases are usually hotly anticipated and a busy time for social planners, but the Social Planning and Research Council of Hamilton has decided that we will not be commenting or reporting on the new NHS data at that time, including trends on Immigration, Aboriginals, Ethnic Origin, Language and Religion that will be released by Statistics Canada.

Since the news of the cancellation of the 2011 Census long form, the SPRC has been clear that the voluntary NHS is a poor replacement. The cancellation of the 2011 Census long form was a hasty decision by the federal government, without any notice or consultation with Canadians, and the negative effects will be felt for years to come. At a community forum on the Census that we organized  in September 2010, Code Red for Neighbourhood Data, organizations across Hamilton voiced their concerns that the cancellation of the long form would harm planning for new schools, immigration settlement services, poverty reduction strategies, youth programming at community centres, services for working parents, and neighbourhood improvements, among many others.

While NHS response rates have been better than the low benchmark of 50% set by Statistics Canada, they are still much lower than for the Census long form. 94% of households completed the Census long form in 2006, while the average NHS response rate for Canada was 68.9% (67.6% for Ontario). In the city of Hamilton, only 65.3% of households who received the survey answered it.

Statistics Canada’s own estimates show that NHS survey respondents are not representative of the population. Their analysis shows that raw NHS data from the Toronto Census Metropolitan Area overreports people earning a high income, those who are university educated, Canadians who work in the public administration industry, or in business, finance or administrative occupations and Chinese Canadians. Conversely, raw NHS data from the Toronto region undercounts the number of low income Canadians, the number without a high school diploma or less, those working in construction and transportation, Black Canadians, non-citizens, Aboriginals, and those who have moved in the last year. Accurate estimates of these populations are critical to social planning.

Our own analysis of response rates by Social Planning Network of Ontario communities (Table 1), shows a general trend whereby communities with low NHS response rates usually have higher poverty rates. The major outliers to this trend are the cities of Toronto, Markham, and Mississauga and the town of Richmond Hill, which all have higher than average poverty rates, yet higher than average NHS response rates. This may be explained in part by the higher NHS response rate of Chinese Canadians (who are a large portion of residents in these cities) as reported by Statistics Canada.

Table 1: Social Planning Network of Ontario Communities, NHS response rate and poverty levels

Community Social Planning Network of Ontario member

NHS raw response rate

Poverty rate (before tax LICO, 2006 Census)

Ajax Community Development Council Durham

72.5%

10.9%

Belleville Community Development Council of Quinte

68.3%

15.1%

Brampton Social Planning Council of Peel

68.9%

13.9%

Burlington Community Development Halton

76.7%

9.5%

Caledon Social Planning Council of Peel

67.9%

4.4%

Cambridge Social Planning Council of Cambridge & North Dumfries

70.0%

9.6%

Clarington Community Development Council Durham

70.8%

5.9%

Cornwall Social Development Council of Cornwall & Area

63.0%

21.1%

Greater Sudbury Social Planning Council of Sudbury

64.8%

12.7%

Halton Hills Community Development Halton

71.9%

5.1%

Hamilton Social Planning and Research Council of Hamilton

65.3%

18.1%

Kingston Social Planning Council of Kingston & District

69.6%

15.4%

Kitchener Social Planning Council of Kitchener-Waterloo

72.0%

11.8%

Markham Social Planning Council of York Region

76.1%

16.1%

Milton Community Development Halton

70.2%

5.0%

Mississauga Social Planning Council of Peel

72.9%

15.7%

Oshawa Community Development Council Durham

66.5%

12.9%

Ottawa Social Planning Council of Ottawa

74.3%

15.2%

Peterborough Peterborough Social Planning Council

63.8%

17.0%

Pickering Community Development Council Durham

72.0%

9.9%

Richmond Hill Social Planning Council of York Region

72.9%

15.8%

Thunder Bay Lakehead Social Planning Council

67.6%

13.8%

Tillsonburg Social Planning Council Oxford

68.8%

9.1%

Timmins Cochrane District Social Planning Council

58.9%

12.7%

Toronto Social Planning Toronto

67.8%

24.5%

Uxbridge North Durham Social Development Council

70.2%

6.2%

Vaughan Social Planning Council of York Region

70.9%

10.6%

Waterloo Social Planning Council of Kitchener-Waterloo

74.8%

10.8%

Whitby Community Development Council Durham

75.0%

7.0%

Woodstock Social Planning Council Oxford

70.0%

10.6%

Ontario

67.6%

14.4%

Data source: National Household Survey: Final response rates (Statistics Canada)

Statistics Canada has not yet indicated which mitigation strategies they have applied to adjust estimates to compensate for lower response rates among these critical groups. They have also not answered our questions about whether they will be releasing NHS data at lower levels of geography (such as census tracts and dissemination areas). Given the geographic concentration of many demographic groups with low response rates, we are quite concerned that neighbourhood-level data will be very unreliable.

We are especially dismayed that detailed information on data quality in the NHS will not be published until May 8th. As part of a conference call with members of the Community Data Program on October 18 2012, Statistics Canada representatives informed data users that they would be publishing a paper on NHS data quality issues in December 2012. Requests to find out why that document has been delayed have not been answered. When it is finally released, we will examine this paper to better understand data quality issues with the NHS.

At the same time, we want to make clear that we continue to support Statistics Canada staff, and the agency as a whole,  who are facing budget cuts that undermine their ability to obtain reliable data about Canadians. High quality Statistics Canada is used in communities across the country to plan for the population’s needs and to improve quality of life.

Starting on May 8th, the SPRC  will take the time to read Statistics Canada’s analysis of NHS data quality and mitigation strategies, and consult with other data users across the country about their judgment on the reliability and comparability of this data. We will update the community when we have formed an opinion on what uses of NHS data will be most prudent for Hamilton.

2011 National Household Survey (NHS) Concept Brief

The following link takes you to a presentation on a concept brief of Immigration and Ethnocultural Diversity in Canada based on the 2011 National Household Survey (NHS) data set to be released on May 8, 2013.

http://www42.statcan.gc.ca/smr10/pdf/smr10_2011_nhs-enm_002-eng.pdf

The NHS replaced the mandatory long form census with a survey whose participation was voluntary. The data on immigration and ethnocultural diversity is the first set of data to be released from the survey. Further information on the NHS and the release schedule can be found at http://www12.statcan.gc.ca/NHS-ENM/index-eng.cfm