Dr. Ivan P. Fellegi, was the speaker at the 2011 Symons Lecture in Charlottetown, PEI on November 8. Dr. Fellegi is an internationally renowned statistician and was Canada’s Chief Statistician from 1985 until 2008.
He spoke on how statistics are intrinsically linked to democracy, and in particular how the decision of the Canadian Government to move the long form census from compulsory to voluntary status has a negative impact on Canadians. His points covered Canada’s international standing and the ability of the Canadian Government to make evidence based policy decisions.
Before discussing his talk, which was very much geared against the Conservative Government’s decision to remove the compulsory status of the long form census, I think it is important to review some of the history of the Canadian Census and the debate over the just completed census.
The history of conducting censuses in Canada dates from 1666 when Intendant Jean Talon tabulated the 3215 inhabitants of New France. The first national census was in 1871 although there were a number of regional censuses in the intervening years. Since then, Canadians have been enumerated at least every 10 and more recently 5 years.
The current debate centres around the Federal Government’s decision to remove the previously compulsory long form census document and replace it with a voluntary National Household Survey.
The Canadian government runs a census of the population every fives years on the authority of the federal Statistics Act (s. 19(1)). Under this statute, it is the job specifically of Statistics Canada to “take the census of population of Canada” (s. 3(c)). The Statistics Act gives the federal cabinet the power and discretion to appoint and remove the Chief Statistician of Canada (s. 4(1)). He and his office are not independent. The Chief Statistician must carry out his duties “under the direction” of the designated minister, which is currently Minister of Industry Tony Clement. Section 7 of the Act empowers the minister to set “rules, instructions, schedules and forms” for Statistics Canada, including for taking a census. Section 21(1) requires the federal cabinet to prescribe census questions by order in council. That is exactly what the cabinet did on June 17 causing the Statistical Society of Canada to criticize the scrapping of the long form.
Industry Minister Tony Clement’s comments July 13, 2010
“In the past, the Government of Canada received complaints about the long-form census from citizens who felt it was an intrusion of their privacy. The government does not think it is necessary for Canadians to provide Statistics Canada with the number of bedrooms in their home, or what time of the day they leave for work, or how long it takes them to get there. The government does not believe it is appropriate to force Canadians to divulge detailed personal information under threat of prosecution.
I cannot reveal and comment on this advice because this information is protected under the law.
However, the government can make this information public if it so wishes. I have always honoured my oath and responsibilities as a public servant as well as those specific to the Statistics Act.
I want to take this opportunity to comment on a technical statistical issue which has become the subject of media discussion. This relates to the question of whether a voluntary survey can become a substitute for a mandatory census.
It can not.
Under the circumstances, I have tendered my resignation to the Prime Minister.
Following Dr. Sheikh’s resignation, Mr. Wayne R. Smith was appointed as Chief Statistician and the 2011 census was conducted with a mandatory short form and the accompanying voluntary National Household Survey.
Dr. Fellegi was one of the group that developed the UN Fundamental Principals of Official Statistics, of which Canada is a signatory.
- Principle 1. Official statistics provide an indispensable element in the information system of a democratic society, serving the Government, the economy and the public with data about the economic, demographic, social and environmental situation. To this end, official statistics that meet the test of practical utility are to be compiled and made available on an impartial basis by official statistical agencies to honor citizens’ entitlement to public information.
- Principle 2. To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.
- Principle 3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.
- Principle 4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.
- Principle 5. Data for statistical purposes may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents.
- Principle 6. Individual data collected by statistical agencies for statistical compilation, whether they refer to natural or legal persons, are to be strictly confidential and used exclusively for statistical purposes.
- Principle 7. The laws, regulations and measures under which the statistical systems operate are to be made public.
- Principle 8. Coordination among statistical agencies within countries is essential to achieve consistency and efficiency in the statistical system.
- Principle 9. The use by statistical agencies in each country of international concepts, classifications and methods promotes the consistency and efficiency of statistical systems at all official levels.
- Principle 10. Bilateral and multilateral cooperation in statistics contributes to the improvement of systems of official statistics in all countries.
Now, on to Dr. Fellegi’s talk.
His first comment raised the question of whether the form should be considered to be a survey or a census form.
Without being compulsory, rather than a census questionnaire, the 2011 long form must be considered a survey. However, he saw several problems with this:
- the form consists of traditional census questions
- it is collected by the same staff
- the scale of the questionnaire is same as previous census forms
- the forms are disseminated with the compulsory part of the census
He then moved into the question of data reliability, a topic he returned to several times.
He argued that since respondents are self-selected and not truly a random sample, this introduces a bias into the results. Traditionally surveys are more likely to be completed by people with more education and higher socio-economic status. This can result in a negative bias against groups such as those with lower levels income and education, especially recent immigrants, aboriginals, and the elderly living alone. Even though these are generally low income/education groups, the bias cannot be quantified, nor can any directionality in bias be determined.
Without this information, we cannot make informed decisions about where to plan the next extension of public transit, or where to target different types of health resources…the knowledge it offers forms the backbone of our society, an information society that needs and wants to know about itself.’
A voluntary survey, Dr. Fellegi pointed out, will mean an unknown quality of data that will affect the compatibility with past and future census data, a problem that becomes more acute as time goes on.
He then moved into the way the decision was made. Despite claims of complaints, there was zero consultation with the public. The decision was made despite Canadian and international concerns. In fact, the majority representatives of the public were strongly against the change. This included cities that accounted for 89% of the Canadian population. According to Dr. Fellegi, the Minister’s arguments included the fact that Statistics Canada works for crown, and so is subject to the demands of government.
In Dr. Fellegi’s opinion, there is a difference between appropriate and inappropriate interference by government in official statistics. Appropriate interference included such things as determining the questions to be asked, as the answers are used to determine government policy, and of course, the budget allotted for the department and the census itself.
The examples he gave for inappropriate interference included the extreme, such as the President of Argentina falsifying data and Josef Stalin suppressing data to the point of having his Chief Statistician assassinated. He also referred to more subtle interferences such as slanting analytical write-ups and interfering in methodology. It is in the interference in the methodology of the census that he sees as inappropriate by the Canadian government, and that it is the first time in Canada there has been any question of inappropriate interference. He lamented the loss of democratic accountability in areas such as determining the effects of the recession, the amount of inflation, education levels (esp of aboriginals), the economic status of new immigrants, the use of the French (or indeed any language), and the amount of income polarization.
Next, he moved into the areas where Statistics Canada loses functionality with government interference.
The report card that Statistics Canada delivers to the government is important for evidence based decision making and rational use of resources by the different levels of government and various businesses. For example, banks use the data for determining risk tolerance, business use it to determine ideal locations, and cities use it for such policies as infrastructure, transportation, demographics and assistance programs. It is also used by national and international financiers to make investment decisions. The information is also used to determine Federal-Provincial transfer payments, and since the data cannot be verified, the potential exists for social disturbance.
In addition to the use of data as described above, over 1000 papers depend upon the accuracy of the data. These papers are on such topics as income, health, single parent households, education vs labour market, and social mobility. All of these depend upon the reliability and replicability of the data.
Also impacted is the reputation of Statistics Canada as honest information brokers with emphasis in areas such as confidence, confidentiality, professional integrity, methodologies, and reliability. In all of these areas, Statistics Canada has been a world leader; to retain that status, it is important that the department be seen to be free from political interference.
He went on to discuss how the nature of the decision violated the UN Fundamental Principals of Official Statistics, three are 10 of these, but Dr. Fellegi commented on 3 of them; specifically two, three, and four.
2. To retain trust in official statistics, the statistical agencies need to decide according to strictly professional considerations, including scientific principles and professional ethics, on the methods and procedures for the collection, processing, storage and presentation of statistical data.
His criticism here was the decision was not based upon professional considerations, and in fact, in spite of criticisms.
3. To facilitate a correct interpretation of the data, the statistical agencies are to present information according to scientific standards on the sources, methods and procedures of the statistics.
4. The statistical agencies are entitled to comment on erroneous interpretation and misuse of statistics.
Dr. Fellegi felt that these two principles were ignored by specifically ordering employees of Statistics Canada not to comment on the decision.
Even with all of these arguments, Dr. Fellegi admitted that according to the Act, the decision was legal, but he emphasized that it violated many years of tradition in the relationship.
Looking forward, he asked what the future of Statistics Canada should be. He outlined three possibilities.
1. The creation of an arms length organizations such as the auditor general
- Although this provides independence, it may remove the department too far from government.
2. The creation of a Crown Corporation
- Again, the department needs to work closely with government due to the determination of questions and the use of the data for policy. This structure may lead to legal issues around protecting the data.
3. The current situation with stronger protections
- This is his recommended course of action, at least partially accomplished by enshrining the UN Principles in the Act. The result of this would make the Chief Statistician more powerful, in that he or she would be able to over-rule others in the government. They would become the public face of Statistics Canada and the defender of non-political integrity. An important aspect of this would be transparency in the selection of the Chief Statistician.
In conclusion, he emphasized that statistics affect lives profoundly, but it is important they be strong, relevant, and objective. He compared statistics to clean water. We can’t just assume the water is clean, we need legal safeguards and public vigilance as preventative maintenance. During the question period he suggested that Canadians do not pay financial taxes voluntarily and a census could be considered to be an information tax, and just as important to the functioning of a well ordered state.
Some of my thoughts and further research.
Critics of the status quo have talked about privacy issues around the amount and type of information that was gathered in the census. Financial and racial information were considered by many to be intrusive or offensive. Also included were condemnations of questions about household work and physical or mental health conditions. The potential consequences of ignoring the census (maximum penalty $500 fine and 3 month in jail) were thought to be onerous.
While Dr. Fellegi refused to speculate upon the motivations of a government that made such a fundamental change in the way census data was gathered, it seems to me to be self evident. It is natural to expect a government with a bit of a libertarian bent to move away from gathering information on the citizens. As a confirmation of the ideological bias, Justice Minister Rob Nicholson stated in reference to the new Omnibus Crime Bill:
“We’re not governing on the basis of the latest statistics,” Nicholson said at a news conference, held earlier in the day near Toronto.
“We’re governing on the basis of what’s right to better protect victims and law-abiding Canadians.”
The controversy and legal dispute over the Insite Program in Vancouver is another example of this government attitude. So, what about the results of the recently completed 2011 census?
The agency says it received the mandatory short form from 98.1 per cent of Canadian households, up a full percentage point from the last census in 2006. StatsCan hasn’t yet analysed the questionnaires to determine how well they were filled out, but statisticians are confident that they’ll have good data.
For the voluntary long form, the results are in as well.
“We ended up with just about 2.6 million respondents,” Census Manager Marc Hamel told QMI Agency. “We have, in effect, more respondents in 2011 than we had under the mandatory census long form in 2006.”
The 2011 National Household Survey ended up with a 69% response rate, which is lower than the approximately 90% rate the mandatory long-form would get. But, anticipating the lower response rate, Statistics Canada sent out the 2011 survey to one in three households, instead of the typical one in five.
Despite the numerical increase in returns, Stats Canada has also been dealing with incomplete returns, something that was considered to be illegal in the past. The results may further compromise already questionable data. What does the future hold for Statistics Canada and the semi-decadal census? According to The Globe and Mail, the short form census questionnaire may be on the block as well. St. Catharines, Ont., Conservative MP Rick Dykstra brought this up last spring.
“We’ve already changed the long-form census so that it is not mandatory, and that is, frankly, the road we are going with the short-form census as well,” Mr. Dykstra told The Standard in St. Catharines. “I frankly don’t think this is the sort of thing a person should be penalized to do.”
Almost immediately, the PMO denied Dykstra’s statement. However,the story may not end there.
Statistics Canada is investigating whether to ditch the mandatory short-form census in favour of alternatives such as data mining, chief statistician Wayne Smith said in a February interview with The Globe and Mail. It was the Harper government who requested this review, he said. The Conservatives, who scrapped the mandatory long-form census on the grounds it was wrong to coerce Canadians into answering intrusive questions, has asked Statistics Canada to rethink the way it collects population data, according to Mr. Smith.
On the face of it, this seems to be a very radical, negative concept. Without any census data at all, how could a country be governed? While many governments, including our own might be able to ignore some aspects of population statistics, but the bulk of the data is critical. However, a little digging unearths something that didn’t make it into the ongoing census debate here in Canada: many other countries are moving away from the combination census that we use here.
Paolo Valente, a statistician with the United Nations Economic Commission for Europe, published a paper in the May 2010 issue of Population & Societies titled “Census taking in Europe: how are populations counted in 2010?” in which he described some of the differing methods countries use to tabulate population data. Valente defines a ‘traditional census’ as one in which all members of the population are asked the full gamut of countries officials deem necessary to govern. The combination of short and long form used here in Canada is just one of several solutions to the effort and cost required to gather and analyze complete data from all residents. Valente describes some of the options being used across Europe and the US.
Combination Long and Short Form
It has the advantage of increasing the range of content while reducing overall respondent burden and the cost of data processing. It may complicate field operations, however, and it reduces the amount of information produced for small geographic areas. This method is used in Canada, for example, and was applied in the United States up to the 2000 census.
Some Northern European countries, such as Denmark and Finland, use administrative registers for statistical purposes and conduct their census on the basis of inforation in the registers, rather than through field enumeration. This approach places no burden on individuals and, once registers are established, is less costly to implement. Data are available from registers on a continuous basis, so the census could potentially be taken every year. The population characteristics considered are limited to those available in the registers, so statistical agencies may combine data from different registers by linking data at the individual level in order to expand the range of data. This demands close cooperation between the statistical agency, register authorities and the public administration, and strict legislative oversight.
Combination of register data with complete enumeration
Some of the characteristics of interest for individuals and households may not be available on existing registers. For this reason, the census is sometimes conducted using registers and field enumeration simultaneously. This approach helps to improve the precision of the count and the quality of the registers. Compared to a register-based census, this approach is more expensive, more complex to apply and places a burden on the public. But compared to the traditional census, it can be more efficient in field operations.
Combination of register data with existing surveys
This is a variant of the previous approach which does not require field operations. Registers are used to provide a total enumeration of the population, and results from existing surveys provide individual characteristics that are not available in registers, provided that information from the different sources can be linked at the unit record level. This approach ensures that census results are consistent with survey results for common variables. It is relatively inexpensive and places no burden on respondents, but the information linking process can be complex.
Traditional enumeration with yearly updates of characteristics on a sample basis
This method was developed in the United States for the 2010 census. It comprises full field enumeration every ten years using a short form, combined with a large household sample survey conducted on an annual basis. The survey provides detailed demographic, social and economic data about households, like in a long form but updated annually. In the United States, the annual American Community Survey covers approximately 2% of the population, which totalled 320 million inhabitants in 2010. Despite the size of the survey, for some small areas there may be insufficient information in each single year to produce estimates. For this reason, information for small areas is aggregated over several years and released only in the form of moving averages
The notion of a moving average over five consecutive years is applied for the rolling census introduced in France in 2004. Small municipalities (fewer than 10,000 inhabitants) are divided into five groups, and a full census is conducted each year in one of the groups. In all large municipalities, a sample survey covering 8% of dwellings is conducted each year. After five consecutive years, the entire population in small municipalities and about 40% of the population in large municipalities has been surveyed. In all, about 70% of the French population is covered in the course of the five-year cycle. This is enough to guarantee robust information at the municipality and neighbourhood levels. The census results are based on moving averages calculated over the five-year cycle, and are updated yearly. This method was developed mainly to improve the frequency of the data releases, and to spread over time the financial and human burden associated with the census.
As Valente points out, there are numerous options to the type of census we have been conditioned to accept here in Canada. Each of these has it’s own strengths and weaknesses, and some are more complicated to understand than others. Some are more personally intrusive while others delve into a number of electronic databases collating data that might have even more privacy issues.
It is entirely appropriate for Canadians to debate the type of census that should be used to enumerate ourselves. After all, the information is ours. Despite the campaigning by Dr. Fellegi and others, I highly doubt we will see a return to the compulsory long form census of the past. It is important that we have an informed discussion on the options available in the future.