MoneySense recently released their list of Best Places to Live 2011, in which they provide rankings of the “livability” of Canadian cities. According to the list, Ottawa-Gatineau is the best place to live and, out of the 180 cities ranked, New Glasgow (Nova Scotia) is the worst.
Of the “big three” Canadian cities, Vancouver was ranked 29th, Toronto 88th, and Montreal 123rd. Now, maybe it’s just because I grew up in and around Toronto, but any list that ranks Guelph (18th) and Newmarket (24th) as better places to live than Toronto makes me a bit skeptical. Fortunately, MoneySense provided a link to the spreadsheet they used for their calculations, allowing me to take a closer look at some of the method behind this madness.
The average MoneySense ratings by province/territory.
The internet seems to be full of rankings and top n lists (among other things) — and for good reason. They’re fun to read, and they boil complex information down into a simple easy-to-read list. But these lists always have limitations. The act of ranking will necessitate a ton of assumptions and attempts at objectively assessing subjective factors. This isn’t to say that these lists have no value; merely that you need to understand what these assumptions are in order to glean any possible value from the list.
This list ranked cities in a number of categories; giving them an overall score out of 105 points, broken down as follows:
WALK/BIKE TO WORK: 7 points
WEATHER: 18 points
AIR QUALITY: 2 points
POPULATION GROWTH: 10 points
UNEMPLOYMENT: 10 points
HOUSING: 15 points
HOUSEHOLD INCOME: 4 points
DISCRETIONARY INCOME: 4 points
NEW CARS: 4 points
INCOME TAXES: 2 points
SALES TAXES: 1 point
CRIME: 5 points
DOCTORS: 6 points
HEALTH PROFESSIONALS: 4 points
TRANSIT: 5 points
AMMENITIES: 3 points
CULTURE: Bonus points – A city could receive up to 5 points based on the percentage of people employed in arts, culture, recreation and sports. Source 2006 Census.
Some things in this list instantly jump out as interesting choices. For instance, weather is (collectively) the most important determining factor in this list of the livability of a city, with 9x the number of points of pollution/air quality. This seems like a highly subjective choice. The second-most important criterion is housing, which appears to be heavily weighted towards the purchase and ownership of houses and, though I could be mistaken, doesn’t take rental properties into account (affordability, quality, tenants’ rights, etc.). Interestingly, the culture category, which is arguably one of the stronger points for larger cities, was given a measly 5 ‘bonus points’ and only took into account the number of people employed in “arts, culture, recreation and sports”.
Taking a closer look at the data reveals some specific questions about the choices made in producing the ranking system:
- Hospitals, colleges and universities were given a binary ranking (’1′ if the city had a hospital, college or university; ’0′ if not). Why was it decided not to include the number and caliber of these institutions? The number and quality of hospitals more-so than the other two (but not exclusively) can make a huge contribution to the quality of life in a city.
- In a list which, to me at least, seems to be focused on an ideal suburban/family lifestyle, it’s curious that they decided to leave schools (number, quality, etc.) out of the rankings.
- Is less precipitation always a good thing? What if you live in or near a farming community, or your water supply is heavily dependent on precipitation? What if you like rain (or snow)?
- Do the “health professionals” (as distinct from doctors) include homeopaths/acupuncturists/etc.? If so, does this raise standard of living and/or is it a good indicator of standard of living?
- This list seems to rank attributes that all of these cities have in common; yet often times some of the things that make a city a great place to live are unique to that city (or geographic region).
Additionally, it would be interesting to know if measures like doctors per capita scales linearly. Is 2 doctors per 1,000 people just as good if you have a small town of 10,000 people (20 doctors) and if you have a larger city of 1,000,000 people (2,000 doctors)? If not, is it better or worse? (Are there economies of scale? Benefits from specialization?)
Furthermore, there were a few subjective calls made in the methodology of compiling this list.
The categories were scored out of a given number of points, for example 10 points for unemployment rate. The top city in each category received the maximum number of points, and the rest of the cities received descending incremental points based on their ranking.
This methodology really means that there’s no consistent weighting between categories, and therefore the chosen weighting by MoneySense is somewhat arbitrary and subjective. This isn’t so much of a problem (how do you compare crime rate to precipitation?) as it is something that we should keep in mind when looking at the list, as it might minimize the impact of the relative disparity between the quality of some of these factors in different cities.
Calculations for some other categories follow a slightly different methodology. For example, in the category of population growth, an annual rate of 7.4% is considered ideal. Anything below or above that rate loses points. The same is true for the subcategory of precipitation which makes up part of the weather category. (The ideal number is 700 ml a year, with anything above or below that losing points accordingly).
This is a more straight-forward assumption, but also important to note. Why is 700 mL of precipitation a year ideal? And what’s the justification behind an ideal annual population growth rate of 7.4 percent?
What can we conclude from this list?
This post isn’t meant to imply that the MoneySense list has no value, but perhaps its criteria for “most livable city” need to be broadened to more accurately reflect the things people actually consider when they think about the “livability” of their city. There are more things I could look at or criticize, but the point isn’t that this list is wrong; merely that it makes certain assumptions, which may or may not reflect our understanding of what makes a city livable. Knowing what these assumptions are makes this list more useful, assuming the data presented is accurate.
The only remaining question is: why did Toronto do so poorly? Toronto scored well or average on the weather categories, but very poorly in housing (1.2/15 points), population (1.8/10 points) and unemployment (2.3/10 points). We also did fairly poorly in crime severity (124th), doctors per 1,000 people (99th), and health professionals (143rd). We did do well in some other categories: transit (2nd), culture (3rd), precipitation (15th), and total crime (28th); but it wasn’t enough, based on the weighting of this particular list. Clearly, the good people at MoneySense severely under-value the number of sushi restaurants per capita.