Visit our sponsor: PACSCanada.com    Visit our sponsor: Your Ad Here    Visit our sponsor: Beaus.ca
Visit our sponsor: Your Ad Here

Math-challenged Transposition of Population not up to usual Elections Canada standards

[Welcome, National Newswatch readers!]

It's probably the last thing Chief Electoral Officer Marc Mayrand needs to hear right now, but the Transposition of Population his office released last Friday as part of the 2013 Redistribution does not add up.

Election data geeks – officially known as "psephologists" – have been exchanging emails, comparing notes on the reliability of the data, and wondering if it can be fixed.

But approaches to Elections Canada so far have proved fruitless.

At issue is the population being transferred from each of the old ridings to each of the new ridings. The population counts add up by new riding, but not by old riding. So for example 102.9% of the old riding of Avalon's population is allocated to new ridings, while only 98.1% of the old Brampton-Springdale's population is accounted for.

The errors are all within plus or minus 5%, but previous transpositions have always added up in both directions. This is not just academic – political parties need to have those percentages accurately, so that EDAs from out-going ridings can transfer their assets in the proper proportions to the new ridings. But, here, the outgoing Avalon EDAs would have to come up with extra money or extra surplus sign stakes, while the outgoing Brampton-Springdale EDAs would have money left over. The ratios are also used by psephologists and political scientists more generally to project results backwards and forwards.

The agency, in conjunction with a contact at Statistics Canada, decided to reproject population counts from old ridings to new ridings using ratios derived from the redistribution of elector counts from old to new. The first problem is: they used the wrong ratios. Viz: if a part of your old riding makes up 50% of my new riding, it does not necessarily follow that I am inheriting 50% of your old riding. I might be getting 75% of your riding, and 25% of another one, even though both chunks represent 50% each of my new one.

However, even using the correct ratios, the old riding populations add up but now the new ones are off. Meaning that populations don't redistribute between ridings in the same proportions as electors, and so the entire assumption behind this Transposition is not exactly iron-clad. This makes sense when we consider that different ridings have differing numbers of non-elector residents (i.e., people who are not yet citizens, or have not yet attained voting age).

By contrast, an exercise that calculated riding population transfers based on summing the population of the 2011 Census Dissemination Blocks contained within them – which must have been the previous methodology used by Elections Canada – was far more precise. I do not know why it wasn't used this time, but I fear budget cuts and/or the reduced timelines resulting from last year's amendments to the Electoral Boundaries Readjustment Act may be the culprit. The populations counts generated in earlier transpositions were always accurate in both directions, but now they are merely estimates. It is as though the Transposition of Population has gone the way of the long-form census.

[click image to open full-sized PDF version]

Demonstration of the Distortions Introduced when Basing Population Transpositions on a Transposition of Elector Counts, 2003 to 2013 Representation Orders

I give a demonstration of the errors using Brampton ridings here, but for those who are interested in greater detail, I've also reproduced the exchange I had with Elections Canada to try and point out the problem and/or get clarification on why the numbers were off.

Here's what I wrote to my contact at the agency's media relations service:

——————————

For example:

–> Look at the old Avalon (2003-10001), which is split up between:
* the new Avalon (2013-10001)
* the new Bonavista-Burin-Trinity (2013-10002), and
* the new St. John's South-Mount Pearl (2013-10007) ridings

and therefore those 3 riding files.

The old population of Avalon is correctly listed in all three files as 78,908, but gets allocated between the 3 new ridings as follows:

* 61,048 to 2013-10001
* 17,123 to 2013-10002
* 2,521 to 2013-10007

This adds up to 80,692 – a difference in allocated population of 1,784 (or 2.9%) !!! I cannot believe that such a magnitude of over-allocation is a mere rounding error.

There are numerous other examples of either over- or under-allocations of the old riding's population to its descendant ridings:

* old Brossard-La Prairie (2003-24011) is under-allocated by 2,432
* old Hull-Aylmer is under-allocated by 2,924
* old Mississauga-Brampton South (2003-35047) is over-allocated by 5,137, while Mississauga-East Cooksville (2003-35048) is under-allocated by 6,988

… And it continues in much the same manner right across the country.

Now, these mis-allocations of population all zero out across the country, but the errors are bigger than I've ever seen in your earlier Transpositions (± 5%).

And – note – the only major errors I see are for the transpositions of Population. The transpositions of Electors and Valid Ballots are all within ±2 at worst (not 2%, but 2 voters), and most are bang on. The transpositions of votes also add up to within tolerable tiny errors.

Which is what makes this truly weird, because your Methodology section says that the transposed population percentages were calculated using the same ratio as transposed electors. Yet the electors match, while the populations don't.

——————————

And here's the answer my contact received from the subject matter experts in the agency:

——————————

As described in section 3.7 of the document, the population counts for the old FEDs [ed. FED=Federal Electoral District] were published by Statistics Canada. The population counts for the new FEDs were derived by independent commissions in consultation with Statistics Canada and are based on the 2011 Census of Population. These are official counts that require no calculation. These old and new FED's population counts are used to derive the population transferred to the new FED or population taken from the old FED.

The population transferred to the new FED or population taken from the FED is derived using the ratio calculated from transposed electors on lists. This ratio is then multiplied by the population count of the new FED to obtain the population transferred to the new FED or population taken from the old FED.

In the case of Avalon (new FED), the calculations were as follow:

1) Electors on lists from section 4.3 tables
a. 48192 electors are transposed from old Avalon
b. 16176 electors are transposed from old St. John's East

2) Ratios:
a. Ratio for old Avalon=48192/(48192+16176)=0.748671
b. Ratio for old St. John's East=16176/(48192+16176)=0.251329

3) New FED's Population for Avalon: 81540
a. For old Avalon=0.748671*81540=61048
b. For old St. John's East=0.251329*81540=20492

——————————

If you were going to use ratios of electors from old to new ridings, here's how those calculations should have been done in my view, in order to faithfully represent what a transposition is supposed to do.

1) Electors on lists from section 4.3 tables
a. 48192 electors are transposed from old Avalon to new Avalon
c. 14200 electors are transposed from old Avalon to new Bonavista–Burin–Trinity
d. 2032 electors are transposed from old Avalon to new St. John’s South–Mount Pearl

b. 16176 electors are transposed from old St. John's East to new Avalon
e. 60248 electors are transposed from old St. John's East to new St. John's East

2) Ratios:
a. Ratio for old Avalon=48192/(48192+14200+2032)=0.748044
b. Ratio for old St. John's East=16176/(16176+60248)=0.211661

3) New FED's Population for Avalon: 81540 || old FED population for Avalon: 78908, for St. John's East: 100,559
a. For old Avalon=0.748044*78908=59027
b. For old St. John's East=0.211661*100559=21284

But, even there, that only adds up to 80,311, which is 1,229 short of the 81,540 census population calculated for Avalon by the Boundary Commission. So, now we know that population doesn't move between old and new ridings in the same proportion as electors. Hence, the Census Dissemination Block population method should have been used.

Using 2011 Census Dissemination Block populations, that we sum using GIS software and best guesses where the Dissemination Blocks crossed riding boundaries, we get the following counts:

 * 59591 in population from old Avalon to new Avalon
 * 16855 in population from old Avalon to new Bonavista–Burin–Trinity
 *   2462 in population from old Avalon to new St. John’s South–Mount Pearl

   (for an accurate old Avalon total population of 78,908)

 * 21992 in population from old St. John's East to new Avalon
 * 78567 in population from old St. John's East to new St. John's East

   (for an accurate old St. John's East total population of 100,559)

 *         5 in population from old St. John’s South–Mount Pearl to new Avalon
 *   3519 in population from old St. John’s South–Mount Pearl to new St. John's East
 * 79327 in population from old St. John’s South–Mount Pearl to new St. John’s South–Mount Pearl

   (for an accurate old St. John’s South–Mount Pearl total population of 82,851)

All of which also add up to:

 * new Avalon population of 81,583 (should be 81,540) +43
 * new St. John's East population of 82,086 (should be 81,936) +150
 * new St. John’s South–Mount Pearl population of 81,789 (should be 81,944) -155

And with the layers and lower-level Block-Face 2011 Census population information available from Statistics Canada, those totals could have been made perfectly accurate.

How Redistribution Information is shown on a Pundits' Guide riding profile page

I'm currently working on getting the Transposed results into the Pundits' Guide database, but that work ground to a halt when things stopped adding up. The section of a riding profile page that shows Redistribution Info for a riding (what percent of each riding it came from, and what percent of its own population went to what other subsequent riding) has always been calculated on the fly using an accurately cross-referenced table of transposed populations from old to new. Now those won't add up to the proper riding populations anymore for the new Representation Order.

Other psephologists have pointed out to me that the documentation of which polling divisions were transferred from old ridings to new ridings was not correct either, inasmuch as they show a minimum and maximum poll number for each section, regardless of whether the section includes polls that belong with a different riding, or overlaps a subsequent section. One of them (known as Krago at the US Election Atlas) provided me with this fictitious example to demonstrate:

Let's say the old riding of Ottawa-Duffy is split between the new ridings of Ottawa-Harb and Ottawa-Wallin as follows:

 * Polls 1-50: Ottawa-Wallin
 * Polls 51-100: Ottawa-Harb
 * Polls 101-150: Ottawa-Wallin
 * Polls 151-200: Ottawa-Harb

It would show on the Transposition of Votes as follows:

 * Ottawa-Wallin: 100% Start: 1 End: 150 Polls 100 Votes …
 * Ottawa-Harb: 100% Start: 51 End: 200 Polls 100 Votes …

It is frickin' useless to check if it is actually correct.

Something does not seem right when the bigger our data, and the better our software, the worse our public datasets are for accuracy. I really wish Elections Canada and Statistics Canada would go back to the drawing board and produce an accurate dataset for the Transposition of Votes and Population for this Representation Order.

Tags: ,

16 Responses to “Math-challenged Transposition of Population not up to usual Elections Canada standards”

  1. CalgaryGrit says:

    Very good work, as always, Alice.

    I’m still a bit fuzzy on the vote transposition though. Is it only a case where it hasn’t been well documented, or do we know that it’s actually wrong?

    From that example, it looks like they still only took 100 polls from each of Wallin and Harb, which is correct – it’s just a case of those polls not being properly identified, making it impossible to double check their math.

  2. Hi Dan, and thanks for your comment.

    I had no reason to question the transposition of votes, until I realized that the transposition of population didn’t add up, and the way the transposition of polling divisions was done didn’t allow for cross-checking.

    There is reason to have more confidence in the reported totals for the transposition of votes, though, given that the transposition of electors – on which it would be based – DOES add up in both directions (as shown in the Brampton example, in the leftmost column).

    Still, one always had full confidence in earlier Transpositions because they always added up in both directions in all elements. Now it’s just that little bit harder to have it. Pundits’ Guide being the “show-me state” of election data blogs, if I can’t add it up, I have to ask questions, and raise the problems for anyone else using the data to be aware of.

  3. CalgaryGrit says:

    Yes, your thoroughness is much appreciated!

  4. BC Voice of Reason says:

    Alice:

    You are exposing a relatively common knowledge fact of life in “The Bureaucracy”.

    While you tread softly “but I fear budget cuts and/or the reduced timelines resulting from last year’s amendments to the Electoral Boundaries Readjustment Act may be the culprit” … high quality results are not demanded or expected from “The Bureaucracy.

    I would venture to guess that Mr. Maynard and his minions are making scads (techie term meaning lots and lots) more money in salary, benefits and pension than you and your fellow psephologists are making.

    The CGI/IBM/Accounting-Type consultants that are hired to do the real work that the Public servants clearly can not handle obviously missed the sanity check on the transposition exercise or wanted to generate more of a mess that they will be able to charge their $300-500 / hr.

    President Obama (a friend of bigger government) clearly understood that the expectations of the Bureaucracy in implement major IT projects is extremely low in his comments on the implementation of his Affordable Care Act.

    Alice : How much would you charge to clean up this mess? and how long would it take?

  5. Shadow says:

    Sigh, EC would have been better off just waiting for the internet to do this for free. There’s enough database/election predictions people out there doing this anyways.

  6. Shadow says:

    Alice I am following twitter for instant reaction but I trust the EC bill will be worthy of a blog post at some point ?

    Approach seems to be to include a few enforcement measures as selling points then ram bill through house before anyone digests it.

    Seems bill will be used to legislate away past legal troubles (in and out, spending limits, signage) and to highlight areas where it was felt EC didn’t do enough against oppo (sponsorship, leadership loans).

    Plus all parties weighing bill through lens of tactical advantage. Liberals won’t mind bigger limits. NDP mad about bequests. Voter ID controversial but feeds into fears of fraud, maybe even vails issue.

    Certainly a lot to take in from many angles.

  7. Shadow, I’m working on it.

    BCVOR, when I wrote that it wasn’t up to their usual standards, I meant that. Elections Canada data might be hard to find on their website, but it has always been reliable and robust in the past.

    Gents, these are public datasets; they should be assembled by the public service, which has the resources and raw data necessary to produce them.

  8. Oh, and they are required by law to do so. So there’s that too.

  9. Shadow says:

    Right, so this data would be used for EC to allocate resources, division of EDA assets, turn out figures, spending limits .. that sort of thing ?

    Its worth debating whether its better to do it on the cheap or just go with the consensus of the internet in an open government/data sort of approach.

    Accuracy doesn’t seem to be an option anymore with a downsized government.

  10. Una says:

    Alice, when will you blog again. I know you are busy on twitter and with your geomatics, but we await financial analysis.

  11. Philip says:

    BCVOR, need I remind you that Harper was the one who brought Mayrand in? If you have any way to prove that he somehow isn’t playing ball, I’m all ears. Until then, I’ll continue to defer to him and EC as a whole.

    I’ll also save myself the hassle of paraphrasing Billy Preston in describing how sparse EC’s budget really is. Try visiting a returning office and you’ll understand.

  12. Shadow says:

    Philip, senator Mike Duffy was also appointed by Harper and CPC partisans have no love or respect for him either.

    It is an essential international norm of democracy that those in charge of elections be seen as completely non-partisan in their application of the rules.

    It isn’t a question of proof, its appearances. And that reputation once lost is unlikely to ever be regained.

    The fact that we are at this point should not be disputed.

    Now, if you want to accuse somebody in the CPC hierarchy of fomenting these feelings in a deliberate campaign to mask their own wrong doing – well go ahead that is one argument out there.

    But as we saw recently with Bill Blair handing over the Ford case, even the accusation of unfair treatment from a group absent any proof whatsoever is still to be treated as a serious matter.

  13. Pat McGrail says:

    The transposition of voters should have been an easier task. However, in looking at 905-West, I noted that the downloaded tables found in section “4.3. Transposition of Votes by Federal Electoral District, from the 2003 to the 2013 Representation Order” do not add up – either vertically or horizontally. The differences are not significant. But it makes one curious about the methodology used – both in the redistribution and the transposition. Do poll boundaries remain static from one election to the next? Is redistribution done by reallocating total poll districts or are poll districts split up? How are mobile polls and SVR1 polls (which presumably do not correspond to poll boundaries) allocated in the transposition process? How are spoiled ballots accounted for?

  14. Pat, I linked to the methodology explanation on the Elections Canada website. Yes, polling divisions had to be split, as did SVR and even some advance polls – jobs for which only Elections Canada and StatsCan would have the correct detailed information to make it accurate. In fact those are accurate, it’s the *POPULATION* that was not transposed correctly.

    If it were up to me, I’d amend the Elections Act and Electoral Boundaries Readjustment Act to stipulate that polling division boundaries had to follow dissemination block boundaries, and could not cross census tract boundaries. And same goes for redrawn riding boundaries. It’s frikkin’ ridiculous how much extra time and work is created later on, by a lack of care to this kind of detail earlier in the process.

    Also, polling division boundaries are not static, but they should be over the life of a representation order, save for (permanent) geographic splits and (urgent) alphabetic ones. A local DRO should not be allowed to just redraw them for every election, trying to meet the Act’s numeric requirements foe electors/poll. Merging and voiding are fine, but the blessed numbering and boundaries should not be messed with!

  15. I’m working on trying to pick apart the transposition data in order to (a) verify the results and (b) repeat the transposition for byelections since 2011. I was wondering if you ended up doing either of those two things, or if you just used the EC transposition data directly for the riding profiles? Give me a shout if you’d like to chat about this.

  16. I just used the Elections Canada transposition data, Matthew.

Leave a Reply