Surrey, BC – February 16, 2016

Some of you might be already familiar with the comments I made through SkyTrain for Surrey on the new LRT survey that was released by the City, claiming 80% of residents are in support of the LRT project. If you aren’t, my chief complaint is that only 600 residents were asked, which means that about 0.1% of residents are being asked to represent a City of over 500,000.

This statement has been met with a mixed response: some people agreed that such a small number shouldn’t represent the city by any means; others disagreed, telling me that I was going up against a professional organization and that the sample size and margin of error was acceptable.

With that said, I was prompted to look into finding even more answers. After turning to my connections in the community, Ipsos Reid’s entire, detailed LRT survey results paper managed to find its way to my e-mail inbox. You can download the results and verify my findings yourself below:

LRT survey screenshot

Download the results

When I opened the PDF document for the first time, the first thing that caught my eyes within the tables and tables of info was the composition of the respondents (this data I am very glad to have collected), followed by the composition of the actual questions. Here are the things that stood out the most to me:

The survey asked only 85 actual transit riders.

A next-generation 96 B-Line bus
A next-generation 96 B-Line bus

Yes, not 85% – 85 out of 600. Out of thousands upon thousands of Surrey transit riders, the surveyors are asking for representation from just 85. All other respondents drive for their commute.

This isn’t only low to begin with, but it’s also lower than the “weighted” base (i.e. if the amount of transit riders asked is to be in-line with the actual percentage of transit users in the city, then the poll should have asked 111 transit riders). For a poll that’s supposed to decide on future transit matters, you’d think that more actual transit riders would be consulted on this – which is sorely disappointing.

Let’s put that into another perspective. Surrey’s 4 SkyTrain stations service 39,169 passenger boardings per weekday. There are many more transit boardings on buses in Surrey, but if we start with the amount of SkyTrain riders, then approximately just 0.2% of Surrey’s transit riders are being asked to decide for all of them on future rapid transit.

I get that there aren’t relatively a lot of people in Surrey who ride transit compared to the amount driving, but neglecting transit rider input for a transit project is absolutely ridiculous. If you agree that it’s ridiculous, then prepare yourself because this is only where I begin…

Many respondents didn’t live near the proposed LRT lines.

LRT survey table 1

The three LRT lines are supposed to travel on 104 Ave, Fraser Highway and King George Blvd. – serving City Centre, Fleetwood, Guildford and Newton. But when compared against the weighted average, the amount of respondents that were from Cloverdale and South Surrey – areas that aren’t necessarily near the proposed LRT lines, requiring connections by bus – was significant in contrast to the amount of respondents that actually live near them and would more likely use them. Both of these areas exceeded their “weighted” base.

Concerningly, very few of the respondents (just 89, compared to a weighted base of 147) live in Whalley or City Centre, which is where one would expect most of Surrey’s transit ridership to come from – since riders here would have access to all 3 proposed lines, SkyTrain and other buses.

The survey weighs these answers in attempt to gather a fairer perspective from these neighbourhoods; regardless, with these numbers on where the respondents are actually from, I definitely don’t feel that accurate information has been collected. The survey neglects people whose lives would actually be affected by the construction and operation of the new LRT lines.

The age of the respondents is out of touch with the city’s composition.

LRT survey table 2

I don’t mean to pick on seniors for any reason, but there were 270 people aged 55+ who responded to this survey – against a significantly lower weighted base of 186. On top of that, forty-five per cent of this group said they would never use an LRT system. Yes, you heard that right – there were more non-transit users aged 55+, than transit users of any age group, polled in this new Surrey LRT survey. Is that misleading or what?

The thing I’m even more concerned about, however, is that very few of the respondents (120) were aged 18-34. That means that the least responses were collected from the age demographic that is statistically the most likely to use transit.

That these respondents were weighted serves as no excuse. This is completely out of touch with the city’s composition, and I would expect the input to be more considerate in its distribution considering that over 25% of the city’s population – by that I mean children and youth aged 0-19, many of who will be moving into the 18-34 age bracket by the time of the LRT system’s launch – was not included in the survey.

Respondents weren’t asked to consider LRT against other alternatives.

For me one the most alarming aspects of this survey is that the question of whether a respondent supports LRT or doesn’t was narrowed down to a simplified yes-or-no question, without any chance to weigh LRT against other alternatives (like SkyTrain and Bus Rapid Transit) – and without any consideration of the LRT project’s own practicalities.

In some cases (like on 104th Avenue, which is served by both the 96 B-Line and a nonstop frequent #337 bus), the future LRT is not as fast as existing buses. If the questions were modified to reveal the future LRT travel times then the conversation would probably have changed immediately. Instead, we’re supposed to rely on answers to vague questions that don’t create the opportunity to consider issues with the LRT proposal.

If a survey is going to conclude a support for LRT technology, it must absolutely consider the alternatives and present them to respondents. I’ve been saying for a long time that the City of Surrey has refused to open a dialogue on LRT benefits/tradeoffs, as well as LRT alternatives, and that it is something that badly needs to be done. Instead, we’re supposed to decide the future based on uneducated opinions supplied by a handful of completely misled people.

Was a phone survey even the right idea?

Phone lulz

My professional day job happens to be in the same field as the people who conducted this survey: canvassing people over the phone. As the client manager for a service-oriented company, having phone conversations with people is something I do all the time. And, while I approach this from a business/sales environment rather than that of a polling company seeking opinions, there is one thing I will say in confidence: this kind of survey should not have been done over the phone.

The thing about phone calls is that they’re unexpected – people don’t want to stay on the phone; they just want to get off it and go back to their day. The telephone is a great place to repeat a written statement, have a quick chat with a friend or land a sale/appointment for your service; but it’s a terrible place to expect a well-thought-out, educated answer from a stranger who’s expected to provide one with very little thinking, on-the-spot.

As a demonstration of this, when respondents were asked some of the more detailed questions, like: “What would be your main question or concern about building this LRT network?” or “What do you think would be the main benefit of building this LRT network?”, most of the answers grouped into specific ones like: “Cost/funding” or “Traffic flow/congestion problems/concerns”, but relatively few of the answers were unique answers in the other categories. There are places for phone surveys, but this clearly wasn’t one. I hate to say it, but we really shouldn’t expect people to spend time and effort thinking about transit issues over the phone.

How many people rejected the survey phone call? Well, the survey numbers I was sent don’t even reveal that number. We will never know whether the 600 respondents were 600 out of 1,000, or if they were actually 600 out of 10,000.

What about the other surveys?

By the way, this isn’t the City of Surrey’s first LRT survey.

Back in the fall of last year, Surrey had an LRT survey done on their internal, online CitySpeaks platform. I took this survey, and in the process made notation on SkyTrain for Surrey of an error in the comparison between rapid buses and the proposed LRT system.

However, the results of this survey were never released. There is mention of the survey on the CitySpeaks page on the City website, but Surrey has never released the survey results or used them anywhere.

It is plausible that the respondents, given room to think (as this was an online survey with no time-limits or on-the-spot pressures), did not respond favourably to the idea of an LRT system. And, it is plausible that this was withheld by the city in favour of paying a pollster to perform another survey with the intention of achieving a favourable result.

In conclusion: The public is being fooled.

Vehicle train collision small header

What in the blazes is going on here?

I can’t even think of where I should start but the numbers that I’ve been given have made it expressly clear that this is a terrible survey. It definitely does not confirm that 80% of Surrey residents support an LRT system, or come to any other conclusion on matters of Surrey transit.

Not only is it unable to effectively conclude that an LRT system would be popular with transit riders (because it doesn’t ask them), but it makes no effort to consider the younger residents who will grow up and be stuck with such a system, by neglecting to include them as part of the conversation and favouring responses from non-transit users aged 55+ instead. It is also using the worst possible format to collect this sort of information (over the phone), and that weakness is visible in many places in the survey results.

The end result is nothing short of unacceptable, and that’s before you even take into account the fact that the 600 respondents makes up approximately just 0.1% of the actual population of this city – a percentage that will get smaller as the city grows ever larger.

Before we come to a conclusion on surveys like the new Ipsos Reid survey, I would like to see more and different surveys – and I would like to see them done fairly, with a consideration of those who actually ride transit, and with the ability to consider LRT against different alternatives including SkyTrain and Bus Rapid Transit.

Deconstructing Surrey’s LRT survey

3 thoughts on “Deconstructing Surrey’s LRT survey

  • February 17, 2016 at 5:29 pm
    Permalink

    Thank you Daryl! Media is ignoring your findings. I guess they are payed off.

    Reply
    • February 17, 2016 at 5:58 pm
      Permalink

      Thanks for the comment! I actually haven’t sent this out to media yet – right now I’m testing the response to the LRT survey on the blog-o-sphere and giving things time to even out before I engage a major media push around this analysis. Have you signed my petition yet? (see: http://skytrainforsurrey.org/petition/) we’re getting close to hitting 1000 supporters so it’d help me if you could share it around for this final push!

      Reply
  • February 25, 2016 at 10:21 am
    Permalink

    Daryl there are ways to check mathematically if these answers are not lining up. Get a copy of the questions to make sure they were not leading questions. Also, did they ask what the person thought LRT actually is as compared to Skytrain? You can check the mathematic relationship between the questions to make sure that they are not asking the same thing and or have no mathematical relation to the answers at all. I am sure that by the time you have hit 20 you have done at least one basic stats and or survey course at school. You can measure if the sample size has a normal distribution or not, this would add value to your claim that the sample size is not indicative of the population as a whole.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *