Convenient and cost-effective: what’s not to like about using online surveys to find out how happy your residents are with your services?

Quite a lot, as it turns out. Not only can the results be skewed in terms of the age profile, but our new analysis shows residents tend to report lower satisfaction rates when answering surveys online rather than by post – and the difference isn’t all down to their age. Unsurprisingly, there is a lot of debate around which method to use, with landlords keen to explore alternatives to postal surveys.

Here at Acuity we’ve got years of expertise in combining survey methods, questions and analysis to ensure clients get as accurate a picture of satisfaction with their services as possible, with the majority opting for the combination of an online questionnaire plus either telephone or postal survey. But this has to be done with great care, as we’ll explain.

Post, phone or screen: the pros and cons?

It’s easy to see why many landlords see online surveys as the obvious alternative to costly paper, but there are quite a few drawbacks, such as low response rates, out-of-date email addresses and – perhaps most important – potential survey bias towards younger residents (HouseMark advises that digital questionnaires should not be the main method for STAR surveys because of the risk of bias). On the other hand, many landlords like going online to boost response rates and reach younger residents alongside more traditional methods.
Phone surveys have the advantage of better engagement with residents, qualitative as well as quantitative results, and the ability to ensure a representative response from the start – but the disadvantage of higher costs as well as disenfranchising some residents.

Response rates and how to improve them

Getting a good response is the holy grail of survey designers, who find themselves balancing many different factors with clients. There can be a trade-off between the number and type of questions you ask, as well as the route you choose to get them to residents, and the response rate. Many landlords prefer online methods, in the belief that they lead to a higher response rate and greater participation from younger tenants.

The average online response rate for the last 10 surveys done by Acuity is 14 per cent, ranging from just two per cent in a London borough to 37 per cent for a small housing association in the South-West. Delve a little deeper into these stats, and we find that actually online surveys have become more popular with residents. Half of the landlords had response rates of over 20 per cent, with just two under 10 per cent. Unsurprisingly higher response rates correlated with the number of email addresses that landlords hold.

Paper versus screen: do they get the same answers?

Half of the landlords got considerably higher ratings for the six core benchmarking questions (overall services, repairs, home, value for money of the rent, neighbourhood and listens to views) from the postal surveys compared with online surveys (averaging 8% to 15% higher across the six questions). The pattern was less obvious for the other half, with some services getting higher approval from postal respondents and others rated better by online residents. The considerable difference between the two survey methods for one landlord is shown below.

Do younger residents respond better to online surveys?

Unsurprisingly, yes. For one landlord, just over a third of residents under 35 completing the survey did so online compared to just nine per cent of those over 65.

It’s easy to assume that satisfaction rates will therefore be lower in online surveys, because they are most likely to be completed by younger residents who are themselves typically less satisfied than their older neighbours. But this doesn’t tell the whole story, which is that older residents who respond online are also less satisfied, and for some benchmarks the difference is marked – as the example below, from one survey, shows.

Useful information: how should you use it?

Our analysis confirms that online bias does exist, and that it is a factor for respondents of all ages, not just the under-35s. Together with the knowledge that earlier, postal, surveys are likely to have inflated satisfaction ratings by under-representing younger residents,  it is clear that care and expertise are needed to properly understand both historical trends and also mixed-method surveys.
Given that mixed-method surveying will be necessary to reach the full resident demographic for years to come, it is increasingly important that landlords consider the implications and complications of such exercises and take expert advice.

We hope this has helped to clarify some of the issues and dilemmas around how to get the most accurate and effective feedback from residents. Our experienced team is here to help. We welcome all questions and enquiries, and can help you through the survey process in the way which best suits your organisation, whether providing specialist consultant support to your team, or running it on your behalf.