Does survey timing affect the results you get?

A frequently asked question when we’re scoping a project is: when is the best time to send out a survey? You might think that the summer is optimal because the sun makes people happier, and there is evidence to support that. But being in a good mood may not mean you rate a product or service higher than on any other day, and the typical British summer can include as many downpours as sunny days. So what’s the answer?

As you might expect, there’s been some research on this.

“Prior studies on the association between weather and psychological changes have produced mixed results. In part, this inconsistency may be because weather’s psychological effects are moderated by two important factors: the season and time spent outside. In two correlational studies and an experiment manipulating participants’ time outdoors (total N = 605), pleasant weather (higher temperature or barometric pressure) was related to higher mood, better memory, and “broadened” cognitive style during the spring as time spent outside increased. The same relationships between mood and weather were not observed during other times of year, and indeed hotter weather was associated with lower mood in the summer. These results are consistent with findings on seasonal affective disorder, and suggest that pleasant weather improves mood and broadens cognition in the spring because people have been deprived of such weather during the winter.

“How do other temperature conditions, such as precipitation, alter respondents’ representativeness and political opinions? Does temperature differentially affect other subpopulations, such as those caring for children, living in different regions, or in rural areas? Considering these factors may pinpoint the particular causal mechanisms leading different groups to react differently to temperature. The answers will help sharpen inferences about how weather shapes mood, opinion, and society.”

So it’s not straightforward.

Let’s look at our own patterns of research – and consider what “best” means in this context. Do we want the highest response rate or the most positive results?

Taking responses to our paper surveys during the last five years only (sadly, we can’t include the telephone surveys which now account for more than half of the STAR projects we undertake), the autumn has been the most popular time to question residents.

Almost a third of the work has been carried out in September and October, with the summer and December the least popular periods.

Chart showing popular survey monhts

What do the response rates tell us?

They varied from one landlord who only managed to achieve 30%, up to an amazing 78%, with an average of 49%. It looks like the highest response rates can be achieved between January and April, with most of the summer months (May, July and August in particular) and November best avoided.

Chart showing response rates by month

Do the highest response times and highest satisfaction correlate?

Yes, there’s a similar pattern but a moderately strong correlation was found between response rate and satisfaction (r=0.542) so this is to be expected.

So what does this mean?

Probably not a lot. The results need to be interpreted with caution as given the relatively small sample the findings might be influenced by the type of landlord involved and the general level of satisfaction each month. Remember, no two landlords are alike.

But it does make you think.

What’s my advice about survey timing?

If you’re doing one-off surveys, stick to the same time of year. Residents can get used to the timing and may expect to complete the form. If you want to make that time of year autumn, then that’s up to you!

Considering a survey?

Now is the perfect time to start planning, especially if you’ve got a quieter period over the summer. Contact us to talk through the possibilities, and we can help you build something which will meet your needs and those of your residents.

About Denise Raine

Denise Raine is a Director of Acuity Ltd, and has specialised in tenant satisfaction and feedback research for more than 20 years.
Comments are closed.