Can your students fix these bad survey questions?

[sue_frantz]

Do you cover survey research in your Intro Psych course? Given the prevalence of bad surveys, I’m starting to think I should spend more time on it.

For a seemingly unlimited supply of bad survey questions, check out the @BadSurveyQ Twitter account. (Thank you to Rachel Soicher at Oregon State University for directing me to this.)

Point out to students that not all surveys are written by researchers who have been trained to conduct surveys. In fact, some survey questions are designed to persuade, not to actually gather data. Other survey questions are written by people with good intentions who may not have thought them all the way through. Can you students spot the difference? More importantly, can your students fix the problems?

@KenFernandezPHD shared this slanted poll question. In small groups, ask students to take a crack at rewriting this question in neutral language.

Do you believe the corrupt leadership of the FBI and DOJ [Department of Justice] now realize President Trump means to end their efforts to subvert his presidency?

Yes
No

@magnatom found another slanted poll question. How would your students fix this one?

Do you think the Government will ever seriously look into proven, practical and effective methods to lower vehicle emissions instead of resorting to raising yet more cash from drivers?

Yes
No
No idea

@t_mabon found this limited option question. Can students identify the problem? And then fix it?

How do you read your books?

Papers
e-reader/tablet
I don’t read
Audio books

@sachinsomaiya found a question that left the interpretation of the rating scale up to the reader. How do your students interpret this? How would they make it better?

What priority would you assign to the candidate for this program? Choose a number between 1 to 10 for the person.

@BadSurveyQ wonders about the “other” option in this question. Other what? What would your students do with this “other” option to fix the question?

Which of the following have you done in the last 2 years?

Rented a house
Rented an apartment
Rented a car
Bought a house
Bought an apartment
Bought a car
Other
None of the above

@t_mabon shared a poll question that had responses only a company could love. What additional options would your students add?

Which of the following statements do you agree with? SELECT ALL THAT APPLY

Uber is a company I’m proud to say I use

Uber is a brand/service for me

Uber sends me relevant communications

And one last question from @BadSurveyQ, another question that only a company could love.

Please select three other statements that according to you also apply to a Tassimo machine [coffee maker].

Freedom
Togetherness
Power
Entertaining
Liberating
Fun
Open-minded

Now, with this blog post completed, I’m going to have a long over-due chat with my coffee maker. If it’s not entertaining and open-minded, it’s out of here.