Can your students fix these bad survey questions?

sue_frantz
Expert
Expert
0 0 4,312

Do you cover survey research in your Intro Psych course? Given the prevalence of bad surveys, I’m starting to think I should spend more time on it.

For a seemingly unlimited supply of bad survey questions, check out the @BadSurveyQ Twitter account. (Thank you to Rachel Soicher at Oregon State University for directing me to this.)

Point out to students that not all surveys are written by researchers who have been trained to conduct surveys. In fact, some survey questions are designed to persuade, not to actually gather data. Other survey questions are written by people with good intentions who may not have thought them all the way through. Can you students spot the difference? More importantly, can your students fix the problems?

@KenFernandezPHD shared this slanted poll question. In small groups, ask students to take a crack at rewriting this question in neutral language.

Do you believe the corrupt leadership of the FBI and DOJ [Department of Justice] now realize President Trump means to end their efforts to subvert his presidency?

Yes
No

@magnatom found another slanted poll question. How would your students fix this one?

Do you think the Government will ever seriously look into proven, practical and effective methods to lower vehicle emissions instead of resorting to raising yet more cash from drivers?

Yes
No
No idea

@t_mabon found this limited option question. Can students identify the problem? And then fix it?

How do you read your books?

Papers
e-reader/tablet
I don’t read
Audio books

@sachinsomaiya found a question that left the interpretation of the rating scale up to the reader. How do your students interpret this? How would they make it better?

What priority would you assign to the candidate for this program? Choose a number between 1 to 10 for the person.

@BadSurveyQ wonders about the “other” option in this question. Other what? What would your students do with this “other” option to fix the question?

Which of the following have you done in the last 2 years?

Rented a house
Rented an apartment
Rented a car
Bought a house
Bought an apartment
Bought a car
Other
None of the above

@t_mabon shared a poll question that had responses only a company could love. What additional options would your students add?

Which of the following statements do you agree with? SELECT ALL THAT APPLY

Uber is a company I’m proud to say I use

Uber is a brand/service for me

Uber sends me relevant communications

And one last question from @BadSurveyQ, another question that only a company could love.

Please select three other statements that according to you also apply to a Tassimo machine [coffee maker].

Freedom
Togetherness
Power
Entertaining
Liberating
Fun
Open-minded

Now, with this blog post completed, I’m going to have a long over-due chat with my coffee maker. If it’s not entertaining and open-minded, it’s out of here.

About the Author
Sue Frantz has taught psychology since 1992. She has served on several APA boards and committees, and was proud to serve the members of the Society for the Teaching of Psychology as their 2018 president. In 2013, she was the inaugural recipient of the APA award for Excellence in the Scholarship of Teaching and Learning at a Two-Year College or Campus. She received in 2016 the highest award for the teaching of psychology--the Charles L. Brewer Distinguished Teaching of Psychology Award. She presents nationally and internationally on the topics of educational technology and the pedagogy of psychology. She is co-author with Doug Bernstein and Steve Chew of Teaching Psychology: A Step-by-Step Guide, 3rd ed. and is co-author with Charles Stangor on Introduction to Psychology, 4.0.