top of page

Agora você conta com um assistente de leitura, o chatGPT responde suas perguntas sobre a postagem que está lendo. Somente abra o chat abaixo e faça perguntas!

Asking questions on surveys: it is not that trivial

Updated: Dec 1, 2022

Generally, we assume that asking questions is trivial: you just need to ask. We have essentially two types of questions settings: feedback section and research sections. They may be somehow different. We are to consider the latter. We are considering asking question to support a primary research.

It becomes hard when those assumptions must be dealt with, on a scientific context. Reproducibility and repeatability are two big concerns in scientific experiments, alongside fairness. An unfair survey can be meaningless and misleading as scientific primary research.

What about if your research is asking questions?

As surprise it may be to you, a consirable amount of research is built from questionnaire. On this type of primary research, someone tries to gather learning from questions, from group of people. For instance, you may want to understand what are the population main concerns. Political polls are well-known examples.

In "Grand Challenges and Inductive Methods: Rigor without Rigor Mortis", it is made a set of imperative discussions on the importance of surveys and alike on creating first-hand accounts. They strive to bring attention to those methods a knowledge building: their core concern, expressed on the title "Rigor without Rigor Mortis", is that surveys may be used as inductive method, as knowledge gathering strategy. Thus, it is imperative to make sure the questionnaires will follow proper planning.

Let me share a small online experience that taught me indirectly the importance of questions on asking what you want: somehow, this episode made me think on the importance to ask properly. Nowadays, it is pretty common online Q&A communities. We learn essentially anything using the Socratic method . As I was learning chess, I used computers to teach me moves; and I would use those moves on daily matches. You may call it cheating, nonetheless, I was learning, and I would think before moving. For me, the line here was fuzzy, since I was not formally competing. Thus, I have decided to ask online whether I was cheating.

"I like to play daily chess. However, before I make each move, I test them against the computer. Would it be cheating?" Quora

See the word "cheating". It attached 'yes' answers, withouth any argumentations. It seems the initial answers is no longer available. After this stressing episode, somehow it got me thinking on the importance to ask properly. Strong words, as example, tend to draw attention and energy to those words.

Before you call me stupid, the book "Asking Questions: The Definitive Guide to Questionnaire Design" is full of samples: including from researchers.

“Are you fairly compensated for your work?” is likely to elicit a very different answer than asking “Does your employer or his representative resort to trickery in order to defraud you of part of your earnings?” Karl Marx

See the strong words.

Brian Wansink et al argues that wording is imperative on asking questions! And my experience tends to agree with their argumentation. As they argument: people tend to believe that asking is easy, and tend to expend little time planning the questions. For a Facebook survey, and we have plenty of them here, we can have the luxury to make planning mistakes: not sure for a PhD, or master of science. An entire research can be invalidated, or may be hard to publish. Saw that with a roommate once, from social sciences.

People do not answer questions directly: heuristic vs. true question

Daniel Kahneman in "Thinking, Fast and Slow" bring a very interesting concept on asking questions: people tend to answer an easier question, called the heuristic questions. Have you ever been in an event and making a scientific question and the person would answer something quite different? Most of the time it happened to me, not sure people really got their questions answers on events or just accept for saving face, respecting the speaker. For me, rarely the speaker really answered the question!



Main reference

Asking Questions: The Definitive Guide to Questionnaire Design -- For Market Research, Political Polls, and Social and Health Questionnaires. Book by Brian Wansink, Norman Bradburn, and Seymour Sudman

62 views0 comments


bottom of page