Audience survey: quiz or phone call, and how to rank pains (MaxDiff in plain words)

Quiz or phone call

Both methods ask the same thing; they differ in who you can reach and how much you learn from each answer.

• A quiz link scales: you can send it through ads, a mailing list, a website or a messenger and collect hundreds of answers at a low cost per answer. It works best for consumer products and for audiences that already click on links.
• A phone call reaches people who never fill in forms: B2B buyers, older customers, existing clients. Each call costs time, but you also hear why someone chose a pain, in their own words, which feeds straight into ad copy.
• Many teams combine them: calls to existing customers first, to check that the list of pains makes sense, then a quiz for the wider audience.

How to word the list of pains

• One idea per item. Slow and expensive delivery is two pains, not one.
• The customer's language, not internal jargon. Take the wording from the review quotes.
• Neutral, similar length. If one item is long and emotional and the others are dry, people pick the vivid one.
• No product names and no hints at your solution.
• Merge items that mean the same thing before you ask, so they do not split the vote.

Why rating scales mislead

Ask people to rate each pain from one to five and most of them will give fours and fives to everything. Nobody wants to say a problem is unimportant, and a real problem feels important when you read it alone. The result is a list of averages that differ by a few tenths, which is not enough to decide where the budget goes. Forcing a choice removes this effect: if only one of two pains can win, the order appears.

Pairwise ranking

The simplest form shows two pains and asks which one is more acute. Each win counts as a point. It is easy to answer on a phone screen and easy to explain. The catch is the number of pairs: eight pains already make 28 possible pairs. So each respondent sees only some of them, and the pairs are rotated so that every combination appears often enough across all respondents.

MaxDiff in plain words

MaxDiff, also called best–worst scaling, shows a small set of pains, usually four or five, and asks two questions: which is the most important to you, and which is the least. Then it shows another set, mixed differently, and repeats a few times. Over the whole survey every pain appears several times next to different neighbours.

The simplest score is counting: the number of times a pain was chosen as most important minus the number of times it was chosen as least important, divided by the number of times it was shown. The score runs from minus one to plus one. Professional survey tools also use statistical models to estimate a score for each respondent, but for deciding which two or three pains to advertise on, the counting score is a fair first reading.

Asking by phone

1. Introduce yourself, say why you are calling and how long it takes, and ask whether it is a good moment.
2. Read the list of pains in a different order for each call, so the first item does not win just because it was first.
3. Ask one main question: which of these problems is the most acute for you?
4. Ask why, and write down the answer as close to word for word as you can.
5. Log the call: date, result (answered, no answer, refused, call back) and the chosen pain.

Worked example (illustrative numbers)

A home cleaning service has six pains from reviews. 100 people complete a MaxDiff quiz; each sees six sets of four pains, so every pain is shown 400 times in total. The counting scores:

• Cleaners arrive late or do not come: 210 times most, 20 times least, score 0.48.
• No time to clean at weekends: 150 most, 50 least, score 0.25.
• Having to explain everything again each time: 110 most, 80 least, score 0.08.
• The final price is unclear until the end: 70 most, 110 least, score minus 0.10.
• Worry about letting strangers into the home: 40 most, 150 least, score minus 0.28.
• Fear that things get damaged: 20 most, 190 least, score minus 0.43.

The team had planned to advertise on saving weekend time. The survey says reliability matters more, with a clear gap. The first ad set is built on punctuality, the second on free weekends, and the rest of the list goes into the landing page and the FAQ.

After the survey

Read the ranking together with the comments from the calls. A pain that wins by a wide margin becomes the first ad set. A pain that many people chose as least important is still useful: it is a weak hook, but it may be a doubt worth answering on the landing page. And if the results of existing customers and of new people differ, that difference may be worth two separate campaigns rather than one compromise message.

How it works in OrgLines

In the Market research module, which our team runs inside the client's cabinet, the pains found in reviews become a survey. It goes out as a quiz link with pairwise or best–worst ranking, or the team asks by phone the question about the most acute problem and logs every call. The answers produce a ranking of pains, and the client receives a report without names, phone numbers or voice recordings. The top pains can then be handed to Creative Studio for ad creatives.

Common mistakes

• Rating scales instead of forced choice.
• Items of different length and tone, so the most vivid wording wins.
• Two pains in one item.
• The same order of items for everyone, which favours whatever comes first.
• Surveying only happy customers and reading the result as the view of the whole market.
• Stopping after a handful of answers because the leader looks obvious.

Checklist

• Five to twelve pains, one idea each, in customers' words.
• Quiz or call chosen according to who you need to reach.
• Forced choice: pairwise or best–worst.
• Order of items rotated.
• Phone calls logged with result and chosen pain.
• Ranking checked for a clear gap between the top pains before money is spent.

Try it free in OrgLines