How many survey responses are enough to launch ads?

There is no magic number of answers. There is a question you can answer with simple arithmetic: is the gap between the top pains bigger than the noise?

The margin of error in plain numbers

If you ask people which pain is the most acute and 40% choose the same one, that 40% is an estimate. With a random sample, the 95% margin of error for a share near 50% is roughly 1 divided by the square root of the number of answers:

  • 50 answers: about ±14 percentage points.
  • 100 answers: about ±10.
  • 200 answers: about ±7.
  • 400 answers: about ±5.
  • 1,000 answers: about ±3.

Notice how slowly it shrinks. Going from 100 to 400 answers costs four times as much and only halves the margin. For choosing the pain to advertise on, the question is not how precise each share is, but whether the order of the top pains is clear.

Look at the gap, not at the total

When the leader is far ahead, a small survey is enough to see it. When two pains are close, no reasonable survey will separate them, and that is a finding too: test both in ads with similar budgets. The difference between two shares from the same people is a bit less certain than each share alone, so ask for a gap clearly larger than the margin, not just equal to it.

A small example (illustrative numbers)

A B2B supplier calls 60 existing customers and asks which problem is the most acute. 27 choose late deliveries (45%), 12 choose unclear invoices (20%), the rest split among four other pains. At 60 answers the margin is about ±13 points. The 25-point gap between the first and second pain is clearly larger, so late deliveries can lead the campaign. The gap between second and third place is a few points, so those two get equal, smaller budgets.

When to stop collecting

  • Look at the ranking after every batch of answers. When new batches stop moving the top two or three pains, more answers will mostly confirm what you know.
  • If you plan separate ads for different segments, each segment needs enough answers on its own.
  • Who answered matters as much as how many. Two hundred answers from people who already love the product describe your fans, not the market. Mix sources where you can and note where each answer came from.

In OrgLines, the Market research module lets our team add new survey answers at any stage, from the quiz or from logged phone calls, so you can watch whether the top pains keep their places before the ad budget is committed.

Try it free in OrgLines

FAQ

Frequently asked questions

Is 30 answers ever enough?

Sometimes, if one pain is far ahead and the audience is small, for example a B2B niche with a few dozen clients. The margin at 30 answers is around ±18 points, so treat the result as a direction to test, not a final ranking.

Does MaxDiff need fewer respondents than a single question?

Each MaxDiff respondent makes several choices, so you get more information per person than from one question. It still pays to collect enough respondents to cover different kinds of buyers, and to check that the ranking is stable.

Do the margins assume a random sample?

Yes. Real surveys rarely are, so the margins are a lower bound on the uncertainty. That is another reason to want a clear gap and to check the ranking against ad results later.

More on Marketing agencies

All guides on this topic