Sample sizes

Luckily, you are just seconds away from some very smart brand marketing solutions. Click here!

Key Point

One of the most common questions business managers ask when wanting to commission some research is "What sample size do I need?".

The response from the Marketing Research Manager is then usually "Why do you want to know?".

This is not as unhelpful a response as you may think.

In more detail…..

When choosing your sample for marketing research, there are two fundamental questions you need to ask:

1. Will the information be valid - will it have measured accurately what I wanted to measure?
2. Will it be reliable - if I were to run the exercise again, would I get the same result?

Different types of research answer these questions in different ways.

Exploratory (qualitative research)

Exploratory research (generally called "qualitative" research) is used when you do not know what answers the interviewees will give. Quantitative research is used when you do know what the range of likely answers is - you simply do not know how many people will give each answer.

Exploratory research is aimed at gaining either a broad or a deep understanding of the market place:

• the broad understanding of the market gives you a general picture from which you can ask more detailed questions during the quantitative research phase. What you are looking for here are not answers, but better defined questions
• the deep understanding of the market is where you are looking for information that you would not necessarily be able to get on a quantitative basis. The most powerful information in research is often unobtainable quantitatively

So, qualitative research is about getting valid knowledge of your market place, without formally counting how many people said what - therefore statistical laws do not apply. What you normally do is carry on interviewing people until you can reliably anticipate what they are likely to say, while trying to ensure representative coverage of the market place.

In individual interviews, this will happen between 5 interviews (in a very homogeneous market) and 75 interviews (in a very heterogeneous market). It is most likely to happen between 10 and 30 interviews.

If you are using focus groups (where you interview 6-10 people at a time), you need to conduct at least 3 focus groups as you are likely to get one "rogue" group where you are given answers that are very different from the norm. If you only commission one focus group, you do not know whether it is a rogue group or not; if you commission two that disagree with one another, you do not know which one is the rogue. As you may be looking at different sets of people in your market place, the usual number of focus groups commissioned is 4, 6, 8, 10 or 12. The number of groups you run is dictated by how many different segments of the market you are examining (e.g. by age, sex, geography, attitude, situation, behavior etc.).

Having suggested that qualitative research does not deliver statistically reliable data, there are types of exploratory research where numbers are counted. In the US, focus groups will usually take scores from the interviewees - leading US marketers to be very sceptical of the results of focus groups. In Europe, some research agencies use rigorous content analysis to generate numerical results, not so as to provide statistically reliable information, but rather to get a more accurate picture of what was being said by the interviewees. One of the problems with exploratory research is that, because you are not collecting numbers, you can pay undue attention to the most articulate people you interview. Rigorous content analysis avoids this problem.

Quantitative research

In quantitative research, you should know the range of answers you will get to each question, but not how many people will give which answer. Because you are now collecting numbers, statistical laws apply.

Statistically, there are two issues here:

1. Have you got a large enough sample to give you a valid representation of the market place?
2. Have you got a large enough sample to give you reliable results?

While you can begin to apply normal distribution statistics to sample sizes of over 50 people (and non-parametric statistics to smaller sample sizes), it is usual to interview a minimum of 150 people, especially if you are conducting tracking research where you will want to know whether your results are better or worse than last time.

If you use sample sizes of 200 interviewees on each occasion, this will allow you to say (with 95% confidence of being correct) that one year's results are better or worse than the previous results if those results are 10% apart or more (where the scores are nearer 50%), or 6% apart or more (if the results are nearer 0% or 100%). If you use a sample size of 1,000 interviewees on each occasion, the results only need to be 5% or more apart or 3% or more apart, respectively.

So, the basic rule is that the larger your sample size, the more likely it is to be representative of the market as a whole, and the smaller the differences need be between the results for you to declare a statistically significant difference.

Of course, in some business-to-business markets, you may not have 200 people in the market place. Unfortunately, normal distribution statistics do not take this factor into account. To be statistically confident of your results, you will need to interview everyone.

The business-to-business market also creates further complications because you often face inequalities of bargaining power. 5 customers may be worth more than the rest of the 500 customers put together. The only way to address this problem statistically is to interview all the key contacts within the top customers, and then a representative sample of the rest, and weight the results accordingly.

Additional statistical treatments

If you want to apply special statistical procedures based on correlation, such as factor analyses (to identify where people answer different questions in the same way) and cluster analyses (to identify where people answer the same questions in the same way), it will again be helpful to have a minimum of a 150-200 sample.

Some statistical techniques prefer even bigger samples. If you want to develop a cluster analysis using Chaid, you should aim for a sample size of over 500.

In summary

The basic rules of sample sizes are therefore these:

1. If you are conducting exploratory research without collecting numbers, try to interview a range of people in your target market place and stop when you have heard enough (probably after 30 interviews, or 4 focus groups)
2. If you want to track numbers over time, or to apply specialist statistical techniques, go for a minimum of 200 interviewees. If you can afford 1,000 (perhaps by using multi-client - omnibus - research), so much the better
3. When it comes to conducting quantitative research, it is usually better to ask far fewer questions of many more people than vice-versa. That way you get more valid and reliable information, and usually a more attentive audience within your own company

Luckily, you are just seconds away from some very smart brand marketing solutions. Click here!

For further information, please contact enquiries@mudvalley.co.uk

© 2004, Mud Valley ™ brand marketing community.

Related answers