If you are just looking for a general view (a feel) on a subject, then the quality of your survey tool can be loose. However, it is important to consider the limitations of your study particularly if you plan to act on its’ output. Whilst there are many aspects that require consideration in constructing a study; the number of respondents is an important component.
Proportion Measurements
In this article the focus is on proportion measurements; a common output for surveys. For proportion measurement; the errors in the % output can be very large for small sample sizes. This is particularly an issue if the measured proportion is small.
For example, if you wanted to gain an understanding on the proportion of customers choosing your product if in blue packaging versus red? If your output is anticipated to be in a 20% range then for a sample of 50 respondents, the error is +/- 11%. Not a very informative result.
If your measured proportion is moving from 20% to 24%, it would be difficult to make a conclusion on making a potentially costly change with only 50 respondents. In this case, you might consider 1500 respondents which would make the error +/- 2% and improve confidence on moving to blue packaging.
If you are in the process of deciding on the number of respondents for a given project, click on Sample Size Error Chart to review possible errors for a given respondent count at a 95% confidence level. This may help you negotiate and choose an appropriate respondent number, particularly where the cost of acquiring them is high. As a general rule, if costs permit, the more respondents the better.
More ideally, you are being guided by someone with expertise through limitations of your full study; either to help restructure it to meet your needs and/or use the results in context of its limitations.