Best Of
How to Easily Calculate Confidence Intervals for Your Study
Article written by Parvathi Vijayamohan
Content Marketer at SurveySparrow
9 min read
30 May 2024
Best Of
Article written by Parvathi Vijayamohan
Content Marketer at SurveySparrow
9 min read
30 May 2024
I have a hard time with targets. Now, what does this have to do with ‘calculate confidence intervals‘?
When asked for a target number, I give ranges. Also, I sometimes have trouble predicting how long a task will take once I start it.
Researchers also face this problem. The results of polls, surveys, quizzes, or experiments are derived from a sample, not the entire population. Therefore, survey research is at its core an educated estimate – the result might not be 100% accurate.
That’s why confidence intervals are always useful when reporting on metrics. By doing this, you will be able to show your stakeholders how accurate your results are.
If you’re new to statistics or just a bit rusty, we’ll show you:
To calculate a confidence interval this way, you will need:
To calculate the sample proportion (P), divide the number of successful items (X) by the sample size (N).
P = X / N
You can choose from the 5 confidence levels below. Each level has a score associated with it.
Okay, you’ve got your sample size, sample proportion, and Confidence Level.
Time to calculate the Margin of Error. Head to our MOE calculator and enter the numbers.
You can easily calculate the confidence intervals when you put together the Margin of Error and observed score.
Suppose we want to find out the most popular beverage at the nearby Starbucks.
We randomly collect a sample of 5,000 people entering Starbucks over a month. So our sample size is:
N = 5,000
Based on the order data, we found out that from this sample, 1,276 people ordered a Pumpkin Spice Latte. So, our sampling proportion will be:
P = 1,276/5,000 = 0.255
Since we’re reasonably confident that the sample order data is reflective of the actual population of Starbucks customers, we choose a Confidence interval of 95%
Time to calculate the Margin of Error. We feed the numbers into the calculator, and voila.
Our margin of error is +/-1%.
To recap: According to our observed score, 25.52% of our Starbucks sample stated that the Pumpkin Spice Latte is their preferred cuppa, and we have a margin of error of +/- 1%.
So we can say that 24% to 26% of Starbucks customers prefer the Pumpkin Spice Latte. And there you go!
There are formulas for calculating confidence intervals in specific scenarios that use other components like mean or standard deviation. As an example, here’s a great blog on calculating confidence intervals for A/B testing.
What I’ve shared here is simply a shortcut to get the data you need.
While you can use SurveySparrow’s free calculator anytime, a free trial gives you unlimited access to 600+ customizable survey templates + features which will massively speed up your data collection process.
Sign up below to get a full-featured free trial.
Let us take a detour to understand what a confidence interval is.
In simple terms, confidence intervals are based on an average score, as we saw in the example above, plus or minus some sampling error. In other words, it provides a ballpark within which you can place your estimate.
For example:
Sample size is the number of items or subjects included in a sample in research. This sample will represent the target population of a study.
For example, suppose we want to predict how customers in California and Singapore will react to a product rebranding. In that case, we can first test it on samples representing customers from these countries. The sample size, in this case, will be the number of people from each country to be surveyed.
According to Piroska Bisits-Bullen, Head of Data at Umbo, “A good maximum sample size is usually around 10% of the population, as long as this does not exceed 1000.” For more easy tips to choose a sample size, check out her blog.
Related: Sample Size Calculator – Slovin’s Formula to Calculate Sample Size
Raluca Budiu, Director of Research, Nielsen Norman Group, explains it well:
“For example, you may want to know the percentage of users, out of your whole target population, who are able to successfully place an order on an ecommerce website.
Unless your audience is tiny, you will have thousands of users and it will be unfeasible to determine that number precisely — after all, to compute it, you would need to ask every single member of your audience to place an order on your site and record whether they were successful.
This value of the metric based on the entire population is called the true score — it is something that cannot be determined exactly. However, it is possible to estimate it.”
This is what we do with sample size. The result from the study then gives us an observed score; we use this to estimate the true score.
Wait, we already have a sample size? So what is this sampling proportion all about?
Well, sampling is an experiment for a particular characteristic, right? So in simple terms, a sampling proportion is the share of subjects who have successfully shown that characteristic.
The ideal sampling proportion varies according to your research goals.
For example, suppose you want to get the proportion of defective items coming off an assembly line. Ideally, you want the ratio to be as low as possible, or it won’t look good for your company! In contrast, if you’re surveying people in a hardware store who make a purchase before leaving, you want the proportion to be high.
To quote from our blog on sample size calculation: “Confidence level is the term used to express how confident you are that your sample size for the survey represents the population size.”
Confidence levels are always represented as a percentage. For example, a 90% confidence level means that if you repeat a survey, 90 percent of the time the results will match the actual results from a larger population.
In other words, your data is reliable!
The Margin of Error reflects the amount of sampling error in your survey results. The larger the margin of error, the less confidence one should have that the observed score reflects the true score of the population.
To quote again from our previous blog:
“For example, if you measure customer experience, let’s assume that if we were to average out the satisfaction score of all your customers, that number out of 10 would be an 8. Now, we obviously cannot survey all your customers, so let’s say we surveyed your sample.
Would you be alright with getting a customer satisfaction score in your survey of 7.5? Or perhaps 8.5? It wouldn’t be precise, but it would be somewhere in the ballpark.
In the above example, your margin of error is about 0.5, expressed in percentage as 5%. Usually, a margin of error between 4% and 8% is acceptable and still gives you meaningful results.”
We sometimes interpret confidence intervals to mean that the true score lies within its range. But this is far from true (see what I did there? No?).
A confidence interval only tells us the range of numbers we can expect to get if we do the same study again, in the same way. So to get an observed score that’s close to the true score, it is better to look at your sample and research methods.
Secondly, a confidence interval is an indicator. It is not meant for drawing any conclusions or insights about the target population.
We hope we could help you with this guide for calculating confidence intervals the easy way. Share your comments or suggestions below. Happy surveying!
Thousands of brands trust SurveySparrow to turn feedback into growth. Try it free today!
Best Of
20 MINUTES
26 September 2020
Best Of
10 MINUTES
8 February 2022
Best Of
14 MINUTES
18 June 2020
Best Of
6 MINUTES
13 May 2023
14-day free trial • Cancel Anytime • No Credit Card Required • No Strings Attached
