Confidence interval for estimating the mean (dispersion is known) in MS EXCEL. Quantitative Analysis Methods: Estimating Confidence Intervals
"Katren-Style" continues to publish a cycle of Konstantin Kravchik on medical statistics. In two previous articles, the author touched on the explanation of such concepts as and.
Konstantin Kravchik
Mathematician-analyst. Specialist in the field of statistical research in medicine and the humanities
Moscow city
Very often in articles on clinical trials you can find a mysterious phrase: "confidence interval" (95% CI or 95% CI - confidence interval). For example, an article might say: "Student's t-test was used to assess the significance of differences, with a 95% confidence interval calculated."
What is the value of the "95% confidence interval" and why calculate it?
What is a confidence interval? - This is the range in which the true mean values in the population fall. And what, there are "untrue" averages? In a sense, yes, they do. In we explained that it is impossible to measure the parameter of interest in the entire population, so the researchers are content with a limited sample. In this sample (for example, by body weight) there is one average value (a certain weight), by which we judge the average value in the entire general population. However, it is unlikely that the average weight in the sample (especially a small one) will coincide with the average weight in the general population. Therefore, it is more correct to calculate and use the range of average values of the general population.
For example, suppose the 95% confidence interval (95% CI) for hemoglobin is between 110 and 122 g/L. This means that with a 95 % probability, the true mean value for hemoglobin in the general population will be in the range from 110 to 122 g/l. In other words, we do not know the average hemoglobin in the general population, but we can indicate the range of values for this feature with 95% probability.
Confidence intervals are particularly relevant to the difference in means between groups, or what is called the effect size.
Suppose we compared the effectiveness of two iron preparations: one that has been on the market for a long time and one that has just been registered. After the course of therapy, the concentration of hemoglobin in the studied groups of patients was assessed, and the statistical program calculated for us that the difference between the average values of the two groups with a probability of 95% is in the range from 1.72 to 14.36 g/l (Table 1).
Tab. 1. Criterion for independent samples
(groups are compared by hemoglobin level)
This should be interpreted as follows: in a part of patients in the general population who take a new drug, hemoglobin will be higher on average by 1.72–14.36 g/l than in those who took an already known drug.
In other words, in the general population, the difference in the average values for hemoglobin in groups with a 95% probability is within these limits. It will be up to the researcher to judge whether this is a lot or a little. The point of all this is that we are not working with one average value, but with a range of values, therefore, we more reliably estimate the difference in a parameter between groups.
In statistical packages, at the discretion of the researcher, one can independently narrow or expand the boundaries of the confidence interval. By lowering the probabilities of the confidence interval, we narrow the range of means. For example, at 90% CI, the range of means (or mean differences) will be narrower than at 95% CI.
Conversely, increasing the probability to 99% widens the range of values. When comparing groups, the lower limit of the CI may cross the zero mark. For example, if we extended the boundaries of the confidence interval to 99 %, then the boundaries of the interval ranged from –1 to 16 g/L. This means that in the general population there are groups, the difference between the averages between which for the studied trait is 0 (M=0).
Confidence intervals can be used to test statistical hypotheses. If the confidence interval crosses the zero value, then the null hypothesis, which assumes that the groups do not differ in the studied parameter, is true. An example is described above, when we expanded the boundaries to 99%. Somewhere in the general population, we found groups that did not differ in any way.
95% confidence interval of difference in hemoglobin, (g/l)
The figure shows the 95% confidence interval of the mean hemoglobin difference between the two groups as a line. The line passes the zero mark, therefore, there is a difference between the means equal to zero, which confirms the null hypothesis that the groups do not differ. The difference between the groups ranges from -2 to 5 g/l, which means that hemoglobin can either decrease by 2 g/l or increase by 5 g/l.
The confidence interval is a very important indicator. Thanks to it, you can see if the differences in the groups were really due to the difference in the means or due to a large sample, because with a large sample, the chances of finding differences are greater than with a small one.
In practice, it might look like this. We took a sample of 1000 people, measured the hemoglobin level and found that the confidence interval for the difference in the means lies from 1.2 to 1.5 g/L. The level of statistical significance in this case p
We see that the hemoglobin concentration increased, but almost imperceptibly, therefore, the statistical significance appeared precisely due to the sample size.
Confidence intervals can be calculated not only for averages, but also for proportions (and risk ratios). For example, we are interested in the confidence interval of the proportions of patients who achieved remission while taking the developed drug. Assume that the 95% CI for the proportions, i.e. for the proportion of such patients, is in the range 0.60–0.80. Thus, we can say that our medicine has a therapeutic effect in 60 to 80% of cases.
Confidence interval
Confidence interval- a term used in mathematical statistics for interval (as opposed to point) estimation of statistical parameters, which is preferable with a small sample size. The confidence interval is the interval that covers the unknown parameter with a given reliability.
The method of confidence intervals was developed by the American statistician Jerzy Neumann, based on the ideas of the English statistician Ronald Fischer.
Definition
Confidence interval parameter θ random variable distribution X with trust level 100 p%, generated by the sample ( x 1 ,…,x n), is called an interval with boundaries ( x 1 ,…,x n) and ( x 1 ,…,x n) which are realizations of random variables L(X 1 ,…,X n) and U(X 1 ,…,X n) such that
.The boundary points of the confidence interval are called confidence limits.
An intuition-based interpretation of the confidence interval would be: if p is large (say 0.95 or 0.99), then the confidence interval almost certainly contains the true value θ .
Another interpretation of the concept of a confidence interval: it can be considered as an interval of parameter values θ compatible with experimental data and not contradicting them.
Examples
- Confidence interval for the mathematical expectation of a normal sample ;
- Confidence interval for the normal sample variance .
Bayesian Confidence Interval
In Bayesian statistics, there is a definition of a confidence interval that is similar but differs in some key details. Here, the estimated parameter itself is considered a random variable with some given a priori distribution (uniform in the simplest case), and the sample is fixed (in classical statistics, everything is exactly the opposite). The Bayesian-confidence interval is the interval covering the parameter value with the posterior probability:
.Generally, classical and Bayesian confidence intervals are different. In the English-language literature, the Bayesian confidence interval is usually called the term credible interval, and the classic confidence interval.
Notes
SourcesWikimedia Foundation. 2010 .
- Baby (film)
- Colonist
See what "Confidence Interval" is in other dictionaries:
Confidence interval- the interval calculated from the sample data, which with a given probability (confidence) covers the unknown true value of the estimated distribution parameter. Source: GOST 20522 96: Soils. Methods of statistical processing of results ... Dictionary-reference book of terms of normative and technical documentation
confidence interval- for a scalar parameter of the general population, this is a segment that most likely contains this parameter. This phrase is meaningless without further clarification. Since the boundaries of the confidence interval are estimated from the sample, it is natural to ... ... Dictionary of Sociological Statistics
CONFIDENCE INTERVAL is a parameter estimation method that differs from point estimation. Let a sample x1, . be given. . ., xn from a distribution with a probability density f(x, α), and a*=a*(x1, . . ., xn) is the estimate α, g(a*, α) is the probability density of the estimate. Are looking for… … Geological Encyclopedia
CONFIDENCE INTERVAL- (confidence interval) The interval in which the confidence of a parameter value for a population derived from a sample survey has a certain degree of probability, such as 95%, due to the sample itself. Width… … Economic dictionary
confidence interval- is the interval in which the true value of the determined quantity is located with a given confidence probability. General chemistry: textbook / A. V. Zholnin ... Chemical terms
Confidence interval CI- Confidence interval, CI * davyaralny interval, CI * confidence interval interval of the sign value, calculated for c.l. distribution parameter (e.g. the mean value of a feature) over the sample and with a certain probability (e.g. 95% for 95% ... Genetics. encyclopedic Dictionary
CONFIDENCE INTERVAL- the concept that arises when estimating the parameter statistich. distribution by interval of values. D. i. for the parameter q corresponding to the given coefficient. confidence P, is equal to such an interval (q1, q2) that for any distribution of the probability of inequality ... ... Physical Encyclopedia
confidence interval- - Telecommunication topics, basic concepts EN confidence interval ... Technical Translator's Handbook
confidence interval- pasikliovimo intervalas statusas T sritis Standartizacija ir metrologija apibrėžtis Dydžio verčių intervalas, kuriame su pasirinktąja tikimybe yra matavimo rezultato vertė. atitikmenys: engl. confidence interval vok. Vertrauensbereich, m rus.… … Penkiakalbis aiskinamasis metrologijos terminų žodynas
confidence interval- pasikliovimo intervalas statusas T sritis chemija apibrėžtis Dydžio verčių intervalas, kuriame su pasirinktąja tikimybe yra matavimo rezultatų vertė. atitikmenys: engl. confidence interval rus. trust area; confidence interval... Chemijos terminų aiskinamasis žodynas
Suppose we have a large number of items with a normal distribution of some characteristics (for example, a full warehouse of vegetables of the same type, the size and weight of which varies). You want to know the average characteristics of the entire batch of goods, but you have neither the time nor the inclination to measure and weigh each vegetable. You understand that this is not necessary. But how many pieces would you need to take for random inspection?
Before giving some formulas useful for this situation, we recall some notation.
First, if we did measure the entire warehouse of vegetables (this set of elements is called the general population), then we would know with all the accuracy available to us the average value of the weight of the entire batch. Let's call this average X cf .g en . - general average. We already know what is completely determined if its mean value and deviation s are known . True, so far we are neither X avg. nor s we do not know the general population. We can only take some sample, measure the values we need and calculate for this sample both the mean value X sr. in sample and the standard deviation S sb.
It is known that if our custom check contains a large number of elements (usually n is greater than 30), and they are taken really random, then s the general population will almost not differ from S ..
In addition, for the case of a normal distribution, we can use the following formulas:
With a probability of 95%
With a probability of 99%
In general, with probability Р (t)
The relationship between the value of t and the value of the probability P (t), with which we want to know the confidence interval, can be taken from the following table:
Thus, we have determined in what range the average value for the general population is (with a given probability).
Unless we have a large enough sample, we cannot claim that the population has s = S sel. In addition, in this case, the closeness of the sample to the normal distribution is problematic. In this case, also use S sb instead s in the formula:
but the value of t for a fixed probability P(t) will depend on the number of elements in the sample n. The larger n, the closer the resulting confidence interval will be to the value given by formula (1). The t values in this case are taken from another table (Student's t-test), which we provide below:
Student's t-test values for probability 0.95 and 0.99
Example 3 30 people were randomly selected from the employees of the company. According to the sample, it turned out that the average salary (per month) is 30 thousand rubles with an average square deviation of 5 thousand rubles. With a probability of 0.99 determine the average salary in the firm.
Decision: By condition, we have n = 30, X cf. =30000, S=5000, P=0.99. To find the confidence interval, we use the formula corresponding to the Student's criterion. According to the table for n \u003d 30 and P \u003d 0.99 we find t \u003d 2.756, therefore,
those. desired trust interval 27484< Х ср.ген
< 32516.
So, with a probability of 0.99, it can be argued that the interval (27484; 32516) contains the average salary in the company.
We hope that you will use this method without necessarily having a spreadsheet with you every time. Calculations can be carried out automatically in Excel. While in an Excel file, click the fx button on the top menu. Then, select among the functions the type "statistical", and from the proposed list in the box - STEUDRASP. Then, at the prompt, placing the cursor in the "probability" field, type the value of the reciprocal probability (that is, in our case, instead of the probability of 0.95, you need to type the probability of 0.05). Apparently, the spreadsheet is designed so that the result answers the question of how likely we can be wrong. Similarly, in the "degree of freedom" field, enter the value (n-1) for your sample.
The confidence interval came to us from the field of statistics. This is a defined range that serves to estimate an unknown parameter with a high degree of reliability. The easiest way to explain this is with an example.
Suppose you need to investigate some random variable, for example, the speed of the server's response to a client request. Each time a user types in the address of a particular site, the server responds at a different rate. Thus, the investigated response time has a random character. So, the confidence interval allows you to determine the boundaries of this parameter, and then it will be possible to assert that with a probability of 95% the server will be in the range we calculated.
Or you need to find out how many people know about the brand of the company. When the confidence interval is calculated, it will be possible, for example, to say that with a 95% probability the share of consumers who know about this is in the range from 27% to 34%.
Closely related to this term is such a value as the confidence level. It represents the probability that the desired parameter is included in the confidence interval. This value determines how large our desired range will be. The larger the value it takes, the narrower the confidence interval becomes, and vice versa. Usually it is set to 90%, 95% or 99%. The value of 95% is the most popular.
This indicator is also influenced by the variance of observations and its definition is based on the assumption that the feature under study obeys. This statement is also known as Gauss' Law. According to him, such a distribution of all probabilities of a continuous random variable, which can be described by a probability density, is called normal. If the assumption of a normal distribution turned out to be wrong, then the estimate may turn out to be wrong.
First, let's figure out how to calculate the confidence interval for Here, two cases are possible. Dispersion (the degree of spread of a random variable) may or may not be known. If it is known, then our confidence interval is calculated using the following formula:
xsr - t*σ / (sqrt(n))<= α <= хср + t*σ / (sqrt(n)), где
α - sign,
t is a parameter from the Laplace distribution table,
σ is the square root of the dispersion.
If the variance is unknown, then it can be calculated if we know all the values of the desired feature. For this, the following formula is used:
σ2 = х2ср - (хр)2, where
х2ср - the average value of the squares of the trait under study,
(xsr)2 is the square of this attribute.
The formula by which the confidence interval is calculated in this case changes slightly:
xsr - t*s / (sqrt(n))<= α <= хср + t*s / (sqrt(n)), где
xsr - sample mean,
α - sign,
t is a parameter that is found using the Student's distribution table t \u003d t (ɣ; n-1),
sqrt(n) is the square root of the total sample size,
s is the square root of the variance.
Consider this example. Assume that, based on the results of 7 measurements, the trait under study was determined to be 30 and the sample variance equal to 36. It is necessary to find, with a probability of 99%, a confidence interval that contains the true value of the measured parameter.
First, let's determine what t is equal to: t \u003d t (0.99; 7-1) \u003d 3.71. Using the above formula, we get:
xsr - t*s / (sqrt(n))<= α <= хср + t*s / (sqrt(n))
30 - 3.71*36 / (sqrt(7))<= α <= 30 + 3.71*36 / (sqrt(7))
21.587 <= α <= 38.413
The confidence interval for the variance is calculated both in the case of a known mean and when there is no data on the mathematical expectation, and only the value of the unbiased point estimate of the variance is known. We will not give here the formulas for its calculation, since they are quite complex and, if desired, they can always be found on the net.
We only note that it is convenient to determine the confidence interval using the Excel program or a network service, which is called so.