Percentile
A percentile (or a centile) is a measure used in statistics indicating the value below which a given percentage of observations in a group of observations falls. For example, the 20th percentile is the value (or score) below which 20% of the observations may be found. Equivalently, 80% of the observations are found above the 20th percentile.
The term percentile and the related term percentile rank are often used in the reporting of scores from norm-referenced tests. For example, if a score is at the 86th percentile, where 86 is the percentile rank, it is equal to the value below which 86% of the observations may be found (carefully contrast with in the 86th percentile, which means the score is at or below the value below which 86% of the observations may be found—every score is in the 100th percentile). The 25th percentile is also known as the first quartile (Q_{1}), the 50th percentile as the median or second quartile (Q_{2}), and the 75th percentile as the third quartile (Q_{3}). In general, percentiles and quartiles are specific types of quantiles.
Applications[edit]
When ISPs bill "burstable" internet bandwidth, the 95th or 98th percentile usually cuts off the top 5% or 2% of bandwidth peaks in each month, and then bills at the nearest rate. In this way infrequent peaks are ignored, and the customer is charged in a fairer way. The reason this statistic is so useful in measuring data throughput is that it gives a very accurate picture of the cost of the bandwidth. The 95th percentile says that 95% of the time, the usage is below this amount: so, the remaining 5% of the time, the usage is above that amount.
Physicians will often use infant and children's weight and height to assess their growth in comparison to national averages and percentiles which are found in growth charts.
The 85th percentile speed of traffic on a road is often used as a guideline in setting speed limits and assessing whether such a limit is too high or low.^{[1]}^{[2]}
In finance, value at risk is a standard measure to assess (in a model dependent way) the quantity under which the value of the portfolio is not expected to sink within a given period of time and given a confidence value.
The normal distribution and percentiles[edit]
The methods given in the definitions section (below) are approximations for use in small-sample statistics. In general terms, for very large populations following a normal distribution, percentiles may often be represented by reference to a normal curve plot. The normal distribution is plotted along an axis scaled to standard deviations, or sigma () units. Mathematically, the normal distribution extends to negative infinity on the left and positive infinity on the right. Note, however, that only a very small proportion of individuals in a population will fall outside the −3 to +3 range. For example, with human heights very few people are above the +3 height level.
Percentiles represent the area under the normal curve, increasing from left to right. Each standard deviation represents a fixed percentile. Thus, rounding to two decimal places, −3 is the 0.13th percentile, −2 the 2.28th percentile, −1 the 15.87th percentile, 0 the 50th percentile (both the mean and median of the distribution), +1 the 84.13th percentile, +2 the 97.72nd percentile, and +3 the 99.87th percentile. This is related to the 68–95–99.7 rule or the three-sigma rule. Note that in theory the 0th percentile falls at negative infinity and the 100th percentile at positive infinity, although in many practical applications, such as test results, natural lower and/or upper limits are enforced.
Definitions[edit]
There is no standard definition of percentile,^{[3]}^{[4]}^{[5]} however all definitions yield similar results when the number of observations is very large and the probability distribution is continuous.^{[6]} In the limit, as the sample size approaches infinity, the 100p^{th} percentile (0<p<1) approximates the inverse of the cumulative distribution function (CDF) thus formed, evaluated at p, as p approximates the CDF. This can be seen as a consequence of the Glivenko–Cantelli theorem. Some methods for calculating the percentiles are given below.
The nearest-rank method[edit]
One definition of percentile, often given in texts, is that the P-th percentile of a list of N ordered values (sorted from least to greatest) is the smallest value in the list such that no more than P percent of the data is strictly less than the value and at least P percent of the data is less than or equal to that value. This is obtained by first calculating the ordinal rank and then taking the value from the ordered list that corresponds to that rank. The ordinal rank n is calculated using this formula
Note the following:
- Using the nearest-rank method on lists with fewer than 100 distinct values can result in the same value being used for more than one percentile.
- A percentile calculated using the nearest-rank method will always be a member of the original ordered list.
- The 100th percentile is defined to be the largest value in the ordered list.
Worked examples of the nearest-rank method[edit]
- Example 1
Consider the ordered list {15, 20, 35, 40, 50}, which contains 5 data values. What are the 5th, 30th, 40th, 50th and 100th percentiles of this list using the nearest-rank method?
Percentile P |
Number in list N |
Ordinal rank n |
Number from the ordered list that has that rank |
Percentile value |
Notes |
---|---|---|---|---|---|
5th | 5 | the first number in the ordered list, which is 15 | 15 | 15 is the smallest element of the list; 0% of the data is strictly less than 15, and 20% of the data is less than or equal to 15. | |
30th | 5 | the 2nd number in the ordered list, which is 20 | 20 | 20 is an element of the list | |
40th | 5 | the 2nd number in the ordered list, which is 20 | 20 | In this example it is the same as the 30th percentile. | |
50th | 5 | the 3rd number in the ordered list, which is 35 | 35 | 35 is an element of the ordered list. | |
100th | 5 | 50, which is the last number in the ordered list | 50 | The 100th percentile is defined to be the largest value in the list, which is 50. |
So the 5th, 30th, 40th, 50th and 100th percentiles of the ordered list {15, 20, 35, 40, 50} using the nearest-rank method are {15, 20, 20, 35, 50}.
- Example 2
Consider an ordered population of 10 data values {3, 6, 7, 8, 8, 10, 13, 15, 16, 20}. What are the 25th, 50th, 75th and 100th percentiles of this list using the nearest-rank method?
Percentile P |
Number in list N |
Ordinal rank n |
Number from the ordered list that has that rank |
Percentile value |
Notes |
---|---|---|---|---|---|
25th | 10 | the 3rd number in the ordered list, which is 7 | 7 | 7 is an element of the list | |
50th | 10 | the 5th number in the ordered list, which is 8 | 8 | 8 is an element of the list. | |
75th | 10 | the 8th number in the ordered list, which is 15 | 15 | 15 is an element of the list. | |
100th | 10 | Last | 20, which is the last number in the ordered list | 20 | The 100th percentile is defined to be the largest value in the list, which is 20. |
So the 25th, 50th, 75th and 100th percentiles of the ordered list {3, 6, 7, 8, 8, 10, 13, 15, 16, 20} using the nearest-rank method are {7, 8, 15, 20}.
- Example 3
Consider an ordered population of 11 data values {3, 6, 7, 8, 8, 9, 10, 13, 15, 16, 20}. What are the 25th, 50th, 75th and 100th percentiles of this list using the nearest-rank method?
Percentile P |
Number in list N |
Ordinal rank n |
Number from the ordered list that has that rank |
Percentile value |
Notes |
---|---|---|---|---|---|
25th | 11 | the 3rd number in the ordered list, which is 7 | 7 | 7 is an element of the list | |
50th | 11 | the 6th number in the ordered list, which is 9 | 9 | 9 is an element of the list. | |
75th | 11 | the 9th number in the ordered list, which is 15 | 15 | 15 is an element of the list. | |
100th | 11 | Last | 20, which is the last number in the ordered list | 20 | The 100th percentile is defined to be the largest value in the list, which is 20. |
So the 25th, 50th, 75th and 100th percentiles of the ordered list {3, 6, 7, 8, 8, 9, 10, 13, 15, 16, 20} using the nearest-rank method are {7, 9, 15, 20}.
The linear interpolation between closest ranks method[edit]
An alternative to rounding used in many applications is to use linear interpolation between adjacent ranks.
Commonalities between the variants of this method[edit]
All of the following variants have the following in common. Given the order statistics
we seek a linear interpolation function that passes through the points . This is simply accomplished by
where uses the floor function to represent the integral part of positive , whereas uses the mod function to represent its fractional part (the remainder after division by 1). (Note that, though at the endpoint , is undefined, it does not need to be because it is multiplied by .) As we can see, is the continuous version of the subscript , linearly interpolating between adjacent nodes.
There are two ways in which the variant approaches differ. The first is in the linear relationship between the rank , the percent rank , and a constant that is a function of the sample size :
There is the additional requirement that the midpoint of the range , corresponding to the median, occur at :
and our revised function now has just one degree of freedom, looking like this:
The second way in which the variants differ is in the definition of the function near the margins of the range of : should produce, or be forced to produce, a result in the range , which may mean the absence of a one-to-one correspondence in the wider region.
First variant, [edit]
(Sources: Matlab "prctile" function,^{[7]}^{[8]})
where
Furthermore, let
The inverse relationship is restricted to a narrower region:
Worked example of the first variant[edit]
Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What are the 5th, 30th, 40th and 95th percentiles of this list using the Linear Interpolation Between Closest Ranks method? First, we calculate the percent rank for each list value.
List value |
Position of that value in the ordered list |
Number of values |
Calculation of percent rank |
Percent rank, |
Notes |
---|---|---|---|---|---|
15 | 1 | 5 | 10 | ||
20 | 2 | 5 | 30 | ||
35 | 3 | 5 | 50 | ||
40 | 4 | 5 | 70 | ||
50 | 5 | 5 | 90 |
Then we take those percent ranks and calculate the percentile values as follows:
Percent rank |
Number of values |
Is ? | Is ? | Is there a percent rank equal to ? |
What do we use for percentile value? | Percentile value |
Notes |
---|---|---|---|---|---|---|---|
5 | 5 | Yes | No | No | We see that P=5, which is less than the first percent rank p1=10, so use the first list value v1, which is 15 | 15 | 15 is a member of the ordered list |
30 | 5 | No | No | Yes | We see that P=30 is the same as the second percent rank p2=30, so use the second list value v2, which is 20 | 20 | 20 is a member of the ordered list |
40 | 5 | No | No | No | We see that P=40 is between percent rank p2=30 and p3=50, so we take k=2, k+1=3, P=40, pk=p2=30, vk=v2=20, vk+1=v3=35, N=5. Given those values we can then calculate v as follows: |
27.5 | 27.5 is not a member of the ordered list |
95 | 5 | No | Yes | No | We see that P=95, which is greater than the last percent rank pN=90, so use the last list value, which is 50 | 50 | 50 is a member of the ordered list |
So the 5th, 30th, 40th and 95th percentiles of the ordered list {15, 20, 35, 40, 50} using the Linear Interpolation Between Closest Ranks method are {15, 20, 27.5, 50}
Second variant, [edit]
(Source: Some software packages, including NumPy^{[9]} and Microsoft Excel^{[5]} (up to and including version 2013 by means of the PERCENTILE.INC function). Noted as an alternative by NIST^{[10]})
Note that the relationship is one-to-one for , the only one of the three variants with this property; hence the "INC" suffix, for inclusive, on the Excel function.
Worked examples of the second variant[edit]
Example 1:
Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What is the 40th percentile of this list using this variant method?
First we calculate the rank of the 40th percentile:
So, x=2.6, which gives us and . So, the value of the 40th percentile is
Example 2:
Consider the ordered list {1,2,3,4} which CONTAINS four data values. What is the 75th percentile of this list using the Microsoft Excel method?
First we calculate the rank of the 75th percentile as follows:
So, x=3.25, which gives us an integral part of 3 and a fractional part of 0.25. So, the value of the 75th percentile is
Third variant, [edit]
(The primary variant recommended by NIST.^{[10]} Adopted by Microsoft Excel since 2010 by means of PERCENTIL.EXC function. However, as the "EXC" suffix indicates, the Excel version excludes both endpoints of the range of p, i.e., , whereas the "INC" version, the second variant, does not; in fact, any number smaller than 1/(N+1) is also excluded and would cause an error.)
The inverse is restricted to a narrower region:
Worked example of the third variant[edit]
Consider the ordered list {15, 20, 35, 40, 50}, which contains five data values. What is the 40th percentile of this list using the NIST method?
First we calculate the rank of the 40th percentile as follows:
So x=2.4, which gives us and . So the value of the 40th percentile is calculated as:
So the value of the 40th percentile of the ordered list {15, 20, 35, 40, 50} using this variant method is 26.
The weighted percentile method[edit]
In addition to the percentile function, there is also a weighted percentile, where the percentage in the total weight is counted instead of the total number. There is no standard function for a weighted percentile. One method extends the above approach in a natural way.
Suppose we have positive weights associated, respectively, with our N sorted sample values. Let
the sum of the weights. Then the formulas above are generalized by taking
- when ,
or
- for general ,
and
The 50% weighted percentile is known as the weighted median.
See also[edit]
References[edit]
- ^ Johnson, Robert; Kuby, Patricia (2007), "Applied Example 2.15, The 85th Percentile Speed Limit: Going With 85% of the Flow", Elementary Statistics (10th ed.), Cengage Learning, p. 102, ISBN 9781111802493.
- ^ "Rational Speed Limits and the 85th Percentile Speed" (PDF). lsp.org. Louisiana State Police. Archived from the original (PDF) on 23 September 2018. Retrieved 28 October 2018.
- ^ Hyndman RH, Fan Y (1996). "Sample quantiles in statistical packages". The American Statistician. 50 (4): 361–365. doi:10.2307/2684934. JSTOR 2684934.
- ^ Lane, David. "Percentiles". Retrieved 2007-09-15.
- ^ ^{a} ^{b} Pottel, Hans. "Statistical flaws in Excel" (PDF). Archived from the original (PDF) on 2013-06-04. Retrieved 2013-03-25.
- ^ Schoonjans F, De Bacquer D, Schmid P (2011). "Estimation of population percentiles". Epidemiology. 22 (5): 750–751. doi:10.1097/EDE.0b013e318225c1de. PMC 3171208. PMID 21811118.
- ^ "Matlab Statistics Toolbox – Percentiles". Retrieved 2006-09-15., This is equivalent to Method 5 discussed here
- ^ Langford, E. (2006). "Quartiles in Elementary Statistics". Journal of Statistics Education. 14 (3). doi:10.1080/10691898.2006.11910589.
- ^ "NumPy 1.12 documentation". SciPy. Retrieved 2017-03-19.
- ^ ^{a} ^{b} "Engineering Statistics Handbook: Percentile". NIST. Retrieved 2009-02-18.