Welch's ttest
In statistics, Welch's ttest, or unequal variances ttest, is a twosample location test which is used to test the hypothesis that two populations have equal means. It is named for its creator, Bernard Lewis Welch, and is an adaptation of Student's ttest,^{[1]} and is more reliable when the two samples have unequal variances and/or unequal sample sizes.^{[2]}^{[3]} These tests are often referred to as "unpaired" or "independent samples" ttests, as they are typically applied when the statistical units underlying the two samples being compared are nonoverlapping. Given that Welch's ttest has been less popular than Student's ttest^{[2]} and may be less familiar to readers, a more informative name is "Welch's unequal variances ttest" — or "unequal variances ttest" for brevity.^{[3]}
Assumptions[edit]
Student's ttest assumes that the sample means (test statistics) of two population distributions being compared are normally distributed with equal variance. Welch's ttest is designed for unequal sample distribution variance, but the assumption of sample distribution normality is maintained.^{[1]} Welch's ttest is an approximate solution to the Behrens–Fisher problem.
Calculations[edit]
Welch's ttest defines the statistic t by the following formula:
where , and are the sample mean, sample standard deviation and sample size, respectively, . Unlike in Student's ttest, the denominator is not based on a pooled variance estimate.
The degrees of freedom associated with this variance estimate is approximated using the Welch–Satterthwaite equation:
Here , the degrees of freedom associated with the first variance estimate. , the degrees of freedom associated with the 2nd variance estimate.
The statistic is approximately from the tdistribution since we have an approximation of the chisquare distribution. This approximation is better done when both and are larger than 5.^{[4]}^{[5]}
Statistical test[edit]
Once t and have been computed, these statistics can be used with the tdistribution to test one of two possible null hypotheses:
 that the two population means are equal, in which a twotailed test is applied; or
 that one of the population means is greater than or equal to the other, in which a onetailed test is applied.
The approximate degrees of freedom are rounded down to the nearest integer.^{[citation needed]}
Advantages and limitations[edit]
Welch's ttest is more robust than Student's ttest and maintains type I error rates close to nominal for unequal variances and for unequal sample sizes under normality. Furthermore, the power of Welch's ttest comes close to that of Student's ttest, even when the population variances are equal and sample sizes are balanced.^{[2]} Welch's ttest can be generalized to more than 2samples,^{[6]} which is more robust than oneway analysis of variance (ANOVA).
It is not recommended to pretest for equal variances and then choose between Student's ttest or Welch's ttest.^{[7]} Rather, Welch's ttest can be applied directly and without any substantial disadvantages to Student's ttest as noted above. Welch's ttest remains robust for skewed distributions and large sample sizes.^{[8]} Reliability decreases for skewed distributions and smaller samples, where one could possibly perform Welch's ttest.^{[9]}
Examples[edit]
The following three examples compare Welch's ttest and Student's ttest. Samples are from random normal distributions using the R programming language.
For all three examples, the population means were and .
The first example is for equal variances () and equal sample sizes (). Let A1 and A2 denote two random samples:
The second example is for unequal variances (, ) and unequal sample sizes (, ). The smaller sample has the larger variance:
The third example is for unequal variances (, ) and unequal sample sizes (, ). The larger sample has the larger variance:
Reference pvalues were obtained by simulating the distributions of the t statistics for the null hypothesis of equal population means (). Results are summarised in the table below, with twotailed pvalues:
Sample A1  Sample A2  Student's ttest  Welch's ttest  

Example  
1  15  20.8  7.9  15  23.0  3.8  −2.46  28  0.021  0.021  −2.46  24.9  0.021  0.017 
2  10  20.6  9.0  20  22.1  0.9  −2.10  28  0.045  0.150  −1.57  9.9  0.149  0.144 
3  10  19.4  1.4  20  21.6  17.1  −1.64  28  0.110  0.036  −2.22  24.5  0.036  0.042 
Welch's ttest and Student's ttest gave identical results when the two samples have identical variances and sample sizes (Example 1). But note that if you sample data from populations with identical variances, the sample variances will differ, as will the results of the two ttests. So with actual data, the two tests will almost always give somewhat different results.
For unequal variances, Student's ttest gave a low pvalue when the smaller sample had a larger variance (Example 2) and a high pvalue when the larger sample had a larger variance (Example 3). For unequal variances, Welch's ttest gave pvalues close to simulated pvalues.
Software implementations[edit]
Language/Program  Function  Documentation 

LibreOffice  TTEST(Data1; Data2; Mode; Type) 
^{[10]} 
MATLAB  ttest2(data1, data2, 'Vartype', 'unequal') 
^{[11]} 
Microsoft Excel pre 2010  TTEST(array1, array2, tails, type) 
^{[12]} 
Microsoft Excel 2010 and later  T.TEST(array1, array2, tails, type) 
^{[13]} 
Minitab  Accessed through menu  ^{[14]} 
SAS (Software)  Default output from proc ttest (labeled "Satterthwaite")
 
Python  scipy.stats.ttest_ind(a, b, equal_var=False) 
^{[15]} 
R  t.test(data1, data2, alternative="two.sided", var.equal=FALSE) 
^{[16]} 
Haskell  Statistics.Test.StudentT.welchTTest SamplesDiffer data1 data2 
^{[17]} 
JMP  Oneway( Y( YColumn), X( XColumn), Unequal Variances( 1 ) ); 
^{[18]} 
Julia  UnequalVarianceTTest(data1, data2) 
^{[19]} 
Stata  ttest varname1 == varname2, welch

^{[20]} 
Google Sheets  TTEST(range1, range2, tails, type)

^{[21]} 
GraphPad Prism  It is a choice on the t test dialog.  
IBM SPSS Statistics  An option in the menu  ^{[22]}^{[23]} 
GNU Octave  welch_test(x, y)

^{[24]} 
See also[edit]
 Student's ttest
 Ztest
 Factorial experiment
 Oneway analysis of variance
 Hotelling's twosample Tsquared statistic, a multivariate extension of Welch's ttest
References[edit]
 ^ ^{a} ^{b} Welch, B. L. (1947). "The generalization of "Student's" problem when several different population variances are involved". Biometrika. 34 (1–2): 28–35. doi:10.1093/biomet/34.12.28. MR 0019277. PMID 20287819.
 ^ ^{a} ^{b} ^{c} Ruxton, G. D. (2006). "The unequal variance ttest is an underused alternative to Student's ttest and the Mann–Whitney U test". Behavioral Ecology. 17 (4): 688–690. doi:10.1093/beheco/ark016.
 ^ ^{a} ^{b} Derrick, B; Toher, D; White, P (2016). "Why Welchs test is Type I error robust" (PDF). The Quantitative Methods for Psychology. 12 (1): 30–38. doi:10.20982/tqmp.12.1.p030.
 ^ The Satterthwaite Formula for Degrees of Freedom in the TwoSample tTest (page 7)
 ^ Yates, Moore, and Starnes, The Practice of Statistics, 3rd ed., p. 792. Copyright 2008 by W.H. Freeman and Company, 41 Madison Avenue, New York, NY 10010
 ^ Welch, B. L. (1951). "On the Comparison of Several Mean Values: An Alternative Approach". Biometrika. 38 (3/4): 330–336. doi:10.2307/2332579. JSTOR 2332579.
 ^ Zimmerman, D. W. (2004). "A note on preliminary tests of equality of variances". British Journal of Mathematical and Statistical Psychology. 57: 173–181. doi:10.1348/000711004849222.
 ^ Fagerland, M. W. (2012). "ttests, nonparametric tests, and large studies—a paradox of statistical practice?". BMC Medical Research Methodology. 12: 78. doi:10.1186/147122881278. PMC 3445820. PMID 22697476.
 ^ Fagerland, M. W.; Sandvik, L. (2009). "Performance of five twosample location tests for skewed distributions with unequal variances". Contemporary Clinical Trials. 30 (5): 490–496. doi:10.1016/j.cct.2009.06.007.
 ^ https://help.libreoffice.org/Calc/Statistical_Functions_Part_Five#TTEST
 ^ http://uk.mathworks.com/help/stats/ttest2.html
 ^ http://office.microsoft.com/enus/excelhelp/ttestHP005209325.aspx
 ^ http://office.microsoft.com/enus/excelhelp/ttestfunctionHA102753135.aspx
 ^ Overview for 2Sample t  Minitab: — official documentation for Minitab version 18. Accessed 20200919.
 ^ http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.ttest_ind.html
 ^ https://stat.ethz.ch/Rmanual/Rdevel/library/stats/html/t.test.html
 ^ http://hackage.haskell.org/package/statistics0.15.0.0/docs/StatisticsTestStudentT.html
 ^ https://www.jmp.com/support/help/
 ^ http://hypothesistestsjl.readthedocs.org/en/latest/index.html
 ^ http://www.stata.com/help.cgi?ttest
 ^ https://support.google.com/docs/answer/6055837?hl=en
 ^ Jeremy Miles: Unequal variances ttest or U MannWhitney test?, Accessed 20140411
 ^ OneSample Test — Official documentation for SPSS Statistics version 24. Accessed 20190122.
 ^ https://octave.sourceforge.io/statistics/function/welch_test.html