| <html> |
| <head> |
| <meta http-equiv="Content-Type" content="text/html; charset=US-ASCII"> |
| <title>Testing a sample mean for difference from a "true" mean</title> |
| <link rel="stylesheet" href="../../../../math.css" type="text/css"> |
| <meta name="generator" content="DocBook XSL Stylesheets V1.77.1"> |
| <link rel="home" href="../../../../index.html" title="Math Toolkit 2.2.0"> |
| <link rel="up" href="../st_eg.html" title="Student's t Distribution Examples"> |
| <link rel="prev" href="tut_mean_intervals.html" title="Calculating confidence intervals on the mean with the Students-t distribution"> |
| <link rel="next" href="tut_mean_size.html" title="Estimating how large a sample size would have to become in order to give a significant Students-t test result with a single sample test"> |
| </head> |
| <body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF"> |
| <table cellpadding="2" width="100%"><tr> |
| <td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../boost.png"></td> |
| <td align="center"><a href="../../../../../../../../index.html">Home</a></td> |
| <td align="center"><a href="../../../../../../../../libs/libraries.htm">Libraries</a></td> |
| <td align="center"><a href="http://www.boost.org/users/people.html">People</a></td> |
| <td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td> |
| <td align="center"><a href="../../../../../../../../more/index.htm">More</a></td> |
| </tr></table> |
| <hr> |
| <div class="spirit-nav"> |
| <a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> |
| </div> |
| <div class="section"> |
| <div class="titlepage"><div><div><h5 class="title"> |
| <a name="math_toolkit.stat_tut.weg.st_eg.tut_mean_test"></a><a class="link" href="tut_mean_test.html" title='Testing a sample mean for difference from a "true" mean'>Testing |
| a sample mean for difference from a "true" mean</a> |
| </h5></div></div></div> |
| <p> |
| When calibrating or comparing a scientific instrument or measurement |
| method of some kind, we want to be answer the question "Does an |
| observed sample mean differ from the "true" mean in any significant |
| way?". If it does, then we have evidence of a systematic difference. |
| This question can be answered with a Students-t test: more information |
| can be found <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">on |
| the NIST site</a>. |
| </p> |
| <p> |
| Of course, the assignment of "true" to one mean may be quite |
| arbitrary, often this is simply a "traditional" method of measurement. |
| </p> |
| <p> |
| The following example code is taken from the example program <a href="../../../../../../example/students_t_single_sample.cpp" target="_top">students_t_single_sample.cpp</a>. |
| </p> |
| <p> |
| We'll begin by defining a procedure to determine which of the possible |
| hypothesis are rejected or not-rejected at a given significance level: |
| </p> |
| <div class="note"><table border="0" summary="Note"> |
| <tr> |
| <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../doc/src/images/note.png"></td> |
| <th align="left">Note</th> |
| </tr> |
| <tr><td align="left" valign="top"><p> |
| Non-statisticians might say 'not-rejected' means 'accepted', (often |
| of the null-hypothesis) implying, wrongly, that there really <span class="bold"><strong>IS</strong></span> no difference, but statisticans eschew this |
| to avoid implying that there is positive evidence of 'no difference'. |
| 'Not-rejected' here means there is <span class="bold"><strong>no evidence</strong></span> |
| of difference, but there still might well be a difference. For example, |
| see <a href="http://en.wikipedia.org/wiki/Argument_from_ignorance" target="_top">argument |
| from ignorance</a> and <a href="http://www.bmj.com/cgi/content/full/311/7003/485" target="_top">Absence |
| of evidence does not constitute evidence of absence.</a> |
| </p></td></tr> |
| </table></div> |
| <pre class="programlisting"><span class="comment">// Needed includes:</span> |
| <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">math</span><span class="special">/</span><span class="identifier">distributions</span><span class="special">/</span><span class="identifier">students_t</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">></span> |
| <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iostream</span><span class="special">></span> |
| <span class="preprocessor">#include</span> <span class="special"><</span><span class="identifier">iomanip</span><span class="special">></span> |
| <span class="comment">// Bring everything into global namespace for ease of use:</span> |
| <span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">math</span><span class="special">;</span> |
| <span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">std</span><span class="special">;</span> |
| |
| <span class="keyword">void</span> <span class="identifier">single_sample_t_test</span><span class="special">(</span><span class="keyword">double</span> <span class="identifier">M</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sm</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sd</span><span class="special">,</span> <span class="keyword">unsigned</span> <span class="identifier">Sn</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">alpha</span><span class="special">)</span> |
| <span class="special">{</span> |
| <span class="comment">//</span> |
| <span class="comment">// M = true mean.</span> |
| <span class="comment">// Sm = Sample Mean.</span> |
| <span class="comment">// Sd = Sample Standard Deviation.</span> |
| <span class="comment">// Sn = Sample Size.</span> |
| <span class="comment">// alpha = Significance Level.</span> |
| </pre> |
| <p> |
| Most of the procedure is pretty-printing, so let's just focus on the |
| calculation, we begin by calculating the t-statistic: |
| </p> |
| <pre class="programlisting"><span class="comment">// Difference in means:</span> |
| <span class="keyword">double</span> <span class="identifier">diff</span> <span class="special">=</span> <span class="identifier">Sm</span> <span class="special">-</span> <span class="identifier">M</span><span class="special">;</span> |
| <span class="comment">// Degrees of freedom:</span> |
| <span class="keyword">unsigned</span> <span class="identifier">v</span> <span class="special">=</span> <span class="identifier">Sn</span> <span class="special">-</span> <span class="number">1</span><span class="special">;</span> |
| <span class="comment">// t-statistic:</span> |
| <span class="keyword">double</span> <span class="identifier">t_stat</span> <span class="special">=</span> <span class="identifier">diff</span> <span class="special">*</span> <span class="identifier">sqrt</span><span class="special">(</span><span class="keyword">double</span><span class="special">(</span><span class="identifier">Sn</span><span class="special">))</span> <span class="special">/</span> <span class="identifier">Sd</span><span class="special">;</span> |
| </pre> |
| <p> |
| Finally calculate the probability from the t-statistic. If we're interested |
| in simply whether there is a difference (either less or greater) or not, |
| we don't care about the sign of the t-statistic, and we take the complement |
| of the probability for comparison to the significance level: |
| </p> |
| <pre class="programlisting"><span class="identifier">students_t</span> <span class="identifier">dist</span><span class="special">(</span><span class="identifier">v</span><span class="special">);</span> |
| <span class="keyword">double</span> <span class="identifier">q</span> <span class="special">=</span> <span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t_stat</span><span class="special">)));</span> |
| </pre> |
| <p> |
| The procedure then prints out the results of the various tests that can |
| be done, these can be summarised in the following table: |
| </p> |
| <div class="informaltable"><table class="table"> |
| <colgroup> |
| <col> |
| <col> |
| </colgroup> |
| <thead><tr> |
| <th> |
| <p> |
| Hypothesis |
| </p> |
| </th> |
| <th> |
| <p> |
| Test |
| </p> |
| </th> |
| </tr></thead> |
| <tbody> |
| <tr> |
| <td> |
| <p> |
| The Null-hypothesis: there is <span class="bold"><strong>no difference</strong></span> |
| in means |
| </p> |
| </td> |
| <td> |
| <p> |
| Reject if complement of CDF for |t| < significance level |
| / 2: |
| </p> |
| <p> |
| <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> |
| <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span> |
| <span class="special"><</span> <span class="identifier">alpha</span> |
| <span class="special">/</span> <span class="number">2</span></code> |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td> |
| <p> |
| The Alternative-hypothesis: there <span class="bold"><strong>is |
| difference</strong></span> in means |
| </p> |
| </td> |
| <td> |
| <p> |
| Reject if complement of CDF for |t| > significance level |
| / 2: |
| </p> |
| <p> |
| <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> |
| <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span> |
| <span class="special">></span> <span class="identifier">alpha</span> |
| <span class="special">/</span> <span class="number">2</span></code> |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td> |
| <p> |
| The Alternative-hypothesis: the sample mean <span class="bold"><strong>is |
| less</strong></span> than the true mean. |
| </p> |
| </td> |
| <td> |
| <p> |
| Reject if CDF of t > 1 - significance level: |
| </p> |
| <p> |
| <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> |
| <span class="identifier">t</span><span class="special">))</span> |
| <span class="special"><</span> <span class="identifier">alpha</span></code> |
| </p> |
| </td> |
| </tr> |
| <tr> |
| <td> |
| <p> |
| The Alternative-hypothesis: the sample mean <span class="bold"><strong>is |
| greater</strong></span> than the true mean. |
| </p> |
| </td> |
| <td> |
| <p> |
| Reject if complement of CDF of t < significance level: |
| </p> |
| <p> |
| <code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> |
| <span class="identifier">t</span><span class="special">)</span> |
| <span class="special"><</span> <span class="identifier">alpha</span></code> |
| </p> |
| </td> |
| </tr> |
| </tbody> |
| </table></div> |
| <div class="note"><table border="0" summary="Note"> |
| <tr> |
| <td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../doc/src/images/note.png"></td> |
| <th align="left">Note</th> |
| </tr> |
| <tr><td align="left" valign="top"><p> |
| Notice that the comparisons are against <code class="computeroutput"><span class="identifier">alpha</span> |
| <span class="special">/</span> <span class="number">2</span></code> |
| for a two-sided test and against <code class="computeroutput"><span class="identifier">alpha</span></code> |
| for a one-sided test |
| </p></td></tr> |
| </table></div> |
| <p> |
| Now that we have all the parts in place, let's take a look at some sample |
| output, first using the <a href="http://www.itl.nist.gov/div898/handbook/eda/section4/eda428.htm" target="_top">Heat |
| flow data</a> from the NIST site. The data set was collected by Bob |
| Zarr of NIST in January, 1990 from a heat flow meter calibration and |
| stability analysis. The corresponding dataplot output for this test can |
| be found in <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">section |
| 3.5.2</a> of the <a href="http://www.itl.nist.gov/div898/handbook/" target="_top">NIST/SEMATECH |
| e-Handbook of Statistical Methods.</a>. |
| </p> |
| <pre class="programlisting">__________________________________ |
| Student t test for a single sample |
| __________________________________ |
| |
| Number of Observations = 195 |
| Sample Mean = 9.26146 |
| Sample Standard Deviation = 0.02279 |
| Expected True Mean = 5.00000 |
| |
| Sample Mean - Expected Test Mean = 4.26146 |
| Degrees of Freedom = 194 |
| T Statistic = 2611.28380 |
| Probability that difference is due to chance = 0.000e+000 |
| |
| Results for Alternative Hypothesis and alpha = 0.0500 |
| |
| Alternative Hypothesis Conclusion |
| Mean != 5.000 NOT REJECTED |
| Mean < 5.000 REJECTED |
| Mean > 5.000 NOT REJECTED |
| </pre> |
| <p> |
| You will note the line that says the probability that the difference |
| is due to chance is zero. From a philosophical point of view, of course, |
| the probability can never reach zero. However, in this case the calculated |
| probability is smaller than the smallest representable double precision |
| number, hence the appearance of a zero here. Whatever its "true" |
| value is, we know it must be extraordinarily small, so the alternative |
| hypothesis - that there is a difference in means - is not rejected. |
| </p> |
| <p> |
| For comparison the next example data output is taken from <span class="emphasis"><em>P.K.Hou, |
| O. W. Lau & M.C. Wong, Analyst (1983) vol. 108, p 64. and from Statistics |
| for Analytical Chemistry, 3rd ed. (1994), pp 54-55 J. C. Miller and J. |
| N. Miller, Ellis Horwood ISBN 0 13 0309907.</em></span> The values result |
| from the determination of mercury by cold-vapour atomic absorption. |
| </p> |
| <pre class="programlisting">__________________________________ |
| Student t test for a single sample |
| __________________________________ |
| |
| Number of Observations = 3 |
| Sample Mean = 37.80000 |
| Sample Standard Deviation = 0.96437 |
| Expected True Mean = 38.90000 |
| |
| Sample Mean - Expected Test Mean = -1.10000 |
| Degrees of Freedom = 2 |
| T Statistic = -1.97566 |
| Probability that difference is due to chance = 1.869e-001 |
| |
| Results for Alternative Hypothesis and alpha = 0.0500 |
| |
| Alternative Hypothesis Conclusion |
| Mean != 38.900 REJECTED |
| Mean < 38.900 NOT REJECTED |
| Mean > 38.900 NOT REJECTED |
| </pre> |
| <p> |
| As you can see the small number of measurements (3) has led to a large |
| uncertainty in the location of the true mean. So even though there appears |
| to be a difference between the sample mean and the expected true mean, |
| we conclude that there is no significant difference, and are unable to |
| reject the null hypothesis. However, if we were to lower the bar for |
| acceptance down to alpha = 0.1 (a 90% confidence level) we see a different |
| output: |
| </p> |
| <pre class="programlisting">__________________________________ |
| Student t test for a single sample |
| __________________________________ |
| |
| Number of Observations = 3 |
| Sample Mean = 37.80000 |
| Sample Standard Deviation = 0.96437 |
| Expected True Mean = 38.90000 |
| |
| Sample Mean - Expected Test Mean = -1.10000 |
| Degrees of Freedom = 2 |
| T Statistic = -1.97566 |
| Probability that difference is due to chance = 1.869e-001 |
| |
| Results for Alternative Hypothesis and alpha = 0.1000 |
| |
| Alternative Hypothesis Conclusion |
| Mean != 38.900 REJECTED |
| Mean < 38.900 NOT REJECTED |
| Mean > 38.900 REJECTED |
| </pre> |
| <p> |
| In this case, we really have a borderline result, and more data (and/or |
| more accurate data), is needed for a more convincing conclusion. |
| </p> |
| </div> |
| <table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr> |
| <td align="left"></td> |
| <td align="right"><div class="copyright-footer">Copyright © 2006-2010, 2012-2014 Nikhar Agrawal, |
| Anton Bikineev, Paul A. Bristow, Marco Guazzone, Christopher Kormanyos, Hubert |
| Holin, Bruno Lalande, John Maddock, Johan Råde, Gautam Sewani, Benjamin Sobotta, |
| Thijs van den Berg, Daryle Walker and Xiaogang Zhang<p> |
| Distributed under the Boost Software License, Version 1.0. (See accompanying |
| file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>) |
| </p> |
| </div></td> |
| </tr></table> |
| <hr> |
| <div class="spirit-nav"> |
| <a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../index.html"><img src="../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../doc/src/images/next.png" alt="Next"></a> |
| </div> |
| </body> |
| </html> |
