Google Git
Sign in
nest-open-source / nest-learning-thermostat / 5.0 / boost / 317601cb979f96545d0e892ce7cfdf59f676ee0a / . / boost_1_45_0 / libs / math / doc / sf_and_dist / html / math_toolkit / dist / stat_tut / weg / st_eg / tut_mean_test.html
blob: 14a2ce95ce71b282b8de4ab517103c0cc7edce72 [file] [log] [blame]
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=US-ASCII">
<title>Testing a sample mean for difference from a "true" mean</title>
<link rel="stylesheet" href="../../../../../../../../../../doc/src/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets V1.74.0">
<link rel="home" href="../../../../../index.html" title="Math Toolkit">
<link rel="up" href="../st_eg.html" title="Student's t Distribution Examples">
<link rel="prev" href="tut_mean_intervals.html" title="Calculating confidence intervals on the mean with the Students-t distribution">
<link rel="next" href="tut_mean_size.html" title="Estimating how large a sample size would have to become in order to give a significant Students-t test result with a single sample test">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/users/people.html">People</a></td>
<td align="center"><a href="http://www.boost.org/users/faq.html">FAQ</a></td>
<td align="center"><a href="../../../../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../../index.html"><img src="../../../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
<div class="section" lang="en">
<div class="titlepage"><div><div><h6 class="title">
<a name="math_toolkit.dist.stat_tut.weg.st_eg.tut_mean_test"></a><a class="link" href="tut_mean_test.html" title='Testing a sample mean for difference from a "true" mean'>
Testing a sample mean for difference from a "true" mean</a>
</h6></div></div></div>
<p>
When calibrating or comparing a scientific instrument or measurement
method of some kind, we want to be answer the question "Does an
observed sample mean differ from the "true" mean in any significant
way?". If it does, then we have evidence of a systematic difference.
This question can be answered with a Students-t test: more information
can be found <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">on
the NIST site</a>.
</p>
<p>
Of course, the assignment of "true" to one mean may be quite
arbitrary, often this is simply a "traditional" method of
measurement.
</p>
<p>
The following example code is taken from the example program <a href="../../../../../../../../example/students_t_single_sample.cpp" target="_top">students_t_single_sample.cpp</a>.
</p>
<p>
We'll begin by defining a procedure to determine which of the possible
hypothesis are rejected or not-rejected at a given significance level:
</p>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../../../doc/src/images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
Non-statisticians might say 'not-rejected' means 'accepted', (often
of the null-hypothesis) implying, wrongly, that there really <span class="bold"><strong>IS</strong></span> no difference, but statisticans eschew
this to avoid implying that there is positive evidence of 'no difference'.
'Not-rejected' here means there is <span class="bold"><strong>no evidence</strong></span>
of difference, but there still might well be a difference. For example,
see <a href="http://en.wikipedia.org/wiki/Argument_from_ignorance" target="_top">argument
from ignorance</a> and <a href="http://www.bmj.com/cgi/content/full/311/7003/485" target="_top">Absence
of evidence does not constitute evidence of absence.</a>
</p></td></tr>
</table></div>
<pre class="programlisting"><span class="comment">// Needed includes:
</span><span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">boost</span><span class="special">/</span><span class="identifier">math</span><span class="special">/</span><span class="identifier">distributions</span><span class="special">/</span><span class="identifier">students_t</span><span class="special">.</span><span class="identifier">hpp</span><span class="special">&gt;</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iostream</span><span class="special">&gt;</span>
<span class="preprocessor">#include</span> <span class="special">&lt;</span><span class="identifier">iomanip</span><span class="special">&gt;</span>
<span class="comment">// Bring everything into global namespace for ease of use:
</span><span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">boost</span><span class="special">::</span><span class="identifier">math</span><span class="special">;</span>
<span class="keyword">using</span> <span class="keyword">namespace</span> <span class="identifier">std</span><span class="special">;</span>
<span class="keyword">void</span> <span class="identifier">single_sample_t_test</span><span class="special">(</span><span class="keyword">double</span> <span class="identifier">M</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sm</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">Sd</span><span class="special">,</span> <span class="keyword">unsigned</span> <span class="identifier">Sn</span><span class="special">,</span> <span class="keyword">double</span> <span class="identifier">alpha</span><span class="special">)</span>
<span class="special">{</span>
<span class="comment">//
</span> <span class="comment">// M = true mean.
</span> <span class="comment">// Sm = Sample Mean.
</span> <span class="comment">// Sd = Sample Standard Deviation.
</span> <span class="comment">// Sn = Sample Size.
</span> <span class="comment">// alpha = Significance Level.
</span></pre>
<p>
Most of the procedure is pretty-printing, so let's just focus on the
calculation, we begin by calculating the t-statistic:
</p>
<pre class="programlisting"><span class="comment">// Difference in means:
</span><span class="keyword">double</span> <span class="identifier">diff</span> <span class="special">=</span> <span class="identifier">Sm</span> <span class="special">-</span> <span class="identifier">M</span><span class="special">;</span>
<span class="comment">// Degrees of freedom:
</span><span class="keyword">unsigned</span> <span class="identifier">v</span> <span class="special">=</span> <span class="identifier">Sn</span> <span class="special">-</span> <span class="number">1</span><span class="special">;</span>
<span class="comment">// t-statistic:
</span><span class="keyword">double</span> <span class="identifier">t_stat</span> <span class="special">=</span> <span class="identifier">diff</span> <span class="special">*</span> <span class="identifier">sqrt</span><span class="special">(</span><span class="keyword">double</span><span class="special">(</span><span class="identifier">Sn</span><span class="special">))</span> <span class="special">/</span> <span class="identifier">Sd</span><span class="special">;</span>
</pre>
<p>
Finally calculate the probability from the t-statistic. If we're interested
in simply whether there is a difference (either less or greater) or
not, we don't care about the sign of the t-statistic, and we take the
complement of the probability for comparison to the significance level:
</p>
<pre class="programlisting"><span class="identifier">students_t</span> <span class="identifier">dist</span><span class="special">(</span><span class="identifier">v</span><span class="special">);</span>
<span class="keyword">double</span> <span class="identifier">q</span> <span class="special">=</span> <span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span> <span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t_stat</span><span class="special">)));</span>
</pre>
<p>
The procedure then prints out the results of the various tests that
can be done, these can be summarised in the following table:
</p>
<div class="informaltable"><table class="table">
<colgroup>
<col>
<col>
</colgroup>
<thead><tr>
<th>
<p>
Hypothesis
</p>
</th>
<th>
<p>
Test
</p>
</th>
</tr></thead>
<tbody>
<tr>
<td>
<p>
The Null-hypothesis: there is <span class="bold"><strong>no difference</strong></span>
in means
</p>
</td>
<td>
<p>
Reject if complement of CDF for |t| &lt; significance level
/ 2:
</p>
<p>
<code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
<span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span>
<span class="special">&lt;</span> <span class="identifier">alpha</span>
<span class="special">/</span> <span class="number">2</span></code>
</p>
</td>
</tr>
<tr>
<td>
<p>
The Alternative-hypothesis: there <span class="bold"><strong>is
difference</strong></span> in means
</p>
</td>
<td>
<p>
Reject if complement of CDF for |t| &gt; significance level
/ 2:
</p>
<p>
<code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
<span class="identifier">fabs</span><span class="special">(</span><span class="identifier">t</span><span class="special">)))</span>
<span class="special">&gt;</span> <span class="identifier">alpha</span>
<span class="special">/</span> <span class="number">2</span></code>
</p>
</td>
</tr>
<tr>
<td>
<p>
The Alternative-hypothesis: the sample mean <span class="bold"><strong>is
less</strong></span> than the true mean.
</p>
</td>
<td>
<p>
Reject if CDF of t &gt; significance level:
</p>
<p>
<code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
<span class="identifier">t</span><span class="special">)</span>
<span class="special">&gt;</span> <span class="identifier">alpha</span></code>
</p>
</td>
</tr>
<tr>
<td>
<p>
The Alternative-hypothesis: the sample mean <span class="bold"><strong>is
greater</strong></span> than the true mean.
</p>
</td>
<td>
<p>
Reject if complement of CDF of t &gt; significance level:
</p>
<p>
<code class="computeroutput"><span class="identifier">cdf</span><span class="special">(</span><span class="identifier">complement</span><span class="special">(</span><span class="identifier">dist</span><span class="special">,</span>
<span class="identifier">t</span><span class="special">))</span>
<span class="special">&gt;</span> <span class="identifier">alpha</span></code>
</p>
</td>
</tr>
</tbody>
</table></div>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../../../../doc/src/images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
Notice that the comparisons are against <code class="computeroutput"><span class="identifier">alpha</span>
<span class="special">/</span> <span class="number">2</span></code>
for a two-sided test and against <code class="computeroutput"><span class="identifier">alpha</span></code>
for a one-sided test
</p></td></tr>
</table></div>
<p>
Now that we have all the parts in place, let's take a look at some
sample output, first using the <a href="http://www.itl.nist.gov/div898/handbook/eda/section4/eda428.htm" target="_top">Heat
flow data</a> from the NIST site. The data set was collected by
Bob Zarr of NIST in January, 1990 from a heat flow meter calibration
and stability analysis. The corresponding dataplot output for this
test can be found in <a href="http://www.itl.nist.gov/div898/handbook/eda/section3/eda352.htm" target="_top">section
3.5.2</a> of the <a href="http://www.itl.nist.gov/div898/handbook/" target="_top">NIST/SEMATECH
e-Handbook of Statistical Methods.</a>.
</p>
<pre class="programlisting"> __________________________________
Student t test for a single sample
__________________________________
Number of Observations = 195
Sample Mean = 9.26146
Sample Standard Deviation = 0.02279
Expected True Mean = 5.00000
Sample Mean - Expected Test Mean = 4.26146
Degrees of Freedom = 194
T Statistic = 2611.28380
Probability that difference is due to chance = 0.000e+000
Results for Alternative Hypothesis and alpha = 0.0500
Alternative Hypothesis Conclusion
Mean != 5.000 NOT REJECTED
Mean &lt; 5.000 REJECTED
Mean &gt; 5.000 NOT REJECTED
</pre>
<p>
You will note the line that says the probability that the difference
is due to chance is zero. From a philosophical point of view, of course,
the probability can never reach zero. However, in this case the calculated
probability is smaller than the smallest representable double precision
number, hence the appearance of a zero here. Whatever its "true"
value is, we know it must be extraordinarily small, so the alternative
hypothesis - that there is a difference in means - is not rejected.
</p>
<p>
For comparison the next example data output is taken from <span class="emphasis"><em>P.K.Hou,
O. W. Lau &amp; M.C. Wong, Analyst (1983) vol. 108, p 64. and from
Statistics for Analytical Chemistry, 3rd ed. (1994), pp 54-55 J. C.
Miller and J. N. Miller, Ellis Horwood ISBN 0 13 0309907.</em></span>
The values result from the determination of mercury by cold-vapour
atomic absorption.
</p>
<pre class="programlisting"> __________________________________
Student t test for a single sample
__________________________________
Number of Observations = 3
Sample Mean = 37.80000
Sample Standard Deviation = 0.96437
Expected True Mean = 38.90000
Sample Mean - Expected Test Mean = -1.10000
Degrees of Freedom = 2
T Statistic = -1.97566
Probability that difference is due to chance = 1.869e-001
Results for Alternative Hypothesis and alpha = 0.0500
Alternative Hypothesis Conclusion
Mean != 38.900 REJECTED
Mean &lt; 38.900 REJECTED
Mean &gt; 38.900 REJECTED
</pre>
<p>
As you can see the small number of measurements (3) has led to a large
uncertainty in the location of the true mean. So even though there
appears to be a difference between the sample mean and the expected
true mean, we conclude that there is no significant difference, and
are unable to reject the null hypothesis. However, if we were to lower
the bar for acceptance down to alpha = 0.1 (a 90% confidence level)
we see a different output:
</p>
<pre class="programlisting">__________________________________
Student t test for a single sample
__________________________________
Number of Observations = 3
Sample Mean = 37.80000
Sample Standard Deviation = 0.96437
Expected True Mean = 38.90000
Sample Mean - Expected Test Mean = -1.10000
Degrees of Freedom = 2
T Statistic = -1.97566
Probability that difference is due to chance = 1.869e-001
Results for Alternative Hypothesis and alpha = 0.1000
Alternative Hypothesis Conclusion
Mean != 38.900 REJECTED
Mean &lt; 38.900 NOT REJECTED
Mean &gt; 38.900 REJECTED
</pre>
<p>
In this case, we really have a borderline result, and more data (and/or
more accurate data), is needed for a more convincing conclusion.
</p>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright &#169; 2006 , 2007, 2008, 2009, 2010 John Maddock, Paul A. Bristow,
Hubert Holin, Xiaogang Zhang, Bruno Lalande, Johan R&#229;de, Gautam Sewani and
Thijs van den Berg<p>
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
</p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="tut_mean_intervals.html"><img src="../../../../../../../../../../doc/src/images/prev.png" alt="Prev"></a><a accesskey="u" href="../st_eg.html"><img src="../../../../../../../../../../doc/src/images/up.png" alt="Up"></a><a accesskey="h" href="../../../../../index.html"><img src="../../../../../../../../../../doc/src/images/home.png" alt="Home"></a><a accesskey="n" href="tut_mean_size.html"><img src="../../../../../../../../../../doc/src/images/next.png" alt="Next"></a>
</div>
</body>
</html>