| <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> |
| |
| <html> |
| <head> |
| <meta http-equiv="Content-Language" content="en-us"> |
| <meta http-equiv="Content-Type" content="text/html; charset=us-ascii"> |
| <meta name="GENERATOR" content="Microsoft FrontPage 6.0"> |
| <meta name="ProgId" content="FrontPage.Editor.Document"> |
| |
| <title>Boost Tokenizer Overview</title> |
| </head> |
| |
| <body bgcolor="#FFFFFF" text="#000000" link="#0000EE" vlink="#551A8B" alink= |
| "#FF0000"> |
| <p><img src="../../boost.png" alt="C++ Boost" width="277" height= |
| "86"><br></p> |
| |
| <h1 align="center">Table Of Contents</h1> |
| |
| <p align="left"> </p> |
| |
| <h2 align="left"><a href="introduc.htm">Introduction</a></h2> |
| |
| <h2 align="left">Containers and Iterators</h2> |
| |
| <ul> |
| <li> |
| <h3 align="left"><a href="tokenizer.htm">tokenizer</a></h3> |
| </li> |
| |
| <li> |
| <h3 align="left"><a href="token_iterator.htm">token iterator</a></h3> |
| </li> |
| </ul> |
| |
| <h2><a href="tokenizerfunction.htm">TokenizerFunction Concept</a></h2> |
| |
| <h2>TokenizerFunction Models</h2> |
| |
| <ul> |
| <li> |
| <h3><a href="char_separator.htm">char_separator</a></h3> |
| </li> |
| |
| <li> |
| <h3><a href= |
| "escaped_list_separator.htm">escaped_list_separator</a></h3> |
| </li> |
| |
| <li> |
| <h3><a href="offset_separator.htm">offset_separator</a></h3> |
| </li> |
| |
| <li><font color="red">Deprecated:</font> <a href= |
| "char_delimiters_separator.htm">char_delimiters_separator</a></li> |
| </ul> |
| |
| <h2> </h2> |
| |
| <h2>Acknowledgements</h2> |
| |
| <p>I wish to thank the members of the boost mailing list, whose comments, |
| compliments, and criticisms during both the development and formal review |
| helped make the Tokenizer library what it is. I especially wish to thank |
| Aleksey Gurtovoy for the idea of using a pair of iterators to specify the |
| input, instead of a string. I also wish to thank Jeremy Siek for his idea |
| of providing a container interface for the token iterators and for |
| simplifying the template parameters for the TokenizerFunctions. He and |
| Daryle Walker also emphasized the need to separate interface and |
| implementation. Gary Powell sparked the idea of using the isspace and |
| ispunct as the defaults for char_delimiters_separator. Jeff Garland |
| provided ideas on how to change to order of the template parameters in |
| order to make tokenizer easier to declare. Thanks to Douglas Gregor who |
| served as review manager and provided many insights both on the boost list |
| and in e-mail on how to polish up the implementation and presentation of |
| Tokenizer. Finally, thanks to Beman Dawes who integrated the final version |
| into the boost distribution.</p> |
| <hr> |
| |
| <p><a href="http://validator.w3.org/check?uri=referer"><img border="0" src= |
| "../../doc/images/valid-html401.png" alt="Valid HTML 4.01 Transitional" |
| height="31" width="88"></a></p> |
| |
| <p>Revised |
| <!--webbot bot="Timestamp" s-type="EDITED" s-format="%d %B, %Y" startspan -->25 |
| December, 2006<!--webbot bot="Timestamp" endspan i-checksum="38518" --></p> |
| |
| <p><i>Copyright © 2000 Jeremy Siek<br> |
| Copyright © 2001 John R. Bandela</i></p> |
| |
| <p><i>Distributed under the Boost Software License, Version 1.0. (See |
| accompanying file <a href="../../LICENSE_1_0.txt">LICENSE_1_0.txt</a> or |
| copy at <a href= |
| "http://www.boost.org/LICENSE_1_0.txt">http://www.boost.org/LICENSE_1_0.txt</a>)</i></p> |
| </body> |
| </html> |
