| [/============================================================================== |
| Copyright (C) 2001-2010 Hartmut Kaiser |
| Copyright (C) 2001-2010 Joel de Guzman |
| |
| Distributed under the Boost Software License, Version 1.0. (See accompanying |
| file LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt) |
| ===============================================================================/] |
| |
| [section:lexer_concepts Lexer Concepts] |
| |
| __lex__ components fall into a couple of generalized __concepts__. The |
| /Lexer/ is the most fundamental concept. All __lex__ components are |
| models of the /Lexer/ concept. /PrimitiveLexer/, /UnaryLexer/, |
| and /NaryLexer/ are all refinements of the /Lexer/ concept. |
| |
| The following sections provide details on these concepts. |
| |
| [/////////////////////////////////////////////////////////////////////////////] |
| [section Lexer] |
| |
| [heading Description] |
| |
| The /Lexer/ is the most fundamental concept. A Lexer has a member |
| function, `collect`, that accepts a token definition container `Def`, and a |
| the name of the lexer state the token definitions of the lexer component need |
| to be added to (a string). It doesn't return anything (return type is `void`). |
| Each Lexer can represent a specific pattern or algorithm, or it |
| can be a more complex lexer component formed as a composition of other Lexer's. |
| Additionally, a Lexer exposes a member `add_actions`, that accepts the token |
| definition container `Def`, while returning nothing (again, the returned type |
| is `void`). |
| |
| [variablelist Notation |
| [[`l`] [A `Lexer`.]] |
| [[`L`] [A `Lexer` type.]] |
| [[`Def`] [A token definition container type.]] |
| [[`State`] [A type used to represent lexer state names.]] |
| ] |
| |
| [heading Valid Expressions] |
| |
| In the expressions below, the behavior of the lexer component, `l`, is left |
| unspecified in the base `Lexer` concept. These are specified in subsequent, |
| more refined concepts and by the actual models thereof. |
| |
| For any Lexer the following expressions must be valid: |
| |
| [table |
| [[Expression] [Semantics] [Return type]] |
| [[`l.collect(def, state)`] [Add all token definitions provided |
| by this Lexer instance to the lexer |
| state `state` of the token definition |
| container `def`.] [`void`]] |
| [[`l.add_actions(def)`] [Add all semantic actions provided |
| by this Lexer instance to the token |
| definition container `def`.] [`void`]] |
| ] |
| |
| [heading Type Expressions] |
| |
| [table |
| [[Expression] [Description]] |
| [[`traits::is_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if |
| a certain type, `L` is a Lexer, `mpl::false_` |
| otherwise (See __mpl_boolean_constant__).]] |
| ] |
| |
| [heading Postcondition] |
| |
| Upon return from `l.collect` the following post conditions should hold: |
| |
| * On return, `def` holds all token definitions defined in the Lexer, `l`. This |
| includes all Lexer's contained inside `l`. |
| |
| Upon return from `l.add_actions` the following post conditions should hold: |
| |
| * On return, `def` holds all semantic actions correctly associated with the |
| corresponding token definitions as defined in the Lexer, `l`. This |
| includes all semantic actions defined by the Lexer's contained inside `l`. |
| |
| [heading Models] |
| |
| All lexer components in __lex__ are models of the /Lexer/ concept. |
| |
| [endsect] [/ Lexer Concept] |
| |
| [/////////////////////////////////////////////////////////////////////////////] |
| [section PrimitiveLexer] |
| |
| [heading Description] |
| |
| /PrimitiveLexer/ is the most basic building block that the client uses |
| to build more complex lexer components. |
| |
| [heading Refinement of] |
| |
| [:__lexer_concept__] |
| |
| [heading Type Expressions] |
| |
| [table |
| [[Expression] [Description]] |
| [[`traits::is_primitive_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if |
| a certain type, `L`, is a PrimitiveLexer, `mpl::false_` |
| otherwise (See __mpl_boolean_constant__).]] |
| ] |
| |
| [heading Models] |
| |
| The following lexer components conform to this model: |
| |
| * character literals (i.e. `'x'`), `char_`, |
| * string literals (`"abc"`), `std::basic_string<>`, `string` |
| |
| __fixme__ Add more links to /PrimitiveLexer/ models here. |
| |
| [endsect] [/ PrimitiveLexer Concept] |
| |
| [/////////////////////////////////////////////////////////////////////////////] |
| [section UnaryLexer] |
| |
| [heading Description] |
| |
| /UnaryLexer/ is a composite lexer component that has a single subject. The |
| UnaryLexer may change the behavior of its subject following the |
| __delegate_pattern__. |
| |
| [heading Refinement of] |
| |
| [:__lexer_concept__] |
| |
| [variablelist Notation |
| [[`l`] [A UnaryLexer.]] |
| [[`L`] [A UnaryLexer type.]] |
| ] |
| |
| [heading Valid Expressions] |
| |
| In addition to the requirements defined in __lexer_concept__, for any |
| UnaryLexer the following must be met: |
| |
| [table |
| [[Expression] [Semantics] [Return type]] |
| [[`l.subject`] [Subject lexer component.] [__lexer_concept__]] |
| ] |
| |
| [heading Type Expressions] |
| |
| [table |
| [[Expression] [Description]] |
| [[`L::subject_type`] [The subject lexer component type.]] |
| [[`traits::is_unary_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if |
| a certain type, `L` is a UnaryLexer, `mpl::false_` |
| otherwise (See __mpl_boolean_constant__).]] |
| ] |
| |
| [heading Invariants] |
| |
| For any UnaryLexer, `L`, the following invariant always holds: |
| |
| * `traits::is_lexer<L::subject_type>::type` evaluates to `mpl::true_` |
| |
| [heading Models] |
| |
| The following lexer components conform to this model: |
| |
| * action lexer component (allowing to attach semantic actions) |
| |
| __fixme__ Add more links to models of UnaryLexer concept |
| |
| [endsect] [/ UnaryLexer Concept] |
| |
| [/////////////////////////////////////////////////////////////////////////////] |
| [section NaryLexer] |
| |
| [heading Description] |
| |
| /NaryLexer/ is a composite lexer component that has one or more subjects. The |
| NaryLexer allows its subjects to be treated in the same way as a single |
| instance of a __lexer_concept__ following the __composite_pattern__. |
| |
| [heading Refinement of] |
| |
| [:__lexer_concept__] |
| |
| [variablelist Notation |
| [[`l`] [A NaryLexer.]] |
| [[`L`] [A NaryLexer type.]] |
| ] |
| |
| [heading Valid Expressions] |
| |
| In addition to the requirements defined in __lexer_concept__, for any |
| NaryLexer the following must be met: |
| |
| [table |
| [[Expression] [Semantics] [Return type]] |
| [[`l.elements`] [The tuple of elements.] [A __fusion__ Sequence of __lexer_concept__ types.]] |
| ] |
| |
| [heading Type Expressions] |
| |
| [table |
| [[Expression] [Description]] |
| [[`l.elements_type`] [Elements tuple type.]] |
| [[`traits::is_nary_lexer<L>::type`] [Metafunction that evaluates to `mpl::true_` if |
| a certain type, `L` is a NaryLexer, `mpl::false_` |
| otherwise (See __mpl_boolean_constant__).]] |
| ] |
| |
| [heading Invariants] |
| |
| For each element, `E`, in any NaryLexer, `L`, the following |
| invariant always holds: |
| |
| * `traits::is_lexer<E>::type` evaluates to `mpl::true_` |
| |
| [heading Models] |
| |
| The following lexer components conform to this model: |
| |
| * lexer sequence component |
| |
| __fixme__ Add more links to models of NaryLexer concept |
| |
| [endsect] [/ NaryLexer Concept] |
| |
| [endsect] |