module Plexing:Lexing for Camlp5 grammars.sig
..end
This module defines the Camlp5 lexer type to be used in extensible
grammars (see module Grammar
). It also provides some useful functions
to create lexers.
typepattern =
string * string
exception Error of string
type 'a
lexer = {
|
tok_func : |
|
tok_using : |
|
tok_removing : |
|
mutable tok_match : |
|
tok_text : |
|
mutable tok_comm : |
'te
is the type of the tokens.tok_func
is the main lexer function. See lexer_func
type below.tok_using
is a function called by the EXTEND
statement to warn the lexer that a rule uses this pattern
(given as parameter). This allow the lexer 1/ to check that
the pattern constructor is really among its possible constructors
2/ to enter the keywords in its tables.tok_removing
is a function possibly called by the
DELETE_RULE
statement to warn the lexer that this pattern
(given as parameter) is no more used in the grammar (the grammar
system maintains a number of usages of all patterns and calls this
function when this number falls to zero). If it is a keyword, this
allow the lexer to remove it in its tables.tok_match
is a function called by the camlp5
grammar system to ask the lexer how the input tokens have to
be matched against the patterns. Warning: for efficiency, this
function has to be written as a function taking patterns as
parameters and, for each pattern value, returning a function
matching a token, *not* as a function with two parameters.tok_text
is a function called by the grammar
system to get the name of the tokens for the error messages,
in case of syntax error, or for the displaying of the rules
of an entry.tok_comm
is a mutable place where the lexer can
put the locations of the comments, if its initial value is not
None
. If it is None
, nothing has to be done by the lexer.type'a
lexer_func =char Stream.t -> 'a Stream.t * location_function
tok_func
of the type
glexer
). The character stream is the input stream to be
lexed. The result is a pair of a token stream and a location
function (see below) for this tokens stream.typelocation_function =
int -> Ploc.t
val lexer_text : pattern -> string
tok_text
function.val default_match : pattern -> string * string -> string
tok_match
function, appling to the token type
(string * string)
The functions below create lexer functions either from a char stream
parser or for an ocamllex
function. With the returned function f
,
it is possible to get a simple lexer (of the type Plexing.glexer
above):
{ Plexing.tok_func = f;
Plexing.tok_using = (fun _ -> ());
Plexing.tok_removing = (fun _ -> ());
Plexing.tok_match = Plexing.default_match;
Plexing.tok_text = Plexing.lexer_text }
Note that a better tok_using
function should check the used tokens
and raise Plexing.Error
for incorrect ones. The other functions
tok_removing
, tok_match
and tok_text
may have other implementations
as well.val lexer_func_of_parser : (char Stream.t * int Pervasives.ref * int Pervasives.ref -> 'a * Ploc.t) ->
'a lexer_func
val lexer_func_of_ocamllex : (Lexing.lexbuf -> 'a) -> 'a lexer_func
ocamllex
val make_stream_and_location : (unit -> 'a * Ploc.t) -> 'a Stream.t * location_function
val eval_char : string -> char
val eval_string : Ploc.t -> string -> string
Failure
if
bad backslash sequence found; Plexing.eval_char (Char.escaped c)
would return c
and Plexing.eval_string (String.escaped s)
would
return s
val restore_lexing_info : (int * int) option Pervasives.ref
val line_nb : int Pervasives.ref Pervasives.ref
val bol_pos : int Pervasives.ref Pervasives.ref
module Lexbuf:sig
..end