MAN page from Old RedHat 7.X perl-Language-Basic-1.44-8.i386.rpm

Language::Basic::Token

Section: User Contributed Perl Documentation (3)
Updated: perl v5.6.0
Index 

NAME

Language::Basic::Token - Module to handle lexing BASIC statements. 

SYNOPSIS

See the Language::Basic manpage for the overview of how the Language::Basic moduleworks. This pod page is more technical.

     # lex a line of basic into a bunch of tokens.     my $token_group = new Language::Basic::Token::Group;     $token_group->lex('PRINT "YES","NO" : A=A+1');

     # Look at tokens     my $tok = $token_group->lookahead && print $tok->text;     # Eat expected tokens     my $tok = $token_group->eat_if_string(",");     my $tok = $token_group->eat_if_class("Keyword");

 

DESCRIPTION

BASIC tokens are pretty simple. They include Keywords, Identifiers (Variableor Function names), String and Numeric Constants, and a few one- ortwo-character operators, like ':' and '<='. Tokens aren't very ambiguous, sofor example, you don't need to know what type of Statement you're looking atin order to lex a line of BASIC. (The only remotely ambiguous thing is that'=' can be either a Relational Operator or an Assignment statement.)

The subclasses of LB::Token represent the various sorts of tokens. TheToken::Group class isn't really a subclass at all; it's a group of tokens. Seethe section on "Language::Basic::Token::Group" for more info.

The ``text'' method returns the text that makes up the token. Note that textis stored in upper case (except for string constants, which are storedexactly as entered). 

class Language::Basic::Token::Group

This important class handles a group of tokens. Text from the BASIC programis lexed and turned into LB::Tokens which are stored in a Token::Group. Anyaccess to these Tokens (including creating them) is through the Token::Groupmethods. Other classes' parse methods will usually eat their way throughthe tokens in the Token::Group until it's empty.

new

This method just creates a new LBT::Group.

lex

This method breaks BASIC text arg1 into LB::Tokens and puts them inToken::Group arg0.

lookahead

This method returns the next token in the Token::Group without removingit from the group. That means lookahead can be called many timesand keep getting the same token (as long as eat is never called).It returns undef if there are no more Tokens left.

eat

This method eats the next Token from the Token::Group and returns it. It returns undef if there are no more Tokens left.

eat_if_string

This method eats the next token from Group arg0 if it matches string arg1If it ate a token, it returns it. Otherwise (or if there are no tokens left)it returns undef.

Note that the string to match should be upper case, since all \w tokensare stored as uppercase.

eat_if_class

This method eats the next token from Group arg0 if the token is of class``Language::Basic::Token::'' . arg1. (I.e., it's called with ``Keyword'' to get a Language::Basic::Token::Keyword Token.) If it ate a token, it returns it.Otherwise (or if there are no tokens left) it returns undef.

slurp

Eats tokens from Group arg1 and puts them in Group arg0 until it getsto a Token whose text matches string arg2 or it reaches the end of arg1. (Thematching Token is left in arg1.)

stuff_left

Returns true if there's stuff left in the Statement we're parsing (i.e. ifthere are still tokens left in the Token::Group and the next token isn't acolon)

print

For debugging purposes. Returns the Tokens in Group arg0 nicely formatted.

 

Other Language::Basic::Token subclasses

The other subclasses are actually kinds of Tokens, unlike Token::Group.There are no ``new'' methods for these classes. Creation of Tokens is doneby Token::Group::lex. In fact, these classes don't have any publicmethods. They're mostly there to use ``isa'' on.

Keyword

A BASIC keyword (reserved word)

Identifier

An Identifier matches /[A-Z][A-Z0-9]*\$?/. It's a variable or functionname.

String_Constant

Stuff inside double quotes.

Numeric_Constant

A float (or integer, currently)

Left_Paren

A ``(''

Right_Paren

A ``)''

Separator

Comma or semicolon (separators in arglists, PRINT statements)

Arithmetic_Operator

Plus or minus

Multiplicative_Operator

Multiply or divide operators ('*' and '/')

Relational_Operator

Greater than, less than, equals, and their combinations. Note thatequals sign is also used to assign values in BASIC.

Logical_Operator

AND, OR, NOT

Comment

REM statement (includes the whole rest of the line, even if there are colonsin it)

Statement_End

End of a statement (i.e., a colon)

---

 

Index

NAME

SYNOPSIS

DESCRIPTION

class Language::Basic::Token::Group

Other Language::Basic::Token subclasses

---

This document was created byman2html,using the manual pages.