MAN page from Old RedHat 6.X perl-HTML-Parser-3.05-6.i386.rpm

lib::HTML::TokeParser

Section: User Contributed Perl Documentation (3)
Updated: perl 5.005, patch 03
Index

NAME

HTML::TokeParser - Alternative HTML::Parser interface

SYNOPSIS

 require HTML::TokeParser; $p = HTML::TokeParser->new("index.html") || die "Can't open: $!"; while (my $token = $p->get_token) {     #... }

DESCRIPTION

The HTML::TokeParser is an alternative interface to the HTML::Parser class.It basically turns the HTML::Parser inside out. You associate a file(or any IO::Handle object or string) with the parser at construction time andthen repeatedly call $parser->get_token to obtain the tags and textfound in the parsed document.

Calling the methods defined by the HTML::Parser base class will beconfusing, so don't do that. Use the following methods instead:

$p = HTML::TokeParser->new( $file_or_doc );

The object constructor argument is either a file name, a file handleobject, or the complete document to be parsed.

If the argument is a plain scalar, then it is taken as the name of afile to be opened and parsed. If the file can't be opened forreading, then the constructor will return an undefined value and $!will tell you why it failed.

If the argument is a reference to a plain scalar, then this scalar istaken to be the literal document to parse. The value of thisscalar should not be changed before all tokens have been extracted.

Otherwise the argument is taken to be some object that theHTML::TokeParser can read() from when it needs more data. Typicallyit will be a filehandle of some kind. The stream will be read() untilEOF, but not closed.

$p->get_token

This method will return the next token found in the HTML document,or undef at the end of the document. The token is returned as anarray reference. The first element of the array will be a (mostly)single character string denoting the type of this token: ``S'' for starttag, ``E'' for end tag, ``T'' for text, ``C'' for comment, ``D'' fordeclaration, and ``pi'' for process instructions. The rest of the arrayis the same as the arguments passed to the corresponding HTML::Parsercallbacks (see the HTML::Parser manpage). Returned tokens look like this:

  ["S",  $tag, %$attr, @$attrseq, $text]  ["E",  $tag, $text]  ["T",  $text]  ["C",  $text]  ["D",  $text]  ["PI", $text]

$p->unget_token($token,...)

If you find out you have read too many tokens you can push them back,so that they are returned the next time $p->get_token is called.

$p->get_tag( [$tag] )

This method returns the next start or end tag (skipping any othertokens), or undef if there are no more tags in the document. If anargument is given, then we skip tokens until the specified tag type isfound. The tag is returned as an array reference in the same form asfor $p->get_token above, but the type code (first element) is missingand the name of end tags are prefixed with ``/''. This means that thetags returned look like this:

  [$tag, %$attr, @$attrseq, $text]  ["/$tag", $text]

$p->get_text( [$endtag] )

This method returns all text found at the current position. It willreturn a zero length string if the next token is not text. Theoptional $endtag argument specifies that any text occurring before thegiven tag is to be returned. Any entities will be converted to theircorresponding character.

The $p->{textify} attribute is a hash that defines how certain tags canbe treated as text. If the name of a start tag matches a key in thishash then this tag is converted to text. The hash value is used tospecify which tag attribute to obtain the text from. If this tagattribute is missing, then the upper case name of the tag enclosed inbrackets is returned, e.g. ``[IMG]''. The hash value can also be asubroutine reference. In this case the routine is called with thestart tag token content as its argument and the return value is treatedas the text.

The default $p->{textify} value is:

  {img => "alt", applet => "alt"}

This means that <IMG> and <APPLET> tags are treated as text, and thatthe text to substitute can be found in the ALT attribute.

$p->get_trimmed_text( [$endtag] )

Same as $p->get_text above, but will collapse any sequences of whitespace to a single space character. Leading and trailing white space isremoved.

EXAMPLES

This example extracts all links from a document. It will print oneline for each link, containing the URL and the textual descriptionbetween the <A>...</A> tags:

  use HTML::TokeParser;  $p = HTML::TokeParser->new(shift||"index.html");

  while (my $token = $p->get_tag("a")) {      my $url = $token->[1]{href} || "-";      my $text = $p->get_trimmed_text("/a");      print "$url\t$text\n";  }

This example extract the <TITLE> from the document:

  use HTML::TokeParser;  $p = HTML::TokeParser->new(shift||"index.html");  if ($p->get_tag("title")) {      my $title = $p->get_trimmed_text;      print "Title: $title\n";  }

COPYRIGHT

This library is free software; you can redistribute it and/ormodify it under the same terms as Perl itself.

This document was created byman2html,using the manual pages.