Section: User Contributed Perl Documentation (3)
Updated: 2008-05-18


HTML::WikiConverter::Dialects - How to add a dialect 


  # In your dialect module:  package HTML::WikiConverter::MySlimWiki;  use base 'HTML::WikiConverter';  sub rules { {    b => { start => '**', end => '**' },    i => { start => '//', end => '//' },    strong => { alias => 'b' },    em => { alias => 'i' },    hr => { replace => "\n----\n" }  } }  # In a nearby piece of code:  package main;  use Test::More tests => 5;  my $wc = new HTML::WikiConverter(    dialect => 'MySlimWiki'  );  is( $wc->html2wiki( '<b>text</b>' ), '**text**', b );  is( $wc->html2wiki( '<i>text</i>' ), '//text//', i );  is( $wc->html2wiki( '<strong>text</strong>' ), '**text**', 'strong' );  is( $wc->html2wiki( '<em>text</em>' ), '//text//', 'em' );  is( $wc->html2wiki( '<hr/>' ), '----', 'hr' );


HTML::WikiConverter (or H::WC, for short) is an HTML to wikiconverter. It can convert HTML source into a variety of wiki markups,called wiki ``dialects''. This manual describes how you to create yourown dialect to be plugged into HTML::WikiConverter. 


Each dialect has a separate dialect module containing rules forconverting HTML into wiki markup specific for that dialect. Currently,all dialect modules are in the "HTML::WikiConverter::" package spaceand subclass HTML::WikiConverter. For example, the MediaWiki dialectmodule is HTML::WikiConverter::MediaWiki, while PhpWiki's isHTML::WikiConverter::PhpWiki. However, dialect modules need not bein the "HTML::WikiConverter::" package space; you may just as easilyuse "package MyWikiDialect;" and H::WC will Do The Right Thing.

From now on, I'll be using the terms ``dialect'' and ``dialect module''interchangeably. 


To interface with H::WC, dialects need to subclass it. This is donelike so at the start of the dialect module:

  package HTML::WikiConverter::MySlimWiki;  use base 'HTML::WikiConverter';

Conversion rules

Dialects guide H::WC's conversion process with a set of rules thatdefine how HTML elements are turned into their wiki counterparts.Each rule corresponds to an HTML tag and there may be any number ofrules. Rules are specified in your dialect's "rules()" method, whichreturns a reference to a hash of rules. Each entry in the hash maps atag name to a set of subrules, as in:

    $tag => \%subrules

where $tag is the name of the HTML tag (e.g., "b", "em",etc.) and %subrules contains subrules that specify how that tagwill be converted when it is encountered in the HTML input.


The following subrules are recognized:

  start  end  preserve  attributes  empty  replace  alias  block  line_format  line_prefix  trim

A simple example

The following rules could be used for a dialect that uses"*asterisks*" for bold and "_underscores_" for italic text:

  sub rules {    b => { start => '*', end => '*' },    i => { start => '_', end => '_' },  }


To add "<strong>" and "<em>" as aliases of "<b>" and"<i>", use the "alias" subrule:

  strong => { alias => 'b' },  em => { alias => 'i' },

(The "alias" subrule cannot be used with any other subrule.)


Many dialects separate paragraphs and other block-level elementswith a blank line. To indicate this, use the "block" subrule:

  p => { block => 1 },

(To better support nested block elements, if a block elements arenested inside each other, blank lines are only added to the outermostelement.)

Line formatting

Many dialects require that the text of an element be contained on asingle line of text, or that it cannot contain any newlines,etc. These options can be specified using the "line_format" subrule,which can be assigned the value "single", "multi", or"blocks".

If the element must be contained on a single line, then the"line_format" subrule should be "single". If the element can spanmultiple lines, but there can be no blank lines contained within, thenuse "multi". If blank lines (which delimit blocks) are allowed,then use "blocks". For example, paragraphs are specified like so inthe MediaWiki dialect:

  p => { block => 1, line_format => 'multi', trim => 'both' },

Trimming whitespace

The "trim" subrule specifies whether leading or trailing whitespace(or both) should be stripped from the element. To strip leadingwhitespace only, use "leading"; for trailing whitespace, use"trailing"; for both, use the aptly named "both"; for neither(the default), use "none".

Line prefixes

Some elements require that each line be prefixed with a particularstring. This is specified with the "line_prefix" subrule. Forexample, preformatted text in MediaWiki is prefixed with a space:

  pre => { block => 1, line_prefix => ' ' },


In some cases, conversion from HTML to wiki markup is as simple asstring replacement. To replace a tag and its contents with aparticular string, use the "replace" subrule. For example, inPhpWiki, three percent signs, "%%%", represents a line break,"<br>", hence:

  br => { replace => '%%%' },

(The "replace" subrule cannot be used with any other subrule.)

Preserving HTML tags

Some dialects allow a subset of HTML in their markup. While H::WCignores unhandled HTML tags by default (i.e., if H::WC encounters atag that does not exist in a dialect's rule specification, then thecontents of the tag is simply passed through to the wiki markup), youmay specify that some be preserved using the "preserve" subrule. Forexample, to allow "<font>" tag in wiki markup:

  font => { preserve => 1 },

Preserved tags may also specify a list of attributes that may alsopassthrough from HTML to wiki markup. This is done with the"attributes" subrule:

  font => { preserve => 1, attributes => [ qw/ style class / ] },

(The "attributes" subrule can only be used if the "preserve" subruleis also present.)

Some HTML elements have no content (e.g., line breaks, images) and thewiki dialect might require them to be preserved in a moreXHTML-friendly way. To indicate that a preserved tag should have nocontent, use the "empty" subrule. This will cause the element to bereplaced with "<tag />" and no end tag. For example,MediaWiki handles line breaks like so:

  br => {    preserve => 1,    attributes => [ qw/ id class title style clear / ],    empty => 1  },

This will convert, for example, "<br clear='both'>" into"<br clear='both' />". Without specifying the "empty"subrule, this would be converted into the (probably undesirable)"<br clear='both'></br>".

(The "empty" subrule can only be used if the "preserve" subrule isalso present.)

Rules that depend on attribute values

In some circumstances, you might want your dialect's conversion rulesto depend on the value of one or more attributes. This can be achievedby producing rules in a conditional manner within "rules()". Forexample:

  sub rules {    my $self = shift;        my %rules = (      em => { start => "''", end => "''" },      strong => { start => "'''", end => "'''" },    );    $rules{i} = { preserve => 1 } if $self->preserve_italic;    $rules{b} = { preserve => 1 } if $self->preserve_bold;    return \%rules;  }

Dynamic subrules

Instead of simple strings, you may use coderefs as values for the"start", "end", "replace", and "line_prefix" subrules. If you do,the code will be called when the subrule is applied, and will bepassed three arguments: the current H::WC object, the currentHTML::Element node being operated on, and a reference to the hashcontaining the dialect's subrules associated with elements of thattype.

For example, MoinMoin handles lists like so:

  ul => { line_format => 'multi', block => 1, line_prefix => '  ' },  li => { start => \&_li_start, trim => 'leading' },  ol => { alias => 'ul' },

It then defines "_li_start()":

  sub _li_start {    my( $self, $node, $subrules ) = @_;    my $bullet = '';    $bullet = '*'  if $node->parent->tag eq 'ul';    $bullet = '1.' if $node->parent->tag eq 'ol';    return "\n$bullet ";  }

This prefixes every unordered list item with "*" and every orderedlist item with "1.", which MoinMoin requires. It also puts eachlist item on its own line and places a space between the prefix andthe content of the list item. 

Subrule validation

Certain subrule combinations are not allowed. Hopefully it's intuitivewhy this is, but in case it's not, prohibited combinations have beenmentioned above parenthetically. For example, the "replace" and"alias" subrules cannot be combined with any other subrules, and"attributes" can only be specified alongside "preserve". Invalidsubrule combinations will trigger a fatal error when the H::WC objectis instantiated. 

Dialect attributes

H::WC's constructor accepts a number of attributes that help determinehow conversion takes place. Dialects can alter these attributes or addtheir own by defining an "attributes()" method, which returns areference to a hash of attributes. Each entry in the hash maps theattribute's name to an attribute specification, as in:

  $attr => \%spec

where $attr is the name of the attribute and %spec is aParams::Validate specification for the attribute.

For example, to add a boolean attribute called "camel_case" which isdisabled by default:

  sub attributes {    camel_case => { default => 0 },  }

Attributes defined liks this are given accessor and mutator methodsvia Perl's "AUTOLOAD" mechanism, so you can later say:

  my $ok = $wc->camel_case;  $wc->camel_case(0);

You may override the default H::WC attributes using thismechanism. For example, while H::WC considers the "base_uri"attribute optional, it is required for the PbWiki dialect. PbWiki canoverride this default-optional behavior by saying:

  sub attributes {    base_uri => { optional => 0 }  }


The first step H::WC takes in converting HTML source to wiki markup isto parse the HTML into a syntax tree using HTML::TreeBuilder. It isoften useful for dialects to preprocess the tree prior to convertingit into wiki markup. Dialects that need to preprocess the tree candefine a "preprocess_node" method that will be called on each node ofthe tree (traversal is done in pre-order). The method receives twoarguments, the H::WC object, and the current HTML::Element nodebeing traversed. It may modify the node or decide to ignore it; itsreturn value is discarded.

Built-in preprocessors

Because they are commonly needed, H::WC automatically carries out twopreprocessing steps, regardless of the dialect: 1) relative URIs inimages and links are converted to absolute URIs (based upon the"base_uri" parameter), and 2) ignorable text (e.g. between a"</td>" and "<td>") is discarded.

H::WC also provides additional preprocessing stepsthat may be explicitly enabled by dialect modules.

Removes any anchor elements that do not contain an "href" attribute.
Removes table captions and reinserts them as paragraphs before thetable.

Dialects may apply these optional preprocessing steps by calling themas methods on the dialect object inside "preprocess_node". Forexample:

  sub preprocess_node {    my( $self, $node ) = @_;    $self->strip_aname($node);    $self->caption2para($node);  }


Once the work of converting HTML is complete, it is sometimes usefulto postprocess the resulting wiki markup. Postprocessing can be usedto clean up whitespace, fix subtle bugs introduced in the markupduring conversion, etc.

Dialects that want to postprocess the wiki markup should define a"postprocess_output" method that will be called just before the"html2wiki" method returns to the client. The method will be passedtwo arguments, the H::WC object and a reference to the wikimarkup. The method may modify the wiki markup that the referencepoints to; its return value is discarded.

For example, to replace a series of line breaks with a pair ofnewlines, a dialect might implement this:

  sub postprocess_output {    my( $self, $outref ) = @_;    $$outref =~ s/<br>\s*<br>/\n\n/gs;  }

(This example assumes that HTML line breaks were replaced with"<br>" in the wiki markup.) 

Dialect utility methods

H::WC defines a set of utility methods that dialect modules may finduseful.


  my $wiki = $wc->get_elem_contents( $node );

Converts the contents of $node into wiki markup and returns theresulting wiki markup.


  my $title = $wc->get_wiki_page( $url );

Attempts to extract the title of a wiki page from the given URL,returning the title on success, "undef" on failure. If "wiki_uri" isempty, this method always return "undef". See``ATTRIBUTES'' in HTML::WikiConverter for details on how the "wiki_uri"attribute is interpreted.


  my $ok = $wc->is_camel_case( $str );

Returns true if $str is in CamelCase, falseotherwise. CamelCase-ness is determined using the same rules thatKwiki's formatting module uses.


  my $attr_str = $wc->get_attr_str( $node, @attrs );

Returns a string containing the specified attributes in the givennode. The returned string is suitable for insertion into an HTML tag.For example, if $node contains the HTML

  <style id="ht" class="head" onclick="editPage()">Header</span>

and @attrs contains "id" and "class", then "get_attr_str()"will return 'id="ht" class="head"'.


  my $value = $wc->_attr( $name );

Returns the value of the named attribute. This is rarely needed sinceyou can access attribute values by treating the attribute name as amethod (i.e., "$wc->$name"). This low-level method of accessingattributes is provided for when you need to override an attribute'saccessor/mutator method, as in:

  sub attributes { {    my_attr => { default => 1 },  } }  sub my_attr {    my( $wc, $name, $new_value ) = @_;    # do something special    return $wc->_attr( $name => $new_value );  }


David J. Iberri <> 


Copyright 2006 David J. Iberri, all rights reserved.

This program is free software; you can redistribute it and/or modify itunder the same terms as Perl itself.



Conversion rules
Dynamic subrules
Subrule validation
Dialect attributes
Dialect utility methods

This document was created byman2html,using the manual pages.