Bug #7320

Wikilinks messed when using international characters

Details

Submitted	2006-04-05 17:39 UTC
From	martin dot ottenwaelter at ensimag dot fr
Assigned	justinpatrin
Status	Closed
Package	Text_Wiki
PHP Version	4.4.1
OS	Irrelevant
Roadmaps	(Not assigned)

Comments

[2006-04-05 17:39 UTC] martin dot ottenwaelter at ensimag dot fr

Description:
------------
** Problem :

The regular expression used to match wikilinks has bugs when
dealing with international characters (é, è, ü, etc...).

For example, the french word "Création" is mis-interpreted
as a wikilink : Text_Wiki will produce the following HTML
sequence wherever the word "Création" appears.

<a href="?wikiword=Cr%C3">Cr?</a>©ation

On the other hand, the word "Creation" is not interpreted as
a wikilink.

** Solution :

Use regular expressions UTF-8 capabilities :
1. instead of using the \xc0-\xfe range for international
characters, use the \pL, \p{Lu} and \p{Ll} for "any letter",
"any uppercase letter" and "any lower case letter"
2. add a "u" at the end of the regex to enable UTF-8 mode :
"/my_regex/u".

I have successfully patched "Wikilink.php" using this method
and it works just fine.

Sources :

http://www.php.net/manual/en/
reference.pcre.pattern.modifiers.php
http://www.php.net/manual/en/
reference.pcre.pattern.syntax.php

[2006-04-05 18:36 UTC] martin dot ottenwaelter at ensimag dot fr

Here goes an example :
The following php code :

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en"
lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8"/>
</head>

<body>

<?php
require_once 'PEAR.php';
require_once 'Text/Wiki.php';

$wiki = & Text_Wiki::singleton('Default');

// UTF8
$wiki->setFormatConf('Xhtml', 'charset',
'UTF-8');

// Extended character set (é à ù, etc...)
$wiki->setParseConf('wikilink', 'ext_chars',
'false');

$result = $wiki->transform("Création", 'Xhtml');
print($result);

</body>
</html>

outputs the follwing HTML

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<body>

</body>
</html>

instead of

...
<p>Création</p>
...

, with PHP 4.4.1 and the file encoding being UTF8.
Haven't had time to test it on 5.1.3, but my patch makes it
work on 4.4.1 !
Martin.