Home » PHP » PHP_LexerGenerator » Bug #9230

[2006-11-04 00:22 UTC] tom at inflatablecookie dot com

Description:
------------
The regex parser (PHP/LexerGenerator/Regex/Parser.php) doesn't handle escape sequences very well in plex file regex definitions when they come before certain other characters.
The main cause of the issue is the fact that the regexes get compiled into double quoted strings rather than single quoted strings (which I can't see a need for).
Characters such as " and $ and { need to be escaped locally in the compiler (which they are), however if you need a backslash before one of those, (as in /[\\"]/ inside a single quote regex) the preceding backslash gets stripped.

Test script:
---------------
/*!lex2php
%input $this->data
%counter $this->N
%token $this->token
%value $this->value
%line $this->line
delim1 = /["]/
delim2 = /[\"]/
delim3 = /[\\"]/
*/

Expected result:
----------------
in /mycompiledlexer.php

$yy_global_pattern = '/^(["]|[\\"]|[\\\\"])/';

Actual result:
--------------
in /mycompiledlexer.php

$yy_global_pattern = "/^([\"]|[\"]|[\\\\\"])/";

[2006-11-04 00:44 UTC] tom at inflatablecookie dot com

This problem also affects octal ranges, but strangely in the opposite manner, and only on the right hand side of the range..

/[a-z_\x7f-\xff]/

will produce

[a-z_\x7f-xff]/

in the compiled lexer.