Manual

Detects the language of a given piece of text. The package attempts to detect the language of a sample of text by correlating ranked 3-gram frequencies to a table of 3-gram frequencies of known languages. It implements a version of a technique originally proposed by Cavnar & Trenkle (1994): "N-Gram-Based Text Categorization".

Detecting the language

At first, you might want to get a list of supported languages. It can be retrieved by calling getLanguages on a Text_LanguageDetect object. It returns an array of strings that represent the languages, e.g. array('albanian', 'arabic', 'azeri').

To actually detect the language of a piece of text, use the detect method on the Text_LanguageDetect object. It takes the text as first parameter, and an optional $limit as second parameter, determining how many (likely) languages shall be returned at most. The method returns a sorted array with the languages as key, and their score as value. If no language is detected, an empty array is returned.

To get the most likely language only, use detectSimple which directly returns the string of the language, or null if none was detected.

To detect the language correctly, the length of the input text should be at least some sentences.

Language names

Text_LanguageDetect works with language names. It accepts language names for a number of methods, e.g. omitLanguages() and returns language names. By default, a "language name" is a lowercase english name of a language.

Often, applications work with ISO 639-1 or ISO 639-2 language codes - two-letter or three letter codes. Text_LanguageDetect supports them since version 0.3.0, and you may enable them with setNameMode():

<?php
$text = 'Das ist ein kleiner deutscher Text';

require_once 'Text/LanguageDetect.php';
$ld = new Text_LanguageDetect();

//default mode: full language name: "german"
echo $ld->detectSimple($text) . "\n";

//two-letter mode: "de"
$ld->setNameMode(2);
echo $ld->detectSimple($text) . "\n";

//three-letter mode: "deu"
$ld->setNameMode(3);
echo $ld->detectSimple($text) . "\n";

?>

The above example gives the following output:

Output

german
de
deu

Example

<?php
require_once 'Text/LanguageDetect.php';
$l = new Text_LanguageDetect();

echo "Supported languages:\n";
try {
    $langs = $l->getLanguages();
    sort($langs);
    echo implode(', ', $langs) . "\n\n";
} catch (Text_LanguageDetect_Exception $e) {
    die($e->getMessage());
}

$text = <<<EOD
Hallo! Das ist ein Text in deutscher Sprache.
Mal sehen, ob die Klasse erkennt, welche Sprache das hier ist.
EOD;

try {
    //return 2-letter language codes only
    $l->setNameMode(2);

    $result = $l->detect($text, 4);
    print_r($result);
} catch (Text_LanguageDetect_Exception $e) {
    die($e->getMessage());
}
?>

The above example would give the following output:

Output

Supported languages:
albanian, arabic, azeri, bengali, bulgarian, cebuano, croatian, czech,
danish, dutch, english, estonian, farsi, finnish, french, german, hausa,
hawaiian, hindi, hungarian, icelandic, indonesian, italian, kazakh, kyrgyz,
latin, latvian, lithuanian, macedonian, mongolian, nepali, norwegian, pashto,
pidgin, polish, portuguese, romanian, russian, serbian, slovak, slovene, somali,
spanish, swahili, swedish, tagalog, turkish, ukrainian, urdu, uzbek, vietnamese,
welsh

Array
(
    [de] => 0.40703703703704
    [nl] => 0.2880658436214
    [en] => 0.28333333333333
    [da] => 0.23452674897119
)