Splits the words from a text by whitespace characters. More...
Public Member Functions | |
tokenise (&$text) |
Splits the words from a text by whitespace characters.
Punctuation marks are also split from accompanying words.
Definition at line 47 of file WhitespaceTok.php.
WhitespaceTok::tokenise | ( | &$ | text | ) |
Implements Tokeniser.
Definition at line 52 of file WhitespaceTok.php.