org.basex.util.ft

Class Tokenizer

java.lang.Object

org.basex.util.ft.FTIterator

org.basex.util.ft.Tokenizer

All Implemented Interfaces:

java.lang.Comparable<org.basex.util.ft.LanguageImpl>, java.util.Iterator<FTSpan>

Direct Known Subclasses:

JapaneseTokenizer, WesternTokenizer

public abstract class Tokenizer
extends FTIterator

Abstract tokenizer.

Author:

BaseX Team 2005-12, BSD License, Jens Erat

Field Summary

Fields

Modifier and Type	Field and Description
static java.util.LinkedList<Tokenizer>	IMPL List of available tokenizers.

Constructor Summary

Constructors

Constructor and Description
Tokenizer()

Method Summary

Methods

Modifier and Type	Method and Description
int	compareTo(org.basex.util.ft.LanguageImpl o)
boolean	equals(java.lang.Object o)
int	hashCode()
protected abstract byte	prec() Returns the precedence of the processor.
static boolean	supportFor(Language l) Checks if the language is supported by the available tokenizers.
boolean	supports(Language ln) Checks if the specified language is supported.

Methods inherited from class org.basex.util.ft.FTIterator

init, nextToken, remove

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

hasNext, next

Field Detail

IMPL

public static final java.util.LinkedList<Tokenizer> IMPL

List of available tokenizers.

Constructor Detail

Tokenizer

public Tokenizer()

Method Detail

supportFor

public static boolean supportFor(Language l)

Checks if the language is supported by the available tokenizers.

Parameters:

l - language to be found

Returns:

result of check

prec

protected abstract byte prec()

Returns the precedence of the processor. If two language implementations support the same language, the processor with the higher precedence will be selected.

Returns:

precedence

supports

public boolean supports(Language ln)

Checks if the specified language is supported.

Parameters:

ln - language

Returns:

true if language is supported

equals

public final boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

compareTo

public final int compareTo(org.basex.util.ft.LanguageImpl o)

Specified by:

compareTo in interface java.lang.Comparable<org.basex.util.ft.LanguageImpl>

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object