- java.lang.Object
-
- org.apache.lucene.analysis.AbstractAnalysisFactory
-
- org.apache.lucene.analysis.TokenizerFactory
-
- Direct Known Subclasses:
ClassicTokenizerFactory
,EdgeNGramTokenizerFactory
,HMMChineseTokenizerFactory
,ICUTokenizerFactory
,JapaneseTokenizerFactory
,KeywordTokenizerFactory
,KoreanTokenizerFactory
,LetterTokenizerFactory
,NGramTokenizerFactory
,PathHierarchyTokenizerFactory
,PatternTokenizerFactory
,SimplePatternSplitTokenizerFactory
,SimplePatternTokenizerFactory
,StandardTokenizerFactory
,ThaiTokenizerFactory
,UAX29URLEmailTokenizerFactory
,WhitespaceTokenizerFactory
,WikipediaTokenizerFactory
public abstract class TokenizerFactory extends AbstractAnalysisFactory
Abstract parent class for analysis factories that createTokenizer
instances.- Since:
- 3.1
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description private static class
TokenizerFactory.Holder
This static holder class prevents classloading deadlock by delaying init of factories until needed.
-
Field Summary
-
Fields inherited from class org.apache.lucene.analysis.AbstractAnalysisFactory
LUCENE_MATCH_VERSION_PARAM, luceneMatchVersion
-
-
Constructor Summary
Constructors Modifier Constructor Description protected
TokenizerFactory()
Default ctor for compatibility with SPIprotected
TokenizerFactory(java.util.Map<java.lang.String,java.lang.String> args)
Initialize this factory via a set of key-value pairs.
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description static java.util.Set<java.lang.String>
availableTokenizers()
returns a list of all available tokenizer names from context classpathTokenizer
create()
Creates a TokenStream of the specified input using the default attribute factory.abstract Tokenizer
create(AttributeFactory factory)
Creates a TokenStream of the specified input using the given AttributeFactorystatic java.lang.String
findSPIName(java.lang.Class<? extends TokenizerFactory> serviceClass)
looks up a SPI name for the specified tokenizer factorystatic TokenizerFactory
forName(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> args)
looks up a tokenizer by name from context classpathstatic java.lang.Class<? extends TokenizerFactory>
lookupClass(java.lang.String name)
looks up a tokenizer class by name from context classpathstatic void
reloadTokenizers(java.lang.ClassLoader classloader)
Reloads the factory list from the givenClassLoader
.-
Methods inherited from class org.apache.lucene.analysis.AbstractAnalysisFactory
defaultCtorException, get, get, get, get, get, getBoolean, getChar, getClassArg, getFloat, getInt, getLines, getLuceneMatchVersion, getOriginalArgs, getPattern, getSet, getSnowballWordSet, getWordSet, isExplicitLuceneMatchVersion, require, require, require, requireBoolean, requireChar, requireFloat, requireInt, setExplicitLuceneMatchVersion, splitAt, splitFileNames
-
-
-
-
Method Detail
-
forName
public static TokenizerFactory forName(java.lang.String name, java.util.Map<java.lang.String,java.lang.String> args)
looks up a tokenizer by name from context classpath
-
lookupClass
public static java.lang.Class<? extends TokenizerFactory> lookupClass(java.lang.String name)
looks up a tokenizer class by name from context classpath
-
availableTokenizers
public static java.util.Set<java.lang.String> availableTokenizers()
returns a list of all available tokenizer names from context classpath
-
findSPIName
public static java.lang.String findSPIName(java.lang.Class<? extends TokenizerFactory> serviceClass)
looks up a SPI name for the specified tokenizer factory
-
reloadTokenizers
public static void reloadTokenizers(java.lang.ClassLoader classloader)
Reloads the factory list from the givenClassLoader
. Changes to the factories are visible after the method ends, all iterators (availableTokenizers()
,...) stay consistent.NOTE: Only new factories are added, existing ones are never removed or replaced.
This method is expensive and should only be called for discovery of new factories on the given classpath/classloader!
-
create
public final Tokenizer create()
Creates a TokenStream of the specified input using the default attribute factory.
-
create
public abstract Tokenizer create(AttributeFactory factory)
Creates a TokenStream of the specified input using the given AttributeFactory
-
-