Class ShingleAnalyzerWrapper
- java.lang.Object
-
- org.apache.lucene.analysis.Analyzer
-
- org.apache.lucene.analysis.AnalyzerWrapper
-
- org.apache.lucene.analysis.shingle.ShingleAnalyzerWrapper
-
- All Implemented Interfaces:
java.io.Closeable
,java.lang.AutoCloseable
public final class ShingleAnalyzerWrapper extends AnalyzerWrapper
A ShingleAnalyzerWrapper wraps aShingleFilter
around anotherAnalyzer
.A shingle is another name for a token based n-gram.
- Since:
- 3.1
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer
Analyzer.ReuseStrategy, Analyzer.TokenStreamComponents
-
-
Field Summary
Fields Modifier and Type Field Description private Analyzer
delegate
private java.lang.String
fillerToken
private int
maxShingleSize
private int
minShingleSize
private boolean
outputUnigrams
private boolean
outputUnigramsIfNoShingles
private java.lang.String
tokenSeparator
-
Fields inherited from class org.apache.lucene.analysis.Analyzer
GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY
-
-
Constructor Summary
Constructors Constructor Description ShingleAnalyzerWrapper()
WrapsStandardAnalyzer
.ShingleAnalyzerWrapper(int minShingleSize, int maxShingleSize)
WrapsStandardAnalyzer
.ShingleAnalyzerWrapper(Analyzer defaultAnalyzer)
ShingleAnalyzerWrapper(Analyzer defaultAnalyzer, int maxShingleSize)
ShingleAnalyzerWrapper(Analyzer defaultAnalyzer, int minShingleSize, int maxShingleSize)
ShingleAnalyzerWrapper(Analyzer delegate, int minShingleSize, int maxShingleSize, java.lang.String tokenSeparator, boolean outputUnigrams, boolean outputUnigramsIfNoShingles, java.lang.String fillerToken)
Creates a new ShingleAnalyzerWrapper
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String
getFillerToken()
int
getMaxShingleSize()
The max shingle (token ngram) sizeint
getMinShingleSize()
The min shingle (token ngram) sizejava.lang.String
getTokenSeparator()
Analyzer
getWrappedAnalyzer(java.lang.String fieldName)
Retrieves the wrapped Analyzer appropriate for analyzing the field with the given nameboolean
isOutputUnigrams()
boolean
isOutputUnigramsIfNoShingles()
protected Analyzer.TokenStreamComponents
wrapComponents(java.lang.String fieldName, Analyzer.TokenStreamComponents components)
Wraps / alters the given TokenStreamComponents, taken from the wrapped Analyzer, to form new components.-
Methods inherited from class org.apache.lucene.analysis.AnalyzerWrapper
attributeFactory, createComponents, getOffsetGap, getPositionIncrementGap, initReader, initReaderForNormalization, normalize, wrapReader, wrapReaderForNormalization, wrapTokenStreamForNormalization
-
Methods inherited from class org.apache.lucene.analysis.Analyzer
close, getReuseStrategy, normalize, tokenStream, tokenStream
-
-
-
-
Field Detail
-
delegate
private final Analyzer delegate
-
maxShingleSize
private final int maxShingleSize
-
minShingleSize
private final int minShingleSize
-
tokenSeparator
private final java.lang.String tokenSeparator
-
outputUnigrams
private final boolean outputUnigrams
-
outputUnigramsIfNoShingles
private final boolean outputUnigramsIfNoShingles
-
fillerToken
private final java.lang.String fillerToken
-
-
Constructor Detail
-
ShingleAnalyzerWrapper
public ShingleAnalyzerWrapper(Analyzer defaultAnalyzer)
-
ShingleAnalyzerWrapper
public ShingleAnalyzerWrapper(Analyzer defaultAnalyzer, int maxShingleSize)
-
ShingleAnalyzerWrapper
public ShingleAnalyzerWrapper(Analyzer defaultAnalyzer, int minShingleSize, int maxShingleSize)
-
ShingleAnalyzerWrapper
public ShingleAnalyzerWrapper(Analyzer delegate, int minShingleSize, int maxShingleSize, java.lang.String tokenSeparator, boolean outputUnigrams, boolean outputUnigramsIfNoShingles, java.lang.String fillerToken)
Creates a new ShingleAnalyzerWrapper- Parameters:
delegate
- Analyzer whose TokenStream is to be filteredminShingleSize
- Min shingle (token ngram) sizemaxShingleSize
- Max shingle sizetokenSeparator
- Used to separate input stream tokens in output shinglesoutputUnigrams
- Whether or not the filter shall pass the original tokens to the output streamoutputUnigramsIfNoShingles
- Overrides the behavior of outputUnigrams==false for those times when no shingles are available (because there are fewer than minShingleSize tokens in the input stream)? Note that if outputUnigrams==true, then unigrams are always output, regardless of whether any shingles are available.fillerToken
- filler token to use when positionIncrement is more than 1
-
ShingleAnalyzerWrapper
public ShingleAnalyzerWrapper()
WrapsStandardAnalyzer
.
-
ShingleAnalyzerWrapper
public ShingleAnalyzerWrapper(int minShingleSize, int maxShingleSize)
WrapsStandardAnalyzer
.
-
-
Method Detail
-
getMaxShingleSize
public int getMaxShingleSize()
The max shingle (token ngram) size- Returns:
- The max shingle (token ngram) size
-
getMinShingleSize
public int getMinShingleSize()
The min shingle (token ngram) size- Returns:
- The min shingle (token ngram) size
-
getTokenSeparator
public java.lang.String getTokenSeparator()
-
isOutputUnigrams
public boolean isOutputUnigrams()
-
isOutputUnigramsIfNoShingles
public boolean isOutputUnigramsIfNoShingles()
-
getFillerToken
public java.lang.String getFillerToken()
-
getWrappedAnalyzer
public final Analyzer getWrappedAnalyzer(java.lang.String fieldName)
Description copied from class:AnalyzerWrapper
Retrieves the wrapped Analyzer appropriate for analyzing the field with the given name- Specified by:
getWrappedAnalyzer
in classAnalyzerWrapper
- Parameters:
fieldName
- Name of the field which is to be analyzed- Returns:
- Analyzer for the field with the given name. Assumed to be non-null
-
wrapComponents
protected Analyzer.TokenStreamComponents wrapComponents(java.lang.String fieldName, Analyzer.TokenStreamComponents components)
Description copied from class:AnalyzerWrapper
Wraps / alters the given TokenStreamComponents, taken from the wrapped Analyzer, to form new components. It is through this method that new TokenFilters can be added by AnalyzerWrappers. By default, the given components are returned.- Overrides:
wrapComponents
in classAnalyzerWrapper
- Parameters:
fieldName
- Name of the field which is to be analyzedcomponents
- TokenStreamComponents taken from the wrapped Analyzer- Returns:
- Wrapped / altered TokenStreamComponents.
-
-