- java.lang.Object
-
- org.apache.lucene.index.SegmentInfos
-
- All Implemented Interfaces:
java.lang.Cloneable
,java.lang.Iterable<SegmentCommitInfo>
public final class SegmentInfos extends java.lang.Object implements java.lang.Cloneable, java.lang.Iterable<SegmentCommitInfo>
A collection of segmentInfo objects with methods for operating on those segments in relation to the file system.The active segments in the index are stored in the segment info file,
segments_N
. There may be one or moresegments_N
files in the index; however, the one with the largest generation is the active one (when older segments_N files are present it's because they temporarily cannot be deleted, or a customIndexDeletionPolicy
is in use). This file lists each segment by name and has details about the codec and generation of deletes.Files:
segments_N
: Header, LuceneVersion, Version, NameCounter, SegCount, MinSegmentLuceneVersion, <SegName, SegID, SegCodec, DelGen, DeletionCount, FieldInfosGen, DocValuesGen, UpdatesFiles>SegCount, CommitUserData, Footer
- Header -->
IndexHeader
- LuceneVersion --> Which Lucene code
Version
was used for this commit, written as threevInt
: major, minor, bugfix - MinSegmentLuceneVersion --> Lucene code
Version
of the oldest segment, written as threevInt
: major, minor, bugfix; this is only written only if there's at least one segment - NameCounter, SegCount, DeletionCount -->
Int32
- Generation, Version, DelGen, Checksum, FieldInfosGen, DocValuesGen -->
Int64
- SegID -->
Int8ID_LENGTH
- SegName, SegCodec -->
String
- CommitUserData -->
Map<String,String>
- UpdatesFiles --> Map<
Int32
,Set<String>
> - Footer -->
CodecFooter
- Version counts how often the index has been changed by adding or deleting documents.
- NameCounter is used to generate names for new segment files.
- SegName is the name of the segment, and is used as the file name prefix for all of the files that compose the segment's index.
- DelGen is the generation count of the deletes file. If this is -1, there are no deletes.
Anything above zero means there are deletes stored by
LiveDocsFormat
. - DeletionCount records the number of deleted documents in this segment.
- SegCodec is the
name
of the Codec that encoded this segment. - SegID is the identifier of the Codec that encoded this segment.
- CommitUserData stores an optional user-supplied opaque Map<String,String> that was
passed to
IndexWriter.setLiveCommitData(Iterable)
. - FieldInfosGen is the generation count of the fieldInfos file. If this is -1, there are no
updates to the fieldInfos in that segment. Anything above zero means there are updates to
fieldInfos stored by
FieldInfosFormat
. - DocValuesGen is the generation count of the updatable DocValues. If this is -1, there are
no updates to DocValues in that segment. Anything above zero means there are updates to
DocValues stored by
DocValuesFormat
. - UpdatesFiles stores the set of files that were updated in that segment per field.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static class
SegmentInfos.FindSegmentsFile<T>
Utility class for executing code that needs to do something with the current segments file.
-
Field Summary
Fields Modifier and Type Field Description long
counter
Used to name new segments.private long
generation
private byte[]
id
Id for this commit; only written starting with Lucene 5.0private int
indexCreatedVersionMajor
The Lucene version major that was used to create the index.private static java.io.PrintStream
infoStream
If non-null, information about loading segments_N files will be printed here.private long
lastGeneration
private Version
luceneVersion
Which Lucene version wrote this commit.private Version
minSegmentLuceneVersion
Version of the oldest segment in the index, or null if there are no segments.(package private) static java.lang.String
OLD_SEGMENTS_GEN
Name of the generation reference file name(package private) boolean
pendingCommit
private java.util.List<SegmentCommitInfo>
segments
java.util.Map<java.lang.String,java.lang.String>
userData
Opaque Map<String, String> that user can specify during IndexWriter.commitlong
version
Counts how often the index has been changed.static int
VERSION_70
The version that added information about the Lucene version at the time when the index has been created.static int
VERSION_72
The version that updated segment name counter to be long instead of int.static int
VERSION_74
The version that recorded softDelCountstatic int
VERSION_86
The version that recorded SegmentCommitInfo IDs(package private) static int
VERSION_CURRENT
-
Constructor Summary
Constructors Constructor Description SegmentInfos(int indexCreatedVersionMajor)
Sole constructor.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description void
add(SegmentCommitInfo si)
Appends the providedSegmentCommitInfo
.void
addAll(java.lang.Iterable<SegmentCommitInfo> sis)
Appends the providedSegmentCommitInfo
s.(package private) void
applyMergeChanges(MergePolicy.OneMerge merge, boolean dropSegment)
applies all changes caused by committing a merge to this SegmentInfosjava.util.List<SegmentCommitInfo>
asList()
Returns all contained segments as an unmodifiableList
view.void
changed()
Call this before committing if changes have been made to the segments.void
clear()
Clear allSegmentCommitInfo
s.SegmentInfos
clone()
Returns a copy of this instance, also copying each SegmentInfo.void
commit(Directory dir)
Writes and syncs to the Directory dir, taking care to remove the segments file on exception(package private) boolean
contains(SegmentCommitInfo si)
Return true if the providedSegmentCommitInfo
is contained.(package private) java.util.List<SegmentCommitInfo>
createBackupSegmentInfos()
java.util.Collection<java.lang.String>
files(boolean includeSegmentsFile)
Returns all file names referenced by SegmentInfo.(package private) java.lang.String
finishCommit(Directory dir)
Returns the committed segments_N filename.static long
generationFromSegmentsFileName(java.lang.String fileName)
Parse the generation off the segments file name and return it.Version
getCommitLuceneVersion()
Returns which LuceneVersion
wrote this commit, or null if the version this index was written with did not directly record the version.long
getGeneration()
Returns current generation.byte[]
getId()
Since Lucene 5.0, every commit (segments_N) writes a unique id.int
getIndexCreatedVersionMajor()
Return the version major that was used to initially create the index.static java.io.PrintStream
getInfoStream()
ReturnsinfoStream
.static long
getLastCommitGeneration(java.lang.String[] files)
Get the generation of the most recent commit to the list of index files (N in the segments_N file).static long
getLastCommitGeneration(Directory directory)
Get the generation of the most recent commit to the index in this directory (N in the segments_N file).static java.lang.String
getLastCommitSegmentsFileName(java.lang.String[] files)
Get the filename of the segments_N file for the most recent commit in the list of index files.static java.lang.String
getLastCommitSegmentsFileName(Directory directory)
Get the filename of the segments_N file for the most recent commit to the index in this Directory.long
getLastGeneration()
Returns last succesfully read or written generation.Version
getMinSegmentLuceneVersion()
Returns the version of the oldest segment, or null if there are no segments.private long
getNextPendingGeneration()
return generation of the next pending_segments_N that will be writtenjava.lang.String
getSegmentsFileName()
Get the segments_N filename in use by this segment infos.java.util.Map<java.lang.String,java.lang.String>
getUserData()
ReturnuserData
saved with this commit.long
getVersion()
version number when this SegmentInfos was generated.(package private) int
indexOf(SegmentCommitInfo si)
Returns index of the providedSegmentCommitInfo
.SegmentCommitInfo
info(int i)
ReturnsSegmentCommitInfo
at the provided index.java.util.Iterator<SegmentCommitInfo>
iterator()
Returns an unmodifiableIterator
of contained segments in order.private static void
message(java.lang.String message)
Prints the given message to the infoStream.private static void
parseSegmentInfos(Directory directory, DataInput input, SegmentInfos infos, int format)
(package private) void
prepareCommit(Directory dir)
Call this to start a commit.private static Codec
readCodec(DataInput input)
static SegmentInfos
readCommit(Directory directory, java.lang.String segmentFileName)
Read a particular segmentFileName.(package private) static SegmentInfos
readCommit(Directory directory, java.lang.String segmentFileName, int minSupportedMajorVersion)
static SegmentInfos
readCommit(Directory directory, ChecksumIndexInput input, long generation)
Read the commit from the providedChecksumIndexInput
.(package private) static SegmentInfos
readCommit(Directory directory, ChecksumIndexInput input, long generation, int minSupportedMajorVersion)
Read the commit from the providedChecksumIndexInput
.static SegmentInfos
readLatestCommit(Directory directory)
Find the latest commit (segments_N file
) and load allSegmentCommitInfo
s.static SegmentInfos
readLatestCommit(Directory directory, int minSupportedMajorVersion)
Find the latest commit (segments_N file
) and load allSegmentCommitInfo
s, as long as the commit'sgetIndexCreatedVersionMajor()
is strictly greater than the provided minimum supported major version.(package private) void
remove(int index)
Remove theSegmentCommitInfo
at the provided index.boolean
remove(SegmentCommitInfo si)
Remove the providedSegmentCommitInfo
.(package private) void
replace(SegmentInfos other)
Replaces all segments in this instance, but keeps generation, version, counter so that future commits remain write once.(package private) void
rollbackCommit(Directory dir)
(package private) void
rollbackSegmentInfos(java.util.List<SegmentCommitInfo> infos)
static void
setInfoStream(java.io.PrintStream infoStream)
If non-null, information about retries when loading the segments file will be printed to this.void
setNextWriteGeneration(long generation)
Set the generation to be used for the next commitvoid
setUserData(java.util.Map<java.lang.String,java.lang.String> data, boolean doIncrementVersion)
Sets the commit data.(package private) void
setVersion(long newVersion)
int
size()
Returns number ofSegmentCommitInfo
s.java.lang.String
toString()
Returns readable description of this segment.int
totalMaxDoc()
Returns sum of all segment's maxDocs.void
updateGeneration(SegmentInfos other)
Carry over generation numbers from another SegmentInfos(package private) void
updateGenerationVersionAndCounter(SegmentInfos other)
private void
write(Directory directory)
void
write(IndexOutput out)
Write ourselves to the providedIndexOutput
-
-
-
Field Detail
-
VERSION_70
public static final int VERSION_70
The version that added information about the Lucene version at the time when the index has been created.- See Also:
- Constant Field Values
-
VERSION_72
public static final int VERSION_72
The version that updated segment name counter to be long instead of int.- See Also:
- Constant Field Values
-
VERSION_74
public static final int VERSION_74
The version that recorded softDelCount- See Also:
- Constant Field Values
-
VERSION_86
public static final int VERSION_86
The version that recorded SegmentCommitInfo IDs- See Also:
- Constant Field Values
-
VERSION_CURRENT
static final int VERSION_CURRENT
- See Also:
- Constant Field Values
-
OLD_SEGMENTS_GEN
static final java.lang.String OLD_SEGMENTS_GEN
Name of the generation reference file name- See Also:
- Constant Field Values
-
counter
public long counter
Used to name new segments.
-
version
public long version
Counts how often the index has been changed.
-
generation
private long generation
-
lastGeneration
private long lastGeneration
-
userData
public java.util.Map<java.lang.String,java.lang.String> userData
Opaque Map<String, String> that user can specify during IndexWriter.commit
-
segments
private java.util.List<SegmentCommitInfo> segments
-
infoStream
private static java.io.PrintStream infoStream
If non-null, information about loading segments_N files will be printed here.- See Also:
setInfoStream(java.io.PrintStream)
-
id
private byte[] id
Id for this commit; only written starting with Lucene 5.0
-
luceneVersion
private Version luceneVersion
Which Lucene version wrote this commit.
-
minSegmentLuceneVersion
private Version minSegmentLuceneVersion
Version of the oldest segment in the index, or null if there are no segments.
-
indexCreatedVersionMajor
private final int indexCreatedVersionMajor
The Lucene version major that was used to create the index.
-
pendingCommit
boolean pendingCommit
-
-
Method Detail
-
info
public SegmentCommitInfo info(int i)
ReturnsSegmentCommitInfo
at the provided index.
-
getLastCommitGeneration
public static long getLastCommitGeneration(java.lang.String[] files)
Get the generation of the most recent commit to the list of index files (N in the segments_N file).- Parameters:
files
- -- array of file names to check
-
getLastCommitGeneration
public static long getLastCommitGeneration(Directory directory) throws java.io.IOException
Get the generation of the most recent commit to the index in this directory (N in the segments_N file).- Parameters:
directory
- -- directory to search for the latest segments_N file- Throws:
java.io.IOException
-
getLastCommitSegmentsFileName
public static java.lang.String getLastCommitSegmentsFileName(java.lang.String[] files)
Get the filename of the segments_N file for the most recent commit in the list of index files.- Parameters:
files
- -- array of file names to check
-
getLastCommitSegmentsFileName
public static java.lang.String getLastCommitSegmentsFileName(Directory directory) throws java.io.IOException
Get the filename of the segments_N file for the most recent commit to the index in this Directory.- Parameters:
directory
- -- directory to search for the latest segments_N file- Throws:
java.io.IOException
-
getSegmentsFileName
public java.lang.String getSegmentsFileName()
Get the segments_N filename in use by this segment infos.
-
generationFromSegmentsFileName
public static long generationFromSegmentsFileName(java.lang.String fileName)
Parse the generation off the segments file name and return it.
-
getNextPendingGeneration
private long getNextPendingGeneration()
return generation of the next pending_segments_N that will be written
-
getId
public byte[] getId()
Since Lucene 5.0, every commit (segments_N) writes a unique id. This will return that id
-
readCommit
public static final SegmentInfos readCommit(Directory directory, java.lang.String segmentFileName) throws java.io.IOException
Read a particular segmentFileName. Note that this may throw an IOException if a commit is in process.- Parameters:
directory
- -- directory containing the segments filesegmentFileName
- -- segment file to load- Throws:
CorruptIndexException
- if the index is corruptjava.io.IOException
- if there is a low-level IO error
-
readCommit
static final SegmentInfos readCommit(Directory directory, java.lang.String segmentFileName, int minSupportedMajorVersion) throws java.io.IOException
- Throws:
java.io.IOException
-
readCommit
public static final SegmentInfos readCommit(Directory directory, ChecksumIndexInput input, long generation) throws java.io.IOException
Read the commit from the providedChecksumIndexInput
.- Throws:
java.io.IOException
-
readCommit
static final SegmentInfos readCommit(Directory directory, ChecksumIndexInput input, long generation, int minSupportedMajorVersion) throws java.io.IOException
Read the commit from the providedChecksumIndexInput
.- Throws:
java.io.IOException
-
parseSegmentInfos
private static void parseSegmentInfos(Directory directory, DataInput input, SegmentInfos infos, int format) throws java.io.IOException
- Throws:
java.io.IOException
-
readCodec
private static Codec readCodec(DataInput input) throws java.io.IOException
- Throws:
java.io.IOException
-
readLatestCommit
public static final SegmentInfos readLatestCommit(Directory directory) throws java.io.IOException
Find the latest commit (segments_N file
) and load allSegmentCommitInfo
s.- Throws:
java.io.IOException
-
readLatestCommit
public static final SegmentInfos readLatestCommit(Directory directory, int minSupportedMajorVersion) throws java.io.IOException
Find the latest commit (segments_N file
) and load allSegmentCommitInfo
s, as long as the commit'sgetIndexCreatedVersionMajor()
is strictly greater than the provided minimum supported major version. If the commit's version is older, anIndexFormatTooOldException
will be thrown.- Throws:
java.io.IOException
-
write
private void write(Directory directory) throws java.io.IOException
- Throws:
java.io.IOException
-
write
public void write(IndexOutput out) throws java.io.IOException
Write ourselves to the providedIndexOutput
- Throws:
java.io.IOException
-
clone
public SegmentInfos clone()
Returns a copy of this instance, also copying each SegmentInfo.- Overrides:
clone
in classjava.lang.Object
-
getVersion
public long getVersion()
version number when this SegmentInfos was generated.
-
getGeneration
public long getGeneration()
Returns current generation.
-
getLastGeneration
public long getLastGeneration()
Returns last succesfully read or written generation.
-
setInfoStream
public static void setInfoStream(java.io.PrintStream infoStream)
If non-null, information about retries when loading the segments file will be printed to this.
-
getInfoStream
public static java.io.PrintStream getInfoStream()
ReturnsinfoStream
.- See Also:
setInfoStream(java.io.PrintStream)
-
message
private static void message(java.lang.String message)
Prints the given message to the infoStream. Note, this method does not check for null infoStream. It assumes this check has been performed by the caller, which is recommended to avoid the (usually) expensive message creation.
-
updateGeneration
public void updateGeneration(SegmentInfos other)
Carry over generation numbers from another SegmentInfos
-
updateGenerationVersionAndCounter
void updateGenerationVersionAndCounter(SegmentInfos other)
-
setNextWriteGeneration
public void setNextWriteGeneration(long generation)
Set the generation to be used for the next commit
-
rollbackCommit
final void rollbackCommit(Directory dir)
-
prepareCommit
final void prepareCommit(Directory dir) throws java.io.IOException
Call this to start a commit. This writes the new segments file, but writes an invalid checksum at the end, so that it is not visible to readers. Once this is called you must callfinishCommit(org.apache.lucene.store.Directory)
to complete the commit orrollbackCommit(org.apache.lucene.store.Directory)
to abort it.Note:
changed()
should be called prior to this method if changes have been made to thisSegmentInfos
instance- Throws:
java.io.IOException
-
files
public java.util.Collection<java.lang.String> files(boolean includeSegmentsFile) throws java.io.IOException
Returns all file names referenced by SegmentInfo. The returned collection is recomputed on each invocation.- Throws:
java.io.IOException
-
finishCommit
final java.lang.String finishCommit(Directory dir) throws java.io.IOException
Returns the committed segments_N filename.- Throws:
java.io.IOException
-
commit
public final void commit(Directory dir) throws java.io.IOException
Writes and syncs to the Directory dir, taking care to remove the segments file on exceptionNote:
changed()
should be called prior to this method if changes have been made to thisSegmentInfos
instance- Throws:
java.io.IOException
-
toString
public java.lang.String toString()
Returns readable description of this segment.- Overrides:
toString
in classjava.lang.Object
-
getUserData
public java.util.Map<java.lang.String,java.lang.String> getUserData()
ReturnuserData
saved with this commit.- See Also:
IndexWriter.commit()
-
setUserData
public void setUserData(java.util.Map<java.lang.String,java.lang.String> data, boolean doIncrementVersion)
Sets the commit data.
-
replace
void replace(SegmentInfos other)
Replaces all segments in this instance, but keeps generation, version, counter so that future commits remain write once.
-
totalMaxDoc
public int totalMaxDoc()
Returns sum of all segment's maxDocs. Note that this does not include deletions
-
changed
public void changed()
Call this before committing if changes have been made to the segments.
-
setVersion
void setVersion(long newVersion)
-
applyMergeChanges
void applyMergeChanges(MergePolicy.OneMerge merge, boolean dropSegment)
applies all changes caused by committing a merge to this SegmentInfos
-
createBackupSegmentInfos
java.util.List<SegmentCommitInfo> createBackupSegmentInfos()
-
rollbackSegmentInfos
void rollbackSegmentInfos(java.util.List<SegmentCommitInfo> infos)
-
iterator
public java.util.Iterator<SegmentCommitInfo> iterator()
Returns an unmodifiableIterator
of contained segments in order.- Specified by:
iterator
in interfacejava.lang.Iterable<SegmentCommitInfo>
-
asList
public java.util.List<SegmentCommitInfo> asList()
Returns all contained segments as an unmodifiableList
view.
-
size
public int size()
Returns number ofSegmentCommitInfo
s.
-
add
public void add(SegmentCommitInfo si)
Appends the providedSegmentCommitInfo
.
-
addAll
public void addAll(java.lang.Iterable<SegmentCommitInfo> sis)
Appends the providedSegmentCommitInfo
s.
-
clear
public void clear()
Clear allSegmentCommitInfo
s.
-
remove
public boolean remove(SegmentCommitInfo si)
Remove the providedSegmentCommitInfo
.WARNING: O(N) cost
-
remove
void remove(int index)
Remove theSegmentCommitInfo
at the provided index.WARNING: O(N) cost
-
contains
boolean contains(SegmentCommitInfo si)
Return true if the providedSegmentCommitInfo
is contained.WARNING: O(N) cost
-
indexOf
int indexOf(SegmentCommitInfo si)
Returns index of the providedSegmentCommitInfo
.WARNING: O(N) cost
-
getCommitLuceneVersion
public Version getCommitLuceneVersion()
Returns which LuceneVersion
wrote this commit, or null if the version this index was written with did not directly record the version.
-
getMinSegmentLuceneVersion
public Version getMinSegmentLuceneVersion()
Returns the version of the oldest segment, or null if there are no segments.
-
getIndexCreatedVersionMajor
public int getIndexCreatedVersionMajor()
Return the version major that was used to initially create the index. This version is set when the index is first created and then never changes. This information was added as of version 7.0 so older indices report 6 as a creation version.
-
-