- java.lang.Object
-
- org.apache.lucene.search.LRUQueryCache
-
- All Implemented Interfaces:
QueryCache
,Accountable
public class LRUQueryCache extends java.lang.Object implements QueryCache, Accountable
AQueryCache
that evicts queries using a LRU (least-recently-used) eviction policy in order to remain under a given maximum size and number of bytes used.This class is thread-safe.
Note that query eviction runs in linear time with the total number of segments that have cache entries so this cache works best with
caching policies
that only cache on "large" segments, and it is advised to not share this cache across too many indices.A default query cache and policy instance is used in IndexSearcher. If you want to replace those defaults it is typically done like this:
final int maxNumberOfCachedQueries = 256; final long maxRamBytesUsed = 50 * 1024L * 1024L; // 50MB // these cache and policy instances can be shared across several queries and readers // it is fine to eg. store them into static variables final QueryCache queryCache = new LRUQueryCache(maxNumberOfCachedQueries, maxRamBytesUsed); final QueryCachingPolicy defaultCachingPolicy = new UsageTrackingQueryCachingPolicy(); indexSearcher.setQueryCache(queryCache); indexSearcher.setQueryCachingPolicy(defaultCachingPolicy);
This cache exposes some global statistics (hit count
,miss count
,number of cache entries
,total number of DocIdSets that have ever been cached
,number of evicted entries
). In case you would like to have more fine-grained statistics, such as per-index or per-query-class statistics, it is possible to override various callbacks:onHit(java.lang.Object, org.apache.lucene.search.Query)
,onMiss(java.lang.Object, org.apache.lucene.search.Query)
,onQueryCache(org.apache.lucene.search.Query, long)
,onQueryEviction(org.apache.lucene.search.Query, long)
,onDocIdSetCache(java.lang.Object, long)
,onDocIdSetEviction(java.lang.Object, int, long)
andonClear()
. It is better to not perform heavy computations in these methods though since they are called synchronously and under a lock.- See Also:
QueryCachingPolicy
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static class
LRUQueryCache.CacheAndCount
Cache of doc ids with a count.private class
LRUQueryCache.CachingWrapperWeight
private class
LRUQueryCache.LeafCache
(package private) static class
LRUQueryCache.MinSegmentSizePredicate
-
Field Summary
Fields Modifier and Type Field Description private java.util.Map<IndexReader.CacheKey,LRUQueryCache.LeafCache>
cache
private long
cacheCount
private long
cacheSize
private java.util.concurrent.atomic.LongAdder
hitCount
private java.util.function.Predicate<LeafReaderContext>
leavesToCache
private long
maxRamBytesUsed
private int
maxSize
private java.util.concurrent.atomic.LongAdder
missCount
private java.util.Set<Query>
mostRecentlyUsedQueries
private long
ramBytesUsed
private java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock
readLock
private float
skipCacheFactor
private java.util.Map<Query,Query>
uniqueQueries
private java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock
writeLock
-
Fields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLE
-
-
Constructor Summary
Constructors Constructor Description LRUQueryCache(int maxSize, long maxRamBytesUsed)
Create a new instance that will cache at mostmaxSize
queries with at mostmaxRamBytesUsed
bytes of memory.LRUQueryCache(int maxSize, long maxRamBytesUsed, java.util.function.Predicate<LeafReaderContext> leavesToCache, float skipCacheFactor)
Expert: Create a new instance that will cache at mostmaxSize
queries with at mostmaxRamBytesUsed
bytes of memory, only on leaves that satisfyleavesToCache
.
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description (package private) void
assertConsistent()
(package private) java.util.List<Query>
cachedQueries()
protected LRUQueryCache.CacheAndCount
cacheImpl(BulkScorer scorer, int maxDoc)
Default cache implementation: usesRoaringDocIdSet
for sets that have a density < 1% and aBitDocIdSet
over aFixedBitSet
otherwise.private static LRUQueryCache.CacheAndCount
cacheIntoBitSet(BulkScorer scorer, int maxDoc)
private static LRUQueryCache.CacheAndCount
cacheIntoRoaringDocIdSet(BulkScorer scorer, int maxDoc)
void
clear()
Clear the content of this cache.void
clearCoreCacheKey(java.lang.Object coreKey)
Remove all cache entries for the given core cache key.void
clearQuery(Query query)
Remove all cache entries for the given query.Weight
doCache(Weight weight, QueryCachingPolicy policy)
Return a wrapper around the providedweight
that will cache matching docs per-segment accordingly to the givenpolicy
.private void
evictIfNecessary()
(package private) LRUQueryCache.CacheAndCount
get(Query key, IndexReader.CacheHelper cacheHelper)
long
getCacheCount()
Return the total number of cache entries that have been generated and put in the cache.long
getCacheSize()
Return the total number ofDocIdSet
s which are currently stored in the cache.java.util.Collection<Accountable>
getChildResources()
Returns nested resources of this class.long
getEvictionCount()
Return the number of cache entries that have been removed from the cache either in order to stay under the maximum configured size/ram usage, or because a segment has been closed.long
getHitCount()
long
getMissCount()
Over thetotal
number of times that a query has been looked up, return how many times this query was not contained in the cache.private static long
getRamBytesUsed(Query query)
long
getTotalCount()
Return the total number of times that aQuery
has been looked up in thisQueryCache
.protected void
onClear()
Expert: callback when the cache is completely cleared.protected void
onDocIdSetCache(java.lang.Object readerCoreKey, long ramBytesUsed)
Expert: callback when aDocIdSet
is added to this cache.protected void
onDocIdSetEviction(java.lang.Object readerCoreKey, int numEntries, long sumRamBytesUsed)
Expert: callback when one or moreDocIdSet
s are removed from this cache.private void
onEviction(Query singleton)
protected void
onHit(java.lang.Object readerCoreKey, Query query)
Expert: callback when there is a cache hit on a given query.protected void
onMiss(java.lang.Object readerCoreKey, Query query)
Expert: callback when there is a cache miss on a given query.protected void
onQueryCache(Query query, long ramBytesUsed)
Expert: callback when a query is added to this cache.protected void
onQueryEviction(Query query, long ramBytesUsed)
Expert: callback when a query is evicted from this cache.private void
putIfAbsent(Query query, LRUQueryCache.CacheAndCount cached, IndexReader.CacheHelper cacheHelper)
long
ramBytesUsed()
Return the memory usage of this object in bytes.(package private) boolean
requiresEviction()
Whether evictions are required.
-
-
-
Field Detail
-
maxSize
private final int maxSize
-
maxRamBytesUsed
private final long maxRamBytesUsed
-
leavesToCache
private final java.util.function.Predicate<LeafReaderContext> leavesToCache
-
mostRecentlyUsedQueries
private final java.util.Set<Query> mostRecentlyUsedQueries
-
cache
private final java.util.Map<IndexReader.CacheKey,LRUQueryCache.LeafCache> cache
-
readLock
private final java.util.concurrent.locks.ReentrantReadWriteLock.ReadLock readLock
-
writeLock
private final java.util.concurrent.locks.ReentrantReadWriteLock.WriteLock writeLock
-
skipCacheFactor
private final float skipCacheFactor
-
hitCount
private final java.util.concurrent.atomic.LongAdder hitCount
-
missCount
private final java.util.concurrent.atomic.LongAdder missCount
-
ramBytesUsed
private volatile long ramBytesUsed
-
cacheCount
private volatile long cacheCount
-
cacheSize
private volatile long cacheSize
-
-
Constructor Detail
-
LRUQueryCache
public LRUQueryCache(int maxSize, long maxRamBytesUsed, java.util.function.Predicate<LeafReaderContext> leavesToCache, float skipCacheFactor)
Expert: Create a new instance that will cache at mostmaxSize
queries with at mostmaxRamBytesUsed
bytes of memory, only on leaves that satisfyleavesToCache
.Also, clauses whose cost is
skipCacheFactor
times more than the cost of the top-level query will not be cached in order to not slow down queries too much.
-
LRUQueryCache
public LRUQueryCache(int maxSize, long maxRamBytesUsed)
Create a new instance that will cache at mostmaxSize
queries with at mostmaxRamBytesUsed
bytes of memory. Queries will only be cached on leaves that have more than 10k documents and have more than half of the average documents per leave of the index. This should guarantee that all leaves from the uppertier
will be cached. Only clauses whose cost is at most 100x the cost of the top-level query will be cached in order to not hurt latency too much because of caching.
-
-
Method Detail
-
onHit
protected void onHit(java.lang.Object readerCoreKey, Query query)
Expert: callback when there is a cache hit on a given query. Implementing this method is typically useful in order to compute more fine-grained statistics about the query cache.
-
onMiss
protected void onMiss(java.lang.Object readerCoreKey, Query query)
Expert: callback when there is a cache miss on a given query.
-
onQueryCache
protected void onQueryCache(Query query, long ramBytesUsed)
Expert: callback when a query is added to this cache. Implementing this method is typically useful in order to compute more fine-grained statistics about the query cache.
-
onQueryEviction
protected void onQueryEviction(Query query, long ramBytesUsed)
Expert: callback when a query is evicted from this cache.
-
onDocIdSetCache
protected void onDocIdSetCache(java.lang.Object readerCoreKey, long ramBytesUsed)
Expert: callback when aDocIdSet
is added to this cache. Implementing this method is typically useful in order to compute more fine-grained statistics about the query cache.
-
onDocIdSetEviction
protected void onDocIdSetEviction(java.lang.Object readerCoreKey, int numEntries, long sumRamBytesUsed)
Expert: callback when one or moreDocIdSet
s are removed from this cache.- See Also:
onDocIdSetCache(java.lang.Object, long)
-
onClear
protected void onClear()
Expert: callback when the cache is completely cleared.
-
requiresEviction
boolean requiresEviction()
Whether evictions are required.
-
get
LRUQueryCache.CacheAndCount get(Query key, IndexReader.CacheHelper cacheHelper)
-
putIfAbsent
private void putIfAbsent(Query query, LRUQueryCache.CacheAndCount cached, IndexReader.CacheHelper cacheHelper)
-
evictIfNecessary
private void evictIfNecessary()
-
clearCoreCacheKey
public void clearCoreCacheKey(java.lang.Object coreKey)
Remove all cache entries for the given core cache key.
-
clearQuery
public void clearQuery(Query query)
Remove all cache entries for the given query.
-
onEviction
private void onEviction(Query singleton)
-
clear
public void clear()
Clear the content of this cache.
-
getRamBytesUsed
private static long getRamBytesUsed(Query query)
-
assertConsistent
void assertConsistent()
-
cachedQueries
java.util.List<Query> cachedQueries()
-
doCache
public Weight doCache(Weight weight, QueryCachingPolicy policy)
Description copied from interface:QueryCache
Return a wrapper around the providedweight
that will cache matching docs per-segment accordingly to the givenpolicy
. NOTE: The returned weight will only be equivalent if scores are not needed.- Specified by:
doCache
in interfaceQueryCache
- See Also:
Collector.scoreMode()
-
ramBytesUsed
public long ramBytesUsed()
Description copied from interface:Accountable
Return the memory usage of this object in bytes. Negative values are illegal.- Specified by:
ramBytesUsed
in interfaceAccountable
-
getChildResources
public java.util.Collection<Accountable> getChildResources()
Description copied from interface:Accountable
Returns nested resources of this class. The result should be a point-in-time snapshot (to avoid race conditions).- Specified by:
getChildResources
in interfaceAccountable
- See Also:
Accountables
-
cacheImpl
protected LRUQueryCache.CacheAndCount cacheImpl(BulkScorer scorer, int maxDoc) throws java.io.IOException
Default cache implementation: usesRoaringDocIdSet
for sets that have a density < 1% and aBitDocIdSet
over aFixedBitSet
otherwise.- Throws:
java.io.IOException
-
cacheIntoBitSet
private static LRUQueryCache.CacheAndCount cacheIntoBitSet(BulkScorer scorer, int maxDoc) throws java.io.IOException
- Throws:
java.io.IOException
-
cacheIntoRoaringDocIdSet
private static LRUQueryCache.CacheAndCount cacheIntoRoaringDocIdSet(BulkScorer scorer, int maxDoc) throws java.io.IOException
- Throws:
java.io.IOException
-
getTotalCount
public final long getTotalCount()
Return the total number of times that aQuery
has been looked up in thisQueryCache
. Note that this number is incremented once per segment so running a cached query only once will increment this counter by the number of segments that are wrapped by the searcher. Note that by definition,getTotalCount()
is the sum ofgetHitCount()
andgetMissCount()
.- See Also:
getHitCount()
,getMissCount()
-
getHitCount
public final long getHitCount()
Over thetotal
number of times that a query has been looked up, return how many times a cachedDocIdSet
has been found and returned.- See Also:
getTotalCount()
,getMissCount()
-
getMissCount
public final long getMissCount()
Over thetotal
number of times that a query has been looked up, return how many times this query was not contained in the cache.- See Also:
getTotalCount()
,getHitCount()
-
getCacheSize
public final long getCacheSize()
Return the total number ofDocIdSet
s which are currently stored in the cache.- See Also:
getCacheCount()
,getEvictionCount()
-
getCacheCount
public final long getCacheCount()
Return the total number of cache entries that have been generated and put in the cache. It is highly desirable to have ahit count
that is much higher than thecache count
as the opposite would indicate that the query cache makes efforts in order to cache queries but then they do not get reused.- See Also:
getCacheSize()
,getEvictionCount()
-
getEvictionCount
public final long getEvictionCount()
Return the number of cache entries that have been removed from the cache either in order to stay under the maximum configured size/ram usage, or because a segment has been closed. High numbers of evictions might mean that queries are not reused or that thecaching policy
caches too aggressively on NRT segments which get merged early.- See Also:
getCacheCount()
,getCacheSize()
-
-