public class WindowCache
extends java.lang.Object
PackFile
in
memory for faster read access.
The WindowCache serves as a Java based "buffer cache", loading segments of a PackFile into the JVM heap prior to use. As JGit often wants to do reads of only tiny slices of a file, the WindowCache tries to smooth out these tiny reads into larger block-sized IO operations.
Whenever a cache miss occurs, load(PackFile, long)
is invoked by
exactly one thread for the given (PackFile,position)
key tuple.
This is ensured by an array of locks, with the tuple hashed to a lock
instance.
During a miss, older entries are evicted from the cache so long as
isFull()
returns true.
Its too expensive during object access to be 100% accurate with a least recently used (LRU) algorithm. Strictly ordering every read is a lot of overhead that typically doesn't yield a corresponding benefit to the application.
This cache implements a loose LRU policy by randomly picking a window comprised of roughly 10% of the cache, and evicting the oldest accessed entry within that window.
Entities created by the cache are held under SoftReferences if option
core.packedGitUseStrongRefs
is set to false
in the git config
(this is the default) or by calling
WindowCacheConfig.setPackedGitUseStrongRefs(boolean)
, permitting the
Java runtime's garbage collector to evict entries when heap memory gets low.
Most JREs implement a loose least recently used algorithm for this eviction.
When this option is set to true
strong references are used which
means that Java gc cannot evict the WindowCache to reclaim memory. On the
other hand this provides more predictable performance since the cache isn't
flushed when used heap comes close to the maximum heap size.
The internal hash table does not expand at runtime, instead it is fixed in size at cache creation time. The internal lock table used to gate load invocations is also fixed in size.
The key tuple is passed through to methods as a pair of parameters rather than as a single Object, thus reducing the transient memory allocations of callers. It is more efficient to avoid the allocation, as we can't be 100% sure that a JIT would be able to stack-allocate a key tuple.
This cache has an implementation rule such that:
load(PackFile, long)
is invoked by at most one thread at a time
for a given (PackFile,position)
tuple.load()
invocation there is exactly one
createRef(PackFile, long, ByteWindow)
invocation to wrap a
SoftReference or a StrongReference around the cached entity.createRef()
there will be
exactly one call to clear(PageRef)
to cleanup any resources associated
with the (now expired) cached entity.
Therefore, it is safe to perform resource accounting increments during the
load(PackFile, long)
or
createRef(PackFile, long, ByteWindow)
methods, and matching
decrements during clear(PageRef)
. Implementors may need to override
createRef(PackFile, long, ByteWindow)
in order to embed additional
accounting information into an implementation specific
WindowCache.PageRef
subclass, as
the cached entity may have already been evicted by the JRE's garbage
collector.
To maintain higher concurrency workloads, during eviction only one thread performs the eviction work, while other threads can continue to insert new objects in parallel. This means that the cache can be temporarily over limit, especially if the nominated eviction thread is being starved relative to the other threads.
Modifier and Type | Class and Description |
---|---|
private static interface |
WindowCache.CleanupQueue |
private static class |
WindowCache.Entry |
private static class |
WindowCache.Lock |
private static interface |
WindowCache.PageRef<T> |
private static class |
WindowCache.SoftCleanupQueue |
private static class |
WindowCache.SoftRef
A soft reference wrapped around a cached object.
|
(package private) static interface |
WindowCache.StatsRecorder
Record statistics for a cache
|
(package private) static class |
WindowCache.StatsRecorderImpl |
private static class |
WindowCache.StrongCleanupQueue |
private static class |
WindowCache.StrongRef
A strong reference wrapped around a cached object.
|
Modifier and Type | Field and Description |
---|---|
private static WindowCache |
cache |
private java.util.concurrent.atomic.AtomicLong |
clock
Access clock for loose LRU.
|
private int |
evictBatch
Number of
table buckets to scan for an eviction window. |
private java.util.concurrent.locks.ReentrantLock |
evictLock
Lock to elect the eviction thread after a load occurs.
|
private WindowCache.Lock[] |
locks
Locks to prevent concurrent loads for same (PackFile,position).
|
private long |
maxBytes |
private int |
maxFiles |
private WindowCache.StatsRecorderImpl |
mbean |
private boolean |
mmap |
private WindowCache.CleanupQueue |
queue
cleanup released and/or garbage collected windows.
|
private static java.util.Random |
rng |
private WindowCache.StatsRecorder |
statsRecorder |
private static int |
streamFileThreshold |
private java.util.concurrent.atomic.AtomicReferenceArray<WindowCache.Entry> |
table
Hash bucket directory; entries are chained below.
|
private int |
tableSize
Number of entries in
table . |
private boolean |
useStrongRefs |
private int |
windowSize |
private int |
windowSizeShift |
Modifier | Constructor and Description |
---|---|
private |
WindowCache(WindowCacheConfig cfg) |
Modifier and Type | Method and Description |
---|---|
private static int |
bits(int newSize) |
private static WindowCache.Entry |
clean(WindowCache.Entry top) |
private void |
clear(WindowCache.PageRef<ByteWindow> ref) |
private void |
close(PackFile pack) |
private WindowCache.PageRef<ByteWindow> |
createRef(PackFile p,
long o,
ByteWindow v) |
private void |
evict() |
private void |
gc() |
(package private) static ByteWindow |
get(PackFile pack,
long offset) |
static WindowCache |
getInstance() |
private ByteWindow |
getOrLoad(PackFile pack,
long position)
Lookup a cached object, creating and loading it if it doesn't exist.
|
WindowCacheStats |
getStats() |
(package private) static int |
getStreamFileThreshold() |
private int |
hash(int packHash,
long off) |
private void |
hit(WindowCache.PageRef r) |
private boolean |
isFull() |
private ByteWindow |
load(PackFile pack,
long offset) |
private WindowCache.Lock |
lock(PackFile pack,
long position) |
private static int |
lockCount(WindowCacheConfig cfg) |
(package private) static void |
purge(PackFile pack) |
static void |
reconfigure(WindowCacheConfig cfg)
Deprecated.
use
cfg.install() to avoid internal reference. |
private void |
removeAll()
Clear every entry from the cache.
|
private void |
removeAll(PackFile pack)
Clear all entries related to a single file.
|
void |
resetStats()
Reset stats.
|
private ByteWindow |
scan(WindowCache.Entry n,
PackFile pack,
long position) |
private int |
slot(PackFile pack,
long position) |
private static int |
tableSize(WindowCacheConfig cfg) |
private long |
toStart(long offset) |
private static final java.util.Random rng
private static volatile WindowCache cache
private static volatile int streamFileThreshold
private final WindowCache.CleanupQueue queue
private final int tableSize
table
.private final java.util.concurrent.atomic.AtomicLong clock
private final java.util.concurrent.atomic.AtomicReferenceArray<WindowCache.Entry> table
private final WindowCache.Lock[] locks
private final java.util.concurrent.locks.ReentrantLock evictLock
private final int evictBatch
table
buckets to scan for an eviction window.private final int maxFiles
private final long maxBytes
private final boolean mmap
private final int windowSizeShift
private final int windowSize
private final WindowCache.StatsRecorder statsRecorder
private final WindowCache.StatsRecorderImpl mbean
private boolean useStrongRefs
private WindowCache(WindowCacheConfig cfg)
private static final int bits(int newSize)
@Deprecated public static void reconfigure(WindowCacheConfig cfg)
cfg.install()
to avoid internal reference.The new configuration is applied immediately. If the new limits are smaller than what is currently cached, older entries will be purged as soon as possible to allow the cache to meet the new limit.
cfg
- the new window cache configuration.java.lang.IllegalArgumentException
- the cache configuration contains one or more invalid
settings, usually too low of a limit.static int getStreamFileThreshold()
public static WindowCache getInstance()
static final ByteWindow get(PackFile pack, long offset) throws java.io.IOException
java.io.IOException
static final void purge(PackFile pack)
public WindowCacheStats getStats()
public void resetStats()
private int hash(int packHash, long off)
private ByteWindow load(PackFile pack, long offset) throws java.io.IOException
java.io.IOException
private WindowCache.PageRef<ByteWindow> createRef(PackFile p, long o, ByteWindow v)
private void clear(WindowCache.PageRef<ByteWindow> ref)
private void close(PackFile pack)
private boolean isFull()
private long toStart(long offset)
private static int tableSize(WindowCacheConfig cfg)
private static int lockCount(WindowCacheConfig cfg)
private ByteWindow getOrLoad(PackFile pack, long position) throws java.io.IOException
pack
- the pack that "contains" the cached object.position
- offset within pack
of the object.java.io.IOException
- the object reference was not in the cache and could not be
obtained by load(PackFile, long)
.private ByteWindow scan(WindowCache.Entry n, PackFile pack, long position)
private void hit(WindowCache.PageRef r)
private void evict()
private void removeAll()
This is a last-ditch effort to clear out the cache, such as before it
gets replaced by another cache that is configured differently. This
method tries to force every cached entry through clear(PageRef)
to
ensure that resources are correctly accounted for and cleaned up by the
subclass. A concurrent reader loading entries while this method is
running may cause resource accounting failures.
private void removeAll(PackFile pack)
Typically this method is invoked during PackFile.close()
, when we
know the pack is never going to be useful to us again (for example, it no
longer exists on disk). A concurrent reader loading an entry from this
same pack may cause the pack to become stuck in the cache anyway.
pack
- the file to purge all entries of.private void gc()
private int slot(PackFile pack, long position)
private WindowCache.Lock lock(PackFile pack, long position)
private static WindowCache.Entry clean(WindowCache.Entry top)