Class ByteBlockPool

  • All Implemented Interfaces:
    Accountable

    public final class ByteBlockPool
    extends java.lang.Object
    implements Accountable
    This class enables the allocation of fixed-size buffers and their management as part of a buffer array. Allocation is done through the use of an ByteBlockPool.Allocator which can be customized, e.g. to allow recycling old buffers. There are methods for writing (append(BytesRef) and reading from the buffers (e.g. readBytes(long, byte[], int, int), which handle read/write operations across buffer boundaries.
    • Field Summary

      Fields 
      Modifier and Type Field Description
      private ByteBlockPool.Allocator allocator  
      private static long BASE_RAM_BYTES  
      byte[] buffer
      Current head buffer.
      private byte[][] buffers
      Array of buffers currently used in the pool.
      private int bufferUpto
      index into the buffers array pointing to the current buffer used as the head
      static int BYTE_BLOCK_MASK
      Use this to find the position of a global offset in a particular buffer.
      static int BYTE_BLOCK_SHIFT
      Use this to find the index of the buffer containing a byte, given an offset to that byte.
      static int BYTE_BLOCK_SIZE
      The size of each buffer in the pool.
      int byteOffset
      Offset from the start of the first buffer to the start of the current buffer, which is bufferUpto * BYTE_BLOCK_SIZE.
      int byteUpto
      Where we are in the head buffer.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void append​(byte[] bytes)
      Append the provided byte array at the current position.
      void append​(byte[] bytes, int offset, int length)
      Append some portion of the provided byte array at the current position.
      void append​(ByteBlockPool srcPool, long srcOffset, int length)
      Append the bytes from a source ByteBlockPool at a given offset and length
      void append​(BytesRef bytes)
      Appends the bytes in the provided BytesRef at the current position.
      private void appendBytesSingleBuffer​(ByteBlockPool srcPool, long srcOffset, int length)  
      byte[] getBuffer​(int bufferIndex)
      Retrieve the buffer at the specified index from the buffer pool.
      long getPosition()
      the current position (in absolute value) of this byte pool
      void nextBuffer()
      Allocates a new buffer and advances the pool to it.
      long ramBytesUsed()
      Return the memory usage of this object in bytes.
      byte readByte​(long offset)
      Read a single byte at the given offset
      void readBytes​(long offset, byte[] bytes, int bytesOffset, int bytesLength)
      Reads bytes out of the pool starting at the given offset with the given length into the given byte array at offset off.
      void reset​(boolean zeroFillBuffers, boolean reuseFirst)
      Expert: Resets the pool to its initial state, while optionally reusing the first buffer.
      (package private) void setBytesRef​(BytesRefBuilder builder, BytesRef result, long offset, int length)
      Fill the provided BytesRef with the bytes at the specified offset and length.
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • BASE_RAM_BYTES

        private static final long BASE_RAM_BYTES
      • BYTE_BLOCK_SHIFT

        public static final int BYTE_BLOCK_SHIFT
        Use this to find the index of the buffer containing a byte, given an offset to that byte.

        bufferUpto = globalOffset >> BYTE_BLOCK_SHIFT

        bufferUpto = globalOffset / BYTE_BLOCK_SIZE

        See Also:
        Constant Field Values
      • BYTE_BLOCK_SIZE

        public static final int BYTE_BLOCK_SIZE
        The size of each buffer in the pool.
        See Also:
        Constant Field Values
      • BYTE_BLOCK_MASK

        public static final int BYTE_BLOCK_MASK
        Use this to find the position of a global offset in a particular buffer.

        positionInCurrentBuffer = globalOffset & BYTE_BLOCK_MASK

        positionInCurrentBuffer = globalOffset % BYTE_BLOCK_SIZE

        See Also:
        Constant Field Values
      • buffers

        private byte[][] buffers
        Array of buffers currently used in the pool. Buffers are allocated if needed.
      • bufferUpto

        private int bufferUpto
        index into the buffers array pointing to the current buffer used as the head
      • byteUpto

        public int byteUpto
        Where we are in the head buffer.
      • buffer

        public byte[] buffer
        Current head buffer.
      • byteOffset

        public int byteOffset
        Offset from the start of the first buffer to the start of the current buffer, which is bufferUpto * BYTE_BLOCK_SIZE. The buffer pool maintains this offset because it is the first to overflow if there are too many allocated blocks.
    • Method Detail

      • reset

        public void reset​(boolean zeroFillBuffers,
                          boolean reuseFirst)
        Expert: Resets the pool to its initial state, while optionally reusing the first buffer. Buffers that are not reused are reclaimed by ByteBlockPool.Allocator.recycleByteBlocks(byte[][], int, int). Buffers can be filled with zeros before recycling them. This is useful if a slice pool works on top of this byte pool and relies on the buffers being filled with zeros to find the non-zero end of slices.
        Parameters:
        zeroFillBuffers - if true the buffers are filled with 0. This should be set to true if this pool is used with slices.
        reuseFirst - if true the first buffer will be reused and calling nextBuffer() is not needed after reset iff the block pool was used before ie. nextBuffer() was called before.
      • nextBuffer

        public void nextBuffer()
        Allocates a new buffer and advances the pool to it. This method should be called once after the constructor to initialize the pool. In contrast to the constructor, a reset(boolean, boolean) call will advance the pool to its first buffer immediately.
      • setBytesRef

        void setBytesRef​(BytesRefBuilder builder,
                         BytesRef result,
                         long offset,
                         int length)
        Fill the provided BytesRef with the bytes at the specified offset and length. This will avoid copying the bytes if the slice fits into a single block; otherwise, it uses the provided BytesRefBuilder to copy bytes over.
      • append

        public void append​(BytesRef bytes)
        Appends the bytes in the provided BytesRef at the current position.
      • append

        public void append​(ByteBlockPool srcPool,
                           long srcOffset,
                           int length)
        Append the bytes from a source ByteBlockPool at a given offset and length
        Parameters:
        srcPool - the source pool to copy from
        srcOffset - the source pool offset
        length - the number of bytes to copy
      • appendBytesSingleBuffer

        private void appendBytesSingleBuffer​(ByteBlockPool srcPool,
                                             long srcOffset,
                                             int length)
      • append

        public void append​(byte[] bytes)
        Append the provided byte array at the current position.
        Parameters:
        bytes - the byte array to write
      • append

        public void append​(byte[] bytes,
                           int offset,
                           int length)
        Append some portion of the provided byte array at the current position.
        Parameters:
        bytes - the byte array to write
        offset - the offset of the byte array
        length - the number of bytes to write
      • readBytes

        public void readBytes​(long offset,
                              byte[] bytes,
                              int bytesOffset,
                              int bytesLength)
        Reads bytes out of the pool starting at the given offset with the given length into the given byte array at offset off.

        Note: this method allows to copy across block boundaries.

      • readByte

        public byte readByte​(long offset)
        Read a single byte at the given offset
        Parameters:
        offset - the offset to read
        Returns:
        the byte
      • ramBytesUsed

        public long ramBytesUsed()
        Description copied from interface: Accountable
        Return the memory usage of this object in bytes. Negative values are illegal.
        Specified by:
        ramBytesUsed in interface Accountable
      • getPosition

        public long getPosition()
        the current position (in absolute value) of this byte pool
      • getBuffer

        public byte[] getBuffer​(int bufferIndex)
        Retrieve the buffer at the specified index from the buffer pool.