Class ShortMappedBigList

java.lang.Object
java.util.AbstractCollection<Short>
it.unimi.dsi.fastutil.shorts.AbstractShortCollection
it.unimi.dsi.fastutil.shorts.AbstractShortBigList
it.unimi.dsi.fastutil.shorts.ShortMappedBigList
All Implemented Interfaces:
it.unimi.dsi.fastutil.BigList<Short>, it.unimi.dsi.fastutil.shorts.ShortBigList, it.unimi.dsi.fastutil.shorts.ShortCollection, it.unimi.dsi.fastutil.shorts.ShortIterable, it.unimi.dsi.fastutil.shorts.ShortStack, it.unimi.dsi.fastutil.Size64, it.unimi.dsi.fastutil.Stack<Short>, Comparable<it.unimi.dsi.fastutil.BigList<? extends Short>>, Iterable<Short>, Collection<Short>

public class ShortMappedBigList extends it.unimi.dsi.fastutil.shorts.AbstractShortBigList
A bridge between byte buffers and type-specific big lists.

Java's memory-mapping facilities have the severe limitation of mapping at most Integer.MAX_VALUE bytes, as they expose the content of a file using a MappedByteBuffer. This class can expose a file of primitive types of arbitrary length as a BigList that is actually based on an array of MappedByteBuffers, each mapping a chunk of CHUNK_SIZE longs.

Mapping can happen with a specified byte order: saving and mapping data in native order using methods from BinIO will enhance performance significantly.

Instances of this class are not thread safe, but the copy() method provides a lightweight duplicate that can be read independently by another thread. Only chunks that are actually used will be duplicated lazily. If you are modifiying the content of list, however, you will need to provide external synchronization.

  • Nested Class Summary

    Nested classes/interfaces inherited from class it.unimi.dsi.fastutil.shorts.AbstractShortBigList

    it.unimi.dsi.fastutil.shorts.AbstractShortBigList.ShortRandomAccessSubList, it.unimi.dsi.fastutil.shorts.AbstractShortBigList.ShortSubList
  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final long
    The size in elements of a chunk created by map(FileChannel, ByteOrder, FileChannel.MapMode).
    static int
    Deprecated.
    static int
    The logarithm of the number of bytes of the primitive type of this list.
  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    ShortMappedBigList(ShortBuffer[] buffer, long size, boolean[] readyToUse)
    Creates a new mapped big list.
  • Method Summary

    Modifier and Type
    Method
    Description
    Returns a lightweight duplicate that can be read independently by another thread.
    void
    getElements(long from, short[] a, int offset, int length)
     
    short
    getShort(long index)
     
    map(FileChannel fileChannel)
    Creates a new mapped big list by read-only mapping a given file channel using the standard Java (i.e., DataOutput) byte order (ByteOrder.BIG_ENDIAN).
    map(FileChannel fileChannel, ByteOrder byteOrder)
    Creates a new mapped big list by read-only mapping a given file channel.
    map(FileChannel fileChannel, ByteOrder byteOrder, FileChannel.MapMode mapMode)
    Creates a new mapped big list by mapping a given file channel.
    short
    set(long index, short value)
     
    long
     

    Methods inherited from class it.unimi.dsi.fastutil.shorts.AbstractShortBigList

    add, add, add, addAll, addAll, addAll, addAll, addElements, addElements, clear, compareTo, contains, ensureIndex, ensureRestrictedIndex, equals, forEach, get, getElements, hashCode, indexOf, indexOf, intSpliterator, iterator, lastIndexOf, lastIndexOf, listIterator, listIterator, peek, peekShort, pop, popShort, push, push, rem, remove, removeElements, removeShort, set, setElements, size, size, subList, top, topShort, toString

    Methods inherited from class it.unimi.dsi.fastutil.shorts.AbstractShortCollection

    add, contains, containsAll, containsAll, remove, removeAll, removeAll, retainAll, retainAll, toArray, toShortArray, toShortArray

    Methods inherited from class java.util.AbstractCollection

    isEmpty, toArray, toArray

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface java.util.Collection

    containsAll, isEmpty, removeAll, retainAll, toArray, toArray, toArray

    Methods inherited from interface it.unimi.dsi.fastutil.shorts.ShortBigList

    addAll, addAll, addAll, addAll, setElements, setElements, spliterator

    Methods inherited from interface it.unimi.dsi.fastutil.shorts.ShortCollection

    add, contains, containsAll, intIterator, intParallelStream, intStream, parallelStream, remove, removeAll, removeIf, removeIf, removeIf, retainAll, stream, toArray, toShortArray, toShortArray

    Methods inherited from interface it.unimi.dsi.fastutil.shorts.ShortIterable

    forEach, forEach

    Methods inherited from interface it.unimi.dsi.fastutil.Stack

    isEmpty
  • Field Details

  • Constructor Details

    • ShortMappedBigList

      protected ShortMappedBigList(ShortBuffer[] buffer, long size, boolean[] readyToUse)
      Creates a new mapped big list.
      Parameters:
      buffer - the underlying buffers.
      size - the overall number of elements in the underlying buffers (i.e., the sum of the capacities of the byte buffers divided by the size of an element in bytes).
      readyToUse - an array parallel to buffer specifying which buffers do not need to be duplicated before being used (the process will happen lazily); the array will be used internally by the newly created mapped big list.
  • Method Details

    • map

      public static ShortMappedBigList map(FileChannel fileChannel) throws IOException
      Creates a new mapped big list by read-only mapping a given file channel using the standard Java (i.e., DataOutput) byte order (ByteOrder.BIG_ENDIAN).
      Parameters:
      fileChannel - the file channel that will be mapped.
      Returns:
      a new read-only mapped big list over the contents of fileChannel.
      Throws:
      IOException
      See Also:
    • map

      public static ShortMappedBigList map(FileChannel fileChannel, ByteOrder byteOrder) throws IOException
      Creates a new mapped big list by read-only mapping a given file channel.
      Parameters:
      fileChannel - the file channel that will be mapped.
      byteOrder - a prescribed byte order.
      Returns:
      a new read-only mapped big list over the contents of fileChannel.
      Throws:
      IOException
      See Also:
    • map

      public static ShortMappedBigList map(FileChannel fileChannel, ByteOrder byteOrder, FileChannel.MapMode mapMode) throws IOException
      Creates a new mapped big list by mapping a given file channel.
      Parameters:
      fileChannel - the file channel that will be mapped.
      byteOrder - a prescribed byte order.
      mapMode - the mapping mode: usually FileChannel.MapMode.READ_ONLY, but if intend to make the list mutable, you have to pass FileChannel.MapMode.READ_WRITE.
      Returns:
      a new mapped big list over the contents of fileChannel.
      Throws:
      IOException
    • copy

      public ShortMappedBigList copy()
      Returns a lightweight duplicate that can be read independently by another thread.

      Only chunks that are actually used will be duplicated lazily.

      Returns:
      a lightweight duplicate that can be read independently by another thread.
    • getShort

      public short getShort(long index)
    • getElements

      public void getElements(long from, short[] a, int offset, int length)
    • set

      public short set(long index, short value)
      Specified by:
      set in interface it.unimi.dsi.fastutil.shorts.ShortBigList
      Overrides:
      set in class it.unimi.dsi.fastutil.shorts.AbstractShortBigList
    • size64

      public long size64()