org.apache.hadoop.util.DirectBufferPool Java Examples

The following examples show how to use org.apache.hadoop.util.DirectBufferPool. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: RemoteReadRequestChain.java    From rubix with Apache License 2.0 5 votes vote down vote up 
public RemoteReadRequestChain(FSDataInputStream inputStream, String localfile, DirectBufferPool bufferPool, int directBufferSize, byte[] affixBuffer, BookKeeperFactory bookKeeperFactory)
{
  this.inputStream = inputStream;
  this.bufferPool = bufferPool;
  this.directBufferSize = directBufferSize;
  this.affixBuffer = affixBuffer;
  this.blockSize = affixBuffer.length;
  this.localFile = localfile;
  this.bookKeeperFactory = bookKeeperFactory;
}
Example #2
Source File: CachedReadRequestChain.java    From rubix with Apache License 2.0 5 votes vote down vote up 
public CachedReadRequestChain(FileSystem remoteFileSystem, String remotePath, DirectBufferPool bufferPool, int directBufferSize,
                              FileSystem.Statistics statistics, Configuration conf, BookKeeperFactory factory)
{
  this.conf = conf;
  this.remotePath = remotePath;
  this.remoteFileSystem = remoteFileSystem;
  this.bufferPool = bufferPool;
  this.directBufferSize = directBufferSize;
  this.statistics = statistics;
  this.factory = factory;
}
Example #3
Source File: DataGen.java    From rubix with Apache License 2.0 5 votes vote down vote up 
public static byte[] readBytesFromFile(String path, int offset, int length) throws IOException
{
  RandomAccessFile raf = new RandomAccessFile(path, "r");
  FileInputStream fis = new FileInputStream(raf.getFD());
  DirectBufferPool bufferPool = new DirectBufferPool();
  ByteBuffer directBuffer = bufferPool.getBuffer(2000);
  byte[] result = new byte[length];
  FileChannel fileChannel = fis.getChannel();

  int nread = 0;
  int leftToRead = length;

  while (nread < length) {
    int readInThisCycle = Math.min(leftToRead, directBuffer.capacity());
    directBuffer.clear();
    int nbytes = fileChannel.read(directBuffer, offset + nread);
    if (nbytes <= 0) {
      break;
    }

    directBuffer.flip();
    int transferBytes = Math.min(readInThisCycle, nbytes);
    directBuffer.get(result, nread, transferBytes);
    leftToRead -= transferBytes;
    nread += transferBytes;
  }

  return result;
}
Example #4
Source File: RemoteReadRequestChain.java    From rubix with Apache License 2.0 4 votes vote down vote up 
@VisibleForTesting
public RemoteReadRequestChain(FSDataInputStream inputStream, String fileName)
{
  this(inputStream, fileName, new DirectBufferPool(), 100, new byte[100], new BookKeeperFactory());
}
Example #5
Source File: CachedReadRequestChain.java    From rubix with Apache License 2.0 4 votes vote down vote up 
@VisibleForTesting
public CachedReadRequestChain(FileSystem remoteFileSystem, String remotePath, Configuration conf, BookKeeperFactory factory)
{
  this(remoteFileSystem, remotePath, new DirectBufferPool(), 100, null, conf, factory);
}