Java Code Examples for org.apache.parquet.bytes.BytesInput#concat()

The following examples show how to use org.apache.parquet.bytes.BytesInput#concat() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: DeltaLengthByteArrayValuesWriter.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up 
@Override
public BytesInput getBytes() {
  try {
    out.flush();
  } catch (IOException e) {
    throw new ParquetEncodingException("could not write page", e);
  }
  LOG.debug("writing a buffer of size {}", arrayOut.size());
  return BytesInput.concat(lengthWriter.getBytes(), BytesInput.from(arrayOut));
}
Example 2
Source File: RunLengthBitPackingHybridValuesWriter.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up 
@Override
public BytesInput getBytes() {
  try {
    // prepend the length of the column
    BytesInput rle = encoder.toBytes();
    return BytesInput.concat(BytesInput.fromInt(Math.toIntExact(rle.size())), rle);
  } catch (IOException e) {
    throw new ParquetEncodingException(e);
  }
}
Example 3
Source File: DeltaBinaryPackingValuesWriterForLong.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up 
/**
 * getBytes will trigger flushing block buffer, DO NOT write after getBytes() is called without calling reset()
 *
 * @return a BytesInput that contains the encoded page data
 */
@Override
public BytesInput getBytes() {
  // The Page Header should include: blockSizeInValues, numberOfMiniBlocks, totalValueCount
  if (deltaValuesToFlush != 0) {
    flushBlockBuffer();
  }
  return BytesInput.concat(
          config.toBytesInput(),
          BytesInput.fromUnsignedVarInt(totalValueCount),
          BytesInput.fromZigZagVarLong(firstValue),
          BytesInput.from(baos));
}
Example 4
Source File: DeltaBinaryPackingValuesWriterForInteger.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up 
/**
 * getBytes will trigger flushing block buffer, DO NOT write after getBytes() is called without calling reset()
 *
 * @return a BytesInput that contains the encoded page data
 */
@Override
public BytesInput getBytes() {
  // The Page Header should include: blockSizeInValues, numberOfMiniBlocks, totalValueCount
  if (deltaValuesToFlush != 0) {
    flushBlockBuffer();
  }
  return BytesInput.concat(
          config.toBytesInput(),
          BytesInput.fromUnsignedVarInt(totalValueCount),
          BytesInput.fromZigZagVarInt(firstValue),
          BytesInput.from(baos));
}
Example 5
Source File: ByteStreamSplitValuesWriter.java    From parquet-mr with Apache License 2.0 5 votes vote down vote up 
@Override
public BytesInput getBytes() {
  BytesInput[] allInputs = new BytesInput[this.numStreams];
  for (int i = 0; i < this.numStreams; ++i) {
    allInputs[i] = BytesInput.from(this.byteStreams[i]);
  }
  return BytesInput.concat(allInputs);
}
Example 6
Source File: DeltaBinaryPackingConfig.java    From parquet-mr with Apache License 2.0 4 votes vote down vote up 
public BytesInput toBytesInput() {
  return BytesInput.concat(
          BytesInput.fromUnsignedVarInt(blockSizeInValues),
          BytesInput.fromUnsignedVarInt(miniBlockNumInABlock));
}
Example 7
Source File: DeltaByteArrayWriter.java    From parquet-mr with Apache License 2.0 4 votes vote down vote up 
@Override
public BytesInput getBytes() {
  return BytesInput.concat(prefixLengthWriter.getBytes(), suffixWriter.getBytes());
}