Java Code Examples for org.apache.beam.sdk.io.fs.EmptyMatchTreatment#DISALLOW

The following examples show how to use org.apache.beam.sdk.io.fs.EmptyMatchTreatment#DISALLOW . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: AvroSource.java    From beam with Apache License 2.0 5 votes vote down vote up 
/**
 * Reads from the given file name or pattern ("glob"). The returned source needs to be further
 * configured by calling {@link #withSchema} to return a type other than {@link GenericRecord}.
 */
public static AvroSource<GenericRecord> from(ValueProvider<String> fileNameOrPattern) {
  return new AvroSource<>(
      fileNameOrPattern,
      EmptyMatchTreatment.DISALLOW,
      DEFAULT_MIN_BUNDLE_SIZE,
      readGenericRecordsWithSchema(null /* will need to be specified in withSchema */, null));
}
Example 2
Source File: TextIOReadTest.java    From beam with Apache License 2.0 5 votes vote down vote up 
private static TextSource prepareSource(
    TemporaryFolder temporaryFolder, byte[] data, byte[] delimiter) throws IOException {
  Path path = temporaryFolder.newFile().toPath();
  Files.write(path, data);
  return new TextSource(
      ValueProvider.StaticValueProvider.of(path.toString()),
      EmptyMatchTreatment.DISALLOW,
      delimiter);
}
Example 3
Source File: BlockBasedSource.java    From beam with Apache License 2.0 4 votes vote down vote up 
/** Like {@link #BlockBasedSource(String, long)}. */
public BlockBasedSource(ValueProvider<String> fileOrPatternSpec, long minBundleSize) {
  this(fileOrPatternSpec, EmptyMatchTreatment.DISALLOW, minBundleSize);
}
Example 4
Source File: TextIO.java    From beam with Apache License 2.0 4 votes vote down vote up 
@Override
public FileBasedSource<String> apply(String input) {
  return new TextSource(
      StaticValueProvider.of(input), EmptyMatchTreatment.DISALLOW, delimiter);
}
Example 5
Source File: FileBasedSource.java    From beam with Apache License 2.0 3 votes vote down vote up 
/**
 * Create a {@code FileBasedSource} based on a single file. This constructor must be used when
 * creating a new {@code FileBasedSource} for a subrange of a single file. Additionally, this
 * constructor must be used to create new {@code FileBasedSource}s when subclasses implement the
 * method {@link #createForSubrangeOfFile}.
 *
 * <p>See {@link OffsetBasedSource} for detailed descriptions of {@code minBundleSize}, {@code
 * startOffset}, and {@code endOffset}.
 *
 * @param fileMetadata specification of the file represented by the {@link FileBasedSource}, in
 *     suitable form for use with {@link FileSystems#match(List)}.
 * @param minBundleSize minimum bundle size in bytes.
 * @param startOffset starting byte offset.
 * @param endOffset ending byte offset. If the specified value {@code >= #getMaxEndOffset()} it
 *     implies {@code #getMaxEndOffSet()}.
 */
protected FileBasedSource(
    Metadata fileMetadata, long minBundleSize, long startOffset, long endOffset) {
  super(startOffset, endOffset, minBundleSize);
  mode = Mode.SINGLE_FILE_OR_SUBRANGE;
  this.singleFileMetadata = checkNotNull(fileMetadata, "fileMetadata");
  this.fileOrPatternSpec = StaticValueProvider.of(fileMetadata.resourceId().toString());

  // This field will be unused in this mode.
  this.emptyMatchTreatment = EmptyMatchTreatment.DISALLOW;
}
Example 6
Source File: FileBasedSource.java    From beam with Apache License 2.0 2 votes vote down vote up 
/**
 * Like {@link #FileBasedSource(ValueProvider, EmptyMatchTreatment, long)}, but uses the default
 * value of {@link EmptyMatchTreatment#DISALLOW}.
 */
protected FileBasedSource(ValueProvider<String> fileOrPatternSpec, long minBundleSize) {
  this(fileOrPatternSpec, EmptyMatchTreatment.DISALLOW, minBundleSize);
}