Java Code Examples for org.apache.beam.sdk.io.fs.EmptyMatchTreatment#ALLOW_IF_WILDCARD

The following examples show how to use org.apache.beam.sdk.io.fs.EmptyMatchTreatment#ALLOW_IF_WILDCARD . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: FileBasedSourceTest.java    From beam with Apache License 2.0 6 votes vote down vote up 
@Test
public void testEmptyFilepatternTreatmentAllow() throws IOException {
  PipelineOptions options = PipelineOptionsFactory.create();
  TestFileBasedSource source =
      new TestFileBasedSource(
          new File(tempFolder.getRoot(), "doesNotExist").getPath(),
          EmptyMatchTreatment.ALLOW,
          64,
          null);
  TestFileBasedSource sourceWithWildcard =
      new TestFileBasedSource(
          new File(tempFolder.getRoot(), "doesNotExist*").getPath(),
          EmptyMatchTreatment.ALLOW_IF_WILDCARD,
          64,
          null);
  assertEquals(0, readFromSource(source, options).size());
  assertEquals(0, readFromSource(sourceWithWildcard, options).size());
}
Example 2
Source File: FileSystems.java    From beam with Apache License 2.0 5 votes vote down vote up 
private static MatchResult maybeAdjustEmptyMatchResult(
    String spec, MatchResult res, EmptyMatchTreatment emptyMatchTreatment) throws IOException {
  if (res.status() == Status.NOT_FOUND
      || (res.status() == Status.OK && res.metadata().isEmpty())) {
    boolean notFoundAllowed =
        emptyMatchTreatment == EmptyMatchTreatment.ALLOW
            || (hasGlobWildcard(spec)
                && emptyMatchTreatment == EmptyMatchTreatment.ALLOW_IF_WILDCARD);
    return notFoundAllowed
        ? MatchResult.create(Status.OK, Collections.emptyList())
        : MatchResult.create(
            Status.NOT_FOUND, new FileNotFoundException("No files matched spec: " + spec));
  }
  return res;
}
Example 3
Source File: FileBasedSourceTest.java    From beam with Apache License 2.0 5 votes vote down vote up 
@Test
public void testEmptyFilepatternTreatmentAllowIfWildcard() throws IOException {
  PipelineOptions options = PipelineOptionsFactory.create();
  TestFileBasedSource source =
      new TestFileBasedSource(
          new File(tempFolder.getRoot(), "doesNotExist").getPath(),
          EmptyMatchTreatment.ALLOW_IF_WILDCARD,
          64,
          null);
  thrown.expect(FileNotFoundException.class);
  readFromSource(source, options);
}