Java Code Examples for org.apache.beam.sdk.annotations.Experimental.Kind#SPLITTABLE_DO_FN

The following examples show how to use org.apache.beam.sdk.annotations.Experimental.Kind#SPLITTABLE_DO_FN . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: AvroIO.java    From beam with Apache License 2.0 5 votes vote down vote up 
/** Like {@link Read#watchForNewFiles}. */
@Experimental(Kind.SPLITTABLE_DO_FN)
public ReadAll<T> watchForNewFiles(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withMatchConfiguration(
      getMatchConfiguration().continuously(pollInterval, terminationCondition));
}
Example 2
Source File: AvroIO.java    From beam with Apache License 2.0 5 votes vote down vote up 
/** Like {@link Read#watchForNewFiles}. */
@Experimental(Kind.SPLITTABLE_DO_FN)
public Parse<T> watchForNewFiles(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withMatchConfiguration(
      getMatchConfiguration().continuously(pollInterval, terminationCondition));
}
Example 3
Source File: AvroIO.java    From beam with Apache License 2.0 5 votes vote down vote up 
/** Like {@link Read#watchForNewFiles}. */
@Experimental(Kind.SPLITTABLE_DO_FN)
public ParseAll<T> watchForNewFiles(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withMatchConfiguration(
      getMatchConfiguration().continuously(pollInterval, terminationCondition));
}
Example 4
Source File: TextIO.java    From beam with Apache License 2.0 5 votes vote down vote up 
/**
 * See {@link MatchConfiguration#continuously}.
 *
 * <p>This works only in runners supporting {@link Kind#SPLITTABLE_DO_FN}.
 */
@Experimental(Kind.SPLITTABLE_DO_FN)
public Read watchForNewFiles(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withMatchConfiguration(
      getMatchConfiguration().continuously(pollInterval, terminationCondition));
}
Example 5
Source File: TextIO.java    From beam with Apache License 2.0 5 votes vote down vote up 
/** Same as {@link Read#watchForNewFiles(Duration, TerminationCondition)}. */
@Experimental(Kind.SPLITTABLE_DO_FN)
public ReadAll watchForNewFiles(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withMatchConfiguration(
      getMatchConfiguration().continuously(pollInterval, terminationCondition));
}
Example 6
Source File: FileIO.java    From beam with Apache License 2.0 4 votes vote down vote up 
/**
 * See {@link MatchConfiguration#continuously}. The returned {@link PCollection} is unbounded.
 *
 * <p>This works only in runners supporting {@link Experimental.Kind#SPLITTABLE_DO_FN}.
 */
@Experimental(Kind.SPLITTABLE_DO_FN)
public Match continuously(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withConfiguration(getConfiguration().continuously(pollInterval, terminationCondition));
}
Example 7
Source File: FileIO.java    From beam with Apache License 2.0 4 votes vote down vote up 
/** Like {@link Match#continuously}. */
@Experimental(Kind.SPLITTABLE_DO_FN)
public MatchAll continuously(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withConfiguration(getConfiguration().continuously(pollInterval, terminationCondition));
}
Example 8
Source File: HBaseIO.java    From beam with Apache License 2.0 4 votes vote down vote up 
/**
 * A {@link PTransform} that works like {@link #read}, but executes read operations coming from a
 * {@link PCollection} of {@link Read}.
 */
@Experimental(Kind.SPLITTABLE_DO_FN)
public static ReadAll readAll() {
  return new ReadAll();
}
Example 9
Source File: AvroIO.java    From beam with Apache License 2.0 3 votes vote down vote up 
/**
 * Continuously watches for new files matching the filepattern, polling it at the given
 * interval, until the given termination condition is reached. The returned {@link PCollection}
 * is unbounded.
 *
 * <p>This works only in runners supporting {@link Kind#SPLITTABLE_DO_FN}.
 */
@Experimental(Kind.SPLITTABLE_DO_FN)
public Read<T> watchForNewFiles(
    Duration pollInterval, TerminationCondition<String, ?> terminationCondition) {
  return withMatchConfiguration(
      getMatchConfiguration().continuously(pollInterval, terminationCondition));
}