Java Code Examples for org.apache.beam.runners.dataflow.options.DataflowPipelineOptions#getExperiments()

The following examples show how to use org.apache.beam.runners.dataflow.options.DataflowPipelineOptions#getExperiments() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: DataflowRunner.java    From beam with Apache License 2.0 5 votes vote down vote up 
@VisibleForTesting
static void validateWorkerSettings(GcpOptions gcpOptions) {
  Preconditions.checkArgument(
      gcpOptions.getZone() == null || gcpOptions.getWorkerRegion() == null,
      "Cannot use option zone with workerRegion. Prefer either workerZone or workerRegion.");
  Preconditions.checkArgument(
      gcpOptions.getZone() == null || gcpOptions.getWorkerZone() == null,
      "Cannot use option zone with workerZone. Prefer workerZone.");
  Preconditions.checkArgument(
      gcpOptions.getWorkerRegion() == null || gcpOptions.getWorkerZone() == null,
      "workerRegion and workerZone options are mutually exclusive.");

  DataflowPipelineOptions dataflowOptions = gcpOptions.as(DataflowPipelineOptions.class);
  boolean hasExperimentWorkerRegion = false;
  if (dataflowOptions.getExperiments() != null) {
    for (String experiment : dataflowOptions.getExperiments()) {
      if (experiment.startsWith("worker_region")) {
        hasExperimentWorkerRegion = true;
        break;
      }
    }
  }
  Preconditions.checkArgument(
      !hasExperimentWorkerRegion || gcpOptions.getWorkerRegion() == null,
      "Experiment worker_region and option workerRegion are mutually exclusive.");
  Preconditions.checkArgument(
      !hasExperimentWorkerRegion || gcpOptions.getWorkerZone() == null,
      "Experiment worker_region and option workerZone are mutually exclusive.");

  if (gcpOptions.getZone() != null) {
    LOG.warn("Option --zone is deprecated. Please use --workerZone instead.");
    gcpOptions.setWorkerZone(gcpOptions.getZone());
    gcpOptions.setZone(null);
  }
}
Example 2
Source File: BatchStatefulParDoOverrides.java    From beam with Apache License 2.0 4 votes vote down vote up 
private static boolean isFnApi(DataflowPipelineOptions options) {
  List<String> experiments = options.getExperiments();
  return experiments != null && experiments.contains("beam_fn_api");
}