org.apache.flink.test.util.TestEnvironment Java Examples

The following examples show how to use org.apache.flink.test.util.TestEnvironment. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: AvroExternalJarProgramITCase.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up 
@Test
public void testExternalProgram() throws Exception {

	String jarFile = JAR_FILE;
	try {
		JobWithJars.checkJarFile(new File(jarFile).getAbsoluteFile().toURI().toURL());
	} catch (IOException e) {
		jarFile = "target/".concat(jarFile);
	}

	TestEnvironment.setAsContext(
		MINI_CLUSTER,
		PARALLELISM,
		Collections.singleton(new Path(jarFile)),
		Collections.emptyList());

	String testData = getClass().getResource(TEST_DATA_FILE).toString();

	PackagedProgram program = new PackagedProgram(new File(jarFile), new String[]{testData});

	program.invokeInteractiveModeForExecution();
}
Example #2
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 6 votes vote down vote up 
@Test
public void testKMeansJobWithCustomClassLoader() throws ProgramInvocationException {
	PackagedProgram kMeansProg = PackagedProgram.newBuilder()
		.setJarFile(new File(KMEANS_JAR_PATH))
		.setArguments(new String[] {
			KMeansData.DATAPOINTS,
			KMeansData.INITIAL_CENTERS,
			"25"})
		.build();

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(KMEANS_JAR_PATH)),
		Collections.emptyList());

	kMeansProg.invokeInteractiveModeForExecution();
}
Example #3
Source File: ClassLoaderITCase.java    From Flink-CEPplus with Apache License 2.0 6 votes vote down vote up 
@Test
public void testKMeansJobWithCustomClassLoader() throws IOException, ProgramInvocationException {
	PackagedProgram kMeansProg = new PackagedProgram(
		new File(KMEANS_JAR_PATH),
		new String[] {
			KMeansData.DATAPOINTS,
			KMeansData.INITIAL_CENTERS,
			"25"
		});

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(KMEANS_JAR_PATH)),
		Collections.<URL>emptyList());

	kMeansProg.invokeInteractiveModeForExecution();
}
Example #4
Source File: AvroExternalJarProgramITCase.java    From flink with Apache License 2.0 6 votes vote down vote up 
@Test
public void testExternalProgram() throws Exception {

	String jarFile = JAR_FILE;
	try {
		JarUtils.checkJarFile(new File(jarFile).getAbsoluteFile().toURI().toURL());
	} catch (IOException e) {
		jarFile = "target/".concat(jarFile);
	}

	TestEnvironment.setAsContext(
		MINI_CLUSTER,
		PARALLELISM,
		Collections.singleton(new Path(jarFile)),
		Collections.emptyList());

	String testData = getClass().getResource(TEST_DATA_FILE).toString();

	PackagedProgram program = PackagedProgram.newBuilder()
		.setJarFile(new File(jarFile))
		.setArguments(new String[]{testData})
		.build();

	program.invokeInteractiveModeForExecution();
}
Example #5
Source File: AvroExternalJarProgramITCase.java    From flink with Apache License 2.0 6 votes vote down vote up 
@Test
public void testExternalProgram() throws Exception {

	String jarFile = JAR_FILE;
	try {
		JobWithJars.checkJarFile(new File(jarFile).getAbsoluteFile().toURI().toURL());
	} catch (IOException e) {
		jarFile = "target/".concat(jarFile);
	}

	TestEnvironment.setAsContext(
		MINI_CLUSTER,
		PARALLELISM,
		Collections.singleton(new Path(jarFile)),
		Collections.emptyList());

	String testData = getClass().getResource(TEST_DATA_FILE).toString();

	PackagedProgram program = new PackagedProgram(new File(jarFile), new String[]{testData});

	program.invokeInteractiveModeForExecution();
}
Example #6
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 6 votes vote down vote up 
@Test
public void testKMeansJobWithCustomClassLoader() throws IOException, ProgramInvocationException {
	PackagedProgram kMeansProg = new PackagedProgram(
		new File(KMEANS_JAR_PATH),
		new String[] {
			KMeansData.DATAPOINTS,
			KMeansData.INITIAL_CENTERS,
			"25"
		});

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(KMEANS_JAR_PATH)),
		Collections.<URL>emptyList());

	kMeansProg.invokeInteractiveModeForExecution();
}
Example #7
Source File: BatchFineGrainedRecoveryITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
private static ExecutionEnvironment createExecutionEnvironment() {
	@SuppressWarnings("StaticVariableUsedBeforeInitialization")
	ExecutionEnvironment env = new TestEnvironment(miniCluster, 1, true);
	env.setRestartStrategy(RestartStrategies.fixedDelayRestart(MAX_JOB_RESTART_ATTEMPTS, Time.milliseconds(10)));
	env.getConfig().setExecutionMode(ExecutionMode.BATCH_FORCED); // forces all partitions to be blocking
	return env;
}
Example #8
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Test
public void testUserCodeTypeJobWithCustomClassLoader() throws ProgramInvocationException {
	PackagedProgram userCodeTypeProg = PackagedProgram.newBuilder()
		.setJarFile(new File(USERCODETYPE_JAR_PATH))
		.build();

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(USERCODETYPE_JAR_PATH)),
		Collections.emptyList());

	userCodeTypeProg.invokeInteractiveModeForExecution();
}
Example #9
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Test
public void testCustomSplitJobWithCustomClassLoaderPath() throws IOException, ProgramInvocationException {
	URL classpath = new File(INPUT_SPLITS_PROG_JAR_FILE).toURI().toURL();
	PackagedProgram inputSplitTestProg2 = PackagedProgram.newBuilder()
		.setJarFile(new File(INPUT_SPLITS_PROG_JAR_FILE))
		.build();

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.emptyList(),
		Collections.singleton(classpath));

	inputSplitTestProg2.invokeInteractiveModeForExecution();
}
Example #10
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Test
public void testCustomSplitJobWithCustomClassLoaderJar() throws ProgramInvocationException {

	PackagedProgram inputSplitTestProg = PackagedProgram.newBuilder()
		.setJarFile(new File(INPUT_SPLITS_PROG_JAR_FILE))
		.build();

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(INPUT_SPLITS_PROG_JAR_FILE)),
		Collections.emptyList());

	inputSplitTestProg.invokeInteractiveModeForExecution();
}
Example #11
Source File: BatchFineGrainedRecoveryITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
private static ExecutionEnvironment createExecutionEnvironment() {
	@SuppressWarnings("StaticVariableUsedBeforeInitialization")
	ExecutionEnvironment env = new TestEnvironment(miniCluster, 1, true);
	env.setRestartStrategy(RestartStrategies.fixedDelayRestart(MAX_JOB_RESTART_ATTEMPTS, Time.milliseconds(10)));
	env.getConfig().setExecutionMode(ExecutionMode.BATCH_FORCED); // forces all partitions to be blocking
	return env;
}
Example #12
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Test
public void testUserCodeTypeJobWithCustomClassLoader() throws IOException, ProgramInvocationException {
	PackagedProgram userCodeTypeProg = new PackagedProgram(new File(USERCODETYPE_JAR_PATH));

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(USERCODETYPE_JAR_PATH)),
		Collections.<URL>emptyList());

	userCodeTypeProg.invokeInteractiveModeForExecution();
}
Example #13
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Test
public void testCustomSplitJobWithCustomClassLoaderPath() throws IOException, ProgramInvocationException {
	URL classpath = new File(INPUT_SPLITS_PROG_JAR_FILE).toURI().toURL();
	PackagedProgram inputSplitTestProg2 = new PackagedProgram(new File(INPUT_SPLITS_PROG_JAR_FILE));

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.<Path>emptyList(),
		Collections.singleton(classpath));

	inputSplitTestProg2.invokeInteractiveModeForExecution();
}
Example #14
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Test
public void testCustomSplitJobWithCustomClassLoaderJar() throws IOException, ProgramInvocationException {

	PackagedProgram inputSplitTestProg = new PackagedProgram(new File(INPUT_SPLITS_PROG_JAR_FILE));

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(INPUT_SPLITS_PROG_JAR_FILE)),
		Collections.<URL>emptyList());

	inputSplitTestProg.invokeInteractiveModeForExecution();
}
Example #15
Source File: ClassLoaderITCase.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Test
public void testUserCodeTypeJobWithCustomClassLoader() throws IOException, ProgramInvocationException {
	PackagedProgram userCodeTypeProg = new PackagedProgram(new File(USERCODETYPE_JAR_PATH));

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(USERCODETYPE_JAR_PATH)),
		Collections.<URL>emptyList());

	userCodeTypeProg.invokeInteractiveModeForExecution();
}
Example #16
Source File: ClassLoaderITCase.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Test
public void testCustomSplitJobWithCustomClassLoaderPath() throws IOException, ProgramInvocationException {
	URL classpath = new File(INPUT_SPLITS_PROG_JAR_FILE).toURI().toURL();
	PackagedProgram inputSplitTestProg2 = new PackagedProgram(new File(INPUT_SPLITS_PROG_JAR_FILE));

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.<Path>emptyList(),
		Collections.singleton(classpath));

	inputSplitTestProg2.invokeInteractiveModeForExecution();
}
Example #17
Source File: ClassLoaderITCase.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Test
public void testCustomSplitJobWithCustomClassLoaderJar() throws IOException, ProgramInvocationException {

	PackagedProgram inputSplitTestProg = new PackagedProgram(new File(INPUT_SPLITS_PROG_JAR_FILE));

	TestEnvironment.setAsContext(
		miniClusterResource.getMiniCluster(),
		parallelism,
		Collections.singleton(new Path(INPUT_SPLITS_PROG_JAR_FILE)),
		Collections.<URL>emptyList());

	inputSplitTestProg.invokeInteractiveModeForExecution();
}
Example #18
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 4 votes vote down vote up 
@After
public void tearDown() {
	TestStreamEnvironment.unsetAsContext();
	TestEnvironment.unsetAsContext();
}
Example #19
Source File: AvroExternalJarProgramITCase.java    From flink with Apache License 2.0 4 votes vote down vote up 
@AfterClass
public static void tearDown() {
	TestEnvironment.unsetAsContext();
	MINI_CLUSTER.closeAsync();
}
Example #20
Source File: AvroExternalJarProgramITCase.java    From flink with Apache License 2.0 4 votes vote down vote up 
@AfterClass
public static void tearDown() {
	TestEnvironment.unsetAsContext();
	MINI_CLUSTER.closeAsync();
}
Example #21
Source File: ClassLoaderITCase.java    From flink with Apache License 2.0 4 votes vote down vote up 
@After
public void tearDown() {
	TestStreamEnvironment.unsetAsContext();
	TestEnvironment.unsetAsContext();
}
Example #22
Source File: ClassLoaderITCase.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up 
@After
public void tearDown() {
	TestStreamEnvironment.unsetAsContext();
	TestEnvironment.unsetAsContext();
}
Example #23
Source File: AvroExternalJarProgramITCase.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up 
@AfterClass
public static void tearDown() {
	TestEnvironment.unsetAsContext();
	MINI_CLUSTER.closeAsync();
}