org.apache.flink.streaming.api.operators.co.CoBroadcastWithNonKeyedOperator Java Examples

The following examples show how to use org.apache.flink.streaming.api.operators.co.CoBroadcastWithNonKeyedOperator. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: ProcessFunctionTestHarnesses.java    From flink with Apache License 2.0 6 votes vote down vote up 
/**
 * Returns an initialized test harness for {@link BroadcastProcessFunction}.
 *
 * @param function instance of a {@link BroadcastProcessFunction} under test
 * @param descriptors broadcast state descriptors used in the {@code function}
 * @param <IN1> type of input stream elements
 * @param <IN2> type of broadcast stream elements
 * @param <OUT> type of output elements
 * @return {@link BroadcastOperatorTestHarness} wrapped around {@code function}
 */
public static <IN1, IN2, OUT>
BroadcastOperatorTestHarness<IN1, IN2, OUT> forBroadcastProcessFunction(
	final BroadcastProcessFunction<IN1, IN2, OUT> function,
	final MapStateDescriptor<?, ?>... descriptors)
	throws Exception {

	BroadcastOperatorTestHarness<IN1, IN2, OUT> testHarness =
		new BroadcastOperatorTestHarness<>(
			new CoBroadcastWithNonKeyedOperator<>(
				Preconditions.checkNotNull(function), Arrays.asList(descriptors)),
			1,
			1,
			0);
	testHarness.open();
	return testHarness;
}
Example #2
Source File: BroadcastConnectedStream.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
/**
 * Assumes as inputs a {@link BroadcastStream} and a non-keyed {@link DataStream} and applies the given
 * {@link BroadcastProcessFunction} on them, thereby creating a transformed output stream.
 *
 * @param function The {@link BroadcastProcessFunction} that is called for each element in the stream.
 * @param outTypeInfo The type of the output elements.
 * @param <OUT> The type of the output elements.
 * @return The transformed {@link DataStream}.
 */
@PublicEvolving
public <OUT> SingleOutputStreamOperator<OUT> process(
		final BroadcastProcessFunction<IN1, IN2, OUT> function,
		final TypeInformation<OUT> outTypeInfo) {

	Preconditions.checkNotNull(function);
	Preconditions.checkArgument(!(inputStream1 instanceof KeyedStream),
			"A BroadcastProcessFunction can only be used on a non-keyed stream.");

	TwoInputStreamOperator<IN1, IN2, OUT> operator =
			new CoBroadcastWithNonKeyedOperator<>(clean(function), broadcastStateDescriptors);
	return transform("Co-Process-Broadcast", outTypeInfo, operator);
}
Example #3
Source File: BroadcastConnectedStream.java    From flink with Apache License 2.0 5 votes vote down vote up 
/**
 * Assumes as inputs a {@link BroadcastStream} and a non-keyed {@link DataStream} and applies the given
 * {@link BroadcastProcessFunction} on them, thereby creating a transformed output stream.
 *
 * @param function The {@link BroadcastProcessFunction} that is called for each element in the stream.
 * @param outTypeInfo The type of the output elements.
 * @param <OUT> The type of the output elements.
 * @return The transformed {@link DataStream}.
 */
@PublicEvolving
public <OUT> SingleOutputStreamOperator<OUT> process(
		final BroadcastProcessFunction<IN1, IN2, OUT> function,
		final TypeInformation<OUT> outTypeInfo) {

	Preconditions.checkNotNull(function);
	Preconditions.checkArgument(!(inputStream1 instanceof KeyedStream),
			"A BroadcastProcessFunction can only be used on a non-keyed stream.");

	TwoInputStreamOperator<IN1, IN2, OUT> operator =
			new CoBroadcastWithNonKeyedOperator<>(clean(function), broadcastStateDescriptors);
	return transform("Co-Process-Broadcast", outTypeInfo, operator);
}
Example #4
Source File: BroadcastConnectedStream.java    From flink with Apache License 2.0 5 votes vote down vote up 
/**
 * Assumes as inputs a {@link BroadcastStream} and a non-keyed {@link DataStream} and applies the given
 * {@link BroadcastProcessFunction} on them, thereby creating a transformed output stream.
 *
 * @param function The {@link BroadcastProcessFunction} that is called for each element in the stream.
 * @param outTypeInfo The type of the output elements.
 * @param <OUT> The type of the output elements.
 * @return The transformed {@link DataStream}.
 */
@PublicEvolving
public <OUT> SingleOutputStreamOperator<OUT> process(
		final BroadcastProcessFunction<IN1, IN2, OUT> function,
		final TypeInformation<OUT> outTypeInfo) {

	Preconditions.checkNotNull(function);
	Preconditions.checkArgument(!(inputStream1 instanceof KeyedStream),
			"A BroadcastProcessFunction can only be used on a non-keyed stream.");

	TwoInputStreamOperator<IN1, IN2, OUT> operator =
			new CoBroadcastWithNonKeyedOperator<>(clean(function), broadcastStateDescriptors);
	return transform("Co-Process-Broadcast", outTypeInfo, operator);
}
Example #5
Source File: BroadcastOperatorTestHarness.java    From flink with Apache License 2.0 5 votes vote down vote up 
public BroadcastOperatorTestHarness(
	CoBroadcastWithNonKeyedOperator<IN1, IN2, OUT> operator,
	int maxParallelism,
	int numSubtasks,
	int subtaskIndex)
	throws Exception {
	super(operator, maxParallelism, numSubtasks, subtaskIndex);
}
Example #6
Source File: BroadcastStateInputFormatTest.java    From flink with Apache License 2.0 4 votes vote down vote up 
private TwoInputStreamOperatorTestHarness<Void, Integer, Void> getTestHarness() throws Exception {
	return new TwoInputStreamOperatorTestHarness<>(
		new CoBroadcastWithNonKeyedOperator<>(
			new StatefulFunction(), Collections.singletonList(descriptor)));
}
Example #7
Source File: BroadcastStateInputFormatTest.java    From flink with Apache License 2.0 4 votes vote down vote up 
private TwoInputStreamOperatorTestHarness<Void, Integer, Void> getTestHarness() throws Exception {
	return new TwoInputStreamOperatorTestHarness<>(
		new CoBroadcastWithNonKeyedOperator<>(
			new StatefulFunction(), Collections.singletonList(descriptor)));
}