Java Code Examples for org.apache.flink.runtime.io.network.partition.ResultSubpartitionView#isReleased()

The following examples show how to use org.apache.flink.runtime.io.network.partition.ResultSubpartitionView#isReleased() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: LocalInputChannel.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Override
Optional<BufferAndAvailability> getNextBuffer() throws IOException, InterruptedException {
	checkError();

	ResultSubpartitionView subpartitionView = this.subpartitionView;
	if (subpartitionView == null) {
		// There is a possible race condition between writing a EndOfPartitionEvent (1) and flushing (3) the Local
		// channel on the sender side, and reading EndOfPartitionEvent (2) and processing flush notification (4). When
		// they happen in that order (1 - 2 - 3 - 4), flush notification can re-enqueue LocalInputChannel after (or
		// during) it was released during reading the EndOfPartitionEvent (2).
		if (isReleased) {
			return Optional.empty();
		}

		// this can happen if the request for the partition was triggered asynchronously
		// by the time trigger
		// would be good to avoid that, by guaranteeing that the requestPartition() and
		// getNextBuffer() always come from the same thread
		// we could do that by letting the timer insert a special "requesting channel" into the input gate's queue
		subpartitionView = checkAndWaitForSubpartitionView();
	}

	BufferAndBacklog next = subpartitionView.getNextBuffer();

	if (next == null) {
		if (subpartitionView.isReleased()) {
			throw new CancelTaskException("Consumed partition " + subpartitionView + " has been released.");
		} else {
			return Optional.empty();
		}
	}

	numBytesIn.inc(next.buffer().getSizeUnsafe());
	numBuffersIn.inc();
	return Optional.of(new BufferAndAvailability(next.buffer(), next.isMoreAvailable(), next.buffersInBacklog()));
}
Example 2
Source File: LocalInputChannel.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Override
Optional<BufferAndAvailability> getNextBuffer() throws IOException, InterruptedException {
	checkError();

	ResultSubpartitionView subpartitionView = this.subpartitionView;
	if (subpartitionView == null) {
		// There is a possible race condition between writing a EndOfPartitionEvent (1) and flushing (3) the Local
		// channel on the sender side, and reading EndOfPartitionEvent (2) and processing flush notification (4). When
		// they happen in that order (1 - 2 - 3 - 4), flush notification can re-enqueue LocalInputChannel after (or
		// during) it was released during reading the EndOfPartitionEvent (2).
		if (isReleased) {
			return Optional.empty();
		}

		// this can happen if the request for the partition was triggered asynchronously
		// by the time trigger
		// would be good to avoid that, by guaranteeing that the requestPartition() and
		// getNextBuffer() always come from the same thread
		// we could do that by letting the timer insert a special "requesting channel" into the input gate's queue
		subpartitionView = checkAndWaitForSubpartitionView();
	}

	BufferAndBacklog next = subpartitionView.getNextBuffer();

	if (next == null) {
		if (subpartitionView.isReleased()) {
			throw new CancelTaskException("Consumed partition " + subpartitionView + " has been released.");
		} else {
			return Optional.empty();
		}
	}

	numBytesIn.inc(next.buffer().getSize());
	numBuffersIn.inc();
	return Optional.of(new BufferAndAvailability(next.buffer(), next.isMoreAvailable(), next.buffersInBacklog()));
}
Example 3
Source File: LocalInputChannel.java    From flink with Apache License 2.0 4 votes vote down vote up 
@Override
Optional<BufferAndAvailability> getNextBuffer() throws IOException, InterruptedException {
	checkError();

	ResultSubpartitionView subpartitionView = this.subpartitionView;
	if (subpartitionView == null) {
		// There is a possible race condition between writing a EndOfPartitionEvent (1) and flushing (3) the Local
		// channel on the sender side, and reading EndOfPartitionEvent (2) and processing flush notification (4). When
		// they happen in that order (1 - 2 - 3 - 4), flush notification can re-enqueue LocalInputChannel after (or
		// during) it was released during reading the EndOfPartitionEvent (2).
		if (isReleased) {
			return Optional.empty();
		}

		// this can happen if the request for the partition was triggered asynchronously
		// by the time trigger
		// would be good to avoid that, by guaranteeing that the requestPartition() and
		// getNextBuffer() always come from the same thread
		// we could do that by letting the timer insert a special "requesting channel" into the input gate's queue
		subpartitionView = checkAndWaitForSubpartitionView();
	}

	BufferAndBacklog next = subpartitionView.getNextBuffer();

	if (next == null) {
		if (subpartitionView.isReleased()) {
			throw new CancelTaskException("Consumed partition " + subpartitionView + " has been released.");
		} else {
			return Optional.empty();
		}
	}

	Buffer buffer = next.buffer();
	CheckpointBarrier notifyReceivedBarrier = parseCheckpointBarrierOrNull(buffer);
	if (notifyReceivedBarrier != null) {
		receivedCheckpointId = notifyReceivedBarrier.getId();
	} else if (receivedCheckpointId < lastRequestedCheckpointId && buffer.isBuffer()) {
		inputGate.getBufferReceivedListener().notifyBufferReceived(buffer.retainBuffer(), channelInfo);
	}

	numBytesIn.inc(buffer.getSize());
	numBuffersIn.inc();
	return Optional.of(new BufferAndAvailability(buffer, next.isDataAvailable(), next.buffersInBacklog()));
}