Java Code Examples for org.apache.flink.runtime.io.network.buffer.Buffer#getRecycler()

The following examples show how to use org.apache.flink.runtime.io.network.buffer.Buffer#getRecycler() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: BufferManager.java    From flink with Apache License 2.0 6 votes vote down vote up 
/**
 * Recycles all the exclusive and floating buffers from the given buffer queue.
 */
void releaseAllBuffers(ArrayDeque<Buffer> buffers) throws IOException {
	// Gather all exclusive buffers and recycle them to global pool in batch, because
	// we do not want to trigger redistribution of buffers after each recycle.
	final List<MemorySegment> exclusiveRecyclingSegments = new ArrayList<>();

	Buffer buffer;
	while ((buffer = buffers.poll()) != null) {
		if (buffer.getRecycler() == this) {
			exclusiveRecyclingSegments.add(buffer.getMemorySegment());
		} else {
			buffer.recycleBuffer();
		}
	}
	synchronized (bufferQueue) {
		bufferQueue.releaseAll(exclusiveRecyclingSegments);
		bufferQueue.notifyAll();
	}

	if (exclusiveRecyclingSegments.size() > 0) {
		globalPool.recycleMemorySegments(exclusiveRecyclingSegments);
	}
}
Example 2
Source File: RemoteInputChannel.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
/**
 * Releases all exclusive and floating buffers, closes the partition request client.
 */
@Override
void releaseAllResources() throws IOException {
	if (isReleased.compareAndSet(false, true)) {

		// Gather all exclusive buffers and recycle them to global pool in batch, because
		// we do not want to trigger redistribution of buffers after each recycle.
		final List<MemorySegment> exclusiveRecyclingSegments = new ArrayList<>();

		synchronized (receivedBuffers) {
			Buffer buffer;
			while ((buffer = receivedBuffers.poll()) != null) {
				if (buffer.getRecycler() == this) {
					exclusiveRecyclingSegments.add(buffer.getMemorySegment());
				} else {
					buffer.recycleBuffer();
				}
			}
		}
		synchronized (bufferQueue) {
			bufferQueue.releaseAll(exclusiveRecyclingSegments);
		}

		if (exclusiveRecyclingSegments.size() > 0) {
			inputGate.returnExclusiveSegments(exclusiveRecyclingSegments);
		}

		// The released flag has to be set before closing the connection to ensure that
		// buffers received concurrently with closing are properly recycled.
		if (partitionRequestClient != null) {
			partitionRequestClient.close(this);
		} else {
			connectionManager.closeOpenChannelConnections(connectionId);
		}
	}
}
Example 3
Source File: RemoteInputChannel.java    From flink with Apache License 2.0 5 votes vote down vote up 
/**
 * Releases all exclusive and floating buffers, closes the partition request client.
 */
@Override
void releaseAllResources() throws IOException {
	if (isReleased.compareAndSet(false, true)) {

		// Gather all exclusive buffers and recycle them to global pool in batch, because
		// we do not want to trigger redistribution of buffers after each recycle.
		final List<MemorySegment> exclusiveRecyclingSegments = new ArrayList<>();

		synchronized (receivedBuffers) {
			Buffer buffer;
			while ((buffer = receivedBuffers.poll()) != null) {
				if (buffer.getRecycler() == this) {
					exclusiveRecyclingSegments.add(buffer.getMemorySegment());
				} else {
					buffer.recycleBuffer();
				}
			}
		}
		synchronized (bufferQueue) {
			bufferQueue.releaseAll(exclusiveRecyclingSegments);
		}

		if (exclusiveRecyclingSegments.size() > 0) {
			memorySegmentProvider.recycleMemorySegments(exclusiveRecyclingSegments);
		}

		// The released flag has to be set before closing the connection to ensure that
		// buffers received concurrently with closing are properly recycled.
		if (partitionRequestClient != null) {
			partitionRequestClient.close(this);
		} else {
			connectionManager.closeOpenChannelConnections(connectionId);
		}
	}
}
Example 4
Source File: TestPooledBufferProvider.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Override
public BufferBuilder requestBufferBuilder() throws IOException {
	Buffer buffer = requestBuffer();
	if (buffer != null) {
		return new BufferBuilder(buffer.getMemorySegment(), buffer.getRecycler());
	}
	return null;
}
Example 5
Source File: TestPooledBufferProvider.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up 
@Override
public BufferBuilder requestBufferBuilderBlocking() throws IOException, InterruptedException {
	Buffer buffer = requestBufferBlocking();
	return new BufferBuilder(buffer.getMemorySegment(), buffer.getRecycler());
}
Example 6
Source File: TestPooledBufferProvider.java    From flink with Apache License 2.0 4 votes vote down vote up 
@Override
public BufferBuilder requestBufferBuilderBlocking() throws IOException, InterruptedException {
	Buffer buffer = requestBufferBlocking();
	return new BufferBuilder(buffer.getMemorySegment(), buffer.getRecycler());
}
Example 7
Source File: TestPooledBufferProvider.java    From flink with Apache License 2.0 4 votes vote down vote up 
@Override
public BufferBuilder requestBufferBuilderBlocking() throws IOException, InterruptedException {
	Buffer buffer = requestBufferBlocking();
	return new BufferBuilder(buffer.getMemorySegment(), buffer.getRecycler());
}