com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessor Java Examples

The following examples show how to use com.amazonaws.services.kinesis.clientlibrary.interfaces.IRecordProcessor. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: ManagedClientProcessorFactory.java    From aws-big-data-blog with Apache License 2.0 5 votes vote down vote up
/**
 * {@inheritDoc}
 */
public IRecordProcessor createProcessor() {
	try {
		LOG.info("Creating new Managed Client Processor");
		ManagedClientProcessor p = this.managedProcessor.copy();
		createdProcessors.put(p.toString(), p);
		return p;
	} catch (Exception e) {
		LOG.error(e);
		return null;
	}
}
 
Example #2
Source File: KinesisConnectorRecordProcessorFactory.java    From amazon-kinesis-connectors with Apache License 2.0 5 votes vote down vote up
@Override
public IRecordProcessor createProcessor() {
    try {
        IBuffer<T> buffer = pipeline.getBuffer(configuration);
        IEmitter<U> emitter = pipeline.getEmitter(configuration);
        ITransformerBase<T, U> transformer = pipeline.getTransformer(configuration);
        IFilter<T> filter = pipeline.getFilter(configuration);
        KinesisConnectorRecordProcessor<T, U> processor =
                new KinesisConnectorRecordProcessor<T, U>(buffer, filter, emitter, transformer, configuration);
        return processor;
    } catch (Throwable t) {
        throw new RuntimeException(t);
    }
}
 
Example #3
Source File: KinesisRecordProcessorFactory.java    From aws-big-data-blog with Apache License 2.0 4 votes vote down vote up
/**
 * {@inheritDoc}
 */
public IRecordProcessor createProcessor() {
    return new KinesisRecordProcessor(redisEndpoint, redisPort);
}
 
Example #4
Source File: StreamsRecordProcessorFactory.java    From aws-big-data-blog with Apache License 2.0 4 votes vote down vote up
@Override
public IRecordProcessor createProcessor() {
	return new StreamsRecordProcessor(cache);
}
 
Example #5
Source File: StreamsRecordProcessorFactory.java    From aws-dynamodb-examples with Apache License 2.0 4 votes vote down vote up
@Override
public IRecordProcessor createProcessor() {
    AmazonDynamoDBClient dynamoDBClient = new AmazonDynamoDBClient(dynamoDBCredentials, new ClientConfiguration());
    dynamoDBClient.setEndpoint(dynamoDBEndpoint);
    return new StreamsRecordProcessor(dynamoDBClient, tableName);
}
 
Example #6
Source File: KinesisClientLibraryPipelinedRecordProcessorFactory.java    From amazon-kinesis-connectors with Apache License 2.0 4 votes vote down vote up
@Override
public IRecordProcessor createProcessor() {
    return new KinesisClientLibraryPipelinedRecordProcessor(recordProcessorFactory.createProcessor(), maxQueueSize, maxQueueWaitTimeMs,
        maxProcessRecordsWaitTimeMs);
}
 
Example #7
Source File: KinesisClientLibraryPipelinedRecordProcessor.java    From amazon-kinesis-connectors with Apache License 2.0 3 votes vote down vote up
/**
 * Constructor. If null values are provided for maxQueueWaitTimeMs and/or maxProcessRecordsWaitTimeMs, default values are used.
 *
 * @param recordProcessor
 *            The record processor to wrap
 * @param maxQueueSize
 *            The maximum queue size
 * @param maxQueueWaitTimeMs
 *            Maximum time to block on the queue waiting for GetRecords result in milliseconds
 * @param maxProcessRecordsWaitTimeMs
 *            Maximum time to wait for the queue consumer to shutdown (finish ProcessRecords call) in milliseconds
 */
public KinesisClientLibraryPipelinedRecordProcessor(IRecordProcessor recordProcessor, int maxQueueSize, Long maxQueueWaitTimeMs,
    Long maxProcessRecordsWaitTimeMs) {
    this.recordProcessor = recordProcessor;
    recordQueue = new LinkedBlockingQueue<Record>(maxQueueSize);
    this.maxQueueWaitTimeMs = (maxQueueWaitTimeMs == null) ? DEFAULT_MAXIMUM_QUEUE_WAIT_TIME_MS : maxQueueWaitTimeMs;
    this.maxProcessRecordsWaitTimeMs = (maxProcessRecordsWaitTimeMs == null) ? DEFAULT_MAXIMUM_PROCESS_RECORDS_WAIT_TIME_MS : maxProcessRecordsWaitTimeMs;
}
 
Example #8
Source File: KinesisClientLibraryPipelinedRecordProcessor.java    From amazon-kinesis-connectors with Apache License 2.0 2 votes vote down vote up
/**
 * Constructor. Default values are used for maximum queue wait time and maximum process records wait time.
 *
 * @param recordProcessor
 *            The record processor to wrap
 * @param maxQueueSize
 *            The maximum queue size
 */
public KinesisClientLibraryPipelinedRecordProcessor(IRecordProcessor recordProcessor, int maxQueueSize) {
    this(recordProcessor, maxQueueSize, DEFAULT_MAXIMUM_QUEUE_WAIT_TIME_MS, DEFAULT_MAXIMUM_PROCESS_RECORDS_WAIT_TIME_MS);
}