Java Code Examples for org.apache.kafka.streams.KeyValue#pair()

The following examples show how to use org.apache.kafka.streams.KeyValue#pair() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example 1

Source File: StockPerformanceMultipleValuesTransformer.java From kafka-streams-in-action with Apache License 2.0

6 votes

@Override
@SuppressWarnings("deprecation")
public KeyValue<String, List<KeyValue<String, StockPerformance>>> punctuate(long timestamp) {
    List<KeyValue<String, StockPerformance>> stockPerformanceList = new ArrayList<>();
    KeyValueIterator<String, StockPerformance> performanceIterator = keyValueStore.all();
    while (performanceIterator.hasNext()) {
        KeyValue<String, StockPerformance> keyValue = performanceIterator.next();
        StockPerformance stockPerformance = keyValue.value;

        if (stockPerformance != null) {
            if (stockPerformance.priceDifferential() >= differentialThreshold ||
                    stockPerformance.volumeDifferential() >= differentialThreshold) {
                stockPerformanceList.add(keyValue);
            }
        }
    }
    return stockPerformanceList.isEmpty() ? null : KeyValue.pair(null, stockPerformanceList);
}

Example 2

Source File: TwitterStreamProcessor.java From talk-kafka-zipkin with MIT License

5 votes

private static KeyValue<String, JsonNode> parseJson(String key, String value) {
	try {
		return KeyValue.pair(key, OBJECT_MAPPER.readTree(value));
	}
	catch (Exception e) {
		e.printStackTrace();
		return KeyValue.pair(key, null);
	}
}

Example 3

Source File: RecordTransformer.java From Kafka-Streams-Real-time-Stream-Processing with The Unlicense

4 votes

@Override
public KeyValue<String, GenericRecord> transform(String key, GenericRecord genericRecord) {
    return KeyValue.pair(context.topic(), genericRecord);
}

Example 4

Source File: TracingFilterTransformer.java From brave with Apache License 2.0

4 votes

@Override KeyValue<K, V> result(K key, V value) {
  return KeyValue.pair(key, value);
}

Example 5

Source File: KafkaStreamsTest.java From brave with Apache License 2.0

4 votes

@Override
public KeyValue<String, String> transform(String key, String value) {
  return KeyValue.pair(key, value);
}

Example 6

Source File: GlobalKTableExample.java From kafka-streams-in-action with Apache License 2.0

2 votes

public static void main(String[] args) throws Exception {


        StreamsConfig streamsConfig = new StreamsConfig(getProperties());

        Serde<String> stringSerde = Serdes.String();
        Serde<StockTransaction> transactionSerde = StreamsSerdes.StockTransactionSerde();
        Serde<TransactionSummary> transactionSummarySerde = StreamsSerdes.TransactionSummarySerde();


        StreamsBuilder builder = new StreamsBuilder();
        long twentySeconds = 1000 * 20;

        KeyValueMapper<Windowed<TransactionSummary>, Long, KeyValue<String, TransactionSummary>> transactionMapper = (window, count) -> {
            TransactionSummary transactionSummary = window.key();
            String newKey = transactionSummary.getIndustry();
            transactionSummary.setSummaryCount(count);
            return KeyValue.pair(newKey, transactionSummary);
        };

        KStream<String, TransactionSummary> countStream =
                builder.stream( STOCK_TRANSACTIONS_TOPIC, Consumed.with(stringSerde, transactionSerde).withOffsetResetPolicy(LATEST))
                        .groupBy((noKey, transaction) -> TransactionSummary.from(transaction), Serialized.with(transactionSummarySerde, transactionSerde))
                        .windowedBy(SessionWindows.with(twentySeconds)).count()
                        .toStream().map(transactionMapper);

        GlobalKTable<String, String> publicCompanies = builder.globalTable(COMPANIES.topicName());
        GlobalKTable<String, String> clients = builder.globalTable(CLIENTS.topicName());


        countStream.leftJoin(publicCompanies, (key, txn) -> txn.getStockTicker(),TransactionSummary::withCompanyName)
                .leftJoin(clients, (key, txn) -> txn.getCustomerId(), TransactionSummary::withCustomerName)
                .print(Printed.<String, TransactionSummary>toSysOut().withLabel("Resolved Transaction Summaries"));


        
        KafkaStreams kafkaStreams = new KafkaStreams(builder.build(), streamsConfig);
        kafkaStreams.cleanUp();


        kafkaStreams.setUncaughtExceptionHandler((t, e) -> {
            LOG.error("had exception ", e);
        });

        CustomDateGenerator dateGenerator = CustomDateGenerator.withTimestampsIncreasingBy(Duration.ofMillis(750));

        DataGenerator.setTimestampGenerator(dateGenerator::get);

        MockDataProducer.produceStockTransactions(2, 5, 3, true);

        LOG.info("Starting GlobalKTable Example");
        kafkaStreams.cleanUp();
        kafkaStreams.start();
        Thread.sleep(65000);
        LOG.info("Shutting down the GlobalKTable Example Application now");
        kafkaStreams.close();
        MockDataProducer.shutdown();
    }