org.apache.crunch.Emitter Java Examples

The following examples show how to use org.apache.crunch.Emitter. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example #1

Source File: CreateSessions.java From kite-examples with Apache License 2.0

5 votes

@Override
public void process(
    Pair<String, Iterable<StandardEvent>> keyAndEvents,
    Emitter<Session> emitter) {
  final Iterator<StandardEvent> events = keyAndEvents.second().iterator();
  if (!events.hasNext()) {
    return;
  }

  // Initialize the values needed to create a session for this group
  final StandardEvent firstEvent = events.next();
  long startTime = firstEvent.getTimestamp();
  long endTime = firstEvent.getTimestamp();
  int numEvents = 1;

  // Inspect each event and keep track of start time, end time, and count
  while (events.hasNext()) {
    final StandardEvent event = events.next();
    startTime = Math.min(startTime, event.getTimestamp());
    endTime = Math.max(endTime, event.getTimestamp());
    numEvents += 1;
  }

  // Create a session. Use the first event for fields that do not change
  emitter.emit(Session.newBuilder()             // same on all events:
      .setUserId(firstEvent.getUserId())        // the user id (grouped by)
      .setSessionId(firstEvent.getSessionId())  // session id (grouped by)
      .setIp(firstEvent.getIp())                // the source IP address
      .setStartTimestamp(startTime)
      .setDuration(endTime - startTime)
      .setSessionEventCount(numEvents)
      .build());
}

Example #2

Source File: Tokenizer.java From tutorials with MIT License

5 votes

@Override
public void process(String line,
    Emitter<String> emitter) {
    for (String word : SPLITTER.split(line)) {
        emitter.emit(word);
    }
}

Example #3

Source File: CrunchDatasets.java From kite with Apache License 2.0

4 votes

@Override
public void process(E entity, Emitter<Pair<E, Void>> emitter) {
  emitter.emit(Pair.of(entity, (Void) null));
}