Java Code Examples for org.elasticsearch.index.seqno.SequenceNumbers#loadSeqNoInfoFromLuceneCommit()

The following examples show how to use org.elasticsearch.index.seqno.SequenceNumbers#loadSeqNoInfoFromLuceneCommit() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: test.java    From vscode-extension with MIT License 6 votes vote down vote up 
private static LocalCheckpointTracker createLocalCheckpointTracker(EngineConfig engineConfig, SegmentInfos lastCommittedSegmentInfos,
    Logger logger, Supplier<Searcher> searcherSupplier, BiFunction<Long, Long, LocalCheckpointTracker> localCheckpointTrackerSupplier) {
    try {
        final SequenceNumbers.CommitInfo seqNoStats =
            SequenceNumbers.loadSeqNoInfoFromLuceneCommit(lastCommittedSegmentInfos.userData.entrySet());
        final long maxSeqNo = seqNoStats.maxSeqNo;
        final long localCheckpoint = seqNoStats.localCheckpoint;
        logger.trace("recovered maximum sequence number [{}] and local checkpoint [{}]", maxSeqNo, localCheckpoint);
        final LocalCheckpointTracker tracker = localCheckpointTrackerSupplier.apply(maxSeqNo, localCheckpoint);
        // Operations that are optimized using max_seq_no_of_updates optimization must not be processed twice; otherwise, they will
        // create duplicates in Lucene. To avoid this we check the LocalCheckpointTracker to see if an operation was already processed.
        // Thus, we need to restore the LocalCheckpointTracker bit by bit to ensure the consistency between LocalCheckpointTracker and
        // Lucene index. This is not the only solution since we can bootstrap max_seq_no_of_updates with max_seq_no of the commit to
        // disable the MSU optimization during recovery. Here we prefer to maintain the consistency of LocalCheckpointTracker.
        if (localCheckpoint < maxSeqNo && engineConfig.getIndexSettings().isSoftDeleteEnabled()) {
            try (Searcher searcher = searcherSupplier.get()) {
                Lucene.scanSeqNosInReader(searcher.getDirectoryReader(), localCheckpoint + 1, maxSeqNo,
                    tracker::markSeqNoAsCompleted);
            }
        }
        return tracker;
    } catch (IOException ex) {
        throw new EngineCreationFailureException(engineConfig.getShardId(), "failed to create local checkpoint tracker", ex);
    }
}
Example 2
Source File: InternalEngine.java    From crate with Apache License 2.0 5 votes vote down vote up 
private LocalCheckpointTracker createLocalCheckpointTracker(
    BiFunction<Long, Long, LocalCheckpointTracker> localCheckpointTrackerSupplier) throws IOException {
    final long maxSeqNo;
    final long localCheckpoint;
    final SequenceNumbers.CommitInfo seqNoStats =
        SequenceNumbers.loadSeqNoInfoFromLuceneCommit(store.readLastCommittedSegmentsInfo().userData.entrySet());
    maxSeqNo = seqNoStats.maxSeqNo;
    localCheckpoint = seqNoStats.localCheckpoint;
    if (logger.isTraceEnabled()) {
        logger.trace("recovered maximum sequence number [{}] and local checkpoint [{}]", maxSeqNo, localCheckpoint);
    }
    return localCheckpointTrackerSupplier.apply(maxSeqNo, localCheckpoint);
}
Example 3
Source File: Store.java    From crate with Apache License 2.0 5 votes vote down vote up 
/**
 * Marks an existing lucene index with a new history uuid.
 * This is used to make sure no existing shard will recovery from this index using ops based recovery.
 */
public void bootstrapNewHistory() throws IOException {
    metadataLock.writeLock().lock();
    try {
        Map<String, String> userData = readLastCommittedSegmentsInfo().getUserData();
        final SequenceNumbers.CommitInfo seqno = SequenceNumbers.loadSeqNoInfoFromLuceneCommit(userData.entrySet());
        bootstrapNewHistory(seqno.maxSeqNo);
    } finally {
        metadataLock.writeLock().unlock();
    }
}
Example 4
Source File: Store.java    From crate with Apache License 2.0 2 votes vote down vote up 
/**
 * Loads the maximum sequence number and local checkpoint from the given Lucene commit point or the latest if not provided.
 *
 * @param commit the commit point to load seqno stats, or the last commit in the store if the parameter is null
 * @return {@link SequenceNumbers.CommitInfo} containing information about the last commit
 * @throws IOException if an I/O exception occurred reading the latest Lucene commit point from disk
 */
public static SequenceNumbers.CommitInfo loadSeqNoInfo(final IndexCommit commit) throws IOException {
    final Map<String, String> userData = commit.getUserData();
    return SequenceNumbers.loadSeqNoInfoFromLuceneCommit(userData.entrySet());
}