org.apache.lucene.codecs.simpletext.SimpleTextCodec Java Examples

The following examples show how to use org.apache.lucene.codecs.simpletext.SimpleTextCodec. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: BaseIndexFileFormatTestCase.java    From lucene-solr with Apache License 2.0 4 votes vote down vote up 
/** This test is a best effort at verifying that checkIntegrity doesn't miss any files. It tests that the
 *  combination of opening a reader and calling checkIntegrity on it reads all bytes of all files. */
public void testCheckIntegrityReadsAllBytes() throws Exception {
  assumeFalse("SimpleText doesn't store checksums of its files", getCodec() instanceof SimpleTextCodec);
  FileTrackingDirectoryWrapper dir = new FileTrackingDirectoryWrapper(newDirectory());
  applyCreatedVersionMajor(dir);

  IndexWriterConfig cfg = new IndexWriterConfig(new MockAnalyzer(random()));
  IndexWriter w = new IndexWriter(dir, cfg);
  final int numDocs = atLeast(100);
  for (int i = 0; i < numDocs; ++i) {
    Document d = new Document();
    addRandomFields(d);
    w.addDocument(d);
  }
  w.forceMerge(1);
  w.commit();
  w.close();

  ReadBytesDirectoryWrapper readBytesWrapperDir = new ReadBytesDirectoryWrapper(dir);
  IndexReader reader = DirectoryReader.open(readBytesWrapperDir);
  LeafReader leafReader = getOnlyLeafReader(reader);
  leafReader.checkIntegrity();

  Map<String, FixedBitSet> readBytesMap = readBytesWrapperDir.getReadBytes();

  Set<String> unreadFiles = new HashSet<>(dir.getFiles());System.out.println(Arrays.toString(dir.listAll()));
  unreadFiles.removeAll(readBytesMap.keySet());
  unreadFiles.remove(IndexWriter.WRITE_LOCK_NAME);
  assertTrue("Some files have not been open: " + unreadFiles, unreadFiles.isEmpty());

  List<String> messages = new ArrayList<>();
  for (Map.Entry<String, FixedBitSet> entry : readBytesMap.entrySet()) {
    String name = entry.getKey();
    FixedBitSet unreadBytes = entry.getValue().clone();
    unreadBytes.flip(0, unreadBytes.length());
    int unread = unreadBytes.nextSetBit(0);
    if (unread != Integer.MAX_VALUE) {
      messages.add("Offset " + unread + " of file " + name + "(" + unreadBytes.length() + "bytes) was not read.");
    }
  }
  assertTrue(String.join("\n", messages), messages.isEmpty());
  reader.close();
  dir.close();
}
Example #2
Source File: MtasSimpleTextCodec.java    From mtas with Apache License 2.0 4 votes vote down vote up 
/**
 * Instantiates a new mtas simple text codec.
 */
public MtasSimpleTextCodec() {
  super("MtasSimpleTextCodec", new SimpleTextCodec());
}