net.jpountz.xxhash.XXHash64 Java Examples

The following examples show how to use net.jpountz.xxhash.XXHash64. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: RpcTester.java    From vespa with Apache License 2.0 6 votes vote down vote up 
void receive(FileReference reference, String filename, byte[] content) {

            log.log(Level.INFO, "Preparing receive call for " + reference.value() + " and file " + filename);

            XXHash64 hasher = XXHashFactory.fastestInstance().hash64();
            Request fileBlob = new Request("filedistribution.receiveFile");

            log.log(Level.INFO, "Calling " + fileBlob.methodName() + " with target " + target);

            fileBlob.parameters().add(new StringValue(reference.value()));
            fileBlob.parameters().add(new StringValue(filename));
            fileBlob.parameters().add(new DataValue(content));
            fileBlob.parameters().add(new Int64Value(hasher.hash(ByteBuffer.wrap(content), 0)));
            fileBlob.parameters().add(new Int32Value(0));
            fileBlob.parameters().add(new StringValue("OK"));
            log.log(Level.INFO, "Doing invokeSync");
            target.invokeSync(fileBlob, 5);
            log.log(Level.INFO, "Done with invokeSync");
        }
Example #2
Source File: FileDirectory.java    From vespa with Apache License 2.0 6 votes vote down vote up 
private Long computeHash(File file) throws IOException {
    XXHash64 hasher = XXHashFactory.fastestInstance().hash64();
    if (file.isDirectory()) {
        return Files.walk(file.toPath(), 100).map(path -> {
            try {
                log.log(Level.FINE, "Calculating hash for '" + path + "'");
                return hash(path.toFile(), hasher);
            } catch (IOException e) {
                log.log(Level.WARNING, "Failed getting hash from '" + path + "'");
                return 0;
            }
        }).mapToLong(Number::longValue).sum();
    } else {
        return hash(file, hasher);
    }
}
Example #3
Source File: FileDownloaderTest.java    From vespa with Apache License 2.0 5 votes vote down vote up 
private void receiveFile(FileDownloader fileDownloader, FileReference fileReference, String filename,
                         FileReferenceData.Type type, byte[] content) {
    XXHash64 hasher = XXHashFactory.fastestInstance().hash64();
    FileReceiver.Session session =
            new FileReceiver.Session(downloadDir, tempDir, 1, fileReference, type, filename, content.length);
    session.addPart(0, content);
    File file = session.close(hasher.hash(ByteBuffer.wrap(content), 0));
    fileDownloader.fileReferenceDownloader().completedDownloading(fileReference, file);
}
Example #4
Source File: FileDirectory.java    From vespa with Apache License 2.0 4 votes vote down vote up 
private long hash(File file, XXHash64 hasher) throws IOException {
    byte[] wholeFile = file.isDirectory() ?  new byte[0] : IOUtils.readFileBytes(file);
    return hasher.hash(ByteBuffer.wrap(wholeFile), hasher.hash(ByteBuffer.wrap(Utf8.toBytes(file.getName())), 0));
}
Example #5
Source File: Jp64Hasher.java    From hash-bench with MIT License 4 votes vote down vote up 
private Jp64Hasher(final XXHash64 delegate) {
  this.delegate = delegate;
}