Java Code Examples for com.amazonaws.services.s3.transfer.Download#waitForCompletion()

The following examples show how to use com.amazonaws.services.s3.transfer.Download#waitForCompletion() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: S3StorageDriver.java    From dcos-cassandra-service with Apache License 2.0 6 votes vote down vote up 
private void downloadFile(TransferManager tx,
                          String bucketName,
                          String sourcePrefixKey,
                          String destinationFile) throws Exception{
    try {
        final File snapshotFile = new File(destinationFile);
        // Only create parent directory once, if it doesn't exist.
        final File parentDir = new File(snapshotFile.getParent());
        if (!parentDir.isDirectory()) {
            final boolean parentDirCreated = parentDir.mkdirs();
            if (!parentDirCreated) {
                LOGGER.error(
                        "Error creating parent directory for file: {}. Skipping to next",
                        destinationFile);
                return;
            }
        }
        snapshotFile.createNewFile();
        final Download download = tx.download(bucketName, sourcePrefixKey, snapshotFile);
        download.waitForCompletion();
    } catch (Exception e) {
        LOGGER.error("Error downloading the file {} : {}", destinationFile, e);
        throw new Exception(e);
    }
}
Example 2
Source File: S3BlobRetriever.java    From hollow-reference-implementation with Apache License 2.0 5 votes vote down vote up 
private File downloadFile(String objectName) throws IOException {
	File tempFile = new File(System.getProperty("java.io.tmpdir"), objectName.replace('/', '-'));
	
	Download download = s3TransferManager.download(bucketName, objectName, tempFile);
	
	try {
	    download.waitForCompletion();
	} catch(SdkBaseException | InterruptedException e) {
	    throw new RuntimeException(e);
	}
	
	return tempFile;
}
Example 3
Source File: S3DownloadStep.java    From pipeline-aws-plugin with Apache License 2.0 5 votes vote down vote up 
@Override
public Void invoke(File localFile, VirtualChannel channel) throws IOException, InterruptedException {
	TransferManager mgr = TransferManagerBuilder.standard()
			.withS3Client(AWSClientFactory.create(this.amazonS3ClientOptions.createAmazonS3ClientBuilder(), this.envVars))
			.build();

	if (this.path == null || this.path.isEmpty() || this.path.endsWith("/")) {
		try {
			final MultipleFileDownload fileDownload = mgr.downloadDirectory(this.bucket, this.path, localFile);
			fileDownload.waitForCompletion();
			RemoteDownloader.this.taskListener.getLogger().println("Finished: " + fileDownload.getDescription());
		}
		finally {
			mgr.shutdownNow();
		}
		return null;
	} else {
		try {
			final Download download = mgr.download(this.bucket, this.path, localFile);
			download.addProgressListener((ProgressListener) progressEvent -> {
				if (progressEvent.getEventType() == ProgressEventType.TRANSFER_COMPLETED_EVENT) {
					RemoteDownloader.this.taskListener.getLogger().println("Finished: " + download.getDescription());
				}
			});
			download.waitForCompletion();
		}
		finally {
			mgr.shutdownNow();
		}
		return null;
	}
}
Example 4
Source File: ReadUtils.java    From micro-server with Apache License 2.0 3 votes vote down vote up 
/**
 * Method returns InputStream from S3Object. Multi-part download is used to
 * get file. s3.tmp.dir property used to store temporary files. You can
 * specify temporary file name by using tempFileSupplier object.
 * 
 * @param bucketName
 * @param key
 *            -
 * @param tempFileSupplier
 *            - Supplier providing temporary filenames
 * @return InputStream of
 * @throws AmazonServiceException
 * @throws AmazonClientException
 * @throws InterruptedException
 * @throws IOException
 */
public InputStream getInputStream(String bucketName, String key, Supplier<File> tempFileSupplier)
        throws AmazonServiceException, AmazonClientException, InterruptedException, IOException {
    File file = tempFileSupplier.get();
    try {
        Download download = transferManager.download(bucketName, key, file);
        download.waitForCompletion();
        return new ByteArrayInputStream(FileUtils.readFileToByteArray(file));
    } finally {
        file.delete();
    }
}
Example 5
Source File: ReadUtils.java    From micro-server with Apache License 2.0 3 votes vote down vote up 
/**
 * Return the InputStream for an S3Object. This API download (via multi-part) S3 object to a local file and return a
 * InputStream to that file. The local file will be deleted when the close is called on the stream.
 *
 * @param bucketName S3 bucket name
 * @param key key for the S3Object
 *
 * @return input stream to the downloaded file
 *
 * @throws AmazonClientException
 * @throws InterruptedException
 * @throws IOException
 */
public InputStream getFileInputStream(String bucketName, String key)
    throws AmazonClientException, InterruptedException, IOException {
    File file = createTmpFile();

    Download download = transferManager.download(bucketName, key, file);
    download.waitForCompletion();
    return Files.newInputStream(file.toPath(), StandardOpenOption.DELETE_ON_CLOSE);
}