org.apache.flink.runtime.jobmaster.KvStateLocationOracle Java Examples

The following examples show how to use org.apache.flink.runtime.jobmaster.KvStateLocationOracle. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: KvStateClientProxyImpl.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Override
public void updateKvStateLocationOracle(JobID jobId, @Nullable KvStateLocationOracle kvStateLocationOracle) {
	if (kvStateLocationOracle == null) {
		kvStateLocationOracles.remove(jobId);
	} else {
		kvStateLocationOracles.put(jobId, kvStateLocationOracle);
	}
}
Example #2
Source File: KvStateClientProxyImpl.java    From Flink-CEPplus with Apache License 2.0 5 votes vote down vote up 
@Nullable
@Override
public KvStateLocationOracle getKvStateLocationOracle(JobID jobId) {
	final KvStateLocationOracle legacyKvStateLocationOracle = kvStateLocationOracles.get(HighAvailabilityServices.DEFAULT_JOB_ID);

	// we give preference to the oracle registered under the default job id
	// to make it work with the legacy code paths
	if (legacyKvStateLocationOracle != null) {
		return legacyKvStateLocationOracle;
	} else {
		return kvStateLocationOracles.get(jobId);
	}
}
Example #3
Source File: KvStateClientProxyImpl.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Override
public void updateKvStateLocationOracle(JobID jobId, @Nullable KvStateLocationOracle kvStateLocationOracle) {
	if (kvStateLocationOracle == null) {
		kvStateLocationOracles.remove(jobId);
	} else {
		kvStateLocationOracles.put(jobId, kvStateLocationOracle);
	}
}
Example #4
Source File: KvStateClientProxyImpl.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nullable
@Override
public KvStateLocationOracle getKvStateLocationOracle(JobID jobId) {
	final KvStateLocationOracle legacyKvStateLocationOracle = kvStateLocationOracles.get(HighAvailabilityServices.DEFAULT_JOB_ID);

	// we give preference to the oracle registered under the default job id
	// to make it work with the legacy code paths
	if (legacyKvStateLocationOracle != null) {
		return legacyKvStateLocationOracle;
	} else {
		return kvStateLocationOracles.get(jobId);
	}
}
Example #5
Source File: KvStateClientProxyImpl.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Override
public void updateKvStateLocationOracle(JobID jobId, @Nullable KvStateLocationOracle kvStateLocationOracle) {
	if (kvStateLocationOracle == null) {
		kvStateLocationOracles.remove(jobId);
	} else {
		kvStateLocationOracles.put(jobId, kvStateLocationOracle);
	}
}
Example #6
Source File: KvStateClientProxyImpl.java    From flink with Apache License 2.0 5 votes vote down vote up 
@Nullable
@Override
public KvStateLocationOracle getKvStateLocationOracle(JobID jobId) {
	final KvStateLocationOracle legacyKvStateLocationOracle = kvStateLocationOracles.get(HighAvailabilityServices.DEFAULT_JOB_ID);

	// we give preference to the oracle registered under the default job id
	// to make it work with the legacy code paths
	if (legacyKvStateLocationOracle != null) {
		return legacyKvStateLocationOracle;
	} else {
		return kvStateLocationOracles.get(jobId);
	}
}
Example #7
Source File: KvStateClientProxyHandler.java    From Flink-CEPplus with Apache License 2.0 4 votes vote down vote up 
/**
 * Lookup the {@link KvStateLocation} for the given job and queryable state name.
 *
 * <p>The job manager will be queried for the location only if forced or no
 * cached location can be found. There are no guarantees about
 *
 * @param jobId              JobID the state instance belongs to.
 * @param queryableStateName Name under which the state instance has been published.
 * @param forceUpdate        Flag to indicate whether to force a update via the lookup service.
 * @return Future holding the KvStateLocation
 */
private CompletableFuture<KvStateLocation> getKvStateLookupInfo(
		final JobID jobId,
		final String queryableStateName,
		final boolean forceUpdate) {

	final Tuple2<JobID, String> cacheKey = new Tuple2<>(jobId, queryableStateName);
	final CompletableFuture<KvStateLocation> cachedFuture = lookupCache.get(cacheKey);

	if (!forceUpdate && cachedFuture != null && !cachedFuture.isCompletedExceptionally()) {
		LOG.debug("Retrieving location for state={} of job={} from the cache.", queryableStateName, jobId);
		return cachedFuture;
	}

	final KvStateLocationOracle kvStateLocationOracle = proxy.getKvStateLocationOracle(jobId);

	if (kvStateLocationOracle != null) {
		LOG.debug("Retrieving location for state={} of job={} from the key-value state location oracle.", queryableStateName, jobId);
		final CompletableFuture<KvStateLocation> location = new CompletableFuture<>();
		lookupCache.put(cacheKey, location);

		kvStateLocationOracle
			.requestKvStateLocation(jobId, queryableStateName)
			.whenComplete(
				(KvStateLocation kvStateLocation, Throwable throwable) -> {
					if (throwable != null) {
						if (ExceptionUtils.stripCompletionException(throwable) instanceof FlinkJobNotFoundException) {
							// if the jobId was wrong, remove the entry from the cache.
							lookupCache.remove(cacheKey);
						}
						location.completeExceptionally(throwable);
					} else {
						location.complete(kvStateLocation);
					}
				});

		return location;
	} else {
		return FutureUtils.completedExceptionally(
			new UnknownLocationException(
					"Could not retrieve location of state=" + queryableStateName + " of job=" + jobId +
							". Potential reasons are: i) the state is not ready, or ii) the job does not exist."));
	}
}
Example #8
Source File: KvStateClientProxyHandler.java    From flink with Apache License 2.0 4 votes vote down vote up 
/**
 * Lookup the {@link KvStateLocation} for the given job and queryable state name.
 *
 * <p>The job manager will be queried for the location only if forced or no
 * cached location can be found. There are no guarantees about
 *
 * @param jobId              JobID the state instance belongs to.
 * @param queryableStateName Name under which the state instance has been published.
 * @param forceUpdate        Flag to indicate whether to force a update via the lookup service.
 * @return Future holding the KvStateLocation
 */
private CompletableFuture<KvStateLocation> getKvStateLookupInfo(
		final JobID jobId,
		final String queryableStateName,
		final boolean forceUpdate) {

	final Tuple2<JobID, String> cacheKey = new Tuple2<>(jobId, queryableStateName);
	final CompletableFuture<KvStateLocation> cachedFuture = lookupCache.get(cacheKey);

	if (!forceUpdate && cachedFuture != null && !cachedFuture.isCompletedExceptionally()) {
		LOG.debug("Retrieving location for state={} of job={} from the cache.", queryableStateName, jobId);
		return cachedFuture;
	}

	final KvStateLocationOracle kvStateLocationOracle = proxy.getKvStateLocationOracle(jobId);

	if (kvStateLocationOracle != null) {
		LOG.debug("Retrieving location for state={} of job={} from the key-value state location oracle.", queryableStateName, jobId);
		final CompletableFuture<KvStateLocation> location = new CompletableFuture<>();
		lookupCache.put(cacheKey, location);

		kvStateLocationOracle
			.requestKvStateLocation(jobId, queryableStateName)
			.whenComplete(
				(KvStateLocation kvStateLocation, Throwable throwable) -> {
					if (throwable != null) {
						if (ExceptionUtils.stripCompletionException(throwable) instanceof FlinkJobNotFoundException) {
							// if the jobId was wrong, remove the entry from the cache.
							lookupCache.remove(cacheKey);
						}
						location.completeExceptionally(throwable);
					} else {
						location.complete(kvStateLocation);
					}
				});

		return location;
	} else {
		return FutureUtils.completedExceptionally(
			new UnknownLocationException(
					"Could not retrieve location of state=" + queryableStateName + " of job=" + jobId +
							". Potential reasons are: i) the state is not ready, or ii) the job does not exist."));
	}
}
Example #9
Source File: KvStateClientProxyHandler.java    From flink with Apache License 2.0 4 votes vote down vote up 
/**
 * Lookup the {@link KvStateLocation} for the given job and queryable state name.
 *
 * <p>The job manager will be queried for the location only if forced or no
 * cached location can be found. There are no guarantees about
 *
 * @param jobId              JobID the state instance belongs to.
 * @param queryableStateName Name under which the state instance has been published.
 * @param forceUpdate        Flag to indicate whether to force a update via the lookup service.
 * @return Future holding the KvStateLocation
 */
private CompletableFuture<KvStateLocation> getKvStateLookupInfo(
		final JobID jobId,
		final String queryableStateName,
		final boolean forceUpdate) {

	final Tuple2<JobID, String> cacheKey = new Tuple2<>(jobId, queryableStateName);
	final CompletableFuture<KvStateLocation> cachedFuture = lookupCache.get(cacheKey);

	if (!forceUpdate && cachedFuture != null && !cachedFuture.isCompletedExceptionally()) {
		LOG.debug("Retrieving location for state={} of job={} from the cache.", queryableStateName, jobId);
		return cachedFuture;
	}

	final KvStateLocationOracle kvStateLocationOracle = proxy.getKvStateLocationOracle(jobId);

	if (kvStateLocationOracle != null) {
		LOG.debug("Retrieving location for state={} of job={} from the key-value state location oracle.", queryableStateName, jobId);
		final CompletableFuture<KvStateLocation> location = new CompletableFuture<>();
		lookupCache.put(cacheKey, location);

		kvStateLocationOracle
			.requestKvStateLocation(jobId, queryableStateName)
			.whenComplete(
				(KvStateLocation kvStateLocation, Throwable throwable) -> {
					if (throwable != null) {
						if (ExceptionUtils.stripCompletionException(throwable) instanceof FlinkJobNotFoundException) {
							// if the jobId was wrong, remove the entry from the cache.
							lookupCache.remove(cacheKey);
						}
						location.completeExceptionally(throwable);
					} else {
						location.complete(kvStateLocation);
					}
				});

		return location;
	} else {
		return FutureUtils.completedExceptionally(
			new UnknownLocationException(
					"Could not retrieve location of state=" + queryableStateName + " of job=" + jobId +
							". Potential reasons are: i) the state is not ready, or ii) the job does not exist."));
	}
}
Example #10
Source File: KvStateClientProxy.java    From Flink-CEPplus with Apache License 2.0 2 votes vote down vote up 
/**
 * Updates the active {@link org.apache.flink.runtime.jobmaster.JobMaster Job Manager}
 * in case of change.
 *
 * <p>This is useful in settings where high-availability is enabled and
 * a failed Job Manager is replaced by a new one.
 *
 * <p><b>IMPORTANT: </b> this method may be called by a different thread than
 * the {@link #getKvStateLocationOracle(JobID)}.
 *
 * @param jobId identifying the job for which to update the key-value state location oracle
 * @param kvStateLocationOracle the key-value state location oracle for the given {@link JobID},
 *                                 or null if there is no oracle anymore
 * */
void updateKvStateLocationOracle(
	JobID jobId,
	@Nullable KvStateLocationOracle kvStateLocationOracle);
Example #11
Source File: KvStateClientProxy.java    From Flink-CEPplus with Apache License 2.0 2 votes vote down vote up 
/**
 * Retrieves a future containing the currently leading key-value state location oracle.
 *
 * <p><b>IMPORTANT: </b> this method may be called by a different thread than the
 * {@link #updateKvStateLocationOracle(JobID, KvStateLocationOracle)}.
 *
 * @param jobId identifying the job for which to request the key-value state location oracle
 * @return The key-value state location oracle for the given {@link JobID} or null if none.
 */
@Nullable KvStateLocationOracle getKvStateLocationOracle(JobID jobId);
Example #12
Source File: KvStateClientProxy.java    From flink with Apache License 2.0 2 votes vote down vote up 
/**
 * Updates the active {@link org.apache.flink.runtime.jobmaster.JobMaster Job Manager}
 * in case of change.
 *
 * <p>This is useful in settings where high-availability is enabled and
 * a failed Job Manager is replaced by a new one.
 *
 * <p><b>IMPORTANT: </b> this method may be called by a different thread than
 * the {@link #getKvStateLocationOracle(JobID)}.
 *
 * @param jobId identifying the job for which to update the key-value state location oracle
 * @param kvStateLocationOracle the key-value state location oracle for the given {@link JobID},
 *                                 or null if there is no oracle anymore
 * */
void updateKvStateLocationOracle(
	JobID jobId,
	@Nullable KvStateLocationOracle kvStateLocationOracle);
Example #13
Source File: KvStateClientProxy.java    From flink with Apache License 2.0 2 votes vote down vote up 
/**
 * Retrieves a future containing the currently leading key-value state location oracle.
 *
 * <p><b>IMPORTANT: </b> this method may be called by a different thread than the
 * {@link #updateKvStateLocationOracle(JobID, KvStateLocationOracle)}.
 *
 * @param jobId identifying the job for which to request the key-value state location oracle
 * @return The key-value state location oracle for the given {@link JobID} or null if none.
 */
@Nullable KvStateLocationOracle getKvStateLocationOracle(JobID jobId);
Example #14
Source File: KvStateClientProxy.java    From flink with Apache License 2.0 2 votes vote down vote up 
/**
 * Updates the active {@link org.apache.flink.runtime.jobmaster.JobMaster Job Manager}
 * in case of change.
 *
 * <p>This is useful in settings where high-availability is enabled and
 * a failed Job Manager is replaced by a new one.
 *
 * <p><b>IMPORTANT: </b> this method may be called by a different thread than
 * the {@link #getKvStateLocationOracle(JobID)}.
 *
 * @param jobId identifying the job for which to update the key-value state location oracle
 * @param kvStateLocationOracle the key-value state location oracle for the given {@link JobID},
 *                                 or null if there is no oracle anymore
 * */
void updateKvStateLocationOracle(
	JobID jobId,
	@Nullable KvStateLocationOracle kvStateLocationOracle);
Example #15
Source File: KvStateClientProxy.java    From flink with Apache License 2.0 2 votes vote down vote up 
/**
 * Retrieves a future containing the currently leading key-value state location oracle.
 *
 * <p><b>IMPORTANT: </b> this method may be called by a different thread than the
 * {@link #updateKvStateLocationOracle(JobID, KvStateLocationOracle)}.
 *
 * @param jobId identifying the job for which to request the key-value state location oracle
 * @return The key-value state location oracle for the given {@link JobID} or null if none.
 */
@Nullable KvStateLocationOracle getKvStateLocationOracle(JobID jobId);