Java Code Examples for org.apache.tez.dag.api.TezConfiguration#DAG_RECOVERY_DATA_DIR_NAME

The following examples show how to use org.apache.tez.dag.api.TezConfiguration#DAG_RECOVERY_DATA_DIR_NAME . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: TestTezCommonUtils.java    From incubator-tez with Apache License 2.0 5 votes vote down vote up 
@Test
public void testTezRecoveryStagingPath() throws Exception {
  String strAppId = "testAppId";
  Path stageDir = TezCommonUtils.getTezSystemStagingPath(conf, strAppId);
  Path confStageDir = TezCommonUtils.getRecoveryPath(stageDir, conf);
  String expectedDir = RESOLVED_STAGE_DIR + File.separatorChar
      + TezCommonUtils.TEZ_SYSTEM_SUB_DIR + File.separatorChar + strAppId + File.separator
      + TezConfiguration.DAG_RECOVERY_DATA_DIR_NAME;
  Assert.assertEquals(confStageDir.toString(), expectedDir);
}
Example 2
Source File: TestTezCommonUtils.java    From incubator-tez with Apache License 2.0 5 votes vote down vote up 
@Test
public void testTezAttemptRecoveryStagingPath() throws Exception {
  String strAppId = "testAppId";
  Path stageDir = TezCommonUtils.getTezSystemStagingPath(conf, strAppId);
  Path recoveryPath = TezCommonUtils.getRecoveryPath(stageDir, conf);
  Path recoveryStageDir = TezCommonUtils.getAttemptRecoveryPath(recoveryPath, 2);

  String expectedDir = RESOLVED_STAGE_DIR + File.separatorChar
      + TezCommonUtils.TEZ_SYSTEM_SUB_DIR + File.separatorChar + strAppId + File.separator
      + TezConfiguration.DAG_RECOVERY_DATA_DIR_NAME + File.separator + "2";
  Assert.assertEquals(recoveryStageDir.toString(), expectedDir);
}
Example 3
Source File: TestTezCommonUtils.java    From incubator-tez with Apache License 2.0 5 votes vote down vote up 
@Test
public void testTezDAGRecoveryStagingPath() throws Exception {
  String strAppId = "testAppId";
  Path stageDir = TezCommonUtils.getTezSystemStagingPath(conf, strAppId);
  Path recoveryPath = TezCommonUtils.getRecoveryPath(stageDir, conf);
  Path recoveryStageDir = TezCommonUtils.getAttemptRecoveryPath(recoveryPath, 2);

  Path dagRecoveryPathj = TezCommonUtils.getDAGRecoveryPath(recoveryStageDir, "dag_123");

  String expectedDir = RESOLVED_STAGE_DIR + File.separatorChar
      + TezCommonUtils.TEZ_SYSTEM_SUB_DIR + File.separatorChar + strAppId + File.separator
      + TezConfiguration.DAG_RECOVERY_DATA_DIR_NAME + File.separator + "2" + File.separator
      + "dag_123" + TezConfiguration.DAG_RECOVERY_RECOVER_FILE_SUFFIX;
  Assert.assertEquals(dagRecoveryPathj.toString(), expectedDir);
}
Example 4
Source File: TestTezCommonUtils.java    From incubator-tez with Apache License 2.0 5 votes vote down vote up 
@Test
public void testTezSummaryRecoveryStagingPath() throws Exception {
  String strAppId = "testAppId";
  Path stageDir = TezCommonUtils.getTezSystemStagingPath(conf, strAppId);
  Path recoveryPath = TezCommonUtils.getRecoveryPath(stageDir, conf);
  Path recoveryStageDir = TezCommonUtils.getAttemptRecoveryPath(recoveryPath, 2);
  Path summaryRecoveryPathj = TezCommonUtils.getSummaryRecoveryPath(recoveryStageDir);

  String expectedDir = RESOLVED_STAGE_DIR + File.separatorChar
      + TezCommonUtils.TEZ_SYSTEM_SUB_DIR + File.separatorChar + strAppId + File.separator
      + TezConfiguration.DAG_RECOVERY_DATA_DIR_NAME + File.separator + "2" + File.separator
      + TezConfiguration.DAG_RECOVERY_SUMMARY_FILE_SUFFIX;
  Assert.assertEquals(summaryRecoveryPathj.toString(), expectedDir);
}
Example 5
Source File: TezCommonUtils.java    From incubator-tez with Apache License 2.0 3 votes vote down vote up 
/**
 * <p>
 * Returns a path to store recovery information
 * </p>
 * 
 * @param tezSysStagingPath
 *          TEZ system level staging directory used for Tez internals
 * @param conf
 *          Tez configuration
 * @return App recovery path
 * @throws IOException
 */
@Private
public static Path getRecoveryPath(Path tezSysStagingPath, Configuration conf)
    throws IOException {
  Path baseReecoveryPath = new Path(tezSysStagingPath,
      TezConfiguration.DAG_RECOVERY_DATA_DIR_NAME);
  FileSystem recoveryFS = baseReecoveryPath.getFileSystem(conf);
  return recoveryFS.makeQualified(baseReecoveryPath);
}