Java Code Examples for org.apache.hadoop.hdfs.server.common.StorageInfo#getLayoutVersion()

The following examples show how to use org.apache.hadoop.hdfs.server.common.StorageInfo#getLayoutVersion() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: NNUpgradeUtil.java    From hadoop with Apache License 2.0 6 votes vote down vote up 
/**
 * Return true if this storage dir can roll back to the previous storage
 * state, false otherwise. The NN will refuse to run the rollback operation
 * unless at least one JM or fsimage storage directory can roll back.
 * 
 * @param storage the storage info for the current state
 * @param prevStorage the storage info for the previous (unupgraded) state
 * @param targetLayoutVersion the layout version we intend to roll back to
 * @return true if this JM can roll back, false otherwise.
 * @throws IOException in the event of error
 */
static boolean canRollBack(StorageDirectory sd, StorageInfo storage,
    StorageInfo prevStorage, int targetLayoutVersion) throws IOException {
  File prevDir = sd.getPreviousDir();
  if (!prevDir.exists()) {  // use current directory then
    LOG.info("Storage directory " + sd.getRoot()
             + " does not contain previous fs state.");
    // read and verify consistency with other directories
    storage.readProperties(sd);
    return false;
  }

  // read and verify consistency of the prev dir
  prevStorage.readPreviousVersionProperties(sd);

  if (prevStorage.getLayoutVersion() != targetLayoutVersion) {
    throw new IOException(
      "Cannot rollback to storage version " +
      prevStorage.getLayoutVersion() +
      " using this version of the NameNode, which uses storage version " +
      targetLayoutVersion + ". " +
      "Please use the previous version of HDFS to perform the rollback.");
  }
  
  return true;
}
Example 2
Source File: NNUpgradeUtil.java    From big-c with Apache License 2.0 6 votes vote down vote up 
/**
 * Return true if this storage dir can roll back to the previous storage
 * state, false otherwise. The NN will refuse to run the rollback operation
 * unless at least one JM or fsimage storage directory can roll back.
 * 
 * @param storage the storage info for the current state
 * @param prevStorage the storage info for the previous (unupgraded) state
 * @param targetLayoutVersion the layout version we intend to roll back to
 * @return true if this JM can roll back, false otherwise.
 * @throws IOException in the event of error
 */
static boolean canRollBack(StorageDirectory sd, StorageInfo storage,
    StorageInfo prevStorage, int targetLayoutVersion) throws IOException {
  File prevDir = sd.getPreviousDir();
  if (!prevDir.exists()) {  // use current directory then
    LOG.info("Storage directory " + sd.getRoot()
             + " does not contain previous fs state.");
    // read and verify consistency with other directories
    storage.readProperties(sd);
    return false;
  }

  // read and verify consistency of the prev dir
  prevStorage.readPreviousVersionProperties(sd);

  if (prevStorage.getLayoutVersion() != targetLayoutVersion) {
    throw new IOException(
      "Cannot rollback to storage version " +
      prevStorage.getLayoutVersion() +
      " using this version of the NameNode, which uses storage version " +
      targetLayoutVersion + ". " +
      "Please use the previous version of HDFS to perform the rollback.");
  }
  
  return true;
}
Example 3
Source File: TestDFSStartupVersions.java    From RDFS with Apache License 2.0 5 votes vote down vote up 
/**
 * Determines if the given Namenode version and Datanode version
 * are compatible with each other. Compatibility in this case mean
 * that the Namenode and Datanode will successfully start up and
 * will work together. The rules for compatibility,
 * taken from the DFS Upgrade Design, are as follows:
 * <pre>
 * 1. The data-node does regular startup (no matter which options 
 *    it is started with) if
 *       softwareLV == storedLV AND 
 *       DataNode.FSSCTime == NameNode.FSSCTime
 * 2. The data-node performs an upgrade if it is started without any 
 *    options and
 *       |softwareLV| > |storedLV| OR 
 *       (softwareLV == storedLV AND
 *        DataNode.FSSCTime < NameNode.FSSCTime)
 * 3. NOT TESTED: The data-node rolls back if it is started with
 *    the -rollback option and
 *       |softwareLV| >= |previous.storedLV| AND 
 *       DataNode.previous.FSSCTime <= NameNode.FSSCTime
 * 4. In all other cases the startup fails.
 * </pre>
 */
boolean isVersionCompatible(StorageInfo namenodeVer, StorageInfo datanodeVer) {
  // check #0
  if (namenodeVer.getNamespaceID() != datanodeVer.getNamespaceID()) {
    LOG.info("namespaceIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #1
  int softwareLV = FSConstants.LAYOUT_VERSION;  // will also be Namenode's LV
  int storedLV = datanodeVer.getLayoutVersion();
  if (softwareLV == storedLV &&  
      datanodeVer.getCTime() == namenodeVer.getCTime()) 
    {
      LOG.info("layoutVersions and cTimes are equal: isVersionCompatible=true");
      return true;
    }
  // check #2
  long absSoftwareLV = Math.abs((long)softwareLV);
  long absStoredLV = Math.abs((long)storedLV);
  if (absSoftwareLV > absStoredLV ||
      (softwareLV == storedLV &&
       datanodeVer.getCTime() < namenodeVer.getCTime())) 
    {
      LOG.info("softwareLayoutVersion is newer OR namenode cTime is newer: isVersionCompatible=true");
      return true;
    }
  // check #4
  LOG.info("default case: isVersionCompatible=false");
  return false;
}
Example 4
Source File: TestDFSStartupVersions.java    From hadoop-gpu with Apache License 2.0 5 votes vote down vote up 
/**
 * Determines if the given Namenode version and Datanode version
 * are compatible with each other. Compatibility in this case mean
 * that the Namenode and Datanode will successfully start up and
 * will work together. The rules for compatibility,
 * taken from the DFS Upgrade Design, are as follows:
 * <pre>
 * 1. The data-node does regular startup (no matter which options 
 *    it is started with) if
 *       softwareLV == storedLV AND 
 *       DataNode.FSSCTime == NameNode.FSSCTime
 * 2. The data-node performs an upgrade if it is started without any 
 *    options and
 *       |softwareLV| > |storedLV| OR 
 *       (softwareLV == storedLV AND
 *        DataNode.FSSCTime < NameNode.FSSCTime)
 * 3. NOT TESTED: The data-node rolls back if it is started with
 *    the -rollback option and
 *       |softwareLV| >= |previous.storedLV| AND 
 *       DataNode.previous.FSSCTime <= NameNode.FSSCTime
 * 4. In all other cases the startup fails.
 * </pre>
 */
boolean isVersionCompatible(StorageInfo namenodeVer, StorageInfo datanodeVer) {
  // check #0
  if (namenodeVer.getNamespaceID() != datanodeVer.getNamespaceID()) {
    LOG.info("namespaceIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #1
  int softwareLV = FSConstants.LAYOUT_VERSION;  // will also be Namenode's LV
  int storedLV = datanodeVer.getLayoutVersion();
  if (softwareLV == storedLV &&  
      datanodeVer.getCTime() == namenodeVer.getCTime()) 
    {
      LOG.info("layoutVersions and cTimes are equal: isVersionCompatible=true");
      return true;
    }
  // check #2
  long absSoftwareLV = Math.abs((long)softwareLV);
  long absStoredLV = Math.abs((long)storedLV);
  if (absSoftwareLV > absStoredLV ||
      (softwareLV == storedLV &&
       datanodeVer.getCTime() < namenodeVer.getCTime())) 
    {
      LOG.info("softwareLayoutVersion is newer OR namenode cTime is newer: isVersionCompatible=true");
      return true;
    }
  // check #4
  LOG.info("default case: isVersionCompatible=false");
  return false;
}
Example 5
Source File: TestDFSStartupVersions.java    From hadoop with Apache License 2.0 4 votes vote down vote up 
/**
 * Determines if the given Namenode version and Datanode version
 * are compatible with each other. Compatibility in this case mean
 * that the Namenode and Datanode will successfully start up and
 * will work together. The rules for compatibility,
 * taken from the DFS Upgrade Design, are as follows:
 * <pre>
 * <ol>
 * <li>Check 0: Datanode namespaceID != Namenode namespaceID the startup fails
 * </li>
 * <li>Check 1: Datanode clusterID != Namenode clusterID the startup fails
 * </li>
 * <li>Check 2: Datanode blockPoolID != Namenode blockPoolID the startup fails
 * </li>
 * <li>Check 3: The data-node does regular startup (no matter which options 
 *    it is started with) if
 *       softwareLV == storedLV AND 
 *       DataNode.FSSCTime == NameNode.FSSCTime
 * </li>
 * <li>Check 4: The data-node performs an upgrade if it is started without any 
 *    options and
 *       |softwareLV| > |storedLV| OR 
 *       (softwareLV == storedLV AND
 *        DataNode.FSSCTime < NameNode.FSSCTime)
 * </li>
 * <li>NOT TESTED: The data-node rolls back if it is started with
 *    the -rollback option and
 *       |softwareLV| >= |previous.storedLV| AND 
 *       DataNode.previous.FSSCTime <= NameNode.FSSCTime
 * </li>
 * <li>Check 5: In all other cases the startup fails.</li>
 * </ol>
 * </pre>
 */
boolean isVersionCompatible(StorageData namenodeSd, StorageData datanodeSd) {
  final StorageInfo namenodeVer = namenodeSd.storageInfo;
  final StorageInfo datanodeVer = datanodeSd.storageInfo;
  // check #0
  if (namenodeVer.getNamespaceID() != datanodeVer.getNamespaceID()) {
    LOG.info("namespaceIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #1
  if (!namenodeVer.getClusterID().equals(datanodeVer.getClusterID())) {
    LOG.info("clusterIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #2
  if (!namenodeSd.blockPoolId.equals(datanodeSd.blockPoolId)) {
    LOG.info("blockPoolIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #3
  int softwareLV = HdfsConstants.DATANODE_LAYOUT_VERSION;
  int storedLV = datanodeVer.getLayoutVersion();
  if (softwareLV == storedLV &&  
      datanodeVer.getCTime() == namenodeVer.getCTime()) 
    {
      LOG.info("layoutVersions and cTimes are equal: isVersionCompatible=true");
      return true;
    }
  // check #4
  long absSoftwareLV = Math.abs((long)softwareLV);
  long absStoredLV = Math.abs((long)storedLV);
  if (absSoftwareLV > absStoredLV ||
      (softwareLV == storedLV &&
       datanodeVer.getCTime() < namenodeVer.getCTime())) 
    {
      LOG.info("softwareLayoutVersion is newer OR namenode cTime is newer: isVersionCompatible=true");
      return true;
    }
  // check #5
  LOG.info("default case: isVersionCompatible=false");
  return false;
}
Example 6
Source File: TestDFSStartupVersions.java    From big-c with Apache License 2.0 4 votes vote down vote up 
/**
 * Determines if the given Namenode version and Datanode version
 * are compatible with each other. Compatibility in this case mean
 * that the Namenode and Datanode will successfully start up and
 * will work together. The rules for compatibility,
 * taken from the DFS Upgrade Design, are as follows:
 * <pre>
 * <ol>
 * <li>Check 0: Datanode namespaceID != Namenode namespaceID the startup fails
 * </li>
 * <li>Check 1: Datanode clusterID != Namenode clusterID the startup fails
 * </li>
 * <li>Check 2: Datanode blockPoolID != Namenode blockPoolID the startup fails
 * </li>
 * <li>Check 3: The data-node does regular startup (no matter which options 
 *    it is started with) if
 *       softwareLV == storedLV AND 
 *       DataNode.FSSCTime == NameNode.FSSCTime
 * </li>
 * <li>Check 4: The data-node performs an upgrade if it is started without any 
 *    options and
 *       |softwareLV| > |storedLV| OR 
 *       (softwareLV == storedLV AND
 *        DataNode.FSSCTime < NameNode.FSSCTime)
 * </li>
 * <li>NOT TESTED: The data-node rolls back if it is started with
 *    the -rollback option and
 *       |softwareLV| >= |previous.storedLV| AND 
 *       DataNode.previous.FSSCTime <= NameNode.FSSCTime
 * </li>
 * <li>Check 5: In all other cases the startup fails.</li>
 * </ol>
 * </pre>
 */
boolean isVersionCompatible(StorageData namenodeSd, StorageData datanodeSd) {
  final StorageInfo namenodeVer = namenodeSd.storageInfo;
  final StorageInfo datanodeVer = datanodeSd.storageInfo;
  // check #0
  if (namenodeVer.getNamespaceID() != datanodeVer.getNamespaceID()) {
    LOG.info("namespaceIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #1
  if (!namenodeVer.getClusterID().equals(datanodeVer.getClusterID())) {
    LOG.info("clusterIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #2
  if (!namenodeSd.blockPoolId.equals(datanodeSd.blockPoolId)) {
    LOG.info("blockPoolIDs are not equal: isVersionCompatible=false");
    return false;
  }
  // check #3
  int softwareLV = HdfsConstants.DATANODE_LAYOUT_VERSION;
  int storedLV = datanodeVer.getLayoutVersion();
  if (softwareLV == storedLV &&  
      datanodeVer.getCTime() == namenodeVer.getCTime()) 
    {
      LOG.info("layoutVersions and cTimes are equal: isVersionCompatible=true");
      return true;
    }
  // check #4
  long absSoftwareLV = Math.abs((long)softwareLV);
  long absStoredLV = Math.abs((long)storedLV);
  if (absSoftwareLV > absStoredLV ||
      (softwareLV == storedLV &&
       datanodeVer.getCTime() < namenodeVer.getCTime())) 
    {
      LOG.info("softwareLayoutVersion is newer OR namenode cTime is newer: isVersionCompatible=true");
      return true;
    }
  // check #5
  LOG.info("default case: isVersionCompatible=false");
  return false;
}