org.apache.hadoop.yarn.webapp.log.AggregatedLogsBlock Java Examples

The following examples show how to use org.apache.hadoop.yarn.webapp.log.AggregatedLogsBlock. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: LogCLIHelpers.java    From hadoop with Apache License 2.0 4 votes vote down vote up 
@Private
@VisibleForTesting
public int dumpAContainersLogs(String appId, String containerId,
    String nodeId, String jobOwner) throws IOException {

  ApplicationId applicationId = ConverterUtils.toApplicationId(appId);
  List<Path> remoteAppLogDirs = AggregatedLogsBlock.getRemoteAppLogDirs(getConf(), applicationId, jobOwner);
  String remoteAppLogDir = StringUtils.join(remoteAppLogDirs, ",");

  RemoteIterator<FileStatus> nodeFiles;
  try {
    nodeFiles = AggregatedLogsBlock.getFileListAtRemoteAppDir(getConf(), remoteAppLogDirs, applicationId, jobOwner);
  } catch (FileNotFoundException fnf) {
    logDirNotExist(remoteAppLogDir.toString());
    return -1;
  }
  boolean foundContainerLogs = false;
  while (nodeFiles.hasNext()) {
    FileStatus thisNodeFile = nodeFiles.next();
    String fileName = thisNodeFile.getPath().getName();
    if (fileName.contains(LogAggregationUtils.getNodeString(nodeId))
        && !fileName.endsWith(LogAggregationUtils.TMP_FILE_SUFFIX)) {
      AggregatedLogFormat.LogReader reader = null;
      try {
        reader =
            new AggregatedLogFormat.LogReader(getConf(),
              thisNodeFile.getPath());
        if (dumpAContainerLogs(containerId, reader, System.out,
            thisNodeFile.getModificationTime()) > -1) {
          foundContainerLogs = true;
        }
      } finally {
        if (reader != null) {
          reader.close();
        }
      }
    }
  }
  if (!foundContainerLogs) {
    containerLogNotFound(containerId);
    return -1;
  }
  return 0;
}
Example #2
Source File: LogCLIHelpers.java    From hadoop with Apache License 2.0 4 votes vote down vote up 
@Private
public int dumpAllContainersLogs(ApplicationId appId, String appOwner,
    PrintStream out) throws IOException {

  List<Path> remoteAppLogDirs = AggregatedLogsBlock.getRemoteAppLogDirs(getConf(), appId, appOwner);
  String remoteAppLogDir = StringUtils.join(remoteAppLogDirs, ",");
  RemoteIterator<FileStatus> nodeFiles;
  try {
    nodeFiles = AggregatedLogsBlock.getFileListAtRemoteAppDir(getConf(), remoteAppLogDirs, appId, appOwner);
  } catch (FileNotFoundException fnf) {
    logDirNotExist(remoteAppLogDir.toString());
    return -1;
  }
  boolean foundAnyLogs = false;
  while (nodeFiles.hasNext()) {
    FileStatus thisNodeFile = nodeFiles.next();
    if (!thisNodeFile.getPath().getName()
      .endsWith(LogAggregationUtils.TMP_FILE_SUFFIX)) {
      AggregatedLogFormat.LogReader reader =
          new AggregatedLogFormat.LogReader(getConf(), thisNodeFile.getPath());
      try {

        DataInputStream valueStream;
        LogKey key = new LogKey();
        valueStream = reader.next(key);

        while (valueStream != null) {

          String containerString =
              "\n\nContainer: " + key + " on " + thisNodeFile.getPath().getName();
          out.println(containerString);
          out.println(StringUtils.repeat("=", containerString.length()));
          while (true) {
            try {
              LogReader.readAContainerLogsForALogType(valueStream, out,
                thisNodeFile.getModificationTime());
              foundAnyLogs = true;
            } catch (EOFException eof) {
              break;
            }
          }

          // Next container
          key = new LogKey();
          valueStream = reader.next(key);
        }
      } finally {
        reader.close();
      }
    }
  }
  if (! foundAnyLogs) {
    emptyLogDir(remoteAppLogDir.toString());
    return -1;
  }
  return 0;
}
Example #3
Source File: HsLogsPage.java    From XLearning with Apache License 2.0 2 votes vote down vote up 
/**
 * The content of this page is the JobBlock
 *
 * @return HsJobBlock.class
 */
@Override
protected Class<? extends SubView> content() {
  return AggregatedLogsBlock.class;
}
Example #4
Source File: AHSLogsPage.java    From hadoop with Apache License 2.0 2 votes vote down vote up 
/**
 * The content of this page is the AggregatedLogsBlock
 * 
 * @return AggregatedLogsBlock.class
 */
@Override
protected Class<? extends SubView> content() {
  return AggregatedLogsBlock.class;
}
Example #5
Source File: HsLogsPage.java    From hadoop with Apache License 2.0 2 votes vote down vote up 
/**
 * The content of this page is the JobBlock
 * @return HsJobBlock.class
 */
@Override protected Class<? extends SubView> content() {
  return AggregatedLogsBlock.class;
}
Example #6
Source File: AHSLogsPage.java    From big-c with Apache License 2.0 2 votes vote down vote up 
/**
 * The content of this page is the AggregatedLogsBlock
 * 
 * @return AggregatedLogsBlock.class
 */
@Override
protected Class<? extends SubView> content() {
  return AggregatedLogsBlock.class;
}
Example #7
Source File: HsLogsPage.java    From big-c with Apache License 2.0 2 votes vote down vote up 
/**
 * The content of this page is the JobBlock
 * @return HsJobBlock.class
 */
@Override protected Class<? extends SubView> content() {
  return AggregatedLogsBlock.class;
}