Java Code Examples for org.apache.flink.table.catalog.stats.CatalogTableStatistics#getTotalSize()

The following examples show how to use org.apache.flink.table.catalog.stats.CatalogTableStatistics#getTotalSize() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: HiveCatalog.java    From flink with Apache License 2.0 6 votes vote down vote up 
/**
 * Determine if statistics is need to be updated, if it needs to be updated and updated its parameters.
 * @param statistics original ``hive table statistics.
 * @param parameters new catalog table statistics parameters.
 * @return needUpdateStatistics flag which indicates whether need to update stats.
 */
private static boolean compareAndUpdateStatisticsProperties(CatalogTableStatistics statistics, Map<String, String> parameters) {
	boolean needUpdateStatistics;
	String oldRowCount = parameters.getOrDefault(StatsSetupConst.ROW_COUNT, HiveStatsUtil.DEFAULT_STATS_ZERO_CONST);
	String oldTotalSize = parameters.getOrDefault(StatsSetupConst.TOTAL_SIZE, HiveStatsUtil.DEFAULT_STATS_ZERO_CONST);
	String oldNumFiles = parameters.getOrDefault(StatsSetupConst.NUM_FILES, HiveStatsUtil.DEFAULT_STATS_ZERO_CONST);
	String oldRawDataSize = parameters.getOrDefault(StatsSetupConst.RAW_DATA_SIZE, HiveStatsUtil.DEFAULT_STATS_ZERO_CONST);
	needUpdateStatistics = statistics.getRowCount() != Long.parseLong(oldRowCount) || statistics.getTotalSize() != Long.parseLong(oldTotalSize)
		|| statistics.getFileCount() != Integer.parseInt(oldNumFiles) || statistics.getRawDataSize() != Long.parseLong(oldRawDataSize);
	if (needUpdateStatistics) {
		parameters.put(StatsSetupConst.ROW_COUNT, String.valueOf(statistics.getRowCount()));
		parameters.put(StatsSetupConst.TOTAL_SIZE, String.valueOf(statistics.getTotalSize()));
		parameters.put(StatsSetupConst.NUM_FILES, String.valueOf(statistics.getFileCount()));
		parameters.put(StatsSetupConst.RAW_DATA_SIZE, String.valueOf(statistics.getRawDataSize()));
	}
	return needUpdateStatistics;
}
Example 2
Source File: HiveCatalog.java    From flink with Apache License 2.0 5 votes vote down vote up 
/**
 * Determine if statistics need to be updated or not.
 * @param newTableStats   new catalog table statistics.
 * @param parameters      original hive table statistics parameters.
 * @return                whether need to update stats.
 */
private boolean statsChanged(CatalogTableStatistics newTableStats, Map<String, String> parameters) {
	return newTableStats.getRowCount() != parsePositiveLongStat(parameters, StatsSetupConst.ROW_COUNT)
			|| newTableStats.getTotalSize() != parsePositiveLongStat(parameters, StatsSetupConst.TOTAL_SIZE)
			|| newTableStats.getFileCount() != parsePositiveIntStat(parameters, StatsSetupConst.NUM_FILES)
			|| newTableStats.getRawDataSize() != parsePositiveLongStat(parameters, StatsSetupConst.NUM_FILES);
}