Java Code Examples for it.unimi.dsi.fastutil.objects.Object2IntOpenHashMap#trim()

The following examples show how to use it.unimi.dsi.fastutil.objects.Object2IntOpenHashMap#trim() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: CategoriesToWIDMap.java    From tagme with Apache License 2.0 5 votes vote down vote up 
@Override
protected Object2IntMap<String> parseSet() throws IOException
{
	final Object2IntOpenHashMap<String> map = new Object2IntOpenHashMap<String>(3000000);
	File input = WikipediaFiles.TITLES.getSourceFile(lang);
	
	SQLWikiParser parser = new SQLWikiParser(log) {
		@Override
		public boolean compute(ArrayList<String> values) throws IOException {
			if (values.get(SQLWikiParser.PAGE_NS).equals(SQLWikiParser.NS_CATEGORY_STRING)){
				
				String category = cleanPageName(values.get(SQLWikiParser.PAGE_TITLE));
				map.put(category, Integer.parseInt(values.get(SQLWikiParser.PAGE_ID)));
				return true;
			} else return false;
		}
		
	};
	InputStreamReader reader = new InputStreamReader(new FileInputStream(input), Charset.forName("UTF-8"));
	parser.compute(reader);
	reader.close();
	
	map.defaultReturnValue(-1);
	map.trim();
	
	return map;
}
Example 2
Source File: TitlesToWIDMap.java    From tagme with Apache License 2.0 5 votes vote down vote up 
@Override
protected Object2IntMap<String> parseSet() throws IOException
{
	final Object2IntOpenHashMap<String> map = new Object2IntOpenHashMap<String>(3000000);
	File input = WikipediaFiles.TITLES.getSourceFile(lang);
	
	SQLWikiParser parser = new SQLWikiParser(log) {
		@Override
		public boolean compute(ArrayList<String> values) throws IOException {
			if (values.get(SQLWikiParser.PAGE_NS).equals(SQLWikiParser.NS_ARTICLE_STRING)){
				
				String title = cleanPageName(values.get(SQLWikiParser.PAGE_TITLE));
				map.put(title, Integer.parseInt(values.get(SQLWikiParser.PAGE_ID)));
				return true;
			} else return false;
		}
		
	};
	InputStreamReader reader = new InputStreamReader(new FileInputStream(input), Charset.forName("UTF-8"));
	parser.compute(reader);
	reader.close();
	
	map.defaultReturnValue(-1);
	map.trim();
	
	return map;
}