org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter Java Examples

The following examples show how to use org.apache.lucene.analysis.compound.HyphenationCompoundWordTokenFilter. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example #1

Source File: HyphenationCompoundWordTokenFilterFactory.java From Elasticsearch with Apache License 2.0

6 votes

@Inject
public HyphenationCompoundWordTokenFilterFactory(Index index, IndexSettingsService indexSettingsService, Environment env, @Assisted String name, @Assisted Settings settings) {
    super(index, indexSettingsService.getSettings(), env, name, settings);

    String hyphenationPatternsPath = settings.get("hyphenation_patterns_path", null);
    if (hyphenationPatternsPath == null) {
        throw new IllegalArgumentException("hyphenation_patterns_path is a required setting.");
    }

    Path hyphenationPatternsFile = env.configFile().resolve(hyphenationPatternsPath);

    try {
        hyphenationTree = HyphenationCompoundWordTokenFilter.getHyphenationTree(new InputSource(Files.newInputStream(hyphenationPatternsFile)));
    } catch (Exception e) {
        throw new IllegalArgumentException("Exception while reading hyphenation_patterns_path: " + e.getMessage());
    }
}

Example #2

Source File: HyphenationCompoundWordTokenFilterFactory.java From crate with Apache License 2.0

6 votes

HyphenationCompoundWordTokenFilterFactory(IndexSettings indexSettings, Environment env, String name, Settings settings) {
    super(indexSettings, env, name, settings);

    String hyphenationPatternsPath = settings.get("hyphenation_patterns_path", null);
    if (hyphenationPatternsPath == null) {
        throw new IllegalArgumentException("hyphenation_patterns_path is a required setting.");
    }

    Path hyphenationPatternsFile = env.configFile().resolve(hyphenationPatternsPath);

    try {
        InputStream in = Files.newInputStream(hyphenationPatternsFile);
        hyphenationTree = HyphenationCompoundWordTokenFilter.getHyphenationTree(new InputSource(in));
    } catch (Exception e) {
        throw new IllegalArgumentException("Exception while reading hyphenation_patterns_path.", e);
    }
}

Example #3

Source File: HyphenationCompoundWordTokenFilterFactory.java From Elasticsearch with Apache License 2.0

5 votes

@Override
public TokenStream create(TokenStream tokenStream) {
    if (version.onOrAfter(Version.LUCENE_4_4_0)) {
        return new HyphenationCompoundWordTokenFilter(tokenStream, hyphenationTree, wordList, minWordSize, 
                                                      minSubwordSize, maxSubwordSize, onlyLongestMatch);
    } else {
        return new Lucene43HyphenationCompoundWordTokenFilter(tokenStream, hyphenationTree, wordList, minWordSize, 
                minSubwordSize, maxSubwordSize, onlyLongestMatch);
    }
}

Example #4

Source File: HyphenationCompoundWordTokenFilterFactory.java From crate with Apache License 2.0

4 votes

@Override
public TokenStream create(TokenStream tokenStream) {
    return new HyphenationCompoundWordTokenFilter(tokenStream, hyphenationTree, wordList, minWordSize,
                                                  minSubwordSize, maxSubwordSize, onlyLongestMatch);
}