Java Code Examples for org.elasticsearch.common.util.CollectionUtils#sortAndDedup()

The following examples show how to use org.elasticsearch.common.util.CollectionUtils#sortAndDedup() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.

Example 1

Source File: BinaryFieldMapper.java From Elasticsearch with Apache License 2.0

6 votes

@Override
public BytesRef binaryValue() {
    try {
        CollectionUtils.sortAndDedup(bytesList);
        int size = bytesList.size();
        final byte[] bytes = new byte[totalSize + (size + 1) * 5];
        ByteArrayDataOutput out = new ByteArrayDataOutput(bytes);
        out.writeVInt(size);  // write total number of values
        for (int i = 0; i < size; i ++) {
            final byte[] value = bytesList.get(i);
            int valueLength = value.length;
            out.writeVInt(valueLength);
            out.writeBytes(value, 0, valueLength);
        }
        return new BytesRef(bytes, 0, out.getPosition());
    } catch (IOException e) {
        throw new ElasticsearchException("Failed to get binary value", e);
    }

}

Example 2

Source File: NumberFieldMapper.java From Elasticsearch with Apache License 2.0

6 votes

@Override
public BytesRef binaryValue() {
    CollectionUtils.sortAndDedup(values);

    // here is the trick:
    //  - the first value is zig-zag encoded so that eg. -5 would become positive and would be better compressed by vLong
    //  - for other values, we only encode deltas using vLong
    final byte[] bytes = new byte[values.size() * ByteUtils.MAX_BYTES_VLONG];
    final ByteArrayDataOutput out = new ByteArrayDataOutput(bytes);
    ByteUtils.writeVLong(out, ByteUtils.zigZagEncode(values.get(0)));
    for (int i = 1; i < values.size(); ++i) {
        final long delta = values.get(i) - values.get(i - 1);
        ByteUtils.writeVLong(out, delta);
    }
    return new BytesRef(bytes, 0, out.getPosition());
}

Example 3

Source File: MinHashFieldMapper.java From elasticsearch-minhash with Apache License 2.0

6 votes

@Override
public BytesRef binaryValue() {
    try {
        CollectionUtils.sortAndDedup(bytesList);
        final int size = bytesList.size();
        final byte[] bytes = new byte[totalSize + (size + 1) * 5];
        final ByteArrayDataOutput out = new ByteArrayDataOutput(bytes);
        out.writeVInt(size); // write total number of values
        for (int i = 0; i < size; i++) {
            final byte[] value = bytesList.get(i);
            final int valueLength = value.length;
            out.writeVInt(valueLength);
            out.writeBytes(value, 0, valueLength);
        }
        return new BytesRef(bytes, 0, out.getPosition());
    } catch (final IOException e) {
        throw new ElasticsearchException("Failed to get MinHash value",
                e);
    }

}

Example 4

Source File: FloatFieldMapper.java From Elasticsearch with Apache License 2.0

5 votes

@Override
public BytesRef binaryValue() {
    CollectionUtils.sortAndDedup(values);

    final byte[] bytes = new byte[values.size() * 4];
    for (int i = 0; i < values.size(); ++i) {
        ByteUtils.writeFloatLE(values.get(i), bytes, i * 4);
    }
    return new BytesRef(bytes);
}

Example 5

Source File: DoubleFieldMapper.java From Elasticsearch with Apache License 2.0

5 votes

@Override
public BytesRef binaryValue() {
    CollectionUtils.sortAndDedup(values);

    final byte[] bytes = new byte[values.size() * 8];
    for (int i = 0; i < values.size(); ++i) {
        ByteUtils.writeDoubleLE(values.get(i), bytes, i * 8);
    }
    return new BytesRef(bytes);
}