Java Code Examples for org.apache.spark.unsafe.Platform#putInt()

The following examples show how to use org.apache.spark.unsafe.Platform#putInt() . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: UnsafeExternalSorter.java    From indexr with Apache License 2.0 6 votes vote down vote up 
/**
 * Write a record to the sorter.
 */
public void insertRecord(Object recordBase, long recordOffset, int length, long prefix)
        throws IOException {

    growPointerArrayIfNecessary();
    // Need 4 bytes to store the record length.
    final int required = length + 4;
    acquireNewPageIfNecessary(required);

    final Object base = currentPage.getBaseObject();
    final long recordAddress = taskMemoryManager.encodePageNumberAndOffset(currentPage, pageCursor);
    Platform.putInt(base, pageCursor, length);
    pageCursor += 4;
    Platform.copyMemory(recordBase, recordOffset, base, pageCursor, length);
    pageCursor += length;
    assert (inMemSorter != null);
    inMemSorter.insertRecord(recordAddress, prefix);
}
Example 2
Source File: UnsafeExternalSorter.java    From indexr with Apache License 2.0 6 votes vote down vote up 
/**
 * Write a key-value record to the sorter. The key and value will be put together in-memory,
 * using the following format:
 *
 * record length (4 bytes), key length (4 bytes), key data, value data
 *
 * record length = key length + value length + 4
 */
public void insertKVRecord(Object keyBase, long keyOffset, int keyLen,
                           Object valueBase, long valueOffset, int valueLen, long prefix)
        throws IOException {

    growPointerArrayIfNecessary();
    final int required = keyLen + valueLen + 4 + 4;
    acquireNewPageIfNecessary(required);

    final Object base = currentPage.getBaseObject();
    final long recordAddress = taskMemoryManager.encodePageNumberAndOffset(currentPage, pageCursor);
    Platform.putInt(base, pageCursor, keyLen + valueLen + 4);
    pageCursor += 4;
    Platform.putInt(base, pageCursor, keyLen);
    pageCursor += 4;
    Platform.copyMemory(keyBase, keyOffset, base, pageCursor, keyLen);
    pageCursor += keyLen;
    Platform.copyMemory(valueBase, valueOffset, base, pageCursor, valueLen);
    pageCursor += valueLen;

    assert (inMemSorter != null);
    inMemSorter.insertRecord(recordAddress, prefix);
}
Example 3
Source File: BytesToBytesMap.java    From indexr with Apache License 2.0 5 votes vote down vote up 
/**
 * Acquire a new page from the memory manager.
 *
 * @return whether there is enough space to allocate the new page.
 */
private boolean acquireNewPage(long required) {
    try {
        currentPage = allocatePage(required);
    } catch (OutOfMemoryError e) {
        return false;
    }
    dataPages.add(currentPage);
    Platform.putInt(currentPage.getBaseObject(), currentPage.getBaseOffset(), 0);
    pageCursor = 4;
    return true;
}
Example 4
Source File: UnsafeRow.java    From indexr with Apache License 2.0 4 votes vote down vote up 
@Override
public void setInt(int ordinal, int value) {
    assertIndexIsValid(ordinal);
    Platform.putInt(baseObject, getFieldOffset(ordinal), value);
}
Example 5
Source File: UnsafeUtil.java    From indexr with Apache License 2.0 4 votes vote down vote up 
public static void setInt(Object base, long addr, int v) {
    Platform.putInt(base, addr, v);
}
Example 6
Source File: Memory.java    From indexr with Apache License 2.0 4 votes vote down vote up 
@Benchmark
public void offheap1() {
    for (int i = 0; i < count; i++) {
        Platform.putInt(null, offheap + (i << 2), Platform.getInt(null, offheap + (i << 2)) + 1);
    }
}