Java Code Examples for org.apache.avro.io.parsing.Symbol#MAP_KEY_MARKER

The following examples show how to use org.apache.avro.io.parsing.Symbol#MAP_KEY_MARKER . You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example 1
Source File: GuidedJsonDecoder.java    From gcp-ingestion with Mozilla Public License 2.0 6 votes vote down vote up
/**
 * Read a string from the current location in parser.
 *
 * <p>This method differs from the original JsonDecoder by serializing all
 * structures captured by the current token into a JSON string. This enables
 * consistent behavior for handling variant types (e.g. field that can be a
 * boolean and a string) and for under-specified schemas.
 *
 * <p>This encoding is lossy because JSON strings are conflated with standard
 * strings. Consider the case where a number is decoded into a string. To
 * convert this Avro file back into the original JSON document, the encoder
 * must parse all strings as JSON and inline them into the tree. Now, if the
 * original JSON represents a JSON object as a string (e.g. `{"payload":
 * "{\"foo\":\"bar\"}"`), then the encoder will generate a new object that is
 * different from the original.
 *
 * <p>There are a few ways to avoid this if it is undesirable. One way is to use
 * a binary encoding for the JSON data such as BSON or base64. A second is to
 * normalize documents to avoid nested JSON encodings and to specify a schema
 * explictly to guide the proper typing.
 */
@Override
public String readString() throws IOException {
  parser.advance(Symbol.STRING);
  if (parser.topSymbol() == Symbol.MAP_KEY_MARKER) {
    parser.advance(Symbol.MAP_KEY_MARKER);
    assertCurrentToken(JsonToken.FIELD_NAME, "map-key");
  }

  String result = null;
  if (in.getCurrentToken() == JsonToken.VALUE_STRING
      || in.getCurrentToken() == JsonToken.FIELD_NAME) {
    result = in.getValueAsString();
  } else {
    // Does this create excessive garbage collection?
    TokenBuffer buffer = new TokenBuffer(in);
    buffer.copyCurrentStructure(in);
    result = mapper.readTree(buffer.asParser()).toString();
    buffer.close();
  }
  in.nextToken();
  return result;
}
 
Example 2
Source File: IOWJsonDecoder.java    From iow-hadoop-streaming with Apache License 2.0 6 votes vote down vote up
@Override
public String readString() throws IOException {
    advance(Symbol.STRING);
    if (parser.topSymbol() == Symbol.MAP_KEY_MARKER) {
        parser.advance(Symbol.MAP_KEY_MARKER);
            if (in.getCurrentToken() != JsonToken.FIELD_NAME) {
                throw error("map-key");
            }
        } else {
            if (in.getCurrentToken() != JsonToken.VALUE_STRING) {
            throw error("string");
        }
    }
    String result = in.getText();
    in.nextToken();
    return result;
}
 
Example 3
Source File: IOWJsonDecoder.java    From iow-hadoop-streaming with Apache License 2.0 5 votes vote down vote up
@Override
public void skipString() throws IOException {
    advance(Symbol.STRING);
    if (parser.topSymbol() == Symbol.MAP_KEY_MARKER) {
        parser.advance(Symbol.MAP_KEY_MARKER);
        if (in.getCurrentToken() != JsonToken.FIELD_NAME) {
            throw error("map-key");
        }
    } else {
          if (in.getCurrentToken() != JsonToken.VALUE_STRING) {
              throw error("string");
          }
    }
    in.nextToken();
}