jdk.nashorn.internal.parser.Lexer Java Examples

The following examples show how to use jdk.nashorn.internal.parser.Lexer. You can vote up the ones you like or vote down the ones you don't like, and go to the original project or source file by following the links above each example. You may check out the related API usage on the sidebar.
Example #1
Source File: Debug.java    From TencentKona-8 with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #2
Source File: Debug.java    From nashorn with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #3
Source File: Debug.java    From jdk8u60 with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #4
Source File: Debug.java    From jdk8u_nashorn with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #5
Source File: Debug.java    From openjdk-jdk8u with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #6
Source File: Debug.java    From openjdk-8 with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #7
Source File: Debug.java    From openjdk-jdk8u-backup with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #8
Source File: Debug.java    From openjdk-8-source with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #9
Source File: Debug.java    From openjdk-jdk9 with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #10
Source File: Debug.java    From hottub with GNU General Public License v2.0 6 votes vote down vote up 
/**
 * Dump a token stream to stdout
 *
 * TODO: most other bugging goes to stderr, change?
 *
 * @param source the source
 * @param lexer  the lexer
 * @param stream the stream to dump
 */
public static void dumpTokens(final Source source, final Lexer lexer, final TokenStream stream) {
    TokenType type;
    int k = 0;
    do {
        while (k > stream.last()) {
            // Get more tokens.
            lexer.lexify();
        }

        final long token = stream.get(k);
        type = Token.descType(token);
        System.out.println("" + k + ": " + Token.toString(source, token, true));
        k++;
    } while(type != EOF);
}
Example #11
Source File: RegExpScanner.java    From nashorn with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #12
Source File: RegExpScanner.java    From jdk8u_nashorn with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #13
Source File: RegExpScanner.java    From openjdk-8 with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #14
Source File: RegExpScanner.java    From openjdk-8-source with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #15
Source File: RegExpScanner.java    From hottub with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #16
Source File: RegExpScanner.java    From openjdk-jdk9 with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #17
Source File: RegExpScanner.java    From openjdk-jdk8u-backup with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #18
Source File: RegExpScanner.java    From openjdk-jdk8u with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #19
Source File: RegExpScanner.java    From jdk8u60 with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #20
Source File: RegExpScanner.java    From TencentKona-8 with GNU General Public License v2.0 5 votes vote down vote up 
private boolean characterClassEscape() {
    switch (ch0) {
    // java.util.regex requires translation of \s and \S to explicit character list
    case 's':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // No nested class required if we already are inside a character class
            if (inCharClass) {
                sb.append(Lexer.getWhitespaceRegExp());
            } else {
                sb.append('[').append(Lexer.getWhitespaceRegExp()).append(']');
            }
            skip(1);
            return true;
        }
        return commit(1);
    case 'S':
        if (RegExpFactory.usesJavaUtilRegex()) {
            sb.setLength(sb.length() - 1);
            // In negative class we must use intersection to get double negation ("not anything else than space")
            sb.append(inNegativeClass ? "&&[" : "[^").append(Lexer.getWhitespaceRegExp()).append(']');
            skip(1);
            return true;
        }
        return commit(1);
    case 'd':
    case 'D':
    case 'w':
    case 'W':
        return commit(1);

    default:
        return false;
    }
}
Example #21
Source File: ScriptRuntime.java    From hottub with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #22
Source File: ScriptRuntime.java    From openjdk-jdk9 with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #23
Source File: ScriptRuntime.java    From openjdk-8-source with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #24
Source File: ScriptRuntime.java    From openjdk-jdk8u-backup with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #25
Source File: ScriptRuntime.java    From openjdk-8 with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #26
Source File: ScriptRuntime.java    From openjdk-jdk8u with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #27
Source File: ScriptRuntime.java    From jdk8u_nashorn with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #28
Source File: ScriptRuntime.java    From jdk8u60 with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #29
Source File: ScriptRuntime.java    From nashorn with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}
Example #30
Source File: ScriptRuntime.java    From TencentKona-8 with GNU General Public License v2.0 2 votes vote down vote up 
/**
 * Test whether a char is valid JavaScript whitespace
 * @param ch a char
 * @return true if valid JavaScript whitespace
 */
public static boolean isJSWhitespace(final char ch) {
    return Lexer.isJSWhitespace(ch);
}