Question

type of tokens in python?

Answer 1

THE TYPES OF TOKENS IN PYTHON ARE GIVEN BELOW
token.tok_name

Dictionary mapping the numeric values of the constants defined in this module back to name strings, allowing more human-readable representation of parse trees to be generated.

token.ISTERMINAL(x)

Return true for terminal token values.

token.ISNONTERMINAL(x)

Return true for non-terminal token values.

token.ISEOF(x)

Return true if x is the marker indicating the end of input.

The token constants are:

token.ENDMARKER
token.NAME
token.NUMBER
token.STRING
token.NEWLINE
token.INDENT
token.DEDENT
token.LPAR
token.RPAR
token.LSQB
token.RSQB
token.COLON
token.COMMA
token.SEMI
token.PLUS
token.MINUS
token.STAR
etc..

Answer 2

Answer:

(i) Keywords

(ii) Identifiers

(iii) Literals

    a) string literals

    b) Numeric literals

    c) Boolean literals

    d) special literal (None)

(iv) Operators

(v) Punctuators