Question

What are tokens. Illustrate at least 4 tokens 1 ip?

Answer 1

● Tokens :- tokens are smallest representable units.

1. keywords - keywords are the reversed word which convey special meaning to the compiler
ex-print,elif , if, for, while, etc

2. identifier - identifier is a vallied set of characters which represent certain value which are also known as variables
ex- any valid variable

3.operators -operators are the symbol which represents sudden operation

4. litreals- literals are the sudden value which do not change through out the programme

5. punctuators- various symbol use in programme
ex- "" , ' ' : , ; ,etc

Answer 2

In general, a token is an object that represents something else, such as another object (either physical or virtual), or an abstract concept as, for example, a gift is sometimes referred to as a token of the giver's esteem for the recipient. In computers, there are a number of types of tokens.