Tokens vs Identifiers vs Keywords in C++
A token is the smallest element of a C++ program that is meaningful to the compiler. The C++ parser recognizes these kinds of tokens: identifiers, keywords, literals, operators, punctuators, and other separators. A stream of these tokens makes up a translation unit. Tokens are usually separated by white space.
The parser recognizes keywords, identifiers, literals, operators, and punctuators. Preprocessing tokens(like #include, #define, #if_def, etc.) are used in the preprocessing phases to generate the token stream passed to the compiler. The preprocessing token categories are header names, identifiers, preprocessing numbers, character literals, string literals, etc. that do not match one of the other categories. Character and string literals can be user-defined literals. Preprocessing tokens can be separated by white space or comments.
An identifier is anything that the user gives a name to. For example method names, variable names, struct names, class names, etc are all identifiers. In C++, keywords are reserved identifiers which cannot be used as names for the variables in a program.
Published on 15-Feb-2018 11:39:45
