Scanner: recognizing language elements
Recognizing keywords, names, operators, and literals in a sequence of characters.
Before understanding an expression, we need to identify its parts. In const hp = 10; the scanner recognizes const, the identifier hp, the symbol =, the numeric literal 10 and ;.
Spaces help separate elements, but the rules cover more than spaces. They also account for strings, comments, Unicode, the context of /, and line breaks that matter for automatic semicolon insertion. Splitting text on spaces is therefore not enough to build a JavaScript parser.
const hp = 10;
// const · hp · = · 10 · ;The scanner answers the question “Which elements are present?”. The parser answers the next question: “Which construct do they form?”. Processing source text and executing a program are different tasks: the scanner and parser do not run game logic.
In the simulation we will examine the tokens one at a time. This is a fixed teaching example, not a general-purpose scanner.