DOCUMENTATION

Parser

The XCX Parser is written structurally as a recursive descent parser combined tightly with a system-level analyzer utilizing top-down operator precedence (Pratt parsing) upon verifying connected logical formations. Its verification deployment loops are placed under src/frontend/parser/ to control syntax structures.

Block Modules

Parser Structure

The Parser tracks state operations across shifted tokens (current and peek) to pipe modified system loops down from the previous token history. This aids error handling context and allows mapped fallback logic reporting back to the compiler. The depth acts as a guard against excessive recursion depth; any call dropping an instance into the parser increments an integration loop, and hitting the depth limit forces compilation out through a panic escape hatch bound specifically to 500 loop captures.

Structural Construction

new loads an optional format block over the pointer Interner structure. A precursor baseline formatting logic for expanders applies new_with_interner injecting sub-parsers back up via parents utilizing the shared interner ID map without allocating independent string arrays across child parser components.

Construction calls loop next_token twice upon initiation to properly allocate the current and peek buffers.

Main Module Program Parser (parse_program)

The pre-defined execution logic loops system inputs reading statements continuously through an empty sequence buffer. Cycle testing reads jump logic modification boundaries checking against EOF. The loop iteratively triggers parse_statement(), assigning success options straight back into Program { stmts }. Any logic failures inside statements drop out triggering specific formatting reports to flag the precursor tag using error recovery synchronize().

Interner Base Extraction (into_interner)

Drops parsing input parameters and discharges the modified interner out of the lexer buffer structure. Extensively deployed within sub-compilation logic blocks via the expander passing string logic IDs natively back onto its parent parser wrapper without duplication.

Token Stream Advance (token_stream.rs)

Rotates the parsed logic pipeline testing modifiers sequentially:

The system steps along predefined boundaries checking operations via functions like expect, updating modifier logic jumps by dropping operational errors back into checking sequences expect_semicolon. The compiler recursively invokes self.advance() internally routing parsed subset blocks into logic branches.

Precedence Ranks (precedence.rs)

Precedence derives PartialOrd so that < comparisons in the Pratt loop work directly. Higher variants = higher binding power.

The precedence of the current token is retrieved via current_precedence() on the parser struct which maps TokenKind to Precedence. (Note: Precedence::for_token(kind) and peek_precedence() were removed as part of parsing consolidation).

Pratt Expression Parser (pratt.rs)

parse_expression(precedence: Precedence) -> Option<Expr>

The core Pratt loop:

1. Call parse_prefix() to get the initial left-hand expression.

2. While the current token is not Semicolon or EOF and precedence < current_precedence(): call parse_infix(left) and update left.

3. Return left.

The depth guard applies here identically to statement parsing to prevent stack overflows on recursive logic drops.

current_precedence()

Inspects the current token and returns the matching Precedence variant, or Lowest if unrecognized. Used during infix parsing to compare lookahead precedence against the caller's constraint boundaries.

Expression Parsing (parse_expr.rs)

parse_prefix

Dispatches on self.current.kind:

parse_infix(left: Expr) -> Option<Expr>

Dispatches on self.current.kind to handle operators that appear after a left-hand expression:

Argument Parsing

Reads comma-separated arguments until ) or EOF. Each argument is either:

• name = expr → Argument::Named(name, expr)
• expr → Argument::Positional(expr)

Named arguments must use a bare identifier followed by = (the name: expr form is not accepted).

Set Literal Content

Parses the interior of set:T{...}. If the first expression is followed by ,,, it is a range (SetRange); otherwise it is an element list. An optional @step follows the end of a range.

Map Literal

Strict structure:

The schema block is mandatory. data may contain EMPTY for an empty map.

Table Literal

Structure:

Column attributes (@pk, @auto, @unique, @optional, @default(expr), @fk(table, col)) are parsed inline after the column type.

Statement Parsing (parse_stmt.rs)

parse_statement_internal

Top-level dispatch on self.current.kind. The parser uses one token of lookahead (self.peek.kind) to disambiguate ambiguous prefixes.

Notable disambiguation rules:

• A token that introduces a type (TypeI, TypeF, Set, Map, etc.) followed by = → assignment. Followed by ( or [ or . → expression statement. Followed by : → variable declaration (or database declaration if the token is Database).
• Identifier followed by = → assignment. Followed by ( → function call statement. Followed by : → variable declaration.
• End → returns None, signalling the end of a block to the calling parse function.

Variable Declaration (parse_var_decl)

Type is parsed with parse_type. The initializer is optional. When the initializer opens with { and a set type was declared, the parser attempts to parse a set literal; otherwise it parses an array literal or a plain expression.

Multiple Variable Declarations (Desugaring):

If a type annotation is followed by a comma-separated list of names (e.g. i: a, b, c;), the parser desugars this into a StmtKind::MultiVarDecl holding individual VarDecl statements in sequence. Each name may carry its own optional initializer (i: a = 1, b = 2;); there is no shared trailing initializer.

Assignment (parse_assignment)

The left-hand side may be a dotted identifier. The right-hand side is any expression.

If Statement (parse_if_statement)

elseif/elif parse the same way (elf and els were removed as keywords in 4.4 and now lex as plain identifiers). The condition must be parenthesized. then (with its optional ;) is consumed when present but not enforced; end is likewise consumed only when present.

Block semicolons are optional (4.4). The semicolon after then, do, else, end, and after the closing } of func/fiber definitions and { }-shaped declarations (map/table/database) is consumed when present and not required. The bare form is the recommended style; statement semicolons inside block bodies remain required. The ; form is removed in XCX 5.0a.

While Statement (parse_while_statement)

The condition must be parenthesized; do is optional and so is the ; after it. A missing end is tolerated (consumed only when present).

For Statement (parse_for_statement)

in is required (the for var = ... form is not accepted); :: also separates range bounds. ForIterType::Range is assigned when to/:: is present, otherwise ForIterType::Array with end = IntLiteral(0) - collection-kind refinement (set/fiber) is not performed at parse time.

Function Definition (parse_func_def)

Parameters are (Type name) or (Type : name) pairs (names may be dotted). The optional return type is placed after an -> arrow inside the parameter list; an -> after ) is rejected with a dedicated error. The block is bounded by {}; a trailing ; after the closing brace is optional (recommended style: without).

Fiber Statements (parse_fiber_statement)

Two forms:

1. Definition: fiber name(Type param, ... -> ReturnType) { ... } - the return type sits inside the parens after -> (or comes from the fiber : Type name(...) spelling); the body is brace-delimited; a trailing ; after } is optional. Stores as FiberDef.

2. Instantiation: fiber [: InnerType] varName = FiberName(args); - stores as FiberDecl.

Halt Statement (parse_halt_stmt)

The . after halt and the >! before the message expression are both required.

Print / Input

• >! expr; → StmtKind::Print
• >? name; → StmtKind::Input (no colon is parsed; the type is read from the following token when it introduces a type, otherwise defaults to Type::String)

Return / Yield

• return [expr];
• yield [expr [as name]]; - bare yield; becomes YieldVoid.
• yield from expr; - YieldFrom.

Include

Stored as StmtKind::Include. The Expander resolves this at pre-semantic time.

Serve

routes is mandatory; port defaults to 8080 if absent.

Net Request

Where method is one of get, post, put, delete, patch, head, options. Without as target, emitted as a fire-and-forget expression statement.

Database Declaration

Wait

Both forms are valid.

Terminal Command

The terminal keyword is mandatory; the optional ! follows it. Parsed as ExprKind::TerminalCommand and wrapped in StmtKind::ExprStmt.

Type Parser (parse_type.rs)

Parses a single type annotation. If array: is present at the start, the inner type is wrapped in Type::Array. Supported types:

is_type_intro(kind) returns true for any TokenKind that can start a type expression. Used to disambiguate map:K<->V from map used as a plain identifier.

Error Recovery (recovery.rs)

error(message: &str)

Creates a Reporter, calls reporter.error(line, col, len, message), and sets has_error = true. Does not abort parsing.

synchronize()

Skips tokens until a likely statement boundary is found:

• A Semicolon is consumed and the function returns.
• Tokens Func, Fiber, Const, If, While, For, Return, GreaterBang stop the skip without consuming (the parser will attempt to re-parse from there).
• EOF terminates the loop.

expect(kind, message) -> bool

Advances and returns true if current.kind == kind. Otherwise calls error(message) and returns false without advancing.

expect_semicolon() -> bool

Specialized expect for ;. The error message is context-aware:

• After end → "Expected ';' after 'end'. Every block must be terminated with 'end;'."
• After then → "Expected ';' after 'then'. Did you forget the semicolon?"
• After do → "Expected ';' after 'do'. Did you forget the semicolon?"
• Otherwise → "Expected ';' at the end of the statement."

Since 4.4, block-position semicolons (after then/do/else, after end, and

after the closing } of func/fiber/data-literal declarations) are consumed

optionally and no longer route through expect_semicolon. The function remains

for flat statement terminators (assignments, calls, return, break,

continue, declarations), where ; is still required.

Identifier Parsing (parse_identifier_as_string_id)

Handles the common case where identifiers, reserved keywords, and type tokens may all appear in name positions. The function maps the current token to its canonical text string, advances past it, and if allow_dots is true, continues consuming .identifier suffixes to build dotted names like math.sin or store.get. The result is interned and returned as a StringId.

Expander (expander.rs)

The Expander is not strictly part of the parser but runs immediately after parsing and before semantic analysis.

expand(program, current_dir) -> Result<Program, String>

Walks the top-level statement list. For each StmtKind::Include:

1. Resolve the path relative to current_dir; then try hardcoded redirects (math.xcx → mathlib/src/math.xcx, pax.xcx → pax/src/pax.xcx, doc.xcx → doc/doc.xcx); then relative to each registered include_path.

2. Detect circular includes via visiting_files.

3. Skip if already included (no alias) via included_files.

4. Lex and parse the included file with a clone of the interner, copying it back into the parent afterwards.

5. Recursively expand the sub-program.

6. If an alias is present, call prefix_program on the sub-AST, then merge.

Alias Prefixing

When include "lib.xcx" as lib; is used, all top-level names in lib.xcx are renamed to lib.name in the AST. Call sites that reference lib.foo(...) or lib.foo are rewritten to FunctionCall { name: "lib.foo" } or Identifier("lib.foo") respectively.

The expansion also handles the edge case of mathematical constants (PI, E,

TAU, PHI, SQRT2, LN2, LN10, INF, INT_MAX, INT_MIN) in aliased

includes. Since 4.4 the rewrite is conditional: for each aliased include the

expander records the module's unprefixed top-level names (alias_exports, in

encounter order), and a bare math-constant identifier is rewritten to

<alias>.<NAME> only when that module actually declares the symbol (first

match in include order). Aliases whose module does not export the name leave

the identifier untouched, so user variables named PI/E/TAU resolve

normally.

Include Path Registration

Adds a directory to search when a relative include path cannot be resolved relative to the including file's directory. Typically set to the compiler's standard library directory.