XCX Parser — Documentation

File: src/parser/pratt.rs
Technique: Pratt Parser (Top-Down Operator Precedence)

Table of Contents

1. Overview
2. Precedence Levels
3. Instruction Dispatch
4. Function Definition Styles
5. Fiber Instructions
6. Expression Parsing
7. AST Nodes — Expr
8. AST Nodes — Stmt
9. Type System
10. Expr Stmt Post-processing

Overview

The XCX Parser uses the Pratt algorithm (Top-Down Operator Precedence).

• File: src/parser/pratt.rs
• Lookahead: One token (current + peek), manual advancement via advance()
• Error Recovery: synchronize() skips tokens until the next semicolon or a keyword starting a statement

The Parser struct borrows the source string for lifetime 'a, and the Scanner<'a> is parameterized with the same lifetime.

Precedence Levels

From lowest to highest:

Instruction Dispatch

parse_statement_internal() dispatches based on the current token:

Function Definition Styles

XCX supports two syntactically different function definition styles:

Curly Brace Style (C-like)

func name(i: x, s: y -> i) {
    return x + 1;
}

XCX Style (Keyword block)

func:i: name(i: x, s: y) do;
    return x + 1;
end;

Both styles produce identical StmtKind::FunctionDef AST nodes. The return type in the curly brace style is declared via -> type inside the parameter list or after the ).

Fiber Instructions

parse_fiber_statement() checks the peek to decide: - peek == Colon → parse_fiber_decl() (instantiation: fiber:T: varname = fiberDef(args);) - otherwise → parse_fiber_def() (definition: fiber name(params) { body })

parse_fiber_decl() also handles the case where after parsing the type and name, the current token is ( — in which case it pivots to finish_fiber_def().

Fiber Definition

fiber myFiber(i: x) {
    yield x;
    yield x + 1;
}

Fiber Instantiation

fiber:i: f = myFiber(10);

Yield

yield expr;       // yield with value
yield from expr;  // delegate to another fiber
yield;            // void yield

Expression Parsing

parse_expression(precedence) calls parse_prefix() for the left side, then loops calling parse_infix(left) as long as the peek token’s precedence exceeds the current minimum.

Prefix Parsers (Selection)

Infix Parsers (Selection)

AST Nodes — Expr

Defined in src/parser/ast.rs as the ExprKind enum:

ExprKind
├── Literals
│   ├── IntLiteral(i64)
│   ├── FloatLiteral(f64)
│   ├── StringLiteral(StringId)
│   ├── BoolLiteral(bool)
│   └── DateLiteral { date_string, format }
├── Identifiers and Access
│   ├── Identifier(StringId)
│   ├── MemberAccess { receiver, member }
│   └── Index { receiver, index }
├── Operations
│   ├── Binary { left, op, right }
│   └── Unary { op, right }
├── Calls
│   ├── FunctionCall { name, args }
│   ├── MethodCall { receiver, method, args, wait_after }
│   └── Lambda { params, return_type, body }
├── Collections
│   ├── ArrayLiteral { elements }
│   ├── ArrayOrSetLiteral { elements }
│   ├── SetLiteral { set_type, elements, range }
│   ├── MapLiteral { key_type, value_type, elements }
│   └── TableLiteral { columns, rows }
├── Networking
│   ├── NetCall { method, url, body }
│   └── NetRespond { status, body, headers }
├── Fiber
│   └── Yield(expr)
├── Randomness
│   ├── RandomChoice { set }
│   ├── RandomInt { min, max, step }
│   └── RandomFloat { min, max, step }
└── Other
    ├── RawBlock(StringId)
    ├── TerminalCommand(cmd, args)
    ├── Tuple(Vec<Expr>)
    ├── As { expr, name }
    └── Tag(StringId)

AST Nodes — Stmt

Key StmtKind variants:

Type System

Type enum in src/parser/ast.rs:

Type
├── Int          — 48-bit integer (NaN-boxed)
├── Float        — 64-bit floating-point
├── String       — UTF-8
├── Bool         — true/false
├── Date         — ms timestamp
├── Json         — any JSON value
├── Array(Box<Type>)
├── Set(SetType) — N/Q/Z/S/C/B
├── Map(Box<Type>, Box<Type>)
├── Table(Vec<ColumnDef>)
├── Database
├── Fiber(Option<Box<Type>>)  — None = void
├── Builtin(StringId)          — json, date, store, etc.
└── Unknown                    — acts as a wildcard

SetType variants: N (Natural), Z (Integer), Q (Rational/Float), S (String), C (Char/String), B (Boolean).

ColumnDef

pub struct ColumnDef {
    pub name:       StringId,
    pub ty:         Type,
    pub attributes: Vec<ColumnAttribute>,
}

Column attributes: Auto (@auto), PrimaryKey (@pk), Unique (@unique), Optional (@optional), Default(Expr) (@default(val)), ForeignKey(table, col) (@fk(Table.col)).

ForIterType

Set by the type checker during semantic analysis:

Expr Stmt Post-processing

After parsing a full statement expression, parse_expr_stmt() checks if the result is a MethodCall:

• Method name bind with 2 arguments, where the second is an Identifier → rewritten as StmtKind::JsonBind
• Method name inject with 2 arguments → rewritten as StmtKind::JsonInject

This enables the syntactic sugar json.bind("path", target); and json.inject(mapping, table); at the statement level.

Language Constructs (Syntax)

--- Variables
i: age = 25;
const s: NAME = "Alice";
var x = 42;

--- Flow Control
if (cond) then;
    ...
elseif (cond2) then;
    ...
else;
    ...
end;

while (cond) do;
    ...
end;

for x in 1 to 10 do;
    ...
end;

for item in myArray do;
    ...
end;

--- Functions
func:i: add(i: a, i: b) do;
    return a + b;
end;

--- HTTP
serve: myServer {
    port = 8080,
    routes = [["GET /api" :: handler]]
};

net.get("https://api.example.com") as resp;

--- Collections
array:i arrOfInts = [1, 2, 3];
set:N mySet = set:N { 1,,100 };
map:s<->i scores = [s::v :: ["Alice" :: 100]];

--- JSON
json: data = <<<{ "name": "Alice" }>>>;
data.bind("/name", nameVar);