PEGO Guides
Task-oriented guides to the PEGO grammar language and runtime. Each guide explains how and why to use one feature, with
complete, runnable examples, and points to the specification for the exact rules.
#Writing Grammars
| Guide |
Contents |
| Rule Expressions |
Writing the matching part of a grammar: ordered choice, repetition and lookahead, cut, white space and tokens, keywords, and the pitfalls of each |
| Shaping Trees |
The tree a grammar builds by default, and shaping it with #atomic, - and captures; performance guidelines |
| Declaring Types |
Struct and join types, optional and list fields, rule result types and inference |
| Actions |
Actions, built-in functions and lambdas, positions of constructed nodes, recipes (foldl, flattening lists) |
| Operators Rule |
Operator expressions: Operators Rule Declaration Statements, left-recursive rules and precedence chains compared, level-restricted calls, operator forms and associativity, pitfalls, a larger example |
| Context-Sensitive Parsing |
Predicates, variables, lookahead and attributes; matching tags, indentation blocks, here-documents and other recipes; the cost of variables |
| Error Messages |
How syntax errors are reported, getting them out on the command line and in Go, readable messages with @error, cuts |
| Recovering from Errors |
Recovering with #recover, what happens to the errors, handling them in Go, inserting missing literals, recipes |
#Running Parsers
| Guide |
Contents |
| Running Parsers |
The Go API: the pipeline from source to parser, compiling, start rules, results and errors, concurrency, choosing a feature |
| Choosing a Backend |
The closure, bytecode and iterative backends and how to choose one, the nesting limit |
| Recognition Mode |
RecognizeOnly: checking input without building a tree, what it still evaluates and where it does not apply |
| Memoization |
What the packrat memo table remembers, when, and what it means for memory, streaming and incremental parsing |
| Cancellation and Work Limits |
WithContext and WithWorkLimits: stopping a parse and what it returns |
| Position Units |
Code points or bytes: what the unit changes, how to choose it, invalid UTF-8 |
| Parsing a Prefix |
ParsePrefix: parsing a fragment of a larger text and reading how much it consumed |
| Building Grammars as ASTs and JSON |
Structural validation, formatting, source metadata and JSON loading for tools that build grammars in Go |
#Using the Results
| Guide |
Contents |
| Reading Trees |
Reading a Node: values and positions, printing and JSON, a whole tree, typed values |
| Walking Trees |
Walk and Cursor: visiting nodes in order and selecting them by position |
| Output Schemas and Structural Queries |
Describing the output of a grammar, comparing schemas, finding nodes by their shape |
| Guide |
Contents |
| Incremental Parsing |
Document for reparsing a text after edits: the model, the API, keeping a result, a walkthrough with Stats |
| What Incremental Parsing Reuses |
Which results survive an edit and why, and writing grammars that reuse well |
| Incremental Parsing Performance |
Measurements of reparsing after an edit, and the limits of a Document |
| Document Change Notifications |
Recording what an edit and a parse changed, for tools that keep an outline, an index or highlights |
| Streaming |
#stream and ParseStream for input that is too big to hold or never ends: what is emitted and when, memory, errors, cancelling |
#Shipping Parsers
| Guide |
Contents |
| Compiled Grammars |
Saving a grammar as a .pegoc file and loading it: the modes and what each serves, sizes and load times, go:embed, versions, untrusted data, and when to use it instead of generated code |
| Generating Parsers |
Standalone parsers with pego gen: why generate, the command and its options, what is supported, keeping generated code up to date, trade-offs |
| The Generated Go API |
The Go package pego gen writes: Parse, ParseRule, nodes and errors, recognition, call options, cancellation and work limits |
| Typed Values in Generated Go |
-types: a Go type for each grammar type, ParseAST, the type mapping, speed and named rules |
| Incremental Documents in Generated Go |
-document: the generated Document, Edit and reparse, what is kept, diagnostics and limits |
| Streams in Generated Go |
-stream: ParseStream, Iterate, diagnostic controls and typed elements |
| Generated TypeScript Parsers |
Standalone TypeScript modules with pego gen -lang ts: the generated API, positions, strings and ints in JavaScript, errors, deep nesting and the JavaScript stack |
| Using the Ready-Made Parsers |
The parsers of parsers/ (CEL, CSV, CUE, DuckDB SQL, Go, JSON, Python, TypeScript, XML, YAML) in a program: the API they share, choosing an entry point, typed trees, positions and units, errors, conformance and limits |
#Developing Grammars
| Guide |
Contents |
The pego Command |
The command-line workflows: parse, format, fix, convert, compile and generate |
| Testing Grammars |
A test suite for a grammar in layers: examples, golden files, fixture suites, properties, generated inputs, fuzzing, reference implementations, equivalence checks, lint and format checks in CI |
| Debugging and Profiling |
Why an input fails (pego explain), what the parser does (pego trace), why a parse is slow (pego profile), WithTrace and WithProfile in Go, costs and limits |
| Linting Grammars |
Finding likely mistakes with pego lint and package lint, what the checks mean and how to fix what they report, suppressing findings |
| Sampling Inputs and Fuzzing |
Generating inputs a grammar accepts with pego sample and package sample: seeds and size bounds, coverage, near-miss invalid inputs, seeding Go fuzz tests |
| Editor Support |
The language server (pego lsp): diagnostics, formatting, navigation, hover, rename, completion and semantic highlighting; setting up VS Code, Neovim, Helix and Emacs |
| The Playground |
The browser playground: inspecting trees and errors, sharing grammar state, its limits and the WebAssembly API |
#Choosing a Guide
PEGO