Why PEGO

PEGO is a grammar compiler and parsing platform designed for every target, engine and use. You write one grammar. Today PEGO checks and compiles it once, generates Go and TypeScript parsers from it, and runs it on three engine backends in Go. It aims to do the same for any target language, engine and optimization profile, and for every use (batch trees, typed values, incremental documents, streaming, recognition and, as a goal, editors), under one contract and with the best measured speed in each, against the fastest tools for each platform. The breadth is in targets and uses, not in grammar class: grammars are PEG-based, with ordered choice.

PEGO is pre-release, and much of that aim is still ahead. Each part is paired with where it stands today:

Best fit today: a Go program, alone or with a TypeScript front end, that parses its own DSL, configuration, query or policy language and wants typed values, declared recovery and incremental reparsing from one grammar. The reason to choose PEGO today is breadth under one contract: typed values, declared recovery, an incremental Document, a default depth cap, opt-in work limits and a TypeScript parser from one grammar, stdlib-only and tested against one engine. Current Limits lists what it does not do yet.

#Choosing a Tool

Choose by the job, not by the parsing algorithm. The PEGO rows state where it fits best today. Where the table names competitors, it is checked against the versions listed under Sources.

#What PEGO Is

PEGO is a parser framework built on Parsing Expression Grammars. Its grammar language keeps PEG's ordered choice, greedy repetition and lookahead. It adds what plain PEG lacks: left recursion (direct and indirect), declarative operator precedence (the Operators Rule), types and value actions, predicates with scoped variables for context-sensitive syntax, cut, and error recovery. It is therefore PEG-based, not plain PEG (language overview).

A .pego grammar is parsed, type-checked and compiled once. The compiled program runs on three engine backends in Go or is saved as bytecode. Generated TypeScript prints the matching code of every rule from the Flow IR, and generated Go prints its typed values and value-free rules from it; the rest of each comes from the grammar or the compiled program.

flowchart LR
  S[".pego source or JSON"] --> G["Checked, typed grammar"]
  G --> P["Compiled program"]
  P --> CL["Closure backend"]
  P --> BC["Bytecode module"]
  BC --> RVM["Recursive VM"]
  BC --> IVM["Iterative VM"]
  BC --> F[".pegoc file"]
  G --> GEN["Go and TypeScript generators"]
  P --> GEN
  G --> CORE["Core IR"]
  CORE --> FLOW["Flow IR"]
  FLOW --> GEN
  GEN --> GO["Generated Go"]
  GEN --> TS["Generated TypeScript"]

The closure backend is the default and the reference. The two bytecode VMs run saved .pegoc files, and the iterative VM handles deep nesting on its own stack (Choosing a Backend). The Core IR and Flow IR feed only the generators; the engine backends compile the grammar by their own routes (intermediate representations).

#Design Principles

Each principle is a commitment with a mechanism behind it. Where the mechanism is incomplete, the gap is stated.

One grammar, one contract. For the features they share, every backend returns the same trees, values, errors and positions, within each backend's limits. The test suite compares the closure backend, both VMs, both position units, memoization on and off, recognition against full parses, and generated Go and TypeScript against the engine (Generated Parsers, development guide). The features each backend lacks are listed rather than approximated (see What One Grammar Gives You).

The best speed is the aim. The runtime and generators avoid allocation (slab storage for nodes, lists and typed values, pooled frames, and typed values built without an intermediate tree). They also defer memoization for rules whose results are never reused, drop values that recognition and discarded captures do not need, and use specialized matching loops. The optimization catalog indexes 85 mechanisms with their backend applicability, and each mechanism has a page with its evidence. It also keeps 123 numbered records of measured changes and lists the experiments that did not pay off. Most mechanisms have no maintained switch for re-measuring them against the same source. The benchmark suite reruns with one command (Performance). Performance separates what is measured today from what is targeted.

Incremental parsing without grammar annotations. A Document keeps the memo table between parses. Each memo entry records the input range its rule examined. After an edit, entries entirely before or after the edit are kept (and shifted), and only overlapping ones are recomputed. Every rule result is a unit of reuse, so how much an edit reuses depends on how the grammar splits into rules and how far they look ahead (What Incremental Parsing Reuses). In that guide's example, an edit in the middle of a 1,000-line list re-evaluates 3 rule bodies when the list is a repetition, but about 500 when it is right-recursive. The specification states that a reparse returns exactly what a fresh parse of the current text returns, errors included (Incremental Documents). pego edit-fuzz tests this: it applies random edits, undos and redos and compares the Document's values and errors with those of a fresh parse at checkpoints (Sampling and Fuzzing); the repository tests run it across backends and position units. The approach builds on incremental packrat parsing (Dubroy and Warth, 2017). gpeg extends the same idea in Go with logarithmic edit application, and tree-sitter's fuzz command tests its own incremental parsing with random edits.

Streaming with memory bounded by the working set. When a start rule has a top-level #stream repetition, each element goes to a callback and is then dropped, along with the input and memo entries before it. Memory is bounded by the largest element, the memo entries of the last few elements, the caller's own state and the list of recovered errors, not by the input length (Streaming).

Typed values from the grammar. Rule types are inferred and checked when the grammar compiles. The node type is node[T]. pego gen -types emits Go structs, TNode[T] values and sealed interfaces for join types, and builds them directly during the parse when every value has a Go type. For TypeScript it emits interfaces and a parseAST conversion (Generated Typed Values). The engine API returns dynamic *pego.Node values.

Recovery is declared in the grammar. #recover(skip=e) resumes after a broken construct and leaves an error value in the tree, on every backend. #missing inserts an absent literal in whole-input parses on the closure backend (Recovering From Errors). Recovery is not automatic: when no #recover rule covers an error, the parse fails with no tree.

Resource use is stated, and bounding it is the caller's choice today. PEGO does not promise linear time. Memoization bounds how often a rule body runs at one position. The specification states which constructs repeat work, the classes of growth a grammar can have (linear, polynomial, exponential) with example grammars, and sufficient conditions for linear work (Time and Memory). Nesting depth is capped by default. Work limits are opt-in today (Cancellation and Work Limits). For a given artifact, input and options, a work limit stops a parse at the same point on every run. Context cancellation follows the caller's clock. Limits on by default are a committed goal (see Commitments and Targets).

One compilation, aimed at every target. A grammar is checked and compiled once. Generated TypeScript prints the matching code of every rule from the Flow IR, and generated Go prints part of its code from it; the engine backends compile the grammar by their own routes today (intermediate representations). Moving every backend onto the Core IR is an accepted design, not implemented yet (design record 062). The conformance tests that hold both generators to the engine's results are the bar for every target and engine that follows.

No cgo and no runtime dependencies. The module uses only the Go standard library. Generated Go needs only the standard library, and generated TypeScript has no dependencies. The toolchain itself runs in the browser as WebAssembly (Playground). The bytecode has a published specification; only the Go VMs implement it today.

#What One Grammar Gives You

Status is per backend. "Engine" covers the closure backend and the two bytecode VMs. A .pegoc file holds one mode (parse, recognize or document) and runs on the VMs.

Around the grammar, the pego command offers lint (14 checks), fmt, trace, profile, explain, sample (inputs a grammar accepts, and near misses), edit-fuzz, a fixture runner for grammar tests, and lsp for editing .pego files (Command-Line Reference). Ten ready-made parsers (CEL, CSV, CUE, DuckDB SQL, Go, JSON, Python 3.14, TypeScript 5.9 and TSX, XML 1.0, YAML 1.2.2) are generated into Go from PEGO grammars and checked against reference implementations and conformance suites. They show what the grammar language can express and serve as large test workloads.

#Performance

The aim is the best measured speed in every target, engine and use, against the fastest tools for each platform and with every loss published, with memory and size chosen per program. This section gives where PEGO stands today and the first targets toward that aim.

#Measured Today

All figures come from Benchmark Results: commit 13b3d186, Go 1.27.1, Apple M3 Max (darwin/arm64, 16 cores), median of three runs, positions in code points, measured on 2026-10-29. Times differ between machines. The standard-library parsers do different work, so the ratios compare different outputs: encoding/json builds values without positions, encoding/xml returns tokens, encoding/csv returns records, and go/parser builds a positioned Go syntax tree. The JSON baseline is encoding/json Unmarshal into any, as built by Go 1.27.1; the report does not record the GOEXPERIMENT setting, and a baseline that decodes with encoding/json/jsontext directly is not measured.

• Incremental. On Minilang, a small language defined in the benchmark suite, a one-character edit and reparse of an 89 KB program takes 116–133 µs with Document. A full parse of the same program on the three engine backends takes 15.6–24.5 ms; generated Go is not part of that comparison.
• Streaming memory. In a separate measurement on a 27 MiB CSV file (Apple silicon, Go 1.27.1), ParseStream peaked at about 13 MB resident against about 1.0 GB for a whole parse (Streaming).

No benchmark against another parser framework exists yet, so this page makes no speed claim relative to any tool it compares.

#Commitments and Targets

These are goals, not results.

• Match, then beat, hand-written parsers. Generated Go is to reach the time of the standard-library parser on each comparable workload while it still returns positioned trees or typed values, and then go below it. The current standard-library baselines are a minimum bar for this goal, not the fastest Go parsers; the JSON comparison moves to a jsontext baseline once it is measured.
• Work limits on by default. Every parse gets a work budget proportional to the input length (c·n). At first, c is one fixed, large constant, calibrated on the repository grammars and ready-made parsers so that ordinary grammars stay well below it, and a lint warns about grammars outside the specification's sufficient conditions for linear work. An unlimited parse has to be requested explicitly. A complexity classifier, which is research, can later refine c per grammar. The overhead of the limits is to shrink alongside, but the default does not wait for it.
• One place for production limits. Optional caps on input length and on estimated memory (memo entries and result trees) fail with the same kind of error as the work limit.
• Choose the trade-off. Generation profiles for speed, memory and size, each a named set of optimization passes, so a program can choose the smallest parser or the fastest one (design record 063, accepted, not implemented).
• Every target and use. Recognition, streaming, Document editing and generated TypeScript each have their own targets, and each target language or engine added later gets its own against the fastest tools for its platform.
• Prove it in public. A cross-tool benchmark with fixed workloads, stated machines and a published harness, starting with the Go tools that this page names as alternatives, so that any comparison can be rerun.

The project strategy gives the measurable targets and the mechanisms planned to reach them.

#Current Limits

• Pre-release. There is no tagged release yet; the language, the Go API and file formats (.pegoc, JSON forms, bytecode) may change between versions with no compatibility shims. The ready-made parser modules cannot be fetched with go get from another module (Ready-Made Parsers). PEGO itself requires Go 1.27.1.
• Not safe by default for untrusted input yet. The nesting cap is on by default; nothing else is. Default work limits and optional input-length and memory caps are committed goals (Commitments and Targets).
  • Work limits and cancellation are opt-in. Generated parsers carry them only when generated with -work-controls, which costs 1.30–1.47× on generated Go Parse (JSON and XML), and the three engine backends take 1.45–1.79× with a context or limits active (Performance).
  • Both limits are needed. MaxSteps alone does not bound a loop that only scans input; MaxInputWork does.
  • The step and input-work counts are deterministic for a given artifact, input and options. Context cancellation is as timely as the caller's clock, and a blocked reader or callback cannot be interrupted.
  • There is no memory or input-size limit; a whole parse of 27 MiB of CSV peaked at about 1.0 GB. Work limits do not bound memory.
  • pego parse has no flag for work limits.
• No linear-time guarantee. Some grammars take quadratic, cubic or exponential time, and no tool classifies a grammar or warns about it yet (design record 064, proposed).
• Not the fastest yet. Generated Go is slower than Go's hand-written parsers on every comparable workload measured, and no benchmark against another parser framework exists yet (see Measured Today).
• Two targets and one engine language. Parsers are generated for Go and TypeScript only, and only the Go VMs implement the bytecode.
• Not yet editor-grade.
  • A Document returns a tree only if the grammar's #recover rules handle each error. #missing is not available in a Document, so a buffer with an unclosed construct that no #recover rule covers gets an error, not a tree.
  • Applying an Edit takes time that grows with the document. At 100,000 lines, replacing one digit takes about 0.33–0.36 ms, mostly copying the text (Apple silicon, Go 1.27.1, guide program). A length-changing edit also shifts the reused entries after it; that cost is not measured separately.
  • At 100,000 key = value lines (1.9 MiB of text), a Document with its tree holds 151 MiB, against 27 MiB for the tree alone (Incremental Parsing Performance).
  • #stream has no effect in a Document, and recognition-only parses cannot use one.
  • Each Parse returns a complete tree. Opt-in change notifications report edits and parse outcomes, but there is no changed-range API.
  • There is no language server, highlighting, formatter or UTF-16 position map for a language defined with PEGO; the language server and the VS Code extension serve .pego grammar sources.
• Backend gaps. Generated TypeScript has no streaming and no Document; generated Go has a Document subset; #missing runs only on the closure backend (Compatibility and Support).
• Language gaps. No imports or multi-file grammars; case-insensitive matching folds ASCII only.
• Evidence base. Benchmarks come from one machine (Apple M3 Max) and compare only with the Go standard library. Recovery quality is not measured.

#How PEGO Compares

Comparisons checked on 2026-10-29. Each row is checked against the tool version it names. "Not documented" means not found in the tool's documentation and source listed under Sources, which is weaker than "absent". The tables describe documented behavior; they are not benchmarks.

#Parsing Model and Features

#Platform and Safety

PEGO's closure backend and recursive VM use the Go stack for rule calls, and generated TypeScript uses the JavaScript stack, which is why PEGO caps nesting depth by default; the iterative VM keeps rule calls on its own stack (Choosing a Backend).

Who covers which part today. Typed values built from a grammar: ANTLR (context classes), participle (your structs) and gotreesitter (typed query matches) cover parts of it. A parser for Go and for the browser from one grammar: ANTLR, tree-sitter and gotreesitter. Recovery: tree-sitter, Lezer and ANTLR recover automatically, while PEGO's is declared. Incremental reparsing in pure Go: gotreesitter and gpeg. Bounded work: gotreesitter (on by default) and PEGO (opt-in today). PEGO's case is that one grammar covers all of these for a language you own, with stdlib-only Go and dependency-free TypeScript. It is not a claim of being better at any one of them.

What other tools do better today.

• ANTLR generates parsers for ten target languages, and tree-sitter's C runtime has bindings for Go, Rust, Node and others and runs in browsers through Wasm. Both cover far more targets than PEGO's two today.
• tree-sitter returns a tree for every input, with ERROR and MISSING nodes and changed ranges. It also has persistent trees, included ranges for embedded languages and a random-edit fuzzer for every grammar author.
• gotreesitter brings 206 tree-sitter grammars to pure Go. Its deterministic work limits and per-parse memory budget are on by default, and it adds timeouts, UTF-16 coordinates and a Wasm runtime.
• Lezer has step-wise parsing with stopAt, mixed-language parsing and compact trees. ANTLR recovers automatically with no grammar annotations, has a large grammar library, and runs one runtime test suite against all its targets. Langium generates a full language server.
• gpeg applies edits in logarithmic time and has a peer-reviewed evaluation up to 100 MB. pest publishes the most candid resource policy of the group. pigeon offers left recursion and labeled-failure recovery in a Go PEG generator.
• Hand-written front ends such as rust-analyzer keep lossless trees that exist for every input. rust-analyzer's own notes say that, in practice, incremental reparsing does not matter much for IDE use and parsing from scratch is fast enough.

#Sources

PEGO facts are from this repository at the commit of this page. Performance figures are from Benchmark Results and Performance. Other tools, accessed on 2026-10-29:

• tree-sitter 0.27.1: documentation and README, api.h, fuzz.rs, static node types, Go binding (import "C"), list of parsers
• gotreesitter 0.56.0: README, API guide, parser_api.go (ParseWorkLimits), v1 design plan
• Lezer: guide, reference (PartialParse, parseMixed), lr README
• ANTLR 4.13.2: targets, left recursion, Go runtime (error_strategy.go), runtime test suite, Parr, Harwell and Fisher, Adaptive LL(*) Parsing (OOPSLA 2014, Theorem 6.3)
• Langium 4.4.0: documentation, DocumentBuilder
• Ohm 17.5.0: API reference, @ohm-js/cli type generation, 18.0 release notes; Dubroy and Warth, Incremental Packrat Parsing (SLE 2017)
• pigeon 1.3.0: doc.go
• participle 2.1.4: README
• gpeg: README and source at commit d601f967 (no tagged release); Yedidia and Chong, Fast Incremental PEG Parsing (SLE 2021; AMD Ryzen 5 1600, Go 1.16; not re-measured on the checked commit)
• pest 2.9.5: SECURITY.md, left-recursion check
• cpp-peglib 1.30.0: README
• grmtools 0.15.0: error recovery; Diekmann and Tratt, Don't Panic! Better, Fewer, Syntax Errors for LR Parsers (ECOOP 2020)
• cel-go 0.30.0: README ("Why not JavaScript, Lua, or WASM?"); Expr 1.17.8: README
• rust-analyzer: syntax notes ("Incremental Reparse")