016. Web Site and Playground

• Status: Implemented
• Author: @ornew
• Date: 2026-10-08

#Summary

PEGO has a static web site, deployed on Netlify at pego.ornew.net: a landing page, the documentation rendered from the repository's Markdown, an API reference generated from the Go source, and a playground where a grammar and an input are edited side by side and the result is shown as you type. The playground runs the PEGO library compiled to WebAssembly in the browser. The site is generated by a Go program in a separate module (site/), so Go is the only tool needed to build it; the generated files are not committed.

How to use the playground is described in the playground guide; site builds and deployment are covered by website maintenance.

#Goals

• An entry point. Someone who finds the project should be able to try a grammar within seconds, without installing anything, and share what they wrote.
• No drift. Every page is generated from the repository at build time: the documentation from its Markdown, the landing page from README.md, the reference from the Go source, the examples from examples/. Nothing is copied by hand.
• Exactly the library. The playground must show what the library and the pego command produce, not an approximation.
• No dependencies for the library. The root module stays free of dependencies; the playground uses only the public API.
• Static, offline-capable pages. No server-side code, no CDN, no web fonts, no analytics; any static server can serve the output.

#Design

#The WebAssembly API (cmd/playground/)

cmd/playground/ is a main package of the root module. Built with GOOS=js GOARCH=wasm, it installs a global object pego with five methods (compile, parse, format, generate, version). Each takes and returns JSON:

• JSON keeps the boundary small and identical for the browser, a Web Worker and Node, and it lets the API be tested natively: the request handling is in a file without build tags, and only the few lines that register the functions with syscall/js are WebAssembly-specific. A native main (behind the opposite build tag) keeps go vet ./... and go test ./... working.
• parse returns the tree as the exact text pego parse prints (an indented json.Encoder) and as the S-expression (Node.String), rather than as a JavaScript object. The page parses the JSON for the tree view, and the smoke test can compare bytes with the command.
• Errors are converted with errors.As into the public SyntaxError and SyntaxErrors. Grammar and compile errors are only available as text through the public API, one error per line in the form line:col: message; the API splits them into diagnostics with positions. This keeps the playground on the public API, at the cost of depending on the format of those messages, which the command line relies on as well.
• The last compiled grammar is cached by its source, so typing in the input does not compile the grammar again.
• A panic in a request is recovered and returned as an error, so a bug does not stop the WebAssembly program.

#Nesting and the Browser's Stack

Go compiled to WebAssembly makes every Go function call a WebAssembly call, so recursion consumes the JavaScript engine's native stack, which is about 1 MB and cannot be raised by a page. When it runs out, the engine throws ("Maximum call stack size exceeded", "too much recursion" in Firefox) and the Go program cannot continue. The engine's own limit of 100,000 nested rule calls is far beyond that, and three things recurse: the closure and bytecode backends (once or more per rule call), encoding/json and Node.String (once per tree level).

Measured with the examples (nested JSON arrays, parenthesized calculator and minilang expressions, nested minilang blocks), with the limits disabled:

The playground sets WithMaxDepth(600) for the backends that recurse, and refuses to encode trees nested more than 400 levels (measured without recursion, and reported with the depth). Both leave a margin of more than two over Chromium's worker, because a rule call takes more stack in rules with deeply nested expressions and other browsers have other stack sizes (Firefox and Safari were not measured). The iterative backend gets no rule limit: its rule calls do not recurse, and its trees are caught by the tree limit. Natively, the tree limit is 4,000, below what encoding/json accepts (10,000 levels of JSON, two per node level). As a last resort, the page restarts the worker when a call fails with a stack overflow and says that the parse needed more stack than the browser provides. The smoke test checks in Node that nesting of 400 and 5,000 levels ends with these errors and leaves the program running.

#The Page

The playground runs pego.wasm in a Web Worker. A long parse cannot freeze the page, and a parse that runs longer than 10 seconds is stopped by terminating the worker. The page compiles the module once (WebAssembly.compileStreaming) and sends the compiled module to each worker, so a restart takes milliseconds.

The client (pego-client.js) queues requests and sends them to the worker one at a time, which is how the worker would run them anyway. This makes the timeout count only the time the worker spends on a request, not the time it waited behind another, and lets a restart lose nothing but the request that timed out: the requests queued behind it, such as the parse of the input the user corrected meanwhile, run on the new worker. A request with a key (parse, generate) replaces a queued request with the same key that has not started, so typing quickly does not queue a parse per keystroke. If a worker cannot start (for example, pego.wasm cannot be downloaded), the queued requests fail and the next request tries again. Requests are also debounced and numbered, and the page drops stale responses.

The editors are textareas with a highlighted copy of the text behind them (the textarea's own text is transparent and its caret visible). This gives syntax highlighting for grammars, marks for errors and for the node under the pointer, and line numbers, while keeping native editing, undo, selection and input methods. A code editor library (CodeMirror, Monaco) would be better at large files but adds a dependency and a build step for the page; the textarea approach is a small class in app.js. Inputs over 200,000 characters fall back to plain text.

Positions in results are in the parse's unit (code points or bytes), and JavaScript strings are indexed in UTF-16 code units. The page builds a map between the two for each parse, so highlighting is exact for any text, including characters outside the Basic Multilingual Plane, in both units.

The tree view renders at most 100 direct children per page. Previous/Next replace the page, and display its range and total; lazy child descriptors avoid building wrappers for every sibling. All node rows and page controls count toward a shared 400-row initial creation budget. Expand shown expands the displayed pages, with a 5,000-row live cap; it does not advance through every sibling page. Each child page has at most 100 child rows plus one control row. Reaching the live cap compacts other materialized branches, then recreates the requested path and its page within the cap. Removed pages release row mappings and selection references, and replacement controls restore focus.

Moving the input caret follows the last containing child at each level, using half-open spans. It materializes only missing path children, so reaching a late sibling does not render the preceding siblings. If the ancestor path itself cannot fit with room for expansion, the view shows the selected subtree and a Show complete tree button. The original model remains available for later caret moves and returning to the bounded initial view. These limits bound DOM work independently of the worker's parse timeout; they do not reduce the parsed tree.

State in the URL. The grammar, input and options are written to the URL fragment after each change, compressed with CompressionStream("deflate-raw") and encoded as base64url (#z=), or as plain JSON where compression is not available (#j=). Compression matters because a grammar of a few kilobytes would otherwise make a link of several; the fragment is never sent to the server. No storage or account is needed to share. Incoming links normalize and apply all state fields, including links that change only parsing options or the tab. Grammar, input, start rule, position unit, backend and recognition changes trigger parsing. Tab-only changes preserve unchanged editors and reuse the result, with code generation on the Go tab. Controlled application-wiring tests exercise the hashchange handler and distinguish parse requests from presentation-only navigation.

Asynchronous URL work belongs to the most recent user action. Editing, changing options or tabs, choosing an example, sharing, and following a link invalidate older encodes and decodes immediately, before debounce timers or compression finish. An incoming link blocks autosaves from older parse completions until it is applied or superseded by a local action. Initial decoding and example fetching follow the same rule, so they cannot replace later edits or navigation. Writes check both the action revision and the adopted fragment: navigation changes the fragment before delivering hashchange, and an old timer must not publish during that interval. Share copies only its own successfully published link; a superseded request does not copy. The landing page's live-example link also keeps only the newest input's encoding. Encoding and decoding failures leave the existing state/link intact and report a console warning.

#The Site Generator (site/)

The generator is a Go program in its own module, github.com/ornew/pego/site, with one dependency: the Markdown renderer goldmark.

The generator:

• Renders each Markdown file under docs/, spec/ and examples/README.md into a page whose URL mirrors its path (docs/guide/runtime.md becomes docs/guide/runtime/, a README.md becomes its directory). Relative links are rewritten: to another rendered file, to a relative URL of its page; to any other file of the repository, to the file on GitHub. A link to a file that does not exist is an error.
• Generates heading anchors like GitHub, so that the many #fragment links in the documentation keep working.
• Builds the landing page from README.md: the tagline, the introduction, the first pego code block (as a live example), the list under "Why PEGO" (as feature cards) and the following sections. If the README stops having that shape, the build fails instead of producing an empty page.
• Writes a search index (the text of every page and its headings, about 0.7 MB, loaded on first use of the search box). A search service or a prebuilt index library would add a dependency or a third party for a site of this size.
• Checks every internal link and anchor of the output, and fails the build if one is broken; it also fails if a Markdown file under docs/ or spec/ belongs to no section of the navigation.

All URLs in the pages are relative, so the output works at any path and from any static server.

#Deployment

netlify.toml builds with sh site/build.sh and publishes site/dist, with GO_VERSION set to the version in go.mod. It sets a Content-Security-Policy that allows only the site's own scripts, styles and data, plus 'wasm-unsafe-eval' for compiling pego.wasm. The pages therefore have no inline scripts or styles (the test checks this): the theme is applied by a small blocking script file, and Markdown tables use align attributes instead of style.

The WebAssembly binary is the one large file of the site, and it changes only when the library or the Go version does. The build names it after a hash of its content (playground/wasm/pego-<hash>.wasm) and records the name in every page (<html data-wasm>), so netlify.toml can let browsers cache it for a year without revalidating (Cache-Control: immutable). Asset retention imports the published current WASM/runtime pair on production builds, retaining one preceding generation without storing binaries in Git. The worker selects its matching content-hashed Go runtime from playground/assets.json; the stable wasm_exec.js supports pre-manifest workers. Pages, application scripts and the manifest revalidate on each visit. Older generations still require a reload; overlapping production promotions need serialized publication or explicit previous-site input.

#Sizes

Stripping symbols saves little for WebAssembly (about 140 KB). Most of the binary is the Go runtime, reflect and encoding/json, and the Go printer and parser packages that gen.Go.Generate uses to format generated code. TinyGo might produce a much smaller binary, but it would be a second compiler to install and to test the library against; it was not evaluated. The landing page loads the binary only when its live example is about to become visible.

#Testing

• cmd/playground/api_test.go tests the API natively.
• TestWasmSmoke builds pego.wasm and the pego command and runs cmd/playground/testdata/smoke.mjs in Node. For every example of cmd/playground/examples.txt it compares the JSON, the S-expression, the errors, recognition, every backend, byte positions, fmt and gen -types with the command. It is skipped without Node and in -short mode.
• site/site_test.go builds the whole site and checks that every document became a page, that the landing page, the reference and the search index are complete, that no page needs inline code, and that no link is broken.
• site/testdata/state_ordering_test.mjs executes the actual playground and landing scripts with deferred encoding, decoding and example fetching. It controls completion order across edits, options, tabs, examples, Share, startup and navigation, including fragment changes before event delivery and error paths. Real browser checks complement these controlled DOM/client/timer tests with the WebAssembly worker.
• site/testdata/tree_test.mjs checks the actual tree module with a controlled DOM: initial and expansion row budgets, lazy sibling access, page replacement, deferred-child reveal, deep-path subtree windows, cap boundaries, row-reference cleanup and page focus. The controlled DOM does not model browser layout; browser checks exercise the real worker and tree controls. bench_tree.mjs measures the actual parent/candidate sources in the same VM; small trees compare equal displayed work, while wide trees intentionally display fewer rows.

#Alternatives Considered

• A server that parses. Simpler for the page, but it would need hosting, would see every grammar and input, and would not work offline. WebAssembly keeps everything in the browser.
• Running the WebAssembly module on the page's main thread. Simpler, but a slow grammar or input would freeze the page with no way to stop it.
• Rendering every sibling when opening a node. Lazy recursion alone does not bound a wide node's DOM work. Replacing child pages gives explicit, bounded navigation; full virtualization would add scroll-height and focus management that the present outline does not need. A subtree window preserves deep-node reachability when its ancestors cannot all coexist under the row cap.
• Committing the built site, or building it with GitHub Actions and Pages. Committing generated files invites drift and large diffs; Netlify builds from the repository on each push and also gives deploy previews for pull requests.
• Linking the reference to pkg.go.dev only. It cannot show unreleased changes, and it does not show the fields and methods of the types that the public API exposes through aliases.

#Limitations and Future Work

• The playground does not offer streaming (ParseStream) or incremental parsing (Document), which have no natural form in a single editor; it could show #stream elements as they are emitted.
• Very large inputs are shown without highlighting, and very large trees are expanded only partly.
• The JavaScript API could be published on its own for other tools that want to run PEGO in the browser.