pagers

Architecture

pagers is a Cargo workspace — a virtual root manifest plus one crate per concern. The dependency direction is acyclic:

pagers-cli pagers-mcp pagers-build pagers-markdown pulldown-cmark pagers-theme embedded assets pagers-graph html / json / dot pagers-site — content model, zero deps pagers-wiki reqwest · scraper · htmd

Crates

Crate Role External deps
pagers-cli pagers binary: clap CLI, build/serve/init/wiki commands clap, pagers-*
pagers-mcp pagers-mcp binary: stdio MCP server (rmcp) rmcp, serde, tokio
pagers-build Build pipeline: discover → render → write → copy → graph pagers-*
pagers-site Content model: discovery, frontmatter, config, URLs, nav, link resolution none
pagers-markdown Markdown → HTML (pulldown-cmark), outlines, alerts, link rewriting pulldown-cmark
pagers-theme Embedded md.css, site chrome, page.html template none
pagers-graph Page-link graph extraction + HTML/JSON/DOT rendering pagers-markdown
pagers-wiki Wikipedia: URL parse → REST fetch → extract → convert → normalize reqwest, scraper, htmd, …

pagers-build exists so both the CLI and the MCP server share one pipeline; pagers-wiki is deliberately independent of the site crates — its stages (url → client → extractor → converter → normalizer → metadata) are reusable for other content sources.

Build pipeline

discover pages + assets + config render markdown → html + edges write pages trailing-slash html copy assets non-markdown files emit graph graph.html · meta files

Everything is deterministic and offline — no network access during a build.

Conventions

  • Edition 2021/2024 idioms, thiserror for library errors, clap derive for the CLI.
  • No unsafe, no unwrap()/expect() on fallible paths in library code.
  • Small focused modules: pagers-wiki splits url/client/extractor/ converter/normalizer/metadata/error into separate files.
  • Tests live beside the code (#[cfg(test)]) plus tests/ integration suites with local fixtures — no live network in tests.

Developing

cargo build                  # debug
cargo build --release        # release → target/release/pagers
cargo test --workspace       # all crate tests
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --check

CI (.github/workflows/ci.yaml) runs fmt/clippy/tests on every push and PR. On master it also builds the release binary, builds the landing site in packages/landing (Next.js static export rendering these docs at /docs), deploys it to GitHub Pages, and publishes the rolling latest release.