Documentaion#
Our documentation spans the mkdocs in /docs and readme.md files in the source tree. It is viewable through the mkdocs site (servable locally), through GitHub or simply in your editor.
Mkdocs and GitHub differ slightly in the way they render content. We are trying to support both for documentation that lives in the source tree.
Mkdocs prerequisites#
Make sure to install documentation dependencies:
uv sync --group docs
How to#
Preview documentation locally#
uv run mkdocs serve
Preview documentation from a compute node#
Forward the port to your local:
ssh -L 8000:localhost:8000 -t <auth> ssh -L 8000:localhost:8000 <username>@<node_tailscale_ip>
where auth might be an ssh configuration alias. You will need to adjust for whatever means of authentication you are using. Replace <username> and <node_tailscale_ip> as well.
Add page to the nav bar#
Edit nav_lines in gen_ref_pages.py
Link a readme.md file#
- source tree: use repository-relative path
- mkdocs: use repository-relative path, then replace slashes with underscores
- see markdown preprocessing phase
- for details, view docstring of docs/hooks.py - transform_readme_links
Custom hooks#
A preprocessing hook system that automatically manages readme files from the source tree and transforms markdown content for proper documentation rendering.
Pre-Build Phase#
create_readme_symlinks - Discovers and symlinks README files
- Searches specified paths recursively, e.g. [
src/agoge/,tests/] - Creates symlinks in
docs/collected/(git ignored) - Gives unique and deterministic file name
- e.g.
src/agoge/module/readme.md→src_agoge_module_readme.md
- e.g.
- Cleans the collected links on each build
Markdown Processing Phase#
on_page_markdown - Transforms all markdown content during build
| Transformation | Example | Purpose |
|---|---|---|
| Remove TOC | Strips ## Table of Contents sections |
MkDocs auto-generates navigation |
Doc *.md links |
docs/getting-started.md → getting-started.md |
Adjust for MkDocs context |
Doc *.py links |
exception list → (GitHub URL) | Link to repository |
| README links | [text](collected/tests_readme.md) → [text](collected/tests_readme.md) |
Point to symlinks |
| Source links | (src/...) → (GitHub URL) |
Link to repository |
| Config links | (configs/...) → (GitHub URL) |
Link to repository |
Configuration#
| Setting | Example Value | Purpose |
|---|---|---|
GITHUB_ORG |
convergence-ai |
GitHub organization for URL generation |
COLLECTED_SYMLINKS_DIR_NAME |
collected |
Directory for symlinks |
README_SEARCH_PATHS |
["src/agoge", "tests"] |
Where to find READMEs |
Limitations & Notes#
- Only processes files in configured search directories
- Symlinks are recreated on every build (not cached)
- All transformations are applied globally to all markdown pages
Known problems#
Header/anchor tags sanitization#
Mkdocs reports:
INFO - Doc file 'contributing.md' contains a link '#communication--support', but there is no such
anchor on this page.
Specific to the markdown backend Mkdocs uses, the link still seems to work ¯\\_(ツ)_/¯. Difference in how escaping and sanitizing headings is handled and later expected to be linked (generating an ID for an anchor tag).
The preview does not update#
It seems that the preview command does not update when the sources change locally, which it should. Could be a problem of the material theme or the plugins. Just terminate the server and restart, it is fast enough.