Versioned Beta Docs Hosting Plan

This document captures the proposed implementation for hosting the VitePress documentation at docs.beta.sqlfluff.com using Netlify for serving and Cloudflare R2 as the persistent store for versioned builds.

Production cutover preparation (2026-10-02)

The beta site now hosts all 54 final releases from 2.0.0 onward. The latest deploy on main succeeded, but docs.sqlfluff.com still serves Read the Docs. The older implementation notes below describe the original rollout plan.

A URL audit compared the 44 document paths in the current Read the Docs stable search index, plus its index utilities and the latest/stable roots: all 48 work on the current production site, while 14 worked on the beta site before the cutover redirects. The other 34 legacy paths now have redirects to built VitePress pages. In particular, the old production-use overview has its own landing page and production/security.html goes to the security guide. The old Sphinx search.html, genindex.html, and py-modindex.html URLs lead to the nearest VitePress starting pages; they do not reproduce the old indexes. The live redirect behavior must be checked after this change is deployed.

Indexing policy:

• docs.sqlfluff.com/en/stable/ is the only indexed documentation version. Its HTML has canonical URLs on docs.sqlfluff.com.
• The site root and /en/ land on /en/stable/ whenever stable is published, including on beta. A latest-only site lands on /en/latest/.
• /en/latest/ and numbered releases remain accessible through the picker and direct links but get X-Robots-Tag: noindex. We do not claim their different content is a duplicate of stable.
• /en/versions.html may be indexed so readers can find older releases.
• After production activation, docs.beta.sqlfluff.com and the site's unique-mooncake-626ae6.netlify.app hostname redirect to the same path on docs.sqlfluff.com with a permanent redirect.

The deployment remains in beta indexing mode by default. To activate the production policy, first attach docs.sqlfluff.com to the Netlify site and cut its DNS over. Then set the GitHub Actions repository variable DOCS_INDEXING_MODE=production and run the docs deployment workflow from main. That run prepares the existing stable HTML even if it builds only latest, publishes the stable-only sitemap and robots.txt, changes the root redirect to stable, and removes beta-wide noindex headers. Verify the stable, archive, beta, and Netlify hostname responses after the deployment.

The goal is to mirror the useful parts of the current Read the Docs model:

• docs.beta.sqlfluff.com/en/latest/ built from main
• docs.beta.sqlfluff.com/en/stable/ built from the newest non-prerelease release
• docs.beta.sqlfluff.com/en/<version>/ built from release tags
• A version picker which works across all published versions

Summary

The recommended model is:

1. GitHub Actions builds one docs target at a time.
2. Cloudflare R2 stores the canonical assembled site tree.
3. Netlify serves the assembled site snapshot.
4. A shared manifest at /en/versions.json drives the version picker.
5. VitePress handles latest, stable, and releases from 4.2.2 onward.
6. Sphinx can later be used for pre-cutover releases.

This avoids rebuilding every historical release on every deployment. Each workflow run rebuilds only the version which changed, merges that output into the assembled site tree, updates the manifest, and republishes the assembled site.

Current Implementation Snapshot

The sections below still describe the target architecture, but the repository now contains a first working slice of that design.

Implemented in the repo so far:

• VitePress can now build against a configurable docs base using SQLFLUFF_DOCS_BASE, with SQLFLUFF_DOCS_NOINDEX adding a beta-friendly robots tag.
• Shared path handling for base-aware assets and redirects now lives in docsv/.vitepress/path-utils.ts, keeping the VitePress config, main theme, and 404 redirect handling aligned.
• The generated docs scripts now write UTF-8 explicitly so Windows builds do not fail on non-ASCII content.
• docsv/package.json now runs the Python prebuild step directly in docs:build, avoiding recursive pnpm invocation problems.
• docsv/scripts/assemble-site.py assembles a minimal published site tree for Stage 1 with /en/latest/, /en/versions.json, Netlify _redirects, and Netlify _headers.
• .github/workflows/publish-docs.yaml now implements a latest-only beta publish workflow that builds VitePress, assembles the site, uploads an artifact, syncs the snapshot to R2, and deploys the same snapshot to Netlify.
• The workflow now passes R2 S3-compatible credentials directly to the aws CLI sync step rather than using the AWS-only aws-actions/configure-aws-credentials action.
• The workflow currently triggers on pushes to main, pushes to ac/docsdeploy, and workflow_dispatch.
• The three trigger paths now resolve through a single Determine docs publish target step rather than repeating the release semantics in each step condition, which is what lets the manual path reuse them.
• workflow_dispatch accepts version, refresh_stable, prerelease, and published_at, so a release tag can be rebuilt and stable repointed without reissuing a release. Refreshing stable from a prerelease is refused, matching the release path and the channel policy below.
• docsv/scripts/smoke-check-assembled-site.py validates the assembled tree before it is uploaded: the manifest parses, every version it advertises has a real index page, the root redirect points at a published default, and the Netlify _headers file exists.
• assemble-site.py rebuilds a manifest entry from scratch on each run, so it now carries an existing release's published_at forward when one is not supplied. Without that, a manual rebuild would silently erase the date that the version picker displays.
• The VitePress theme carries the version picker and the stale version notice, both reading /en/versions.json at runtime. They are described under Version Picker Design below. When the manifest cannot be reached they degrade differently: the picker falls back to naming the current version as plain text, with nothing to switch to, while the notice renders nothing at all, since it cannot know whether a better version exists.
• The docs base defaults to /en/latest/ rather than an unversioned path, so a local run matches the published layout. The picker reads the current version out of the base, so an unversioned base cannot show it at all.
• Because the manifest lives at the language root, above any single version's base, the dev server cannot serve it from public/. A dev-only Vite plugin serves .vitepress/dev-versions.json at that path so both components can be worked on locally.

Validated locally so far:

• A Windows build of corepack pnpm run docs:build succeeded with SQLFLUFF_DOCS_BASE=/en/latest/ and SQLFLUFF_DOCS_NOINDEX=1.
• The assembled site helper produced the expected site/en/latest/, site/en/versions.json, _redirects, and _headers outputs.

Validated live so far:

• A successful GitHub Actions publish using the repository secrets deployed the Stage 1 latest-only snapshot.
• https://docs.beta.sqlfluff.com/ responds with an HTTP 302 redirect to /en/latest/.
• https://docs.beta.sqlfluff.com/en/latest/ serves the assembled VitePress snapshot with a noindex,nofollow robots tag.
• https://docs.beta.sqlfluff.com/en/versions.json serves the expected minimal manifest with a single latest entry.

Still not confirmed from this repository alone:

• Release-driven /en/<version>/ and /en/stable/ publishing.
• A live rollback or rebuild flow using the assembled snapshot in R2.
• The manual rebuild path. The dispatch inputs and the publish-target logic have been tested in isolation, but no run has yet rebuilt a real tag or repointed stable against the live site.
• The version picker and notice against more than one published version. Both have only been exercised locally, where the manifest comes from a fixture.

Architecture

Responsibilities

• GitHub Actions: Build docs, update the assembled site, and trigger deploys.
• Cloudflare R2: Durable store for the published multi-version site tree.
• Netlify: Serve the assembled snapshot at docs.beta.sqlfluff.com.
• VitePress: Render latest, stable, and future release docs.
• Sphinx: Later fallback for versions older than the VitePress cutover.

Why R2 Is The Source Of Truth

Netlify is good at serving static content, but each deploy effectively replaces the deployed site. That is not a good fit for long-lived version archives.

R2 is a better canonical store because it lets the workflows:

• keep old built versions indefinitely
• update only one version subtree at a time
• recover older versions without rebuilding everything else
• survive dependency drift in historical tags

Canonical Site Layout

The assembled site stored in R2 should look like this:

site/
  en/
    latest/
    stable/
    4.3.0/
    4.2.1/
    versions.json
    shared/
      version-picker.js
      version-picker.css

Notes:

• latest is built from main.
• stable is built from the newest final release.
• Release folders are built from git tags.
• shared/ holds assets reused by both VitePress and Sphinx outputs.

Snapshot Storage Strategy

R2 should retain both of the following:

• an exploded assembled site tree for the current published view
• immutable assembled snapshot archives for rollback, audit, and manual imports

This keeps day-to-day publishing simple while preserving a stronger rollback and recovery path.

Build And Deploy Model

Each deploy follows the same high-level pattern:

1. Checkout the target ref.
2. Download the current assembled site snapshot from R2.
3. Build only the requested docs variant.
4. Replace only the relevant subtree in the assembled snapshot.
5. Regenerate /en/versions.json.
6. Upload the updated snapshot back to R2.
7. Deploy that assembled snapshot to Netlify.

This keeps the build incremental while still publishing a complete static site.

Deploy From main

Trigger: push to main

Expected result:

• Build VitePress once with base /en/latest/
• Replace only /en/latest/
• Regenerate versions.json
• Leave /en/stable/ and all release folders untouched
• Sync the assembled tree to R2
• Deploy the assembled tree to Netlify

Deploy From Release Tag

Trigger: GitHub release published

Expected result:

• Checkout the release tag
• Build VitePress for /en/<version>/
• Replace only /en/<version>/
• If the release is not a prerelease, also build VitePress for /en/stable/
• Regenerate versions.json
• Sync the assembled tree to R2
• Deploy the assembled tree to Netlify

Important detail:

stable should be built separately rather than implemented as a redirect or a copy of /en/<version>/. VitePress bakes the base path into links and asset paths, so /en/stable/ wants a dedicated build with its own base.

Manual Rebuild Of Older Release

Trigger: workflow_dispatch

Expected result:

• Accept a version tag as input
• Validate the tag exists
• Determine whether to use VitePress or Sphinx
• Build only that version
• Replace only /en/<version>/
• Optionally refresh /en/stable/ when explicitly requested and appropriate
• Regenerate versions.json
• Republish the assembled snapshot

Version Picker Design

Registry Model

Published versions should not need to register themselves with each other at build time. Instead, the picker should read a shared runtime manifest from /en/versions.json.

That manifest should be updated by the publishing workflow each time a version is added, rebuilt, promoted to stable, or hidden.

Initial Manifest Shape

The exact schema can evolve, but a practical initial contract is:

{
  "default": "stable",
  "latest": "latest",
  "stable": "4.3.0",
  "versions": [
    {
      "key": "latest",
      "label": "latest",
      "title": "Development",
      "path": "/en/latest/",
      "kind": "channel",
      "builder": "vitepress",
      "prerelease": false
    },
    {
      "key": "stable",
      "label": "stable",
      "title": "Stable",
      "path": "/en/stable/",
      "kind": "channel",
      "builder": "vitepress",
      "prerelease": false
    },
    {
      "key": "4.3.0",
      "label": "4.3.0",
      "title": "4.3.0",
      "path": "/en/4.3.0/",
      "kind": "release",
      "builder": "vitepress",
      "prerelease": false,
      "published_at": "2026-05-19"
    }
  ]
}

Picker Behavior

The plan was to switch to the selected version root first and preserve the current page path only once redirect parity was good enough. The shipped picker preserves the page path from the outset: a reader switching version stays on the page they were reading, and if that page does not exist in the target version they get that version's own 404.

That trade was taken deliberately. Losing your place on every version switch is a certain cost on every use, where a missing page is a possible cost on some. It is worth revisiting for mixed VitePress and Sphinx path compatibility, where the paths differ structurally rather than occasionally.

Two details are easy to break and worth knowing before changing this:

• Cross-version links must carry a target attribute. VitePress intercepts every same-origin link for client-side routing, which would resolve another version's page against the current build; a link with a target is skipped.
• The page path is taken from the current route rather than read once on mount, so it stays correct after a client-side navigation.

The picker is a disclosure of links rather than a select or a role="menu" widget. A native select cannot style its popup, so the closed control and the open list could not be made to match. Links, rather than a change handler, keep middle-click and open-in-new-tab working, and a list of links after the trigger is already reachable with Tab, which avoids reimplementing arrow-key movement and typeahead.

Stale Version Notice

Readers usually arrive from a search engine, which favours whichever version has accumulated links rather than the current one, so the picker alone does not help someone who does not know they need it. Published docs therefore also carry a notice naming the version being read, with a link to the same page in the recommended version:

• latest is flagged as development documentation, since it tracks main.
• A release which is not the newest is flagged as superseded.
• Nothing is shown when there is nowhere better to go, which also keeps the notice off a single-channel deployment.

The notice is driven by the same manifest as the picker, and decides from the manifest's ordering rather than from any date: version_sort_key in assemble-site.py sorts releases newest-first by version number, and a release which is not the first is treated as superseded. published_at is only used by the picker, to label each entry.

That means the ordering is load-bearing. A change to how releases are sorted changes which versions readers are warned about.

Shared Runtime Assets

The final picker should not be VitePress-only. Instead, publish shared assets under /en/shared/ so both VitePress and Sphinx versions can load the same runtime picker and manifest.

This avoids having to rebuild every historical version whenever a new version is published.

Workflow Layout

The workflows should be organized around one reusable deploy workflow plus thin trigger workflows.

Reusable Workflow

Purpose: perform the shared assembly and deploy steps.

Inputs should include:

• git ref or tag to build
• output target such as latest, stable, or 4.3.0
• builder type: vitepress or sphinx
• whether to refresh stable
• whether the target is prerelease

Trigger Workflows

1. Main docs deploy

   • Trigger: push to main
   • Builder: VitePress
   • Target: latest

2. Release docs deploy

   • Trigger: release published
   • Builder: VitePress or Sphinx depending on cutover policy
   • Target: <version>
   • Also refresh stable for non-prereleases

3. Manual docs rebuild

   • Trigger: workflow_dispatch
   • Builder: auto-detected from version and cutover policy
   • Target: explicit version tag

Compatibility And Operational Policy

Redirect Compatibility Scope

• Hard requirement: documentation URLs emitted by CLI and runtime surfaces in SQLFluff 2.0.0 and later must continue to resolve.
• Strong parity target: preserve all currently declared Sphinx permalinks and redirects.
• Best-effort target: preserve broader historical site structure where it helps with SEO or older inbound links.

This is a link-resolution guarantee, not a promise that every historical version is fully hosted on day one. When an older CLI or runtime link already targets a stable or permalink route, preserving that canonical route is sufficient.

Rebuild And Import Policy

• Historical rebuilds should default to rebuilding the tagged source with the currently approved builder image.
• A rebuild is considered acceptable when it preserves materially equivalent user-facing content, version identity, and important permalinks or redirects. It does not need to reproduce byte-identical HTML or search output.
• If a historical tag no longer builds cleanly with the maintained toolchain, maintainers may publish an archived static snapshot for that version instead.
• If neither a rebuild nor a snapshot import is practical, the version remains unpublished until a manual artifact is supplied.
• The one-time backfill of all 54 final releases from 2.0.0 onward was uploaded to R2 and deployed to beta on 2026-09-29. Later rebuilds update individual versions.

Release Channel Policy

• Final releases should publish /en/<version>/ and automatically refresh /en/stable/.
• Prereleases are published at their direct version URLs and are listed in the version picker, marked as prereleases rather than hidden. Hiding them was the original intent, but a version a reader can reach by URL and cannot find in the picker is harder to explain than one which is labelled, and the label is what stops it being mistaken for a final release. They are still kept off /en/stable/, which only ever points at a final release.
• latest is the only channel where edit links are important. Other channels may either omit edit links or point to main; version-accurate edit links are not required for the first rollout.
• The beta site may be publicly reachable, but it should remain noindex until cutover work begins.

Recommended Serving Defaults

• Treat HTML pages and /en/versions.json as mutable content and serve them with no-cache or must-revalidate semantics.
• Treat latest and stable as mutable channels.
• Cache fingerprinted static assets aggressively as immutable.
• If shared picker assets are not fingerprinted, keep them on a short cache.

Stable Promotion Safeguard

• Automatic promotion of stable is the default behavior for final releases.
• Maintainers should retain a manual override so stable can be repointed to an older version or snapshot if needed.

Staged Implementation Plan

Stage 0: Infrastructure Bootstrap

Status: complete for the current Stage 1 slice. The R2 bucket, Netlify site, custom domain, and repository secrets are now wired well enough to publish the live beta docs site.

Deliverables:

• Create the R2 bucket
• Create an R2 API token for GitHub Actions
• Create or reuse the Netlify site for beta docs
• Wire docs.beta.sqlfluff.com
• Add repository secrets
• Set the first VitePress-native release tag to 4.2.2, the first release actually published with the versioned VitePress pipeline.
• Decide whether prereleases appear in the picker — decided: they appear, marked as prereleases. See the release channel policy above.

Stopping point:

• No code deployed yet, but infrastructure and policies are ready

Stage 1: latest Only On Beta

Status: complete for the initial beta rollout. The repository workflow now publishes latest successfully and the beta site is live.

Deliverables:

• Make the VitePress base configurable at build time
• Replace the current GitHub Pages-only docs deploy path
• Add a workflow for pushes to main
• Build and publish only /en/latest/
• Generate a minimal versions.json

Stopping point:

• docs.beta.sqlfluff.com/en/latest/ is live
• R2 is storing the canonical assembled tree
• Netlify is serving the assembled tree

Stage 2: Manifest And Initial Version Picker

Status: complete for VitePress. The shared manifest is live and the picker reads it at runtime, alongside the stale version notice described above. Only the cross-builder work in Stage 5 remains before older Sphinx versions can use it.

Deliverables:

• Define the versions.json schema — done
• Add a basic picker to the VitePress theme — done
• Read version data from the shared manifest at runtime — done
• Keep initial switching behavior simple by targeting version roots only — superseded; the picker preserves the current page path instead, for the reasons under Picker Behavior above

Stopping point:

• latest has a working picker framework
• The registry model is validated before release builds depend on it

Stage 3: Tagged Releases And stable

Deliverables:

• Add release-triggered docs deployment
• Build and publish /en/<version>/
• For final releases, also build and publish /en/stable/
• Promote stable automatically using the existing non-prerelease semantics already used by the release process
• Retain a manual override so maintainers can repoint stable if required
• Update the manifest so release versions appear in the picker

Stopping point:

• /en/latest/, /en/stable/, and /en/<version>/ all work for VitePress releases
• Releases before 4.2.2 use Sphinx so their original content and links are preserved.

Stage 4: Manual Rebuild Workflow

Status: mostly complete. A chosen tag can be rebuilt from workflow_dispatch, and stable can be repointed, both validated before upload. Snapshot import and immutable snapshot archives are still outstanding.

Deliverables:

• Add workflow_dispatch for rebuilding a chosen version tag — done
• Validate the tag exists before build — done implicitly; the tag is the checkout ref, so a tag which does not exist fails the run before anything is built
• Rebuild only the requested version subtree — done; the existing tree is pulled from R2 first, so only the rebuilt subtree is replaced
• Support importing an archived static snapshot when rebuilding is not practical
• Keep immutable assembled snapshots so rollback and manual imports use the same artifact model
• Optionally refresh stable in controlled cases — done, and refused for prereleases

Stopping point:

• Historical versions can be republished without reissuing releases

Stage 5: Shared Picker Assets For Cross-Builder Support

Deliverables:

• Move picker runtime logic into shared JS and CSS under /en/shared/
• Keep versions.json as the single registry source of truth
• Load the same picker assets in VitePress and future Sphinx outputs

Stopping point:

• Older published versions do not need rebuilding just to learn about new versions

Stage 6: Sphinx Backfill For Pre-Cutover Versions

Deliverables:

• Use 4.2.2 as the first VitePress-native release
• Build versions older than that cutoff using the existing Sphinx toolchain
• Inject shared picker assets into Sphinx output
• Validate the assembled historical archive before its one-time upload
• Treat the initial proof as successful once latest, stable, and one older Sphinx-hosted version are live under the beta domain

Stopping point:

• Beta can host a mixed set of VitePress and Sphinx versions under one domain

Stage 7: Redirect And Compatibility Hardening

Deliverables:

• Review parity with current docs.sqlfluff.com URLs
• Preserve documentation URLs emitted by CLI and runtime surfaces in supported SQLFluff versions from 2.0.0 onward
• Preserve or replace important .html, permalink, and declared redirect routes
• Add or refine Netlify redirects where needed
• Confirm page-path preservation on version switch holds up against the redirect set, and decide how it should behave across the VitePress and Sphinx boundary where paths differ structurally
• Review edit links, 404 behavior, and search behavior across versions

Stopping point:

• Beta is close enough to production parity to be considered for cutover

Stage 8: Final Cutover

Deliverables:

• Point docs.sqlfluff.com at the Netlify-hosted assembled site
• Preserve the latest, stable, and versioned URL structure
• Retire or reduce dependency on Read the Docs

Stopping point:

• The new docs hosting becomes the primary production site

Repository Changes Expected In Early Stages

These files are likely to be touched first:

• docsv/.vitepress/config.ts
• docsv/.vitepress/theme/index.ts
• docsv/package.json
• .github/workflows/publish-docs.yaml
• one or more new helper scripts for manifest generation and site assembly

These files are likely to be touched later:

• pyproject.toml
• util.py
• docs/source/conf.py
• docs/generate-auto-docs.py

Secrets And Configuration

Expected repository secrets:

• R2_ACCOUNT_ID
• R2_ACCESS_KEY_ID
• R2_SECRET_ACCESS_KEY
• R2_BUCKET
• NETLIFY_AUTH_TOKEN
• NETLIFY_SITE_ID

Expected repository or project-level configuration values:

• first VitePress-native release tag, 4.2.2
• redirect compatibility floor, starting at SQLFluff 2.0.0
• whether manual rebuilds may promote stable
• whether the beta environment should emit noindex

Historical Backfill Scope

All final release tags from 2.0.0 onward are in scope. Prereleases are not part of this historical backfill; new prereleases continue to publish through the release workflow. The one-time local build mirrored existing Read the Docs Sphinx pages where available and built the remaining releases from their tags.

On 2026-09-29, a local dry run against the latest assembled deployment artifact completed all three major-series batches. The resulting manifest includes all 54 final releases (21 from 2.x, 22 from 3.x, and 11 from 4.x), and each has an index page. The major-series smoke checks passed. The assembled artifact was uploaded to R2 and deployed to the beta site.

Remaining Cutover Work

The historical backfill is complete. Before replacing the production docs site:

1. Recheck permalinks, archived Sphinx navigation and search, the version picker, and the beta noindex header after further docs deployments.
2. Complete the URL parity review against docs.sqlfluff.com.

Success Criteria

The plan should be considered successful when:

• main automatically publishes to /en/latest/
• each release automatically publishes to /en/<version>/
• the newest final release is available at /en/stable/
• the picker lists available versions from a shared manifest
• prereleases can be published directly and are listed in the picker as prereleases, without ever becoming stable
• rebuilding a historical version does not require rebuilding every other version
• documentation URLs emitted by SQLFluff 2.0.0 and later continue to resolve
• pre-cutover versions can still be hosted under the same beta domain
• the beta site can remain public while staying noindex until cutover
• the beta site is close enough to replace docs.sqlfluff.com when desired