intent/docs/plan.md

# Intent V1 implementation plan

Companion to [concept.md](concept.md). The concept says what Intent is; this says how V1 gets built, in what order, and which details were decided by default rather than in the concept. Where the two disagree, fix the concept first.

## 1. V1 scope

**In:** organizations, projects, features (markdown, revisions, hierarchy, files), issues, comment threads (feature, line, file, image coordinate, issue), implementations as plain URLs with per-feature manual status, auth via Forgejo/GitHub/Discord/magic link with account linking, sysadmin via Django admin, three project visibility levels, open contribution toggle, invites.

**Out (V1.1 or later):** notifications, search, public API or MCP, Forgejo API access of any kind (issue creation, release links, visibility propagation, commit-trailer status), status inheritance through the feature hierarchy, real-time updates.

## 2. Decisions and defaults

Decisions made in the concept are not repeated. These fill the gaps.

### Model

| Topic | Default |
|---|---|
| Primary keys | Opaque, non-sequential: ULID stored as a 26-char string, generated in Python (`python-ulid`), set as `default` on a shared abstract base model. Never exposed as a sequence; used in URLs only where no human-readable key exists (files, file versions, threads, comments). |
| Feature stable ID | Human-facing, separate from the PK: per-project prefix + sequence, e.g. `AUTH-12`. Prefix is set on project creation, uppercase, 2–8 chars, unique within the org. Number never reused. |
| Issue ID | Per-project sequence `#N`, separate from feature numbers. |
| Feature revision | Integer starting at 1, incremented by every commit to the feature: description edit, file add/replace/remove, parent change, status change. Each revision stores the full description text (not a diff); diffs are computed on display. |
| Commit note | Required, short, free text. Optionally linked to one or more issues. |
| Issue ↔ feature | Many-to-many. An issue promoted from a comment thread is linked to that thread's feature. Accepting an issue that creates a feature links both. |
| Issue statuses | `open`, `accepted`, `rejected`, `withdrawn` (author closes), `duplicate` (links to another issue). Collaborators can reopen any closed issue; authors can withdraw their own open issue and reopen it only if they withdrew it. Duplicate chains can't loop. Every status change, feature link and edit is an `IssueEvent` shown in the issue's timeline between comments; edits keep the previous title and text. Closed issues still take comments. |
| Issue → feature | From an issue, a collaborator follows "Create a feature from #N" or "Update INT-3 for #N". The feature form opens prefilled (title and description from the issue, or a change note naming it), ties the new revision to the issue, links the feature, and offers a checked "Mark #N accepted" box. Accept and Reject buttons also work on their own. |
| Feature statuses | `active`, `withdrawn`, `superseded` (requires `superseded_by` feature). |
| Implementation status | `active`, `archived`. |
| Feature × implementation status | `n/a`, `not_started`, `in_progress`, `shipped`, `retired`; stamped with the feature revision it applies to. Missing row displays as `not_started @ —`. If stamped revision < current revision, display "behind". Each implementation page is one form: a status select per feature, plus a "Still … at revision N" checkbox on rows that are behind, which re-stamps without changing the status. Only changed or confirmed rows are written. Implementation links must be http(s). Collaborators create, edit and archive implementations; project owners delete them (typed name to confirm). |
| Hierarchy | Single optional parent within the same project. Cycles rejected. Moving a feature records a revision. Statuses independent in V1. |
| Files | Belong to a feature, typed `prototype` or `documentation`. Adding, replacing or removing a file is a feature revision with a required change note; the revision records what happened ("Added prototype “a.png”") and shows it in history. Replacing creates a new file version; old versions stay (so old image-pin threads still resolve). Removed files stay too, so old revisions list the files they had. Size cap 25 MB; allowed types: images (png, jpg, gif, webp, avif, svg), PDF, Markdown/text, HTML, zip. The content type comes from the extension, never the browser. Stored under opaque paths (`files/<ulid>/<name>`). Markdown files render on their page; text shows as text; PDFs and zips open or download. |
| Line-anchored threads | Anchor = feature revision + start/end source line of one top-level Markdown block (paragraph, list, heading, code block). Each block has its own `<details>` comment form, so it works without JS. A thread goes "outdated" only when the description text has changed since its revision (a status-only revision doesn't outdate it); outdated threads move to "Comments on earlier revisions", quote their original lines, and link to the revision. No attempt to re-map lines. A line comment posted against a stale revision is refused and the draft is shown back. |
| Image threads | Anchor = file version + x/y as fractions (0–1). Pins are drawn as an SVG overlay using percentage coordinates (no inline styles), numbered per version, and each is a link to its thread. Placing a pin: two number fields (across %, down %) work without JS; clicking the image fills them in. New comments go on the latest version only; threads on other versions are listed separately with a link to that version. |
| File threads | Anchor = file version, like image threads without a point. |
| Comments | Edits keep history (visible "edited" link to a history page). Author or project owner may delete; a deleted comment shows as a tombstone and its text and history are removed. Delete asks for confirmation in a disclosure, so it works without JS. Resolving a feature thread is allowed for the thread author or any collaborator; resolved threads start collapsed. An issue has exactly one thread, its discussion, which isn't resolvable (the issue status says that). |
| Thread → issue | Original commenter (while they can contribute) or any project collaborator. Thread gets `promoted_to` issue and is resolved; the issue is linked to the thread's feature, its body is prefilled with the quoted lines and comments, and its page links back to the thread. |
| Deleted users | Content remains, attributed to a "ghost" user. |
| Moderation | Project owners can delete comments and close issues in their project. Sysadmin can deactivate a user site-wide. |
| Invites | By email or existing username. Username invites appear on the invitee's Invitations page (no notifications in V1). Email invites send a link carrying a random token (only its hash is stored); the link requires sign-in and the first signed-in visitor claims the invite with whatever account they used. Claiming rotates the token, so the link (which appears in request logs) is dead after one use; the claimant then accepts or declines from their Invitations page. Unclaimed email invites also appear on the Invitations page of a user with that verified email. Re-inviting replaces the pending invite. Sending is throttled (30 per sender per hour, 3 per recipient per org per hour). Expire after 14 days. The last org owner cannot be removed, demoted or leave. |
| URLs | Organizations live at the root: `/<org>/`, `/<org>/<project>/`, `/<org>/<project>/features/INT-12/`. App routes are reserved slugs at both levels. |
| Project slug and prefix | Fixed after creation, since they appear in URLs and feature IDs. Name, visibility and open contribution stay editable. |
| Hidden projects | A project the user can't see returns 404 (not 403), so private projects don't reveal they exist. Signed-out visitors are sent to sign in instead, for missing and hidden projects alike, so the redirect reveals nothing either. Markdown drops remote images (`<img>`) until uploads exist, so an author can't log readers' IP addresses. |
| Project owners | No last-owner guard: org owners can always administer every project. Leaving the org removes a user's project roles in it. |
| Feature editing | One form edits title, description, parent, status and replacement, with a required change note. Saving with no changes is refused. A "Preview description" button renders the Markdown server-side, so preview works without JS. Django admin shows features read-only so every change goes through revisions. |

### Stack

| Topic | Default |
|---|---|
| Runtime | Python 3.13, Django 5.2 LTS, `uv` for dependencies, `ruff` for lint/format. |
| Auth | `django-allauth`: GitHub, Discord, OpenID Connect provider per Forgejo instance (Forgejo supports OIDC). Sysadmins register each provider as a SocialApp in Django admin, so adding a Forgejo instance needs no deploy. Email sign-in is allauth's login-by-code: the "magic link" is an emailed one-time code, not a clickable link. Signup asks for email and username only; regular users never get a password, and allauth's password URLs are not routed at all. Adding an email address requires re-entering a fresh code, and existing addresses are notified. Multiple social accounts per user via allauth's connect flow (no automatic linking by email). Usernames: letters, digits, `-`, `_`, max 39. |
| Sysadmins | Django `is_staff` users with passwords, Django admin at `/admin/`. A model constraint rejects staff users as org members. Regular users have no password (unusable). Only superusers can grant staff or superuser status or permissions in the admin. |
| Cache | Django's database cache (`createcachetable` runs at start). It holds allauth's rate limits and the invite throttle, so it must be shared by all gunicorn workers and survive restarts; the database is the only shared store on a one-machine deploy. Dev and tests use the in-memory cache. |
| Database | SQLite on a Fly volume. `transaction_mode = "IMMEDIATE"`, WAL mode, `busy_timeout` 5 s. Litestream replicates to B2 continuously. |
| Files | Tigris via `django-storages` S3 backend, private bucket, time-limited signed URLs. Every file URL is an app view (`…/files/<id>/v<n>/<name>`) that checks project access first, then redirects to a 5-minute signed URL. Where the bucket isn't configured (dev), the view streams the file itself with `nosniff` and a `Content-Security-Policy: sandbox` header, so an uploaded SVG or HTML file can't act as the app. HTML prototypes are shown in an `<iframe sandbox="allow-scripts allow-modals allow-popups">` (no `allow-same-origin`): checked in Chromium, a prototype can't read cookies, reach the parent page, or make network requests. |
| Markdown | `markdown-it-py` with source line mapping (`data-source-lines` attributes on blocks) for line threads; output sanitized with `nh3`. Descriptions render block by block so each block can carry its threads. Diffs via `difflib` rendered server-side. |
| Frontend | Django templates, HTMX for partial updates, and a few lines of plain JS (`static/js/intent.js`) for focus after swaps and pin placement. Alpine turned out unnecessary, and its standard build needs `unsafe-eval`, which the Phase 5 CSP would forbid. CodeMirror 6 as a progressive enhancement on the markdown textarea, optional. No build step beyond copying vendored JS into static files. |
| Email | `django-anymail` with the provider pluggable; console backend in development. **Provider TBD; launch blocker for magic links and email invites.** |
| Background work | None. Email is sent synchronously. |
| Observability | Sentry (free tier), Fly logs. |
| Tests | `pytest-django` for models, permissions and views; a handful of Playwright smoke tests for HTMX flows. |
| Local dev | Docker Compose: one app container (Django runserver), SQLite in a volume. Uploads go to the local filesystem unless `BUCKET_NAME` is set, in which case they go to a real bucket (e.g. a Tigris dev bucket). No MinIO: its community images stopped in Oct 2025 and the repo was archived in 2026. `.env` for secrets. |
| Deploy | Fly.io, one machine, one volume. `bin/start` restores from Litestream if the volume is empty, runs migrations, then runs gunicorn under `litestream replicate`. Migrations can't use Fly's `release_command` because release machines don't mount volumes. CI assumed to be Forgejo Actions in the Intent repo: lint, test, `fly deploy` on main. |

### Styling and accessibility

The app should look close to unstyled HTML. Guiding rule: if the stylesheet fails to load, only layout suffers.

- Semantic HTML first: `<main>`, `<nav>`, `<article>`, `<section>`, `<h1>`–`<h6>` in order, `<button>` for actions, `<a>` for navigation, `<table>` for tabular data, `<details>` for collapsibles, `<dialog>` where a modal is needed.
- One small stylesheet (target under 5 KB). It sets: `color-scheme: light dark` on `:root`, a readable max line width, spacing, focus-visible outline, and a few utilities. Colors come from `light-dark()` or system colors; no custom palette beyond a link color and a muted text color.
- Dark mode via `prefers-color-scheme` from day one, with a three-way toggle (system/light/dark) stored in `localStorage` and applied via `data-theme` on `<html>`.
- Every HTMX interaction has a non-JS fallback: forms post normally, links navigate. HTMX enhances, never gates.
- Keyboard: all actions reachable by tab; skip link to `<main>`; visible focus. Image pins get a list alternative (threads listed below the image with coordinates). Line threads have a per-line "comment" button, not hover-only.
- Contrast and semantics checked with axe in the Playwright smoke tests.
- Until then, unit tests run a structural check (`tests/a11y.py`) on every page: `lang`, one `<h1>`, no skipped heading levels, unique ids, labelled form controls, `alt` on images. New pages get added to `tests/test_a11y.py`.
- Markdown headings are re-levelled so the shallowest heading in the text lands one below the surrounding page heading (`<h2>` in descriptions and issue bodies, `<h3>` in comments), keeping relative structure. User text never adds a second `<h1>`, and a comment starting at `##` doesn't skip a level.
- After an HTMX swap, focus moves to the new or edited comment, or to the thread (`data-autofocus`, `static/js/intent.js`).

## 3. Data model

Fields are indicative, not exhaustive. All models have a ULID `id` primary key plus `created_at`/`updated_at` via a common abstract base. Human-facing keys (`Organization.slug`, `Project.slug`, feature `AUTH-12`, issue `#N`) are separate columns used in URLs; the ULID appears in URLs only for models without one. Soft deletes only where noted.

```
User                 allauth user; no password for regular users; is_staff = sysadmin
Organization         slug, name
OrgMembership        org, user, role ∈ {owner, collaborator}, can_create_projects: bool
OrgInvite            org, email | user, role, token, expires_at, accepted_at
Project              org, slug, name, feature_prefix, visibility ∈ {private, authenticated, public},
                     open_contribution: bool, next_feature_number, next_issue_number
ProjectMembership    project, user, role ∈ {owner, collaborator}
Feature              project, number, parent (nullable, same project), title, status, superseded_by,
                     current_revision (int), created_by
FeatureRevision      feature, revision, description_md, note, author, parent_at_revision,
                     status_at_revision   (issues via Issue.revisions)
FeatureFile          feature, kind ∈ {prototype, documentation}, name, current_version,
                     added_in_revision, removed_in_revision
FeatureFileVersion   file, version, blob, original_name, content_type, size,
                     created_in_revision, uploaded_by
Implementation       project, name, url, status ∈ {active, archived}
FeatureImplStatus    feature, implementation, status, feature_revision, updated_by   (unique feature+impl)
Issue                project, number, title, body_md, author, status, duplicate_of, closed_at,
                     features (M2M), revisions (M2M FeatureRevision)
IssueEvent           issue, actor, kind ∈ {status, linked, unlinked, revision, edited}, status,
                     previous_status, duplicate_of, feature, revision, old_title, old_body
CommentThread        project, kind ∈ {feature, line, issue, file, image}, file_version, x, y,
                     feature, issue, revision, line_start, line_end, author,
                     resolved_by, resolved_at, promoted_to (OneToOne issue)
Comment              thread, author, body_md, edited_at, deleted_at, deleted_by
CommentEdit          comment, body_md (text before the edit), created_at
```

Notes:
- `FeatureRevision` is append-only. `Feature` holds denormalized current values for cheap listing.
- Feature number and issue number are allocated inside a transaction with `select_for_update` on the project row (fine under SQLite's single-writer model).
- Staff users are kept out of orgs in three places: `OrgMembership.clean()` (admin forms), the membership services, and `intent/access.py`, which grants staff nothing even if a row exists. A database constraint can't express this because it spans tables. Staff signing in through allauth are redirected to the admin login.
- Thread target is validated in `clean()`: exactly the fields required for its `target` type are set.

## 4. Permissions

Resolved per request into a small `Access` object (`can_view`, `can_comment`, `can_create_issue`, `can_edit_features`, `can_manage_impl`, `can_admin`) and passed to templates. One module, well tested, used everywhere.

| Action | Anonymous | Logged-in non-member | Org collaborator (no project grant) | Project collaborator | Project owner / org owner |
|---|---|---|---|---|---|
| View public project | ✓ | ✓ | ✓ | ✓ | ✓ |
| View authenticated project | | ✓ | ✓ | ✓ | ✓ |
| View private project | | | | ✓ | ✓ |
| Comment, create issue | | if open contribution | if open contribution | ✓ | ✓ |
| Promote own thread to issue | | if open contribution | if open contribution | ✓ | ✓ |
| Promote any thread to issue | | | | ✓ | ✓ |
| Accept/reject/reopen issue | | | | ✓ | ✓ |
| Create/edit feature, files, status | | | | ✓ | ✓ |
| Link implementations, set statuses | | | | ✓ | ✓ |
| Resolve feature threads | | own | own | ✓ | ✓ |
| Edit comments, withdraw issues | | own | own | own | own |
| Delete comments | | own | own | own | ✓ |
| Manage project members, delete project | | | | | ✓ |
| Create project | | | if `can_create_projects` | | org owner ✓ |
| Manage org members, grants | | | | | org owner |
| Create organization | | | | | sysadmin only |

Org owners are treated as project owners on every project in the org without needing a membership row.

## 5. Phases

Each phase ends deployable. Rough sizes assume one person, part-time; adjust freely.

### Phase 0 — Skeleton and deploy (small)

- Repo, `uv` project, Django 5.2, settings split (base/dev/prod), `ruff`, `pytest-django`.
- Abstract base model with ULID primary key and timestamps; all models inherit it.
- Docker Compose for local dev.
- Custom `User` model with ULID key (must exist before the first migration).
- Base template: semantic layout, skip link, nav, the minimal stylesheet, dark mode toggle.
- Fly app, volume, Litestream to B2, migrations in the start script, Sentry.
- CI: lint + test + deploy on main.
- **Exit:** empty app with a health page running on Fly; restore from Litestream tested once.

### Phase 1 — Accounts and organizations (medium)

- allauth with GitHub, Discord, one Forgejo OIDC provider, email login-by-code (console backend).
- Account linking UI (allauth connections page, restyled minimally).
- Sysadmin: Django admin, create orgs with initial owner.
- Org pages: member list, roles, `can_create_projects`, invites by email/username, accept/decline, leave org, last-owner guard.
- Permissions module with tests.
- **Exit:** sysadmin creates an org; owner invites a user; user logs in with two linked providers.

### Phase 2 — Projects and features (large)

- Project create (prefix, visibility, open contribution), member management, delete.
- Feature CRUD as revisions: create, edit description with required note, change parent, change status, `superseded_by`.
- Revision history page, diff view between any two revisions, per-revision read-only view.
- Feature tree view (nested `<ul>`/`<details>`), flat list, feature page with children listed.
- Markdown rendering with line mapping and sanitization.
- **Exit:** full feature lifecycle works without JS enabled.

### Phase 3 — Issues and comments (large)

- Issue CRUD, statuses, duplicate linking, M2M to features, "accept and create feature" flow prefilling the feature from the issue.
- Comment threads on issues and on features (general), with resolve, edit history, tombstone delete.
- Line-anchored threads: gutter buttons per rendered block, thread list beside the description, outdated marking on newer revisions.
- Promote thread to issue.
- HTMX partials for posting comments and resolving threads, with form fallbacks.
- **Exit:** the example flow in concept.md (steps 1–7) works end to end.
- Done: `tests/test_example_flow.py` drives steps 1–7 (and the rejected alternative) through the views. HTMX reply, edit, resolve and the issue timeline were also checked once in headless Chromium; Playwright tests proper are Phase 5.

### Phase 4 — Files and implementations (medium)

- File upload to Tigris, versions, signed URLs, sandboxed iframe for HTML, image display.
- Image-coordinate threads (plain JS for pin placement, number-field fallback).
- File-level threads.
- Implementations: CRUD, URL, status; per-feature status table on the implementation page and per-implementation status on the feature page; "behind" indicator against current revision.
- **Exit:** a feature with a prototype image has a pinned thread; an implementation shows one feature shipped and one behind.
- Done: `tests/test_phase4_exit.py` covers the exit through the views. Pin placement, pin alignment, keyboard access to pins, the prototype sandbox and the upload form were checked once in headless Chromium.

### Phase 5 — Launch hardening (small)

- Pick email provider, configure anymail, verify magic link and invite emails.
- Moderation: deactivate user (sysadmin), delete comments (project owner), ghost user on account deletion.
- Playwright smoke tests with axe checks on the main pages in both color schemes.
- Rate limiting on comment endpoints (allauth already limits its own views; invites are throttled), CSP header, security headers review, second factor for `/admin/`.
- Backup restore drill repeated against the real volume.
- **Exit:** launch checklist (below) complete.

Done in code; what remains needs production (see the checklist and `docs/operations.md`):

- **Email:** `django-anymail` installed. The provider is chosen entirely by environment (`EMAIL_BACKEND` plus `ANYMAIL_*` variables), so picking one needs no code change. Still TBD.
- **Moderation:** deactivating a user in the admin blocks sign-in and ends their sessions; their content stays. Deleted accounts' content shows as "ghost" (authors are `SET_NULL`; no placeholder user row), and `ghost` and other official-looking usernames are reserved. The admin refuses to delete an organization's last owner, and bulk delete is off for users so that check always applies.
- **Rate limits:** one fixed-window helper (`intent/core/throttle.py`) in the shared database cache: comments and new threads 30 per 10 minutes, issues 20 per hour, uploads 30 per hour, invites as before. A limited HTMX reply keeps the draft.
- **CSP and headers:** every page gets `default-src 'self'`, no inline script or style, `object-src 'none'`, `base-uri 'none'`, `frame-ancestors 'self'`, and the bucket origin for images and frames when one is configured. No `form-action`, because sign-in with an external provider posts a form that redirects off-site. htmx is configured not to inject styles or eval. Also `Permissions-Policy`. File responses keep their own stricter sandbox CSP.
- **Admin second factor:** `django-otp` TOTP on the admin site, with `manage.py add_admin_totp <user>` to enrol. Off by default only in local development.
- **Browser tests** (`tests/browser`, marked `browser`): axe WCAG 2.1 A/AA on every page in light and dark, signed in and out; no CSP violations or script errors on any page including the admin; a keyboard-only walkthrough (skip link, line comment, HTMX reply with focus); the theme toggle. axe found that the page had no explicit background, so tools assumed white in dark mode; `:root` now sets `Canvas`/`CanvasText`. CI installs Chromium.
- **Restore drill:** written up in `docs/operations.md`; needs the real volume and B2.

## 6. Launch checklist

- [ ] Email provider configured; magic link and invite emails tested in production
- [ ] Litestream restore verified from B2 into a fresh volume
- [ ] Fly volume snapshots enabled as a second backup
- [ ] Sentry receiving errors from production
- [ ] At least one Forgejo OIDC provider registered and tested
- [x] axe passes on feature, issue, project and org pages, light and dark (`tests/browser`; recheck against production once deployed)
- [ ] Keyboard-only walkthrough of the example flow (the comment and reply path is automated; do the full flow by hand once)
- [ ] Privacy check: private project content not reachable via file URLs, feeds, or admin leaks
- [ ] `EMAIL_BACKEND` set in production (prod settings refuse the console backend)
- [ ] Container confirmed running as the `app` user on the Fly volume
- [ ] First administrator created with `add_admin_totp`, and admin sign-in tested with a code
- [ ] HSTS raised to a year once the domain is settled (`SECURE_HSTS_SECONDS` in prod settings)

## 7. Hooks for V1.1

Decisions made now so V1.1 does not need a migration redesign:

- `Implementation.url` is kept separate from future `forgejo_instance` and `repo_path` fields; the URL stays the display link.
- `FeatureImplStatus` already records `feature_revision`, which is what commit trailers will set (`Intent: AUTH-12@7`).
- Org-level `ForgejoCredential` (instance URL, service-user token) is the planned home for sync credentials; nothing in V1 depends on it.
- Feature stable IDs and revision numbers are never reused or renumbered, so external references stay valid.