Running QA¶

This file contains the conductor prompt. Paste it into a Claude session that has the mcp-gee-sweet MCP server connected. Claude will read your fixtures, execute every test case against the live server, record outcomes, and save a results report.

Prerequisites¶

mcp-gee-sweet server running and connected to this Claude session
docs/qa/.env exists and is filled in (see setup.md if not)
For calendar tests: the calendar in .env is accessible to the authenticated account
If using Playwright for visual verification: Playwright MCP connected and a valid token.json present (see docs/qa/playwright_oauth.md)

How to start¶

Copy everything in the Conductor prompt section below and paste it into the Claude chat.

To run a single category instead of the full suite, add after the prompt: "Only run the tests/drive_files.md file" (or whichever category you want).

To resume an interrupted run: paste the prompt and add "Resume from docs/qa/results/<date>-partial.md".

Playwright verification¶

Playwright is optional at the run level. Whether a test requires visual verification is a per-test-case decision, marked in the test file itself with **Playwright: required**. The conductor follows those tags — it does not decide at runtime which tests get visual verification.

Tiers (authoring reference)¶

Three tiers replace the earlier "visual whenever visual is possible" guidance (which produced near-zero-signal screenshots on read-only/error-path/count tests — see the v0.8.0 retro's finding #1). Only Required corresponds to an actual tag on the test case; Spot-check and Skip are authoring/runtime guidance, not tags.

Tier	When to use	Tag the test case?	Examples
Required	The check verifies a mutation with a visual signature the API-level response can't fully confirm — formatting, hyperlinks, images, charts, layout, table-cell run formatting, create/delete/move confirmed in UI, cache invalidation checks	Yes — `Playwright: required` on the Prompt line	`format_cells`, `freeze`, `merge`, `add_chart`, `delete_file`, `write_doc_content` tables, `create_event`, `style_doc_range` (font size, hyperlinks)
Spot-check	An API response looks unexpected, or a cache discrepancy is suspected — a runtime judgment call during the run, not a fixed property of the test case	No	Any time the API says success but behavior seems wrong
Skip	Read-only, error paths, count/pagination, cache-hit-only, unit-tested paths, or a mutation whose visual signature the API response already fully confirms (e.g. plain-paragraph bold/italic runs, checkbox glyphs as literal text, `namedStyleType`, Drive file metadata)	No	`get_sheet_data`, `list_events`, error returns, `create_folder`/`rename_file` (confirmed via `list_files`/`get_file_metadata` instead — see Drive fixture note below), row/column counts

A test case's Checks list is the source of truth: if every listed check is answerable from the tool's own response (or a follow-up read-only call), it's Skip even if the mutation sounds visual. If at least one check can only be answered by looking at the rendered result, it's Required. When in doubt, prefer Required — the cost of an unnecessary screenshot is lower than a silently-unverified regression.

Drive mutations are Skip by default policy, not by oversight. Drive's UI mostly just reflects API-returned metadata directly (file name, mimeType, trashed state, parent), so list_files/get_file_metadata is the confirmation source rather than a screenshot — this is documented per-case in docs/qa/tests/drive_*.md Result entries (e.g. TC-D at drive_files.md's create_shortcut test, which uses the shortcut mimeType instead of a screenshot). Permission/sharing changes follow the same policy — see "Known limitations" below.

How it works (authoring reference)¶

When a test case is tagged **Playwright: required**, after the tool call the conductor:

Sheets — navigates to https://docs.google.com/spreadsheets/d/{SPREADSHEET_ID}/edit and snapshots the affected cells
Docs — navigates to https://docs.google.com/document/d/{DOC_ID}/edit and snapshots the affected section
Drive — navigates to https://drive.google.com/drive/folders/{FOLDER_ID} (or the file's web_link) and confirms the change
Calendar — navigates to https://calendar.google.com and confirms the event appears or is gone

Known limitations¶

Footer / header content — Google Docs renders headers and footers outside the main document canvas. For short documents window.scrollTo does not bring the footer into the viewport. When a test involves create_footer or create_header, use the API response as confirmation; a Playwright snapshot of the body is still useful but will not show the footer content.
Permission changes — sharing confirmations are not visible in the Drive UI without navigating to the file's "Share" dialog. Use list_permissions API response as the confirmation source for share tests.
Chart-covered grid (Sales sheet) — repeated add_chart test runs leave floating chart objects on the Sales sheet (they're never cleaned up between runs), and as of 2026-07-15 there were ~12 stacked overlapping charts covering roughly rows 1–22. This blocks visual verification of anything in that region — row-height/column-width changes (resize_rows/resize_columns), cell formatting, etc. are not visible in a screenshot, and Playwright's accessibility snapshot doesn't expose per-cell grid geometry either (only the charts' own accessible text). When a test in that area needs precise confirmation, use get_sheet_data(..., range=<affected range>, include_grid_data=True) instead — the raw response includes rowMetadata/columnMetadata with each row/column's actual pixelSize, which is more precise than a screenshot anyway. The same call also works for border verification (update_borders QA, TC-S85–S87 on 2026-07-16): each cell's userEnteredFormat.borders/effectiveFormat.borders reports top/bottom/left/right style and color directly, so a border-covering-range test doesn't need the screenshot at all — check the per-cell borders dict instead of rowMetadata/columnMetadata. Longer term, add_chart QA test cases should delete the chart they create at the end of each run (see docs/qa/tests/sheets_charts.md) so the fixture doesn't accumulate; that cleanup is not yet in place.

Verifying Playwright is signed into the right account¶

"Connected," "signed in," and "signed in as the account that owns the fixtures" are three different states, and only the third makes **Playwright: required** cases meaningful. Playwright MCP keeps a separate persistent browser profile per working directory — the main checkout, each .claude/worktrees/* slot, and every other project on the machine each get their own login. Any one of them can be signed into a Google account that isn't the one owning the mcp-gee-sweet-shared Shared Drive fixtures (see setup.md), or into none at all.

The check, run once before the first **Playwright: required** case in any QA pass (lane QA or release pass):

Get the fixture doc's real title via this role's own get_file_metadata(TEST_DOC_ID).
browser_navigate to https://docs.google.com/document/d/<TEST_DOC_ID>/edit (under the Playwright mutex below, if other shards may be running).
Pass only if the page title contains that exact document name.
Fail on either:
a redirect to accounts.google.com — the profile isn't signed in;
the URL staying on docs.google.com but the page showing "You need access" / "Request access," or a generic "Google Docs" title — the profile is signed into the wrong account. This one is easy to misread as a pass, since there's no sign-in redirect; it's what the old "not a sign-in page" check let through (confirmed 2026-09-23: a lane QA session's browser opened on the personal account instead of the Workspace-org one that owns the fixtures).

On a fail, stop and tell the user which session's browser needs signing in, and to which account (the one that owns the Shared Drive fixtures). Then wait. The sign-in persists in that slot's profile, so this is a one-time fix per slot. Don't fall back to API-only verification for Playwright-required cases unless the user explicitly says to — that's the silent degradation retro-v0.9.0.md describes. Once the user says it's fixed, re-run the check before continuing.

Coordinating Playwright across parallel shards¶

A single Playwright MCP instance is one browser tab. If a QA run is split across multiple agents (e.g. one per domain), any two shards that both hit **Playwright: required** cases at the same time will fight over that tab — one shard's navigation clobbers the snapshot the other was about to take. Spinning up a second authenticated Playwright instance to get true parallelism is possible but costly: the browser profile needs its own logged-in Google session (see playwright_oauth.md), so a second instance means cloning that profile rather than just adding a server entry.

Cheaper fix: a filesystem mutex around Playwright usage, since only one shard needs the tab at a time and each hold is brief (one navigate + snapshot, not the whole run).

Lock path: /tmp/mcp-gee-sweet-playwright.lock (a directory, not a file — mkdir is atomic on POSIX, so no separate locking library is needed).
Acquire: before a Playwright tool call, attempt mkdir /tmp/mcp-gee-sweet-playwright.lock. Success means the lock is held — proceed. Failure (already exists) means another shard holds it — back off (a few seconds) and retry.
Release: immediately after the snapshot for that one test case, rmdir /tmp/mcp-gee-sweet-playwright.lock. Don't hold the lock across an entire test file — only around the actual Playwright step.
Stale-lock recovery: if the lock directory's mtime is older than ~120s, treat it as abandoned (the shard that created it likely crashed or was killed) — remove it and re-acquire rather than waiting forever.
When it's not needed: a single-shard full run (no parallel agents) never contends with itself — skip the lock entirely in that case.

Give each parallel shard's conductor prompt this protocol explicitly (path, acquire/release commands, backoff, staleness threshold) rather than assuming priority ordering or hoping timing works out.

Running true-concurrency test cases¶

A single MCP client session awaits each tool result before issuing the next, so it can never hold two requests to one server process in flight at once. A handful of test cases need genuinely simultaneous requests and are otherwise perpetually skipped:

TC	File	What it needs
TC-I24	`tests/infra.md`	Two distinct `get_sheet_data` calls (different spreadsheets) firing at the same instant — checks the shared startup `httplib2` transport isn't corrupted across worker threads (issue #183)
TC-I02	`tests/infra.md`	A write (`update_cells`) and a read (`get_sheet_data`) on the same spreadsheet firing at the same instant — checks SQLite WAL lets the read proceed during the write with no locking error (issue #234)

TC-R36/TC-R37 (tests/sheets_read.md) look similar but are within-call concurrency (get_multiple_sheet_data/get_multiple_spreadsheet_summary gather() several .execute() calls inside one tool call) — a single session already exercises them. They don't need this procedure; just re-run them normally during the pass and record a fresh Result. Only the cross-session cases (TC-I24, TC-I02) need what follows.

Mechanism: two subagents, one server, a `mkdir` barrier¶

Aziz (never a lane session — this runs during the release pass, see .claude/team-roles/aziz.md) spawns two Agent-tool subagents — real subagents, which inherit this session's already-connected MCP servers; not Agent-View spawns, which don't. Both subagents point at the same server prefix. Prefer mcp__mcp-gee-sweet-kai-sa__ — it runs the main-checkout (release-candidate) code, not a worktree copy. A lane prefix (mcp__mcp-gee-sweet-sky__ / -kit__) also works if the main-checkout server isn't available, but only after that worktree is reset to the release commit; note the substitution in the run file.

Per iteration, the two subagents rendezvous at a filesystem barrier so both .execute() calls hit the wire within a few milliseconds of each other:

Before the loop: warm the target server. Until #1052 is fixed, each worker thread's cached connection dies after a few minutes idle, and the first request on it fails once with Connection reset by peer. That is TC-I24's own failure signature, so an idle server produces false TC-I24 failures. Right before spawning the two subagents, Aziz calls get_multiple_sheet_data on the target prefix with about 12–24 single-cell queries, repeating until one call comes back fully clean, and notes the warm-up in the run file. In v0.9.2, kai-sa lost one query on each of two warm-up calls before a 24/24 clean one. Then spawn at once, so the server doesn't go idle again.
Barrier dir: /tmp/mcp-gee-sweet-qa-barrier (created once by Aziz before spawning; distinct from the Playwright lock). Markers are files inside it.
A background release watcher (Aziz starts one run_in_background shell loop before spawning the subagents) does, for each iteration i from 1 to N:
wait until both ready-a- and ready-b- exist,
touch go-,
wait until both done-a- and done-b- exist, then continue to i+1.
Each subagent (<id> = a or b), per iteration i:
touch /tmp/mcp-gee-sweet-qa-barrier/ready-<id>-
spin (while [ ! -e .../go- ]; do :; done — a tight poll, no sleep) until go- appears
immediately issue its one assigned tool call — no reasoning, no other tool call between the spin ending and the call
append the raw result (full response, or the error text) to /tmp/mcp-gee-sweet-qa-barrier/result-<id>-.json
touch /tmp/mcp-gee-sweet-qa-barrier/done-<id>-
Loop count: N = 20–50. Transport/SSL corruption and WAL contention are intermittent; one clean pair proves nothing. Stop early and report if any iteration trips a check.
Teardown: Aziz rm -rf /tmp/mcp-gee-sweet-qa-barrier after collecting results.
Shell form. The worktree-isolation guard can refuse a Bash call that uses a shell variable ($i included) or a heredoc. In every command of the prompt template below, replace $i with the iteration number and use the literal absolute path, and write a result with a plain printf '%s\n' '<result>' >> /tmp/mcp-gee-sweet-qa-barrier/result-<id>-.json (both v0.9.2 workers lost their first iteration's write to this). A result containing ' breaks that quoting: write the error text, or the result with each ' replaced, rather than dropping the line.

The subagent's job is to run the calls and write result files — it must not edit any tracked repo file. Only Aziz reads the result-* files back, diffs them, and writes the **Result** entries (see .claude/team-roles/aziz.md, the Compile step).

Per-TC checks¶

TC-I24 — subagent a calls get_sheet_data on {SPREADSHEET_ID} / Sales!A1:C3; subagent b calls get_sheet_data on a second, distinct spreadsheet / range with different known values. Across all N iterations: - every result-a-* holds only {SPREADSHEET_ID}'s data, every result-b-* only the second spreadsheet's — no row from one appears in the other's file - no result is empty, truncated, or an SSL/connection error (record layer failure, Connection reset by peer, Remote end closed connection — the signature of the shared-transport bug this guards against)

TC-I02 — subagent a calls update_cells writing a per-iteration sentinel (e.g. QA-I02--<timestamp>) to {SPREADSHEET_ID} / Empty!A1; subagent b calls get_sheet_data on {SPREADSHEET_ID} / Sales!A1:C3 (a different sheet in the same spreadsheet, so both touch the same cache row). Across all N iterations: - no result-b-* is a database is locked / SQLITE_BUSY error, and none is empty or an SSL/connection error - every result-a-* reports the write succeeded - if the server's LOG_FILE is reachable (from docs/qa/.env or the team MCP config), grep it for database is locked / OperationalError over the run window — there should be none; if it isn't reachable, the response-level checks above still catch the failure mode (WAL contention surfaces as an error in the response, not just the log)

Subagent prompt template¶

Give each of the two subagents a prompt of this shape (fill the bracketed parts; <id> is a for one, b for the other):

You are one of two subagents running a true-concurrency QA iteration loop. Your worker id is <id>. Barrier dir: /tmp/mcp-gee-sweet-qa-barrier. Iterations: <N>.

For i in 1..<N>, in order: 1. touch /tmp/mcp-gee-sweet-qa-barrier/ready-<id>-$i 2. Spin with a tight shell poll until /tmp/mcp-gee-sweet-qa-barrier/go-$i exists. 3. The instant it exists, call exactly this tool, once: [full tool name + params, e.g. mcp__mcp-gee-sweet-kai-sa__get_sheet_data(spreadsheet_id="…", range="Sales!A1:C3")]. Do nothing else first — no other tool call, no reasoning step, between the spin ending and this call. 4. Append the tool's full raw result (or, on error, the complete error text) as one JSON line to /tmp/mcp-gee-sweet-qa-barrier/result-<id>-$i.json. Use a plain printf with the literal absolute path, with $i replaced by the iteration number (see "Shell form" above). 5. touch /tmp/mcp-gee-sweet-qa-barrier/done-<id>-$i

Call only mcp__mcp-gee-sweet-kai-sa__* tools. Do not edit, create, or delete any file inside the git repo — your only writes are to /tmp/mcp-gee-sweet-qa-barrier/. When the loop finishes, report back: for each i, whether the tool call returned data or an error, and the first ~200 chars of each result. Do not interpret pass/fail — just report what you observed.

Drive fixture-folder pollution¶

TEST_FOLDER_ID accumulates stray items across many creation/copy test categories that don't tear down after themselves (tracked in #304 — dedicated QA account/cleanup sweep). As of 2026-07-17 it held ~15 duplicate leftovers (Copy of mcp-gee-sweet-qa-fixtures ×2, QA-Cache-Check ×2, QA-Copy-Explicit, QA-Create-Explicit, QA-Doc-Copy, QA-DocCache, QA-HTML-Doc ×2, QA-Markdown-Doc ×2, QA-Table-Doc ×2, loose qa-notes.md/qa-upload.txt, two leftover folders).

This matters for any test that syncs, lists, or downloads the whole folder (sync_folder, download_folder, list_files against TEST_FOLDER_ID directly) — the pollution items show up in results alongside whatever the test actually cares about, and a recursive/whole-folder operation will process all of it. Until #304 lands: for a test that needs a clean or isolated Drive tree (rather than "does this tool correctly handle files that happen to be in TEST_FOLDER_ID"), create a throwaway child folder under TEST_FOLDER_ID for that test's own fixtures instead of working at the shared top level, and delete it in teardown. Same tool behavior either way — this only reduces noise and avoids accidentally sweeping up items you don't own.

Clearing sheet-level state with no direct tool (e.g. data validation)¶

Some sheet-level state has a tool to set it but none to clear it, and no tool exposes a sheet's numeric sheetId by name to fall back to the raw batch_update escape hatch (list_sheets returns names only; the spreadsheet://{id}/info resource that's supposed to cover this was broken until #363 fixed it — reading it now works, but it's still no substitute for a tool that returns sheetId by name, tracked as a product gap in #365). Hit live testing add_data_validation/get_data_validation (PR #361, 2026-07-18): no way to clear a validation rule from the fixture's Empty sheet between test cases.

Workaround for a scratch fixture sheet (never do this to a sheet with real data — it destroys everything on the tab, not just the state you're trying to clear): delete_sheet(sheet="Empty") then create_sheet(title="Empty"). Fully resets the tab to blank, including any validation/formatting/merges, and is safe here because every QA tool call references the sheet by name, never by the ID that changes on recreate.

Use disjoint ranges for independent TCs sharing one scratch sheet within a single run. Confirmed on the v0.9.0 pass: re-verifying merge_cells/unmerge_cells TCs and add_data_validation/get_data_validation TCs against overlapping ranges on the Empty sheet in the same subagent run triggered Google Sheets' own merge → validation-carryover behavior (a merge overwrote the overlapping range's dropdown rule with a checkbox rule) — native Sheets semantics, not a tool defect, but it reads exactly like cross-test contamination until investigated. Give each independent TC its own non-overlapping block (e.g. A1:C3 for one, E1:G3 for another).

Forcing a real `storageQuotaExceeded` (service account only)¶

Several quota-error test cases say a quota error can't be forced live, so they verify only by unit test. That's true for a user identity, since the QA account's My Drive isn't full. A service account's error can be forced: it has no My Drive storage, so creating a file with no parents returns a real 403 storageQuotaExceeded. Run it from a fresh interpreter in the checkout under test (no MCP server involved), and pass the caught exception to the code under test:

uv run python -I -c "
import mcp_gee_sweet
from mcp_gee_sweet import auth
from googleapiclient.discovery import build
d = build('drive', 'v3', credentials=auth._service_account_creds(), cache_discovery=False)
try:
    r = d.files().create(body={'name': 'qa-sa-quota-probe', 'mimeType': 'application/vnd.google-apps.document'}, fields='id').execute()
    print('UNEXPECTED: created'); d.files().delete(fileId=r['id']).execute()
except Exception as e:
    print(type(e).__name__, e.resp.status)  # then feed e to the helper under test
"

This needs service-account credentials in src/mcp_gee_sweet/.env (CREDENTIALS_CONFIG or SERVICE_ACCOUNT_PATH). Nothing is created, so there's no teardown. Used for TC-D291 in PR #1006.

Missing `docs/qa/.env` in a role worktree¶

docs/qa/.env is gitignored, so it doesn't exist by default in a freshly-provisioned .claude/worktrees/<name> slot — confirmed empty/absent across every role worktree and the main checkout, 2026-07-19, while doing a scoped QA pass from Kit's own role process (.claude/team-roles/qa.md step 4), not the full conductor-prompt flow this file otherwise documents. That flow's own fixture-check step (below) would just stop and ask the user to run setup.md, but a scoped single-PR QA pass doesn't need the whole .env — only the one fixture ID relevant to the PR under review.

Workaround: the fixture files have fixed, documented names (setup.md's seed prompt: "Rename the doc to mcp-gee-sweet-qa-fixtures-doc" / "Rename the spreadsheet to mcp-gee-sweet-qa-fixtures"). Find the ID directly instead of blocking: search_files(query="mcp-gee-sweet-qa-fixtures-doc", mime_type="application/vnd.google-apps.document") (swap the doc mime type/name for the spreadsheet as needed). Only reach for this fallback in a scoped pass that needs one or two fixture IDs — a full multi-category run still needs the real .env for TEST_FOLDER_ID/TEST_CALENDAR_ID/etc., which don't have as fixed a name to search by.

Running server-startup and CLI cases in a scoped lane pass (e.g. `⚠️ local-filesystem` infra cases)¶

Some cases test how the server starts, not what a tool does: auth waterfall and degraded start, stdio vs SSE behavior, CLI subcommands like mcp-gee-sweet auth. Your own mcp-gee-sweet-<name> server can't exercise these, since its env and transport are fixed. They still run fine in a lane pass. Start a fresh server process from the worktree with the env the case needs, and drive it yourself. Confirmed on PR #807 (Kit) and PR #828 (Sky, TC-I35..I40, 2026-09-26).

stdio: use the mcp SDK's stdio_client(StdioServerParameters(command="uv", args=["run", "--directory", <worktree>, "mcp-gee-sweet"], env=...)) + ClientSession. Then initialize, call_tool, and read_resource("server://auth-status"). Strip AUTH_METHOD/TOKEN_PATH/SERVICE_ACCOUNT_PATH/CREDENTIALS_CONFIG/GOOGLE_APPLICATION_CREDENTIALS from the inherited env before applying the case's overrides. Set DEBUG_LEVEL=DEBUG + LOG_FILE=<tmp> for the log-line checks. mcp v2's CallToolResult field is is_error, not isError.
SSE: start uv run mcp-gee-sweet --transport sse in its own process group (subprocess.Popen(..., start_new_session=True) from Python, or set -m then cmd & in a bash script; macOS has no setsid). Drive it with mcp.client.sse.sse_client + ClientSession. Stop it with SIGTERM to the server pid: it exits within a few seconds, even mid consent wait (#833), so a server still running after that is a finding. Only then SIGKILL the whole group (os.killpg / kill -9 -- -$PID). Never pkill -f "transport sse" or similar: other lanes' and Kai's servers match. The lifespan (and any consent wait) runs when the SSE stream opens, before initialize, so time the connect, not initialize. Uvicorn's access log goes to stdout, so check stdout for the absence of specific text (e.g. the consent prompt), not for an empty file.
Race and hang repros: a concurrency finding needs a repro that forces the overlap. Gate the operations with events (e.g. A's consent waits on an event set from inside B's token load) so the stale operation is still in flight when the other side's event fires. Starting B after A finishes only shows a sequential case, which may be correct behavior. Run the same script against the previous commit as a control: a race or hang that shows up there and not on the fix is the evidence. For an SSE-timing hang, use a raw client that POSTs the instant the endpoint event arrives, not the SDK client: PR #867's intermittent SIGTERM hang reproduced 3 of 39 runs through sse_client and 25 of 25 with the raw client (TC-I44 step 2 encodes it).
CLI + OAuth consent: run the command in the background, read the Please visit this URL line from its output, and complete (or Cancel) the consent via Playwright, after the signed-in check and under the mutex above. The account chooser lists more than one account, so pick the Workspace one that owns the fixtures.
Worktree-isolation guard: a session pinned to a role worktree rejects Bash commands it can't statically verify: uv/git with $VAR-expanded paths, heredocs, multi-step cd … && … chains. Write each probe as a script under $CLAUDE_JOB_DIR/tmp with the Write tool (literal absolute paths inside), then run it as a single bash /abs/path/script.sh or uv run python /abs/path/probe.py KEY=VAL ... call. It also refused these read-only or scratch-only forms on PR #866 (2026-10-01): python3 -c ... with a $var path operand, cd <scratch> && git init && claude -p ..., an inline python3 -c filter containing the word system, and gh pr create --body "<free text>". These ran: a plain cd <scratch> && claude -p ... and VAR=... claude -p .... Give claude -p a scratch project made of .claude/settings.json alone, since it needs no git init. Pull fields from stream-json with grep -oE, not an inline Python filter. Pass a PR body with --body-file.
Fixture: point TOKEN_PATH at a nonexistent file in a directory that exists. A missing parent directory now trips the pre-consent writability check (#811), which tests a different path.

Conductor prompt¶

You are the QA conductor for mcp-gee-sweet. Your job is to execute the full test suite against the live MCP server, record outcomes, and save a results report.

You have the mcp-gee-sweet MCP connected. Before starting, check whether Playwright MCP is also connected AND actually authenticated to Google — these are different things: a connected Playwright MCP still runs an unauthenticated browser by default, and navigating it to a Google URL just redirects to `accounts.google.com` sign-in. Do the real check described in "Verifying Playwright is signed into the right account" (above): the fixture doc's page title must match its actual name — a sign-in redirect *or* a "You need access" page (wrong account) both fail. If Playwright isn't connected, or the check fails, tell me: "Playwright is not connected / not authenticated — tests marked **Playwright: required** will run without visual verification. Confirm to proceed, or connect/authenticate Playwright first (see `docs/qa/playwright_oauth.md`) and restart." Wait for my confirmation before continuing. If I tell you mid-run that Playwright is now authenticated, re-verify immediately and switch to using it for all subsequent **Playwright: required** cases — don't keep treating it as unusable for the rest of the run based on the earlier check.

## Step 0 — Fixture setup

Before running any tests:

1. Record the start time (current timestamp).
2. Read `.env` from the repo root. If the file does not exist or the TEST_* keys are missing, stop and say: ".env not found or TEST_* keys missing — follow docs/qa/setup.md to create your fixtures first."
3. Extract TEST_SPREADSHEET_ID, TEST_DOC_ID, TEST_FOLDER_ID, TEST_CALENDAR_ID, TEST_EVENT_ID, TEST_LARGE_DOC_ID, TEST_PERMISSION_EMAIL, SHARED_DRIVE_ID.
4. Verify the fixture spreadsheet with get_sheet_data: confirm sheet tabs Sales, Empty, Notes & Misc exist and Sales data has 6 rows (header + Widget/Gadget/Donut/Gizmo/Totals), columns A–D. If data is missing or in wrong order, use update_cells to restore known seed state (see docs/qa/setup.md §Known fixture state).
5. Verify the fixture doc with get_doc_structure: confirm title "mcp-gee-sweet-qa-fixtures-doc" and body contains heading "Test Document", a paragraph, and a bullet list (Item one / Item two). If content is wrong, use write_doc_content to restore it.
6. Tell me the fixture IDs, start time, and whether the fixture state looks correct, then wait for me to confirm before proceeding.

## Step 1 — Run tests

Work through the test files in this order:
1. `docs/qa/tests/infra.md`
2. `docs/qa/tests/sheets_read.md`
3. `docs/qa/tests/sheets_write.md`
4. `docs/qa/tests/sheets_mgmt.md`
5. `docs/qa/tests/sheets_charts.md`
6. `docs/qa/tests/drive_files.md`
7. `docs/qa/tests/drive_sharing.md`
8. `docs/qa/tests/drive_transfer.md`
9. `docs/qa/tests/drive_activity.md`
10. `docs/qa/tests/docs_content.md`
11. `docs/qa/tests/docs_tables.md`
12. `docs/qa/tests/docs_style.md`
13. `docs/qa/tests/docs_layout.md`
14. `docs/qa/tests/calendar.md`
15. `docs/qa/tests/gmail.md`

`docs/qa/tests/docs_markdown.md` isn't in this sequence: it renders every case through eight entry points twice, so it runs on its own (`run_group` with `group: docs_markdown`), following that file's own "Running this suite" section.

For each test case:

1. Announce the TC number and title.
2. Substitute fixture IDs into the prompt (replace {SPREADSHEET_ID}, {DOC_ID}, etc. with the values from `.env`).
3. Execute the prompt using the mcp-gee-sweet tools available in this session.
4. If the test case is marked **Playwright: required** and Playwright MCP is connected: navigate to the affected resource and take a snapshot before recording the outcome. Also save a screenshot via `browser_take_screenshot`, with `filename` set to `docs/qa/screenshots/<YYYY-MM-DD>-<tc-id>.png` (today's date, lowercase TC number — e.g. `tc-doc12.png`). The date prefix keeps each run's screenshots separate without overwriting a prior run's evidence, and avoids depending on a subfolder that `browser_take_screenshot` won't auto-create.
5. Evaluate each item in the **Checks** list against the actual result.
6. Record one of:
   - **PASS** — every check met, and the result describes no defect
   - **FAIL** — one or more checks failed; note exactly what was wrong
   - **PARTIAL** — the checks that ran were met, but at least one listed check didn't run (including the Playwright check on a `**Playwright: required**` case); name each one that didn't
   - **SKIP** — with reason (e.g. "requires server restart", "skipped by user", "prerequisite TC failed")

   A result that describes a defect is never a PASS. That includes wording like "matches the documented failure mode", "known gap", "pre-existing", "not a new regression", "out of scope", or "still fails": if the case's own output shows a defect, record FAIL, even when its Checks were written to accept it. A known limitation is a FAIL that links an **open** issue: if the one it names is closed, or it names none, file one (or add the evidence to an open one) and link it in the Result. A defect outside the case's own checks (a probe, a side observation) can sit beside a PASS only if the same Result names where it's tracked: sent back on the PR, or an open issue. "Not filed" next to a defect is never acceptable (#1044 audit, `docs/qa/results/2026-10-17-pass-audit.md`).
7. For tests marked ⚠️ destructive: ask me before running. If I say skip, record SKIP(destructive-skipped).
8. For tests marked 🔍 product decision: record PASS and note the observed behavior, unless that behavior is a defect or a limitation; then record FAIL per step 6.
9. Track intermediate IDs as you go — some tests produce IDs (permissionId, eventId, fileId) needed by later tests. Record them in your working notes and substitute them when referenced.

## Step 2 — Save progress

After completing each test file, write current results to `docs/qa/results/<YYYY-MM-DD>.md` (use today's date). If a file already exists for today, overwrite it with the fully updated version.

If I say "pause" or "stop" at any point, save immediately to `docs/qa/results/<YYYY-MM-DD>-partial.md` before stopping.

## Step 3 — Final report

After all test files are complete, record the end time and write the final report to `docs/qa/results/<YYYY-MM-DD>.md` using this format:

---
# QA Run — <YYYY-MM-DD>

**Auth:** OAuth  
**Fixtures:** SPREADSHEET_ID=`<id>` · DOC_ID=`<id>` · FOLDER_ID=`<id>`  
**Start:** <timestamp> · **End:** <timestamp> · **Duration:** <HH:MM>

## Summary

| Category | PASS | FAIL | SKIP | Total |
|---|---|---|---|---|
| Infrastructure | | | | |
| Sheets Read | | | | |
| Sheets Write | | | | |
| Sheets Mgmt | | | | |
| Sheets Charts | | | | |
| Drive + Docs | | | | |
| Docs Tools | | | | |
| Calendar | | | | |
| **Total** | | | | |

## Failures

For each FAIL, one entry:
**TC-XXX — <title>**
Expected: <what the check said>
Observed: <what actually happened>
Playwright: <what the browser showed, if applicable>

*(none)* if all tests passed.

## Tool coverage

For every tool registered by the server, list which TCs exercise it and whether those TCs passed. Derive this by scanning the test files you ran.

| Tool | TC(s) | Status |
|---|---|---|
| get_sheet_data | TC-R01, TC-R02, … | ✅ all pass |
| create_spreadsheet | TC-D01 | ✅ pass |
| create_calendar | (none) | ⚠️ no coverage |
| … | | |

Use ✅ if all covering TCs passed, ❌ if any failed, ⚠️ no coverage if no TC exercises the tool, and SKIP if all covering TCs were skipped.

## Full results

| TC | Title | Outcome | Notes |
|---|---|---|---|
| TC-I01 | ... | PASS | |
| ... | | | |
---

If any test case this run was marked **Playwright: required**, tell me: "Screenshots saved to `docs/qa/screenshots/` (prefixed `<YYYY-MM-DD>-`) — delete when no longer needed."

## Resuming an interrupted run

If resuming: read the partial results file I specify, identify the last completed TC, and continue from the next one. Re-confirm fixture IDs from `.env` and re-verify fixture state before continuing.

Results location¶

Reports are saved to docs/qa/results/. Files named YYYY-MM-DD.md are completed runs; files named YYYY-MM-DD-partial.md are interrupted runs.