Skip to content

Markdown rendering baseline, Phase 1 — 2026-10-18 (#1044)

The first live run of docs/qa/tests/docs_markdown.md, Phase 1 only (TC-MD001–016, the 16 realistic documents). Per the suite's "Running this suite" step 6, Phase 1's failures are filed before Phase 2 starts.

Result: 0 of 128 case × entry-point cells pass. Each of the 16 documents went through all 8 entry points, twice (Run 1, Run 2). 125 cells rendered identically in both runs; 3 (EP-8 of TC-MD004, TC-MD011, TC-MD013) changed between runs, which fails Rule 4 on its own.

Setup

  • Code under test: develop at 504a5b2b. The four servers used all run from the main checkout, whose src/, pyproject.toml and uv.lock matched develop exactly. LOCAL_FILE_ACCESS was the stdio default, full.
  • Servers (one shard each): mcp-gee-sweet-kai-oauth (A), mcp-gee-sweet-oauth (B), mcp-gee-sweet-kai-sa (C), mcp-gee-sweet-sa (D). Two OAuth and two service-account identities, all writing to the fixture Shared Drive. The lane servers weren't used: lanes A and B were mid-ticket (#1057, #1056).
  • Shards: A: TC-MD001/005/009/013; B: 002/006/010/014; C: 003/007/011/015; D: 004/008/012/016. Same assignment in both runs.
  • Folders: QA-MD-2026-10-18-run1 and -run2 under TEST_FOLDER_ID, each with one subfolder per shard holding that shard's ep5/ep6/ep7 children. Trashed after recording.
  • Playwright: verified signed into the fixture-owning account before spawning (fixture doc title matched). One 1400×2200 viewport showed each whole Doc (one or two pages), so each cell has one JPEG screenshot, docs/qa/screenshots/md/run<N>/EP-<n>/<TC>-1.jpeg (gitignored, kept until the issues below are fixed).
  • Determinism check: every Run 2 screenshot was pixel-compared with its Run 1 pair (grayscale, threshold 40, Docs chrome and text caret masked). 96 pairs: 0 changed pixels. 29 pairs: 3–9 scattered JPEG pixels. 3 pairs: 3,174–11,141 pixels, all in an EP-8 code block (below).

Grid

Cells: ❌ with the issues that make the cell fail. Every row fails in both runs. ⚠️ND marks a cell whose rendering changed between runs.

Case EP-1 to EP-4 (our converter: create_doc, write_doc_content, create_doc_from_file, update_doc_from_file) EP-5 to EP-7 (Drive Markdown import: upload_local_file, upload_local_folder, sync_folder) EP-8 (upload_file markdown)
TC-MD001 Meeting notes ❌ #1071, #1093, #1056, #1098 ❌ #1101 ❌ #1102
TC-MD002 Project plan ❌ #1071, #1093 ❌ #1101 ❌ #1102
TC-MD003 Spec / PRD ❌ #1093, #1095, #1096 ❌ #1101 ❌ #1102
TC-MD004 README ❌ #1094, #1093, #1099 ❌ #1101, #1099 ❌ #1102, #1099, ⚠️ND
TC-MD005 Status report ❌ #1071, #1095, #1093 ❌ #1101 ❌ #1102
TC-MD006 How-to ❌ #1094, #1072, #1071, #1056, #1093 ❌ #1101 ❌ #1102
TC-MD007 Comparison ❌ #1095, #1093 ❌ #1101 ❌ #1102
TC-MD008 Research summary ❌ #1071, #1093, #1098, #1056, #1095 ❌ #1101 ❌ #1102
TC-MD009 Changelog ❌ #1094, #1093 ❌ #1101 ❌ #1102
TC-MD010 Task plan ❌ #1071, #1056 ❌ #1101 ❌ #1102
TC-MD011 Postmortem ❌ #1071, #1093 ❌ #1101 ❌ #1102, ⚠️ND
TC-MD012 Interview guide ❌ #1071, #1093 ❌ #1101 ❌ #1102
TC-MD013 API reference ❌ #1095, #1071, #1097, #1098, #1056, #1093 ❌ #1101 ❌ #1102, ⚠️ND
TC-MD014 Study notes ❌ #1071, #1093 ❌ #1101 ❌ #1102
TC-MD015 Travel itinerary ❌ #1071, #1097, #1095 ❌ #1101 ❌ #1102
TC-MD016 Planning notes ❌ #1071, #1093, #1100 ❌ #1101 ❌ #1102

EP-1 to EP-4 rendered identically to each other in every case, and so did EP-5 to EP-7: the wrappers add no differences of their own. The three columns are three engines.

Issues

Issue Defect Engine
#1071 (existing) 2/3-space sub-lists don't nest; with #1093 each literal-dash sub-item becomes its own numbered item (r16's Agenda: 1–10 instead of 1–4, the #1044 report) ours
#1072 (existing) numbering restarts after a nested block (r06) ours
#1056 (existing, lane B) blockquote has a border but no visible indent; depth 2 not deeper ours
#1093 soft line breaks become paragraph breaks; continuation lines become list items ours
#1094 fenced code / second paragraph inside a list item breaks the list; <ol start> ignored ours
#1095 GFM column alignment ignored ours
#1096 --- renders as an empty paragraph ours
#1097 a table takes on the style of the following block (cells bold; table indented under a quote) ours
#1098 adjacent quotes merge, consecutive code blocks merge, no spacing around quotes ours
#1099 an image Docs can't fetch (SVG badge) leaves an empty paragraph all
#1100 a document ending in a table can get a blank last page ours
#1101 Drive's Markdown import: - bullets, code not monospace, quotes without border, nested numbering 1./1-2-2, numbered tasks lose numbers Drive import
#1102 EP-8: nesting flattened, inline code plain, literal [ ], header not bold, quotes plain, bare URLs unlinked, nondeterministic code font EP-8
#1103 (read side) get_doc_as_markdown exports the pre-table spacer as an empty heading —
#1104 (read side) get_doc_as_markdown exports Drive-imported lists as > 1. —

Nondeterminism (EP-8)

The same fixture rendered through upload_file(source_format="markdown", convert_to_doc=True) sets its fenced code blocks in a monospace font one time and in Arial the next:

Fixture Run 1 Run 2 Extra renders
r04 monospace Arial —
r11 Arial "Courier" (font box, read in the live Doc) —
r13 monospace Arial 3 more, all Arial (1 of 5 monospace)
r03, r16 monospace monospace —

Each shard re-shot the changed cell after a reload to rule out a font-loading race. get_doc_as_markdown exports the Arial renders without a fence, consistent with the runs not being Courier New. Text, line breaks and indentation were identical across runs. Filed under #1102.

Findings that weren't filed

  • Pagination in Drive-imported Docs (EP-5 to EP-7): tables split with a repeated header row, and a heading moves to the next page with its table. Ordinary Docs pagination, not a rendering defect.
  • Uneven column widths in r11's action-items table (EP-1 to EP-4): auto-sized columns; column widths are #809's territory.
  • Drive's task items render as native Docs checklists rather than ● ☐ text. Equivalent on screen; recorded under #1101 as an engine difference, not as a failure of its own.

Process notes

  • Lock slip (Run 1, shard A): one acquire and a browser_navigate went out in the same parallel batch, so a navigate ran at 15:38:50Z while another shard held the Playwright lock. The screenshots taken around that moment (TC-MD006 EP-1 at 15:38:45Z, TC-MD008 EP-1 at 15:38:57Z) show the right Docs. Run 2's rules required the acquire in its own call before any Playwright call; no recurrence.
  • Aziz's worktree token is a personal Gmail account, not the fixture owner, so it can't read the Shared Drive. The pre-flight in aziz.md checks only that the token refreshes. The mechanical structure diff planned for this run was dropped for the pixel comparison above; structural questions went through get_doc_structure on kai-oauth.
  • Worktree-isolation guard: shards had a few Bash calls refused (shell variables, $((…)), compound commands) and re-ran them as plain commands or scratch scripts. One shard's rm -rf of its own scratch dir was blocked by the safety check; it removed the empty dirs with rmdir. No classifier denials of any approved write.
  • #1052: no connection resets in either run.

Resource usage

Token figures are each shard's subagent_tokens at its final stop.

Run Shard Prefix Tokens Tool uses Wall time
1 A kai-oauth 363,283 287 ~27 min
1 B oauth 338,032 231 ~27 min
1 C kai-sa 329,121 181 ~19 min
1 D sa 339,798 228 ~27 min
2 A kai-oauth 280,406 251 ~30 min
2 B oauth 267,609 217 ~24 min
2 C kai-sa 291,651 251 ~32 min
2 D sa 451,788 270 ~30 min
— Aziz (setup, repeats, compare, compile, filing) — not measured per step

Total across shards: about 2.66M tokens. The user reported 6% of a 5-hour session used before the shards started.

Next

Phase 2 (TC-MD101–117, the matrices TC-MD201–204), then TC-MD301–303 and TC-MD401–404, once the user gives the go-ahead. Phase 2 needs a share approval for the image cases (IM-04 to IM-06) that Phase 1 didn't.